Clear all data from local Qdrant

Clear all data from the local Qdrant database

Clear all conversations from Qdrant storage.

Check if a collection exists in Milvus. Args: collection_name: Name of the collection to check embedding_siz

Compare the two items and return the results.

Compare different description strategies.

Compute per-example quality weights (0-1) based on noise analysis.

Compute satisficer metrics on a validation/test split.

Convert string numbers to proper numeric types.

Send an async chat completion request via snowflake.cortex.complete and return parsed JSON output. See: https://docs.snowflake.com/en/user-g

Embed text using snowflake.cortex.embed. See: https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-llm-rest-api#label-cortex-llm

Crawl a list of URLs incrementally.

Crawl a single page and extract meaningful content.

Crawl a site and return sitemap URLs and content.

Crawl a single URL with retry logic

* Create intermediate message HTML

* Create a safe copy of DOM element using cloneNode to break data flow * @param {Element} element - The DOM element to copy * @returns {Element}

* Creates a title row for an item * * @param {Object} item - The item data * @param {HTMLElement} contentDiv - The content div

Create the web application

Create and configure the argument parser.

Create a Milvus collection with the specified settings. Args: collection_name: Name of the collection (defaults to confi

Create prompt for LLM to enhance site description.

Create an error message. Args: error_text: The error message text handler: Handler instance (provides conversation_id and se

Create expired token.

Create a single search index with the specified name and embedding size. Args: index_name: Name of the index to create (

Create the Elasticsearch index with proper vector mapping if it doesn't exist. Args: index_name: Optional index name (de

Create the initial user query message as a Message object. This message represents the user's original query. Returns:

Create token with invalid signature.

Create a Notion-style database with unstructured meeting notes.

Create a Qdrant filter for site filtering.

Create a status/intermediate message. Args: status_text: The status message text handler: Handler instance (provides convers

Factory for creating test conversations.

Create the HTML for a Data Commons web component.

Log a critical message.

Use GPT-4.1 to decompose query into aspects and generate partial queries.

Delete a specific conversation. Args: conversation_id: Conversation ID to delete user_id: Optional user ID for access contro

Delete the Elasticsearch index if it exists.

Delete all entries for a specific site from the database. Args: site: Site identifier to delete database: Specific database

Detect the encoding of a file, with special handling for UTF-16. Args: file_path: Path to the file to analyze Returns:

Parse a line with URL, JSON, and embedding into document objects. Args: line: Tab-separated line with URL, JSON, and embedding

Download checkpoint from URL to a temp file. Returns local path.

Download RSS feed content

Dump all conversations from Qdrant storage.

Use LLM to enhance the site description.

Enhance a single site's description.

Ensure that a collection exists, creating it if necessary. Args: collection_name: Name of the collection (defaults to co

Simulate connection pool exhaustion.

* Extracts an image URL from a schema object * * @param {Object} schema_object - The schema object * @returns {string|null} - The image URL

* Extract image URL from various schema formats

Extract the best URL for an RSS item. Args: item: RSS item element feed_url: URL of the feed itself (for fallback) Retu

Extract meaningful content from HTML.

Extract and display descriptions from JSONL file.

Extract and normalize duration from various formats. Args: duration_str: Duration string from feed Returns: Normalized

Extract GUID from an RSS item element. Args: item: RSS item element Returns: GUID string or None

Extract metadata from the response.

Extract query strings from the JavaScript sample queries file.

Extract query strings from the JavaScript sample queries file.

Extract RSS feed URL from various possible locations

Recursively extract all @type values from a JSON object

Extract all possible URL candidates from an RSS item. Args: item: RSS item element Returns: List of URL candidates

Extract URLs from a sitemap (limited to avoid overwhelming the LLM).

Convert an RSS/Atom feed to Schema.org format. Args: feed_path: Path to the feed file Returns: List of Schema.org forma

* Fetch UI resource from uri://

Fetch RSS feed and save to temporary file

* Formats a value as HTML * * @param {*} value - The value to format * @param {number} indent - The indentation level * @returns {string}

Format rows and metadata for LLM consumption.

Format Qdrant search results to match expected API: [url, text_json, name, site].

Format rows as a text table.

Forward pass. Args: bert_embedding: (batch, bert_dim) — [CLS] embeddings rubric_features: dict mapping group name → (

* Generates a star rating display * * @param {number} rating - The rating value (0-5) * @returns {string} - Star characters representi