Load tools from XML file.

Load tools from XML if not already cached.

Run per-entity LLM calls in parallel for assessment. Takes joined {key: {entity, associated}} or list of results. Returns list of ass

Recursively merge two dictionaries. Rules: - If attribute exists in only one dict, copy it - If attribute exists in both: -

Check if NLWebScorer checkpoints exist and config is enabled (cached).

Normalize a group's features using saved stats.

Print current crawl status on a single line.

Process a single URL: fetch, extract, embed, and upload.

Query a single site and stream results as they arrive.

Rank results from a single query. Args: results: List of 4-tuples (url, json_str, name, site) from search original_qu

Rank a single item for relevance to the query. Args: result_tuple: 4-tuple (url, json_str, name, site) original_query

Rank a single site and cache the result. Args: query: The user's query site: Site information dictionary

Background task to receive messages.

Receive messages in background.

Filter, sort, and limit assessed results.

Refresh the sites cache in the background.

Returns (select_expression, list_of_new_join_clauses).

Retrieve and rank results for a single query.

Internal method to retrieve top n records filtered by site and ranked by vector similarity Args: sites: Site or list of

Fallback to LLM planner

Execute a soft join query - analyze notes with LLM to derive risk

Select top N results from each query's ranked results. Returns: List of dicts, each containing: - item: the original

Send cricket statistics result to the client. Args: formatted_response: Formatted table string for display

Send an error message.

Send final summary of the multi-site query.

Send message when no recipes are found.

Send message when recipes already meet requirements.

Send notification that a site query completed.

Send notification that a site query failed.

Send status about a site being queried.

Send intermediate message with list of all sites that will be searched.

Send a status update message.

Send the substitution suggestions to the client.

Send tool selection messages in debug mode.

Setup SSL context if enabled

Generate narrative summary from assessed results.

Jaccard similarity between two texts (word-level).

Update TEST_METRICS.md with baseline measurements.

Warm the cache for common types.

* Adds an image to the item if available * * @param {Object} item - The item data * @param {HTMLElement} container - The container element

* Adds a visible URL to the content div * * @param {Object} item - The item data * @param {HTMLElement} contentDiv - The content div

Add a conversation to storage. Args: user_id: User ID (can be anonymous ID) site: Site context message_id: Message I

Analyze average absolute contribution of each rubric group.

Analyze average absolute contribution of each feature subnet.

Find items that appear across queries with inconsistent scores.

Find queries where all items score similarly (no ranking signal).

Analyze a single query for duplicates and source distribution

Analyze a JSONL file containing schema.org markup and return all types found with counts

Check if scores correlate with simple text relevance signals.

Use LLM to analyze URL patterns and predict content.

Find items in the same query with similar text but different scores.

Apply a specific profile's settings

Make a streaming request to NLWebServer and yield results as they arrive. Args: endpoint: The full URL of the endpoint query

Process multiple cricket queries concurrently. Args: queries: List of natural language queries Returns: List of formatt

Build [bert_score, rubric_feat_1, ..., rubric_feat_16] matrix.

Build the text input for ModernBERT: [query] [SEP] [readable item text].

Check if this provider can handle a query for the given site(s). Implements caching with stale-while-revalidate pattern. Arg

Check if this Shopify MCP client can handle a query for the given site(s). Returns True if the site contains 'shopify' in the domain

Categorize the type of error encountered

Check if at least half of the results have addresses and send a map message if so.

Check core dependencies from requirements.txt.

Check Inception API connectivity

Check Inception API connectivity

Check dependencies for enabled LLM providers.

Check dependencies for enabled retrieval backends.

Check if the database table schema is correctly set up. This helps diagnose issues with table schema that could prevent document uplo

Remove newlines from JSON string

Clean organization name for use as filename

Clean and extract JSON content from response text.

Strip markdown fences and extract the first JSON object.

Clean and parse Ollama response

Clean and extract JSON content from OpenAI response. Args: content: The content to clean. May be None. Returns:

Strip markdown fences and extract the first JSON object.

Strip markdown fences and extract the first JSON object.

Clean and parse DeepSeek response

Strip markdown fences and extract the first JSON object.

Strip markdown fences and extract the first JSON object.

Clean and parse Llama response

Clean a single data split. Returns cleaned examples and stats.

Clear all data from local Qdrant