Initializes a dedicated AI-based reranker.

Initializes the reranker for hybrid search score fusion.

Define service configurations based on mode.

Retrieves a window of chunks around a central chunk using LanceDB.

Handle query using direct Ollama client with thinking disabled for speed. Returns: tuple: (response_text, empty_

Handle query using the full RAG pipeline (delegates to the advanced RAG API running on port 8001). Returns: tuple[str, L

Attempt to authenticate with Hugging Face Hub using an env token. We support both the new canonical env var name (HF_TOKEN) and the two histo

Load configuration from file or use default.

Load batch indexing configuration from file.

Load and aggregate overviews for the given index IDs. Strategy: 1. Attempt to load each index's dedicated overview file.

Merge batch config with default pipeline config.

Print final indexing statistics

Report current progress

🎯 Use document overviews and LLM to make intelligent routing decisions. Returns True if RAG should be used, False for direct

Use document overviews and a small model to decide routing. Returns 'rag_query', 'direct_answer', or None if unsure/disabled.

🧠 ENHANCED: Determine if a query should use RAG pipeline using document overviews. Args: message: The user's que

Special handling for Ollama startup.

Stop a single service.

Add a new SSE connection

Adds a document file path to a session.

Create index from batch configuration file.

Test agent initialization

Validate configurations

Test database access

Test embedding model

Test basic imports

Check all pipeline configurations for completeness.

Check if all required tools are available.

Test a sample query if tables exist

Chunks the Markdown text and injects metadata. Args: text: The Markdown text to chunk. document_id: The iden

Walk a DoclingDocument and emit chunks. Tables / Code / Figures are emitted as atomic chunks. Paragraph-like nodes are sentence-packe

Interactive processing configuration.

Converts a document to a single Markdown string, preserving layout and tables. Supports PDF, DOCX, HTML, and other formats.

Run the interactive index creation process.

Create multiple indexes based on configuration.

Create a sample batch configuration file.

Create a comprehensive sample configuration file.

Create a single index from configuration.

Decompose *query* into standalone sub-queries. Parameters ---------- query : str The latest user message.

Delete an index and its related records (documents, session links). Returns True if deleted.

Delete a session and all its messages

Demonstrate various indexing features.

Extract text from PDF bytes

Mark progress as finished and send final update

Generate embeddings for all chunks using batch processing

Generates a completion using Watson X foundation models. Args: model: The name/ID of the Watson X model (e.g., '

Asynchronous version of generate_completion using httpx.

Get conversation history in the format expected by Ollama

Factory function to get an instance of the Indexing Pipeline.

Handle legacy chat requests (without sessions)

Handles a chat query by calling the agentic RAG pipeline.

Handles a chat query by calling the agentic RAG pipeline.

Stream internal phases and final answer using SSE (text/event-stream).

Clean up empty sessions

Create a new chat session

Remove an index, its documents, links, and the underlying LanceDB table.

Delete a session and its messages

Handle file uploads, save them, and associate with the session.

Get available models from both Ollama and HuggingFace, grouped by capability

Get a specific session with its messages

Return documents and basic info for a session.

Get all chat sessions

Triggers the document indexing pipeline for specific files.

Triggers indexing for all documents in a session.

Reuse file upload logic but store docs under index.

Triggers the document indexing pipeline with real-time progress tracking.

Return a list of locally installed Ollama models and supported HuggingFace models, grouped by capability.