Extract token usage from a LangChain response object.

Process a PDF file through the full ingestion pipeline. Steps: 1. Extract text from PDF pages 2. Split pages into chunks

Retrieve relevant text chunks from a vector database using embeddings, using traditional BM25 search and vector search then uses a reranker

Get or initialize the LangChain Chroma vectorstore.

Split pages into chunks while preserving metadata.

Combine BM25 and vector search results using Reciprocal Rank Fusion (RRF).

Convert raw retrieval chunks into Source objects.

Call LLM providers with retry + exponential backoff + failover. Tries each provider in order (Groq → Anthropic → OpenAI). Each provider gets

Return providers that have API keys configured, in failover order. Order: Groq (free) → Anthropic → OpenAI. Only includes providers whose API

Return a Langfuse client, or None if tracing is disabled or unavailable.

Rerank chunks against a query using a cross-encoder model.

Core RAG pipeline: retrieve → build prompt → call LLM → validate citations.

Return the total number of chunks in the vectorstore without loading them.

Instantiate the LangChain chat model for the given provider. Imports are local so users only need to install the package for providers they a

Generate embedding for a single query text. Args: text: The text to embed. Returns: List of float embedding values.

Call this once at app startup. Crashes immediately with a clear message if critical vars are missing. Much better than cra

Convert chunk dictionaries to LangChain Documents.

Get or initialize the embedding model.

Generate a cited answer for the given query using the RAG pipeline.

Check if any chunks exist in the vectorstore without loading them.

Semantic similarity search in vector store. Uses embed_query to embed the query.

Convert LangChain Documents back to chunk dictionaries.

Search the BM25 index using LangChain retriever.

Build a BM25 index from chunks using LangChain BM25Retriever.

Build the citation prompt for the RAG system.

Load all chunks from the vectorstore. Uses the underlying collection for efficient retrieval.

Generate embeddings for chunks in batches.

Returns the underlying Chroma collection for compatibility. Accesses the internal _collection from LangChain Chroma.

Add or update chunks in the vectorstore using LangChain add_documents.

Inject provider API keys from sidebar UI into Config.

Call a single provider with retry + exponential backoff.

Run the RAG pipeline with Langfuse tracing spans around each step.

Display answer with expandable citations.

Get a BM25Retriever for use in chains.

Get the LangChain Chroma vectorstore for use in chains. Returns: Chroma vectorstore instance.

Handle user query and generate response.

Initialize session state variables.

Main app entry point.

Process PDF through ingestion pipeline.

Render main chat area.

Render left sidebar with upload and document list.

Resets the singleton vectorstore (useful for tests).

Save uploaded PDF to data/raw/ directory.

Test batch chunk embedding preserves metadata.

Test single query embedding returns correct format.

Test that model is lazy-loaded and reused.

Test that similar texts have higher similarity.

Reset module-level state before each test.

Verify doc_id and page_num survive from extract through chunk to embed.

Verify chunk_pages adds total_chunks metadata.