Helper to create a StageTimer with optional key/value tags.

Importance notes: 1. Changes will be persisted to disk during the next index_done_callback 2. Only one process should updatin

Call LLM function with cache support If cache is available and enabled (determined by handle_cache based on mode), retrieve result from cache

Process a single extraction result (either initial or gleaning) Args: result (str): The extraction result to process c

Wrap a coroutine with timing so it can be scheduled via create_task().

Retrieve text chunks from the vector database without reranking or truncation. This function performs vector search to find relevant text ch

Process a single chunk Args: chunk_key_dp (tuple[str, TextChunkSchema]): ("chunk-xxxxxx", {"tokens": int, "content

Load a JSON file and return its content as a dict.

Merge nodes and edges from extraction results Args: chunk_results: List of tuples (maybe_nodes, maybe_edges) containing extracted entitie

Split *page_numbers* into *chunks* balanced sub‑lists.

Async Insert documents with checkpoint support Args: input: Single document string or list of document strings split_

Process pending documents by splitting them into chunks, processing each chunk for entity and relation extraction, and updating the

Perform an async query.

Clean text by removing null bytes (0x00) and whitespace Args: text: Input text to clean Returns: Cleaned text

Collect every embedded image (including table snapshots) on a single page and return their absolute (string) paths.

Merge every textual element that belongs to the same page. Returns a dict: {page_idx: {"text": <concatenated text>}}

Enable JSONL persistence. Call once at startup (we auto-call below).

Read queries from either: - JSONL: lines with {"question": "..."} (preferred) - Markdown-like lines: "- Question N: text"

Initialise a :class:`MegaRAG` instance for *working_dir* using *addon_params*.

Initialise a MegaRAG instance for *working_dir* using *addon_params*.

Sync Insert documents with checkpoint support Args: input: Single document string or list of document strings split_b

Insert the JSON contexts into the RAG store, retrying on failure.

Complete *prompt* with OpenAI, supporting optional image inputs. Args: model: Vision‑capable model (e.g. ``"gpt-4o-mini"``)

Render a waterfall chart for a given trace_id from the JSONL timing file. Requirements: pandas, matplotlib. Install: pip install pandas matplotli

Assemble per-page assets: pages_content[page_idx] = { "text": <merged text>, "page_image": <main pag

Render *page_numbers* (1‑based) of *pdf_path* to *output_dir*. Each worker opens its own document instance → no cross‑process state.

Generate embeddings for a list of texts using OpenAI's API. Args: texts: List of texts to embed. model: The OpenAI embedding mode

Process a single document end-to-end.

Run a single refinement round: extract again then merge.

Verify if storage implementation is compatible with specified storage type Args: storage_type: Storage type (KV_STORAGE, GRAPH_STORAGE et