Parse document with caching support Args: file_path: Path to the file to parse output_dir: Output directory

Run pipeline from specified stage. Args: base_dir: Base directory for this document/project config_dir: Config-specific direc

Configure logging with console output.

Generate a summary of the batch processing results

Convert to markdown format with caption.

Generate the final summary by combining relevant sections. Args: include_titles: Whether to include section titles in the

Create with custom paths.

Initialize context extractor Args: config: Context extraction configuration tokenizer: Tokenizer for accurate token c

Basic cleanup for common JSON issues

Add belongs_to relations for multimodal entities

Use LightRAG's extract_entities for batch entity relation extraction

Use LightRAG's merge_nodes_and_edges for batch merge

Build prompt for multiple file extraction (assumes independent papers).

Build prompt for poster.

Build simple prompt for single file extraction.

Build VLM message format, using markers to correspond images with text positions Args: enhanced_prompt: Enhanced prompt

Call the official Google Gemini API for image generation.

Call the image generation model with retry logic.

Call VLM to process multimodal content Args: messages: VLM message format Returns: str: VLM respons

Check which conversion backends are available

Convert multimodal data to LightRAG standard chunks format

Generate equation description for query

Generate generic content description for query

Generate image description for query

Generate table description for query

Encode image file to base64 string

Extract context based on content chunks Args: content_list: List of content items current_item_info: Current item wit

Extract required fields using regex as last resort

Extract context from dictionary-based content source Args: dict_source: Dictionary containing content current_item_in

Extract context from simple text chunks list Args: text_chunks: List of text strings current_item_info: Current item

Extract plain text from markdown file, removing image links. Args: md_path: Path to markdown file max_chars: Maximum cha

Search for Figure caption near an image. Search strategy: First search forward (more common), then backward. Stops if encountering a

Fallback: Search for Figure caption by character position. This is used when line-based search fails, typically when the markdown ha

Search for Table caption near an HTML table. Search strategy: First search backward (more common), then forward. Stops if encounteri

Fallback: Search for Table caption by character position. This is used when line-based search fails, typically when the markdown has

Format ProcessedStyle into style hints string for poster.

Format ProcessedStyle into style hints string for slide.

Format all sections as markdown.

Generate cache key based on file path and parsing configuration Args: file_path: Path to the file parse_meth

Generate cache key for multimodal query Args: query: Base query text multimodal_content: List of multimodal

Generate 1 poster image.

Generate content description for query Args: processor: Multimodal processor content: Content data

Generate N slide images (slides 1-2 sequential, 3+ parallel).

Get cached parsing result if available and valid Args: cache_key: Cache key to look up file_path: Path to th

Get default CSS styling

Get MIME type for image file based on extension

Load figure images as base64.

Load figure images as base64 with caption.

Parse equation analysis response with robust JSON handling

Parse generic analysis response

Parse LLM response to extract query strings.

Parse model response

Parse table analysis response

Plan poster sections.

Plan slides sections.

Process chunk for entity and relationship extraction

Process image paths in prompt, keeping original paths and adding VLM markers Args: prompt: Original prompt Retu

Process Markdown content with extensions

Type-aware batch processing that selects correct processors based on content type. This is the corrected implementation that handles

Process multimodal content individually (fallback method) Args: multimodal_items: List of multimodal items f

Process multimodal query content to generate enhanced query text Args: base_query: Base query text multimoda

Remove markdown format tables from content. A valid markdown table must have: - At least 2 consecutive lines starting and ending wit

Replace image references in markdown with base64 encoded images, preserving position Args: markdown_content: Markdown text conte

Fast mode: Direct GPT-4o queries with markdown content and images in original positions Args: client: OpenAI client mark

Store parsing result in cache Args: cache_key: Cache key to store under content_list: Content list to cache

Store chunks to storage

Store multimodal main entities to full_entities storage. Args: entities_to_store: Dictionary of entities to store

Store multimodal main entities to entities_vdb and full_entities. This ensures that entities like "TableName (table)" are properly in

Update document status with multimodal chunks

Update state.json when background pipeline fails

VLM enhanced query - replaces image paths in retrieved context with base64 encoded images for VLM processing Args: query

Multimodal query - combines text and multimodal content for querying Args: query: Base query text multimodal

Query multiple questions with sliding window concurrency. Args: questions: List of questions to query.

Execute batch queries organized by category with sliding window concurrency. All queries across all categories are executed

Cancel a running session. Returns True if session was running

Release resources.

Convert Markdown file to PDF Args: input_path: Input Markdown file path output_path: Output PDF file path (o

Convert markdown content to PDF Args: markdown_content: Markdown content to convert output_path: Output PDF

Convert Office document (.doc, .docx, .ppt, .pptx, .xls, .xlsx) to PDF. Requires LibreOffice to be installed. Args:

Convert text file (.txt, .md) to PDF using ReportLab with full markdown support. Args: text_path: Path to the text file

Convert using WeasyPrint (best for styling)

Encode image file to base64 string Args: image_path: Path to the image file Returns: str: Base64 encoded string, empty

End a session

Extract context for current item from content source Args: content_source: Source content (list, dict, or other format)

Extract figures from markdown content. Args: content: Full markdown content lines: Content split by lines search

Extract structured content from RAG results for a general document. Args: rag_results: List of query results llm_client:

Extract structured content from RAG results for a paper. Args: rag_results: RAG query results llm_client: OpenAI client

Extract paper metadata (title, authors, affiliations) directly from markdown files. Bypasses RAG queries and extracts from raw markdown text.

Extract tables from markdown content. Args: content: Full markdown content lines: Content split by lines search_

Generate images from ContentPlan. Args: plan: ContentPlan from ContentPlanner gen_input: GenerationI

Generate text description and entity info only, without entity relation extraction. Used for batch processing stage 1. Args: