Extract citations from text and organize by paragraphs and sentences. Args: text: Text containing citations in [n] format

Get configuration for a specific dataset. Args: dataset_name: Name of the dataset (Researchy-GEO, E-commerce, or GEO-Bench)

r"""Return the list of all elements.

Get or create a logger instance. Args: log_dir: Directory to save log files task_name: Name of the task log_level: Lo

Get the model

Reward function that penalizes overlong completions. It is used to penalize overlong completions, but not to reward shorter completions. Refe

r"""Get the token ids of thought words.

Get the tokenizer for the model.

Calculate impression score based on position only. Args: sentences: Nested list of sentences with citations n: Number of

Calculate impression score based on word count only. Args: sentences: Nested list of sentences with citations n: Number o

Calculate impression score based on word count and position. Args: sentences: Nested list of sentences with citations n:

r"""Initialize the adapters. Support full-parameter, freeze and LoRA training. Note that the trainable parameters must be cast to float32.

r"""Extract a chat template from the tokenizer.

Call LLM with retry logic and JSON parsing. Args: prompt: Prompt text for LLM llm_args: Dictionary of LLM arguments

r"""Configure root logger using a stdout stream handler with an explicit format.

r"""Encode formatted inputs to pairs of token ids. Turn 0: prefix + system + query resp Turn t: query

r"""Get arguments to load config/tokenizer/model. Note: including inplace operation of model_args.

r"""Preprocesses the dataset, including format checking and tokenization.

r"""Validate the configuration.

r"""Update gradio components according to the (elem_id, properties) mapping.

Evaluate AutoGEO API rewritten documents. Args: num_examples: Number of examples to process data_dir: Directory containing da

Call Gemini API with retry logic. Args: user_prompt: User prompt text system_prompt: System prompt text (default: COMMON_SYST

r"""Get a list of responses of the chat model.

r"""Create reference model for PPO/DPO training. Evaluation mode is not supported. The valuehead parameter is randomly initialized since it is us

r"""Return multiple pairs of token ids representing prompts and responses respectively.

Estimate token count for a list of rules. Args: rules_list: List of rule strings Returns: Estimated token

r"""Find all available modules to apply LoRA, GaLore or APOLLO.

r"""Fix proxy settings for gradio ui.

r"""Fix the valuehead checkpoint files. The model is already unwrapped. There are three cases: 1. full tuning without ds_zero3: state_di

r"""Convert dataset examples to messages.

Reward function that checks if the reasoning process is enclosed within <think> and </think> tags, while the final answer is enclosed within <answer>

Generate answer using Claude with RAG. Args: query: User query text sources: List of source document texts model

Generate answer using Gemini with RAG. Args: query: User query text sources: List of source document texts model

Generate answer using GPT with RAG. Args: query: User query text sources: List of source document texts model_na

r"""Get the base elements that are commonly used.

r"""Get a dataset converter.

r"""Freeze vision tower and language model for VLM full/freeze tuning.

Get response from appropriate LLM based on model name. This function uses the centralized LLM clients from utils, with support for pas

Get the rewrite prompt template without calling the API. Useful for testing or when you only need the prompt. Args: dataset:

r"""Get a list of scores of the reward model.

Helper function for setting up Weights & Biases logging tools.

r"""Load model config.

Load dataset from Hugging Face and reconstruct into local JSON files. Args: config: Configuration dictionary with dataset settings

Parse rewrite method name to extract dataset name and LLM name. Args: rewrite_method_name: Method name in format 'autogeo_{dataset}_{

Process prediction text to extract rewritten content. Args: text: Raw prediction text that may contain introductory phrases l

r"""Remove thought from assistant message.

r"""Add or replace eos token to the tokenizer.

r"""Build a dictionary containing the current training configuration.

r"""Check if the video is nested images.

r"""Remove args with NoneType or False or empty string value.

r"""Clean up the server process when the engine is destroyed.

r"""Convert elements to token ids.

r"""Get the path to user config.

Get default rules for a dataset when extracted rules are unavailable. These are fallback rules used when extracted rules are not available.

r"""Return the jinja template.

r"""Return the merged datasets in the standard format.

Get or create a snapshot with the necessary dependencies and scripts for evaluation.

r"""Compute video sample indices according to fps.

Initialize or get the current event loop.

r"""Start the training process.