Save evaluation results to files

Setup computation device

平衡分布的自适应混合，增强数值稳定性

修复版本：确保plan_logits不产生梯度

修复版本：确保random_logits不产生梯度

Attempt to classify the ALFWorld task type Args: task_description: Task description text task_item: Original

Clean up distributed resources

Given an image, return the base64 encoded image string.

Create a specialized prompt for ALFWorld task planning Args: task_description: Description of the household task

Create and configure argument parser

Create and configure argument parser

Create metadata about the generation process Args: task_item: Original dataset item Returns: Dictio

Create metadata about the generation process Args: task_item: Original trajectory item Returns: Dic

Create a specialized prompt for mathematical planning Args: problem: The mathematical problem statement Returns

Create evaluation prompt for a math question Override this method for different prompt formats

Create prompt for mathematical reasoning task Args: task_item: Dictionary containing problem data Returns:

Create prompt for ALFWorld task planning Args: task_item: Dictionary containing conversation trajectory Returns

Simulate group-wise dequantization.

Automatically detect the best supported precision on the current device. Returns: "bf16", "fp16", or "no"

Distribute dataset across processes

Estimate memory size of a sample in bytes

Evaluate model performance

Evaluate model performance; use base_offset to ensure globally unique task_id

Evaluate if model output matches ground truth Returns: Tuple of (is_correct, predicted_answer, normalized_ground_truth)

Evaluate model on a math dataset Args: dataset_name: HuggingFace dataset name split: Dataset split to use

Extract metadata from a math task item Args: task_item: Original dataset item task_index: Local index of the

Extract metadata from an ALFWorld task item Args: task_item: Original trajectory item task_index: Local inde

Each process saves its own data; the main process merges them

Each process saves its own data; the main process merges them

Complete the distributed data collection process

生成更合理的随机hidden states

Generate model response for a given prompt

Get predefined configuration by name

Get a predefined configuration by name Args: config_name: Name of the predefined configuration Returns: DataCollectionC