Return the device.

Return the device.

Helper function to draw bounding boxes on video frames.

Returns a dictionary representing the task's config. :returns: str The fewshot context.

Evaluate the given request and return a response Args: request: JudgeRequest containing the evaluation context

Extract and validate position data. Args: position: The position value Returns: A tuple of (x, y) coordinates

result_full: [{'title', 'snippet', 'content','screenshot_path'}]

Gets the current git branch name (if in a repo).

Get job status and results. Args: job_id: The job ID returned from evaluate() Returns: Dict with jo

Get model output for the given messages. Args: messages: Messages in OpenAI chat format Returns: Mo

Creates a dictionary of task objects from either a name of task, config, or prepared Task object. :param task_name_list: List[Union[str, Dict, Ta

Encode PIL image to base64 JPEG.

Convert PIL Image or image path to base64 string

Check if a facing value is valid (one of: up, down, left, right, inner, outer). Args: facing: The facing value to check Returns:

Check if a position value is valid (a list of 2 numeric values). Args: position: The position value to check Returns: True i

Calculate the Jaccard Index.

Return canonical model ids.

Load data from a JSONL file. Args: jsonl_path: Path to the JSONL file Returns: List of loaded JSON objects

Return the mean squared error.

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Normalize the str to lower case and make them float numbers if possible. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445

Parse the JSON object, including nested JSON strings.

Parse multiple choice answers from various formats. Handles formats like: "A", "A.", "A)", "(A)", "The answer is A", "A: xxx", etc.

A wrapper for pronouncingpy's phones_for_word to handle out-of-vocab issues.