Modify ``x`` in place to replace ``float("-inf")`` with the minimum value of the dtype when ``check_neg_inf`` is ``True`` and to replace ``fl

Modify ``x`` in place to replace ``float("-inf")`` with the minimum value of the dtype when ``check_neg_inf`` is ``True`` and to replace ``

Modify ``x`` in place to replace ``float("-inf")`` with the minimum value of the dtype when ``check_neg_inf`` is ``True`` and to replace ``

Estimates pass@k of each problem and returns them in an array.

Calculates 1 - comb(n - c, k) / comb(n, k).

Evaluate a multiple choice instance. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#

Evaluate a multiple choice instance. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#

Evaluate a multiple choice instance. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#

Evaluate a multiple choice instance. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#

Evaluate an open question instance https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#L1

Evaluate an open question instance https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#L1

Evaluate an open question instance https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#L1

Evaluate an open question instance https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eval/eval_utils.py#L1

Instantiate and evaluate a model on a list of tasks. :param lm: obj Language Model :param task_dict: dict[str, Task] Dictiona

Evaluates the generation and populates the score and explanation fields.

Batch evaluation for multiple choice and open questions. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794

Batch evaluation for multiple choice and open questions. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794

Batch evaluation for multiple choice and open questions. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794

Batch evaluation for multiple choice and open questions. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794

Evaluates whether the answer key and student response are semantically the same. Returns a score of 1 if they match, otherwise 0.

evaluation_imports: Dictionary ( key = module name , value = alias ) with python modules used in the evaluation.

Brought from MMStar

Brought from MMStar

Extract the number from a response containing {N} format.

Extracts the answer from the output string based on the task type. Parameters: output_string (str): The output string. task_type (st

Extract the answer letter from model response.

Extract coordinates and text content from the given prediction string, handling potential format issues. Args: predict_str (str): Mo

Exact all forms of numbers from a string with regex.

Exact all forms of numbers from a string with regex. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eva

Exact all forms of numbers from a string with regex. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eva

Exact all forms of numbers from a string with regex. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eva

Exact all forms of numbers from a string with regex. https://github.com/MMMU-Benchmark/MMMU/blob/51ce7f3e829c16bb44bc5445782686b4c3508794/eva

Returns a fewshot context string that is made up of a prepended description (if provided), the `num_fewshot` number of examples, and an append

:return: Iterable[obj] A iterable of any object, that doc_to_text can handle