Get the module of a metric from a metric repository on the Hub.

Helper class to avoid loading a dataset column into memory when accessing it.

Base class to store information about an evaluation used for `MetricInfo`, `ComparisonInfo`, and `MeasurementInfo`. `EvaluationModuleInfo` do

Get the module of a local metric. The metric script is loaded from a local script.

Get the module of a metric that has been loaded once already and cached. The script that is loaded from the cache is the most recent one with

Thread lock until a file **cannot** be locked

Question answering evaluator. This evaluator handles [**extractive** question answering](https://huggingface.co/docs/transformers/task_summar

Audio classification evaluator. This audio classification evaluator can currently be loaded from [`evaluator`] using the default task name

Automatic speech recognition evaluator. This automatic speech recognition evaluator can currently be loaded from [`evaluator`] using the defa

Create a complex radar chart with different scales for each variable Args: fig (`matplotlib.figure`) : A matplotlib figure object to add the

Helper class to avoid loading two dataset columns into memory when accessing it.

Dummy tqdm which doesn't do anything.

Image classification evaluator. This image classification evaluator can currently be loaded from [`evaluator`] using the default task name

Text2Text generation evaluator. This Text2Text generation evaluator can currently be loaded from [`evaluator`] using the default task name

Text classification evaluator. This text classification evaluator can currently be loaded from [`evaluator`] using the default task name

Text generation evaluator. This Text generation evaluator can currently be loaded from [`evaluator`] using the default task name `text-ge

Token classification evaluator. This token classification evaluator can currently be loaded from [`evaluator`] using the default task name

Helper class to wrap a callable into a class with a `tokenize` method as used by rouge-score.

StringIO that throws an exception when it's read from

A base dummy tokenizer to derive from.

A base dummy tokenizer to derive from.

CharacTer is a character-level metric inspired by the commonly applied translation edit rate (TER).

Character-based MT evaluation.

A Comparison is the base class and common API for all comparisons. Args: config_name (`str`): This is used to define a hash s

Information about a comparison. `EvaluationModuleInfo` documents a comparison, including its name, version, and features. See the constructor

Accuracy metric for the MATH dataset.

A `EvaluationModule` is the base class and common API for metrics, comparisons, and measurements. Args: config_name (`str`):

This base class exposes some attributes of EvaluationModuleInfo at the base level of the EvaluationModule for easy access.

This class instantiates an evaluation suite made up of multiple tasks, where each task consists of a dataset and an associated metric, and ru

The [`Evaluator`] class is the class from which all evaluators inherit. Refer to this class for methods shared across different evaluators.

A Measurement is the base class and common API for all measurements. Args: config_name (`str`): This is used to define a hash

Information about a measurement. `EvaluationModuleInfo` documents a measurement, including its name, version, and features. See the construct

A Metric is the base class and common API for all metrics. Args: config_name (`str`): This is used to define a hash specific

Information about a metric. `EvaluationModuleInfo` documents a metric, including its name, version, and features. See the constructor argumen

A wrapper around NLTK's NIST implementation.