Returns model ranking.

Determine the truncation point in the completion based on the last line of the prompt, remove all content before that line in the comp

Convert a role prompt to a string, given an updated role_dict. Args: role_prompt (Dict): The role prompt to be converted.

Runs the compiled executable and interprets the result.

Splits punctuation on a piece of text.

Strips accents from a piece of text.

Poll rjob status until both active and pending are 0. Break if no dict line is found.

Save evaluation results to file.

Support for single prompt inference. Args: inputs (List[str]): A list of strings. max_out_len (int): The maximum leng

Split the prompt template into rounds, based on single round template. Return the index ranges of each round. Specifically, p

Split the prompt template into rounds, based on single round template. Return the index ranges of each round. Specifically, p

Produces the prefix of decoded_string that ends at the first occurrence of a stop_token. WARNING: the decoded_string *must not* inclu

Adds whitespace around any CJK character.

Trims the prediction string based on the length of the reference string. Args: prediction (str): The prediction string.

The Gumbel max is a method for sampling categorical distributions. According to arXiv:2409.02908, for MDM, low-precision Gumbel Max improves

The Gumbel max is a method for sampling categorical distributions. According to arXiv:2409.02908, for MDM, low-precision Gumbel Max improves

The Gumbel max is a method for sampling categorical distributions. According to arXiv:2409.02908, for MDM, low-precision Gumbel Max improve

Add a message pair to the history and return the updated history

:param line: :return:

:param line: :return:

Create original dataset and concat to the end.

Calculate the similarity between predicted plan and golden answer, A plan can be regarded a sequence of actions, and each act

Check if the brackets in a LaTeX string are balanced Args: s(str): the input string Return: bool: True if the brackets ar

Check if the binary search solution is valid. :param instance: The instance dictionary with array and target value. :param solution:

Build LLM Inference.

Build LLM Inference.

This function is used to calculate the BPB (Bits Per Byte) for the data. The correct BPB is obtained directly from the values in the '

Check correctness of code generation with a global timeout. The global timeout is to catch some extreme/rare cases not handled by the

Check correctness of code generation with a global timeout. The global timeout is to catch some extreme/rare cases not handled by the

A tricky way to check internet.

Check position bias for judgellm's judgement. Args: judged_answers: The successfully extracted judgement. references: The referen

Exclude response without real answer.

Check if a specific URL is accessible through optional proxy. Args: url: str, target URL to check timeout: float, timeout in seco

Clean answer format specifically for QwQ-32B-Preview model. Args: text: Raw prediction text Returns: Standardized name forma