Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Calculates 1 - comb(n - c, k) / comb(n, k).

Find the first available file with given extensions.

Find first valid option for text.

Get the computational cost of inferring on the dataset. Args: dataset (ConfigDict): The dataset config. get_raw_facto

Get the computational cost of inferring on the dataset. Args: dataset (ConfigDict): The dataset config. get_raw_facto

Load data from a JSON or JSONL file.

Match the config file in workdir recursively given the pattern. Additionally, if the pattern itself points to an existing file, it will be di

提取 <material> 标签内容

Lower text and remove punctuation, storys and extra whitespace.

in Elo we want numpy arrays for matchups and outcomes matchups: int32 (N,2) contains model ids for the competitors in a match outcom

Cilin 詞林 is a thesaurus with semantic information

Calculate scores. Args: predictions (List): List of predictions of each sample. references (List): List of targets fo

Apply the transformation on the input coordinates Args: pos (): the 3-D coordinates to transforms, of s

Wait till the next query can be sent. Applicable in both single-thread and multi-thread environments.

Runs basic whitespace cleaning and splitting on a peice of text.

Extract the protein str between <protein> and </protein> in the sentences

Create a new one or in-place assignment Args: translation: the translation (center) of the frame rotation: t

Cleans up the generated code.

深度优先遍历，获取最小编辑距离相同的所有序列

Evaluate code with retry mechanism. Args: input_data: Can be either: - dict: Dictionary containing code and test

Extract code from markdown-formatted text. Args: text (str): Text that may contain code blocks in markdown format. Retur

NOTE: This is a replacement of ERRANTS lemma cost for Chinese 计算基于字符相似度的编辑距离cost

Get the condition probability of next token. Args: input_texts (List[str]): All the input prompt to be tested. target

Get loglikelihood scores given input string and continuation string. Args: inputs (str): string. conts (str): strings

Get perplexity scores given a list of inputs. Args: inputs (List[str]): A list of strings. mask_length (Optional[List

NOTE: Based on the paper: Improved-Edit-Distance Kernel for Chinese Relation Extraction 获取每个词语的语义类别（基于大词林，有三个级别）

计算基于语义信息的替换操作cost :param a: 单词a的语义类别 :param b: 单词b的语义类别 :return: 替换编辑代价