计算基于词性信息的编辑距离cost :param a_pos: 单词a的词性 :param b_pos: 单词b的词性 :return: 替换编辑代价

计算两个单词拼写相似度，分别由字形相似度和字音相似度组成 :param a: 单词a :param b: 单词b，且单词a的长度小于等于b :param pinyin_a: 单词a的拼音 :param pinyin_b

Handle streaming response and collect all chunks into final response. Args: response_stream: The streaming response from

Build resource when request in real.

Checks whether CP is the codepoint of a CJK character.

Checks whether `chars` is a control character.

Checks whether `chars` is a punctuation character.

Checks whether `chars` is a whitespace character.

Launch a single task. Args: task (BaseTask): Task to launch. Returns: tuple[str, int]: Task name and exit co

Launch a single task. Args: cfg (ConfigDict): Task config. random_sleep (bool): Whether to sleep for a random time be

Launch a single task. Args: task_cfg (ConfigDict): Task config. random_sleep (bool): Whether to sleep for a random ti

Launch a single task via rjob bash script.

Launch a single task. Args: cfg (ConfigDict): Task config. random_sleep (bool): Whether to sleep for a random time be

Load test dataset and apply postprocessing if needed.

Load model predictions from files.

Output summary dataframe to file. Args: output_path (str): Output path timestamp (str): Timestamp for file suffix

The function reads the numerical results of evaluations from the output folder based on the configuration file, and ultimately returns

The function reads the numerical results of evaluations from the output folder based on the configuration file, and ultimately returns

Postprocess for final scores. Args: scores (dict): Dict of calculated scores of metrics. Returns: dict: post

Prepare evaluation dataset by calculating total number of characters and from original text and concatenating encodings into a single

Preprocess the final predictions and references to needed format. Args: predictions (List): List of predictions of each sample.

Process code completions to extract the relevant code. Args: completion (str): Code completion string. Returns:

Apply various processing steps to predictions.

Process the response to needed format. Args: datum(dict): inputs. Returns: dict: Processed response data sampl

Process the response to needed format. Args: datum(dict): inputs. Returns: dict: Processed response data sampl

Process the response to needed format. Args: datum(dict): inputs. Returns: dict: Processed response data sampl

Process the response to needed format. Args: datum(dict): inputs. Returns: dict: Processed response data sam