Functions — github.com/Mwie1024/Extra-CoT

hub / github.com/Mwie1024/Extra-CoT / functions

Functions1,850 in github.com/Mwie1024/Extra-CoT

⨍ Functions 1,850◇ Types & classes 174↳ Endpoints 4

↓ 5 callersFunctionload_json

(file, encoding='utf-8')

src/data/compressor_longformer/validate/Tokenskip_pipeline/compression_llmlingua2/build_llamafactory_input_numeric.py:6

↓ 5 callersFunctionrewrite_item

构造一条 SFT 三元组；add_token_io=True 表示 input 末尾追加 token、output 头部前置 token。

src/data/sft/longformer_pipeline/query_result/autofollow_sft_dataset/rewrite_follow_to_unified_json_v3.py:57

↓ 5 callersFunctionrun_exp

(args: Optional[dict[str, Any]] = None, callbacks: Optional[list["TrainerCallback"]] = None)

llamafactory/src/llamafactory/train/tuner.py:94

↓ 5 callersMethodstream_chat

r"""Get the response token-by-token of the chat model.

llamafactory/src/llamafactory/chat/chat_model.py:94

↓ 4 callersFunction_as_ratio

Accept '20' or '0.2' → 0.2

src/training/qwen1p7b_rl/dataset/rl_4k/convert_data_to_verl.py:93

↓ 4 callersFunction_as_ratio

Accept '20' or '0.2' → 0.2

src/training/qwen1p7b_rl/dataset/rl_tiktoken_10k/convert_data_to_verl.py:93

↓ 4 callersMethod_convert_slots_to_ollama

r"""Convert slots to ollama template.

llamafactory/src/llamafactory/data/template.py:274

↓ 4 callersFunction_escape_html

r"""Escape HTML characters.

llamafactory/src/llamafactory/webui/chatter.py:41

↓ 4 callersFunction_get_decay_parameter_names

r"""Return a list of names of parameters with weight decay. (weights in non-layernorm layers).

llamafactory/src/llamafactory/train/trainer_utils.py:188

↓ 4 callersMethod_jinja_escape

r"""Escape single quotes in content.

llamafactory/src/llamafactory/data/template.py:210

↓ 4 callersFunction_latex_to_plain_basic

无 SymPy 时的极简归一：去包裹、把 \frac{a}{b} -> a/b，取等号右侧做值比较友好。

src/eval/qwen1p7b_eval/recheck_accuracy.py:112

↓ 4 callersFunction_latex_to_plain_basic

(s: str)

src/eval/qwen1p7b_eval/mmlu-stem/recheck_acc.py:101

↓ 4 callersFunction_parse_args

( parser: "HfArgumentParser", args: Optional[Union[dict[str, Any], list[str]]] = None, allow_extra_keys: b

llamafactory/src/llamafactory/hparams/parser.py:74

↓ 4 callersFunction_to_numeric

(s: str)

src/eval/qwen1p7b_eval/recheck_accuracy.py:129

↓ 4 callersFunction_to_numeric

(s: str)

src/eval/qwen1p7b_eval/mmlu-stem/recheck_acc.py:115

↓ 4 callersMethodbackward

(ctx: "torch.autograd.Function", grad_output: "torch.Tensor")

llamafactory/src/llamafactory/model/model_utils/checkpointing.py:64

↓ 4 callersFunctioncalculate_pixels

r"""Calculate the number of pixels from the expression.

llamafactory/src/llamafactory/webui/common.py:220

↓ 4 callersFunctioncount_tokens

(s: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/build_diff_by_model_ratio.py:13

↓ 4 callersFunctioncount_tokens

(s: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/build_diff.py:12

↓ 4 callersFunctioncount_tokens

(s: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/filter_by_ratio_window.py:13

↓ 4 callersFunctionensure_1d_long

(x)

src/data/compressor_longformer/train/train_longformer_v2.py:92

↓ 4 callersFunctionensure_1d_tensor

(x)

src/data/compressor_longformer/train/train_longformer.py:111

↓ 4 callersFunctionextract_think_inner

(text: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/build_diff_by_model_ratio.py:26

↓ 4 callersFunctionextract_think_inner

(text: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/build_diff.py:25

↓ 4 callersFunctionextract_think_inner

(text: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/filter_by_ratio_window.py:50

↓ 4 callersFunctionfirst_nonempty

(obj: dict, keys: list[str])

src/data/sft/longformer_pipeline/query_result/autofollow_sft_dataset/rewrite_follow_to_unified_json_v1.py:67

↓ 4 callersFunctionfirst_nonempty

(obj: dict, keys: list[str])

src/data/sft/longformer_pipeline/query_result/autofollow_sft_dataset/rewrite_follow_to_unified_json_v2.py:72

↓ 4 callersMethodget_elem_list

r"""Return the list of all elements.

llamafactory/src/llamafactory/webui/manager.py:37

↓ 4 callersFunctionget_first

(obj: Dict[str, Any], keys: List[str])

src/data/compressor_longformer/validate/eval_utils/tmp.py:28

↓ 4 callersMethodget_thought_word_ids

r"""Get the token ids of thought words.

llamafactory/src/llamafactory/data/template.py:106

↓ 4 callersFunctioninit_adapter

r"""Initialize the adapters. Support full-parameter, freeze and LoRA training. Note that the trainable parameters must be cast to float32.

llamafactory/src/llamafactory/model/adapter.py:283

↓ 4 callersFunctionlatex_spans

(s: str)

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/math_tokenizer_no_space.py:49

↓ 4 callersFunctionload_json

(file, encoding='utf-8')

src/data/compressor_longformer/validate/llama_factory_0.5_0.7_0.9.py:6

↓ 4 callersFunctionmust_no_replacement

(where: str, s: str)

src/data/compressor_longformer/eval/preview_longformer.py:44

↓ 4 callersFunctionnearest_bucket

(r: float, buckets: List[float])

src/data/compressor_longformer/validate/build_mixture_llamafactory.py:116

↓ 4 callersFunctionnormalize

(text: str, case_sensitive: bool)

src/data/sft/metamath_145k_query/filter_jsonl_by_query.py:20

↓ 4 callersFunctionnormalize_for_string_eq

用于最先进行的宽松字符串比较：去空白、部分等价符号、统一大小写。

src/training/qwen1p7b_rl/dataset/rl_dataset/check_basemodel_accuracy.py:79

↓ 4 callersFunctionnormalize_unicode

(s: str)

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/math_tokenizer_no_space.py:31

↓ 4 callersFunctionparse_template

r"""Extract a chat template from the tokenizer.

llamafactory/src/llamafactory/data/template.py:530

↓ 4 callersFunctionpick_answer

(rec: Dict[str, Any])

src/data/compressor_longformer/validate/build_llamafactory_input.py:60

↓ 4 callersFunctionsave_text

(path: str, text: str)

src/data/compressor_longformer/eval/compare_longformer_llmlingua2/compare.py:42

↓ 4 callersMethodstart

(self)

src/data/compressor/dataset_preparation/API/api_result_completiness/retry_empty_chunks.py:364

↓ 4 callersFunctionstrip_latex_wrappers

(s: str)

src/training/qwen1p7b_rl/dataset/rl_dataset/check_basemodel_accuracy.py:57

↓ 4 callersFunctiontry_parse_sympy

(expr_str: str)

src/eval/qwen1p7b_eval/recheck_accuracy.py:142

↓ 4 callersFunctiontry_parse_sympy

(expr_str: str)

src/eval/qwen1p7b_eval/mmlu-stem/recheck_acc.py:128

↓ 4 callersFunctiontry_parse_sympy

使用 SymPy 解析字符串到符号表达式： 1) 先尝试 LaTeX 解析 2) 失败则用 sympify 解析普通表达式返回解析成功的表达式或 None。

src/training/qwen1p7b_rl/dataset/rl_dataset/check_basemodel_accuracy.py:113

↓ 4 callersMethodzero_grad

(self, set_to_none: bool = True)

llamafactory/src/llamafactory/train/trainer_utils.py:77

↓ 3 callersMethod__call__

(self, features: list[dict[str, Any]])

llamafactory/src/llamafactory/data/collator.py:285

↓ 3 callersFunction_agg

(sub: List[Dict[str, Any]])

src/eval/qwen1p7b_eval/vllm_eval.py:409

↓ 3 callersFunction_agg

(sub: List[Dict[str, Any]])

src/eval/qwen1p7b_eval/special_token_vllm.py:522

↓ 3 callersFunction_agg

(sub: List[Dict[str, Any]])

src/eval/qwen1p7b_eval/mmlu-stem/eval_vllm.py:444

↓ 3 callersFunction_alias_from_messages

(rec: Dict[str, Any])

src/data/compressor_longformer/validate/datasets/sample_exlusive_1k.py:115

↓ 3 callersFunction_aliases_from_query_fields

(rec: Dict[str, Any])

src/data/compressor_longformer/validate/datasets/sample_exlusive_1k.py:107

↓ 3 callersFunction_check_extra_dependencies

( model_args: "ModelArguments", finetuning_args: "FinetuningArguments", training_args: Optional["T

llamafactory/src/llamafactory/hparams/parser.py:135

↓ 3 callersMethod_close_thread_pool

(self)

llamafactory/src/llamafactory/train/callbacks.py:222

↓ 3 callersFunction_configure_library_root_logger

r"""Configure root logger using a stdout stream handler with an explicit format.

llamafactory/src/llamafactory/extras/logging.py:100

↓ 3 callersFunction_create_stream_chat_completion_chunk

( completion_id: str, model: str, delta: "ChatCompletionMessage", index: Optional[int] = 0,

llamafactory/src/llamafactory/api/chat.py:172

↓ 3 callersMethod_encode

r"""Encode formatted inputs to pairs of token ids. Turn 0: prefix + system + query resp Turn t: query

llamafactory/src/llamafactory/data/template.py:129

↓ 3 callersFunction_first_non_empty_str

(d: Dict[str, Any], keys: List[str])

src/data/compressor_longformer/validate/Tokenskip_pipeline/llmlingua2_compression.py:58

↓ 3 callersFunction_fmt

(m)

src/eval/qwen1p7b_eval/vllm_eval.py:450

↓ 3 callersFunction_fmt

(m)

src/eval/qwen1p7b_eval/special_token_vllm.py:563

↓ 3 callersFunction_fmt

(m)

src/eval/qwen1p7b_eval/mmlu-stem/eval_vllm.py:485

↓ 3 callersFunction_get_init_kwargs

r"""Get arguments to load config/tokenizer/model. Note: including inplace operation of model_args.

llamafactory/src/llamafactory/model/loader.py:56

↓ 3 callersFunction_get_library_root_logger

()

llamafactory/src/llamafactory/extras/logging.py:96

↓ 3 callersFunction_get_preprocessed_dataset

r"""Preprocesses the dataset, including format checking and tokenization.

llamafactory/src/llamafactory/data/loader.py:230

↓ 3 callersMethod_initialize

r"""Validate the configuration.

llamafactory/src/llamafactory/webui/runner.py:74

↓ 3 callersFunction_is_close

(batch_a: dict[str, Any], batch_b: dict[str, Any])

llamafactory/tests/data/test_mm_plugin.py:110

↓ 3 callersMethod_peek_nonspace

(self, s: str, i: int, step: int)

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/gpt_chunk_camel_data.py:198

↓ 3 callersMethod_peek_nonspace

(self, s: str, i: int, step: int)

src/data/compressor/dataset_preparation/camel/camel_chunk_only_cot/gpt_chunk_camel_data.py:198

↓ 3 callersFunction_set_env_vars

()

llamafactory/src/llamafactory/hparams/parser.py:98

↓ 3 callersFunction_set_transformers_logging

()

llamafactory/src/llamafactory/hparams/parser.py:91

↓ 3 callersFunction_stable_id_from_fields

(rec: Dict[str, Any])

src/data/compressor_longformer/validate/inference_metamath.py:202

↓ 3 callersMethod_update_component

r"""Update gradio components according to the (elem_id, properties) mapping.

llamafactory/src/llamafactory/webui/engine.py:40

↓ 3 callersFunction_verify_model_args

( model_args: "ModelArguments", data_args: "DataArguments", finetuning_args: "FinetuningArguments"

llamafactory/src/llamafactory/hparams/parser.py:106

↓ 3 callersFunctionbatch_confmat

Return dict{total, correct, tp1, fp1, fn1} on valid positions.

src/data/compressor_longformer/train/train_longformer_v2.py:171

↓ 3 callersFunctionbuild_output_text_with_leading_token

输出文本 = <COMP_xx> + compressed_cot (或回落 cot) + optional final answer 返回: (文本, 是否含答案)

src/data/compressor_longformer/validate/build_mixture_llamafactory.py:138

↓ 3 callersMethodcalculate_sentence_split_score

计算在某个句子后分割的评分

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/chunk_camel_data.py:180

↓ 3 callersMethodcalculate_sentence_split_score

计算在某个句子后分割的评分

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/claude_chunk_camel_data.py:178

↓ 3 callersFunctioncanon_text

更强的文本规范化（用于构造去重键）： - 可选移除 [asy]...[/asy] - NFKC 归一 - 统一常见标点、移除零宽 - 合并空白 - 可选小写

src/data/compressor_longformer/validate/datasets/sample_exlusive_1k.py:77

↓ 3 callersFunctionchange_name

(name: str, old_index: int, new_index: int)

llamafactory/scripts/llama_pro.py:36

↓ 3 callersFunctionchar_maps_for_tokens

(s: str, toks: List[Token])

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/math_tokenizer_no_space.py:359

↓ 3 callersMethodchat

r"""Get a list of responses of the chat model.

llamafactory/src/llamafactory/chat/chat_model.py:65

↓ 3 callersFunctioncheck_lora_model

(model: "LoraModel")

llamafactory/src/llamafactory/train/test_utils.py:46

↓ 3 callersFunctionchoose_source_record

按优先级选择该压缩比的素材记录

src/data/compressor_longformer/validate/build_mixture_llamafactory.py:121

↓ 3 callersMethodchunk_text

(self, text: str, validate: bool = True)

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/gpt_chunk_camel_data.py:418

↓ 3 callersMethodchunk_text

(self, text: str, validate: bool = True)

src/data/compressor/dataset_preparation/camel/camel_chunk_only_cot/gpt_chunk_camel_data.py:418

↓ 3 callersFunctionclean_answer

(s: str)

src/data/compressor_longformer/validate/eval_utils/tmp.py:55

↓ 3 callersMethodclose

(self)

llamafactory/src/llamafactory/extras/logging.py:62

↓ 3 callersMethodcopyfrom

(cls, source: "Self", **kwargs)

llamafactory/src/llamafactory/hparams/model_args.py:417

↓ 3 callersFunctioncreate_ref_model

r"""Create reference model for PPO/DPO training. Evaluation mode is not supported. The valuehead parameter is randomly initialized since it is us

llamafactory/src/llamafactory/train/trainer_utils.py:111

↓ 3 callersMethodencode_multiturn

r"""Return multiple pairs of token ids representing prompts and responses respectively.

llamafactory/src/llamafactory/data/template.py:74

↓ 3 callersFunctionextract_output_text

(rec: Dict[str, Any])

src/data/sft/longformer_pipeline/query_result/Compression/RL_actual_ratio/filter_bad_sample.py:80

↓ 3 callersFunctionfind_all_linear_modules

r"""Find all available modules to apply LoRA, GaLore or APOLLO.

llamafactory/src/llamafactory/model/model_utils/misc.py:28

↓ 3 callersFunctionfind_formula_spans_nolatex

Return tight spans of math-like expressions, **ignoring LaTeX** regions.

src/data/compressor/dataset_preparation/camel/camel_chunk_cot_and_answer/math_tokenizer_no_space.py:441

↓ 3 callersFunctionfirst_nonempty

(obj: dict, keys: list[str])

src/data/sft/longformer_pipeline/query_result/autofollow_sft_dataset/rewrite_autofollow.py:36

↓ 3 callersFunctionfix_proxy

r"""Fix proxy settings for gradio ui.

llamafactory/src/llamafactory/extras/misc.py:325

↓ 3 callersFunctionfix_valuehead_checkpoint

r"""Fix the valuehead checkpoint files. The model is already unwrapped. There are three cases: 1. full tuning without ds_zero3: state_di

llamafactory/src/llamafactory/train/callbacks.py:56

↓ 3 callersMethodformat_example

r"""Convert dataset examples to messages.

llamafactory/src/llamafactory/eval/template.py:36

↓ 3 callersMethodget_base_elems

r"""Get the base elements that are commonly used.

llamafactory/src/llamafactory/webui/manager.py:57

↓ 3 callersFunctionget_dataset_converter

r"""Get a dataset converter.

llamafactory/src/llamafactory/data/converter.py:385

← previous next →101–200 of 1,850, ranked by callers