Functions — github.com/RUC-NLPIR/SmartSearch

hub / github.com/RUC-NLPIR/SmartSearch / functions

Functions2,379 in github.com/RUC-NLPIR/SmartSearch

⨍ Functions 2,379◇ Types & classes 411↳ Endpoints 33

↓ 8 callersFunctionload_corpus

(corpus_path: str)

src/flashrag/retriever/utils.py:121

↓ 8 callersFunctionload_reference_model

( model_path: str, lora_path: Optional[str] = None, use_lora: bool = False, use_pissa: bool =

LLaMA-Factory/src/llamafactory/train/test_utils.py:75

↓ 8 callersMethodpostprocess_data

(self, data: DataProto)

src/verl/workers/sharding_manager/base.py:32

↓ 8 callersMethodrank

(self)

src/verl/single_controller/base/worker.py:208

↓ 7 callersFunction_megatron_calc_global_rank

given TP,DP,PP rank to get the global rank.

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_saver.py:27

↓ 7 callersFunction_megatron_calc_global_rank

given TP,DP,PP rank to get the global rank.

src/verl/models/llama/megatron/checkpoint_utils/llama_saver.py:27

↓ 7 callersFunctionappend_to_dict

(data: Dict, new_data: Dict)

src/verl/utils/py_functional.py:41

↓ 7 callersFunctionclean_traceback

(error_traceback)

src/verl/utils/reward_score/prime_code/testing_util.py:101

↓ 7 callersFunctioncompare_model

(model_a: "torch.nn.Module", model_b: "torch.nn.Module", diff_keys: list[str] = [])

LLaMA-Factory/src/llamafactory/train/test_utils.py:34

↓ 7 callersMethodencode

Converts a string to a sequence of ids (integer), using the tokenizer and vocabulary. Args: text (`str`, `List[str]`

src/verl/workers/rollout/tokenizer.py:70

↓ 7 callersFunctionget_current_device

r"""Get the current available device.

LLaMA-Factory/src/llamafactory/extras/misc.py:144

↓ 7 callersFunctionload_config

r"""Load user config if exists.

LLaMA-Factory/src/llamafactory/webui/common.py:74

↓ 7 callersFunctionload_infer_model

(add_valuehead: bool = False, **kwargs)

LLaMA-Factory/src/llamafactory/train/test_utils.py:69

↓ 7 callersFunctionlogprobs_from_logits

See: https://github.com/pytorch/pytorch/issues/563#issuecomment-330103591

src/verl/utils/torch_functional.py:48

↓ 7 callersFunctionoffload_fsdp_optimizer

(optimizer)

src/verl/utils/fsdp_utils.py:152

↓ 7 callersFunctiontorch_gc

r"""Collect the device memory.

LLaMA-Factory/src/llamafactory/extras/misc.py:254

↓ 7 callersMethodunion

Union with another DataProto. Union batch and meta_info separately. Throw an error if - there are conflict keys in batch and they

src/verl/protocol.py:542

↓ 6 callersMethod__init__

( self, config, threshold=0.2, look_ahead_steps=64, max_generatio

src/flashrag/pipeline/active_pipeline.py:820

↓ 6 callersFunction_broadcast_tensor

broadcast tensor from rank0 across mp_group

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_loader_depracated.py:100

↓ 6 callersFunction_broadcast_tensor

broadcast tensor from rank0 across mp_group

src/verl/models/mcore/loader.py:100

↓ 6 callersFunction_broadcast_tensor

broadcast tensor from rank0 across mp_group

src/verl/models/llama/megatron/checkpoint_utils/llama_loader_depracated.py:104

↓ 6 callersFunction_fetch_tensor

fetch tensor

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_loader.py:100

↓ 6 callersFunction_fetch_tensor

fetch tensor

src/verl/models/llama/megatron/checkpoint_utils/llama_loader.py:104

↓ 6 callersMethodcalculate_rouge

(self, pred, golden_answers)

src/flashrag/evaluator/metrics.py:299

↓ 6 callersFunctioncombined_int_check

(val)

src/verl/utils/reward_score/prime_code/testing_util.py:97

↓ 6 callersFunctioncreate_custom_optimizer

( model: "PreTrainedModel", training_args: "TrainingArguments", finetuning_args: "FinetuningArgume

LLaMA-Factory/src/llamafactory/train/trainer_utils.py:525

↓ 6 callersFunctioncreate_custom_scheduler

( training_args: "TrainingArguments", num_training_steps: int, optimizer: Optional["torch.optim.Op

LLaMA-Factory/src/llamafactory/train/trainer_utils.py:549

↓ 6 callersFunctiongather_heads_scatter_seq

A func to sync attention result with alltoall in sequence parallel gather head dimension and scatter seq dim: e.g. seq_dim: 1, head_di

src/verl/utils/ulysses.py:85

↓ 6 callersMethodget_scores

r"""Get a list of scores of the reward model.

LLaMA-Factory/src/llamafactory/chat/chat_model.py:155

↓ 6 callersFunctionis_non_local

(path)

src/verl/utils/fs.py:32

↓ 6 callersFunctionis_torch_mps_available

Check if MPS is available.

LLaMA-Factory/src/llamafactory/v1/accelerator/helper.py:132

↓ 6 callersFunctionis_torch_xpu_available

Check if XPU is available.

LLaMA-Factory/src/llamafactory/v1/accelerator/helper.py:142

↓ 6 callersFunctionjudge_zh

(input_str: str)

src/flashrag/retriever/utils.py:27

↓ 6 callersFunctionmasked_mean

Compute mean of tensor with a masked values.

src/verl/utils/torch_functional.py:118

↓ 6 callersMethodmerge

(self, other)

src/verl/utils/seqlen_balancing.py:37

↓ 6 callersFunctionnumpify

r"""Cast a torch tensor or a numpy array to a numpy array.

LLaMA-Factory/src/llamafactory/extras/misc.py:236

↓ 6 callersMethodpostprocess_prediction

(self, pred)

src/flashrag/pipeline/active_pipeline.py:479

↓ 6 callersMethodput_item

(self, item: dict[str, any])

LLaMA-Factory/src/llamafactory/v1/utils/batching_queue.py:172

↓ 6 callersFunctionrun_exp

(args: Optional[dict[str, Any]] = None, callbacks: Optional[list["TrainerCallback"]] = None)

LLaMA-Factory/src/llamafactory/train/tuner.py:110

↓ 6 callersMethodsample

( self, input_ids: torch.LongTensor, logits_processor: Optional[LogitsProcessorList

src/flashrag/pipeline/replug_utils.py:29

↓ 6 callersFunctiontruncatefn

(s, length=300)

src/verl/utils/reward_score/prime_code/testing_util.py:42

↓ 6 callersFunctionulysses_pad_and_slice_inputs

Pad and slice input_ids to be divisible by sp_size Pad position_ids to be divisible by sp_size. Note both input_ids_rmpad and positi

src/verl/utils/ulysses.py:252

↓ 5 callersMethod__init__

(self, config)

src/flashrag/generator/generator.py:22

↓ 5 callersMethod__init__

(self, config)

src/flashrag/retriever/retriever.py:118

↓ 5 callersMethod__init__

(self, config: Qwen2Config, megatron_config: ModelParallelConfig)

src/verl/models/qwen2/megatron/modeling_qwen2_megatron.py:83

↓ 5 callersMethod__init__

(self, config: LlamaConfig, megatron_config: ModelParallelConfig)

src/verl/models/llama/megatron/modeling_llama_megatron.py:83

↓ 5 callersMethod__post_init__

(self)

LLaMA-Factory/src/llamafactory/hparams/model_args.py:444

↓ 5 callersFunction_broadcast_tensor

broadcast tensor across mp_group

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_saver.py:123

↓ 5 callersFunction_broadcast_tensor

broadcast tensor across mp_group

src/verl/models/llama/megatron/checkpoint_utils/llama_saver.py:123

↓ 5 callersFunction_convert_str_dict

Parse string representation inside the dictionary. Args: data: The string or dictionary to convert. Returns: The converted d

LLaMA-Factory/src/llamafactory/v1/config/arg_utils.py:53

↓ 5 callersFunction_hdfs_cmd

(cmd: str)

src/verl/utils/hdfs_io.py:139

↓ 5 callersFunction_parse_args

( parser: "HfArgumentParser", args: dict[str, Any] | list[str] | None = None, allow_extra_keys: bool = Fal

LLaMA-Factory/src/llamafactory/hparams/parser.py:85

↓ 5 callersFunction_run_cmd

(cmd: str, timeout=None)

src/verl/utils/hdfs_io.py:135

↓ 5 callersMethod_search_or_batch_search

(self, query: Union[str, list], target_modal, num, return_score, method, retriever_list)

src/flashrag/retriever/retriever.py:653

↓ 5 callersFunctionagg_loss

Aggregate the loss matrix into a scalar. Args: loss_mat: `(torch.Tensor)` shape: (bs, response_length) loss_

src/verl/trainer/ppo/core_algos.py:309

↓ 5 callersMethodall_gather

Gather tensor across specified parallel group.

LLaMA-Factory/src/llamafactory/v1/accelerator/interface.py:212

↓ 5 callersMethodapply

(cls, model, **kwargs)

LLaMA-Factory/src/llamafactory/v1/plugins/model_plugins/kernels/mlp/npu_fused_moe.py:214

↓ 5 callersFunctioncreate_modelcard_and_push

( trainer: "Trainer", model_args: "ModelArguments", data_args: "DataArguments", training_args:

LLaMA-Factory/src/llamafactory/train/trainer_utils.py:85

↓ 5 callersMethodeval

(self)

LLaMA-Factory/src/llamafactory/eval/evaluator.py:79

↓ 5 callersMethodevaluate

Calculate all metric indicators and summarize them.

src/flashrag/evaluator/evaluator.py:46

↓ 5 callersFunctiongather_outpus_and_unpad

(x: Tensor, gather_dim: int, unpad_dim: int = None,

src/verl/utils/ulysses.py:233

↓ 5 callersMethodgenerate

Get responses from the generater. Args: input_list: it contains input texts, each item represents a sample. Returns

src/flashrag/generator/generator.py:51

↓ 5 callersMethodget

(self)

src/verl/utils/rendezvous/ray_backend.py:30

↓ 5 callersMethodget_cur_token_micro_bsz

(self)

LLaMA-Factory/src/llamafactory/v1/utils/batching_queue.py:185

↓ 5 callersFunctionget_init_weight_context_manager

(use_meta_tensor=True, mesh: DeviceMesh = None)

src/verl/utils/fsdp_utils.py:39

↓ 5 callersFunctionget_model

Build the model.

src/verl/utils/megatron_utils.py:42

↓ 5 callersMethodget_ppl

( self, text: str, granularity: str = "sentence", input_ids=None,

src/flashrag/refiner/llmlingua_compressor.py:303

↓ 5 callersFunctionget_response_mask

end of sentence token can be int or list: 1 or [1, 2] e.g. response_id = torch.tensor([[20, 10, 34, 1, 0, 0, 0],

src/verl/utils/torch_functional.py:150

↓ 5 callersMethodget_samples

Get samples from the buffer that fit within the token budget. Args: max_tokens_per_iteration: Maximum number of tokens to retriev

LLaMA-Factory/src/llamafactory/v1/utils/batching_queue.py:45

↓ 5 callersMethodinit_model

(self)

src/verl/workers/fsdp_workers.py:805

↓ 5 callersFunctionload_docs

(corpus, doc_idxs: List[int])

src/flashrag/retriever/utils.py:148

↓ 5 callersFunctionrearrange_micro_batches

Split the batch into a list of micro_batches, where the max_token_len is smaller than max_token_len and the number of valid tokens in each micro

src/verl/utils/seqlen_balancing.py:224

↓ 5 callersMethodselect_idxs

Select specific indices from the DataProto. Args: idxs (torch.Tensor or numpy.ndarray or list): Indices to s

src/verl/protocol.py:403

↓ 5 callersMethodsignature

(self)

src/flashrag/evaluator/_bleu.py:74

↓ 5 callersMethodstream_chat

r"""Get the response token-by-token of the chat model.

LLaMA-Factory/src/llamafactory/chat/chat_model.py:120

↓ 5 callersMethodtruncate_prompt

(self, prompt)

src/flashrag/prompt/base_prompt.py:59

↓ 5 callersFunctionupdate_model_config

(module_config, override_config_kwargs)

src/verl/utils/model.py:42

↓ 5 callersFunctionvalidate_ulysses_config

(num_heads, ulysses_sequence_size)

src/verl/utils/ulysses.py:291

↓ 5 callersMethodwarning_rank0_once

(self, *args, **kwargs)

LLaMA-Factory/src/llamafactory/extras/logging.py:76

↓ 4 callersMethod__compress

( self, context_list: list, reduce_rate: float = 0.5, token_to_word: str =

src/flashrag/refiner/llmlingua_compressor.py:2227

↓ 4 callersMethod__init__

(self, config)

src/flashrag/refiner/refiner.py:12

↓ 4 callersMethod__init__

(self, dim, max_position_embeddings=2048, base=10000, device=None)

src/verl/models/llama/megatron/layers/parallel_attention.py:37

↓ 4 callersFunction_broadcast_tensor

broadcast tensor across mp_group

src/verl/models/mcore/saver.py:137

↓ 4 callersFunction_broadcast_tp_shard_tensor

broadcast tensor in tp shards across mp_group

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_saver.py:162

↓ 4 callersFunction_broadcast_tp_shard_tensor

broadcast tensor in tp shards across mp_group

src/verl/models/mcore/saver.py:176

↓ 4 callersFunction_broadcast_tp_shard_tensor

broadcast tensor in tp shards across mp_group

src/verl/models/llama/megatron/checkpoint_utils/llama_saver.py:162

↓ 4 callersFunction_escape_html

r"""Escape HTML characters.

LLaMA-Factory/src/llamafactory/webui/chatter.py:41

↓ 4 callersFunction_get_cpu_tensor

(tensor: torch.Tensor)

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_saver.py:116

↓ 4 callersFunction_get_cpu_tensor

(tensor: torch.Tensor)

src/verl/models/mcore/saver.py:130

↓ 4 callersFunction_get_cpu_tensor

(tensor: torch.Tensor)

src/verl/models/llama/megatron/checkpoint_utils/llama_saver.py:116

↓ 4 callersFunction_get_decay_parameter_names

r"""Return a list of names of parameters with weight decay. (weights in non-layernorm layers).

LLaMA-Factory/src/llamafactory/train/trainer_utils.py:191

↓ 4 callersFunction_get_gpt_model

(model)

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_loader_depracated.py:66

↓ 4 callersFunction_get_gpt_model

(model)

src/verl/models/qwen2/megatron/checkpoint_utils/qwen2_loader.py:66

↓ 4 callersFunction_get_gpt_model

(model)

src/verl/models/mcore/loader.py:64

↓ 4 callersFunction_get_gpt_model

(model)

src/verl/models/llama/megatron/checkpoint_utils/llama_loader.py:70

↓ 4 callersFunction_get_gpt_model

(model)

src/verl/models/llama/megatron/checkpoint_utils/llama_loader_depracated.py:70

↓ 4 callersFunction_is_non_local

(path: str)

src/verl/utils/hdfs_io.py:143

↓ 4 callersFunction_pre_process_inputs

(pad_token_id, prompt_token_ids: torch.Tensor)

src/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py:54

↓ 4 callersFunction_repeat_interleave

(value: Union[torch.Tensor, np.ndarray], repeats: int)

src/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py:62

↓ 4 callersFunction_split_args_kwargs_data_proto

(chunks, *args, **kwargs)

src/verl/single_controller/base/decorator.py:45

← previous next →101–200 of 2,379, ranked by callers