MCPcopy Create free account

hub / github.com/AkariAsai/self-rag / functions

Functions256 in github.com/AkariAsai/self-rag

⨍ Functions 256◇ Types & classes 23
Methodencode_corpus
(self, corpus: List[Dict[str, str]], batch_size: int, **kwargs)
retrieval_lm/src/beir_utils.py:85
Methodencode_queries
(self, queries: List[str], batch_size: int, **kwargs)
retrieval_lm/src/beir_utils.py:46
Functionencode_with_messages_format
Here we assume each example has a 'messages' field Each message is a dict with 'role' and 'content' fields. We concatenate all messages with
retrieval_lm/finetune.py:294
Functionencode_with_prompt_completion_format
Here we assume each example has 'prompt' and 'completion' fields. We concatenate prompt and completion and tokenize them together because oth
retrieval_lm/finetune.py:250
Functioneval_batch
(scores, inversions, avg_topk, idx_topk)
retrieval_lm/src/evaluation.py:163
Functionevaluate_model
( query_encoder, doc_encoder, tokenizer, dataset, batch_size=128, add_special_tokens=T
retrieval_lm/src/beir_utils.py:125
Functionexact_match_score
(prediction, ground_truth)
retrieval_lm/metrics.py:8
Functionexact_match_score
(prediction, ground_truths)
retrieval_lm/src/evaluation.py:156
Functionf1
(decoded_preds, decoded_labels)
retrieval_lm/metrics.py:28
Functionf1_score
(prediction, ground_truths)
retrieval_lm/src/evaluation.py:153
Functionfind_entity_tags
(sentence)
retrieval_lm/metrics.py:69
Functionforward
Input shape: Batch x Time x Channel attention_mask: [bsz, q_len]
retrieval_lm/llama_flash_attn_monkey_patch.py:19
Methodforward
(ctx, x: torch.tensor)
retrieval_lm/src/dist_utils.py:9
Methodforward
(self, q_tokens, q_mask, k_tokens, k_mask, stats_prefix="", iter_stats={}, **kwargs)
retrieval_lm/src/inbatch.py:59
Methodforward
( self, input_ids=None, attention_mask=None, token_type_ids=None, posi
retrieval_lm/src/contriever.py:64
Methodforward
(self, q_tokens, q_mask, k_tokens, k_mask, stats_prefix="", iter_stats={}, **kwargs)
retrieval_lm/src/moco.py:104
Functiongather
(x: torch.tensor)
retrieval_lm/src/dist_utils.py:21
Functiongather_nograd
(x: torch.tensor)
retrieval_lm/src/dist_utils.py:30
Methodgenerate_offset
(self)
retrieval_lm/src/data.py:71
Methodget_encoder
(self)
retrieval_lm/src/inbatch.py:56
Methodget_encoder
(self, return_encoder_k=False)
retrieval_lm/src/moco.py:68
Functionget_parameters
(net, verbose=False)
retrieval_lm/src/utils.py:145
Functionget_varsize
gather tensors of different sizes along the first dimension
retrieval_lm/src/dist_utils.py:64
Functionget_world_size
()
retrieval_lm/src/dist_utils.py:89
Functioninit_distributed_mode
Handle single and multi-GPU / multi-node / SLURM jobs. Initialize the following variables: - local_rank - global_rank
retrieval_lm/src/slurm.py:43
Functioninit_logger
(args, stdout_only=False)
retrieval_lm/src/utils.py:18
Functioninit_signal_handler
Handle signals sent by SLURM for time limit / pre-emption.
retrieval_lm/src/slurm.py:35
Functioninit_tb_logger
(output_dir)
retrieval_lm/src/utils.py:201
Functionjdump
Dump a str or dictionary to a file in json format. Args: obj: An object to be written. f: A string path to the location on disk.
data_creation/train_special_tokens.py:107
Functionload
(model_class, dir_path, opt, reset_params=False)
retrieval_lm/src/utils.py:66
Functionload_data
(data_path)
retrieval_lm/passage_retrieval.py:212
Functionload_data
(opt, tokenizer)
retrieval_lm/src/data.py:20
Functionload_hf
(object_class, model_name)
retrieval_lm/src/utils.py:193
Functionload_json
(fn)
data_creation/generator/postprocess_data.py:48
Functionload_passages
(path)
retrieval_lm/src/data.py:227
Functionload_retriever
(model_path, pooling="average", random_init=False)
retrieval_lm/src/contriever.py:103
Methodlr_lambda
(self, step)
retrieval_lm/src/utils.py:98
Methodlr_lambda
(self, step)
retrieval_lm/src/utils.py:115
Functionnormalize
(text)
retrieval_lm/src/normalize_text.py:133
Methodparse
(self)
retrieval_lm/src/options.py:129
Functionpostprocess_output
(input_instance, prediction, task, intermediate_results=None)
retrieval_lm/utils.py:145
Functionpreprocess_input
(input_data, task)
retrieval_lm/utils.py:117
Methodprint_options
(self, opt)
retrieval_lm/src/options.py:112
Functionprocess_arc_instruction
(item, instruction)
retrieval_lm/utils.py:174
Methodreset
(self)
retrieval_lm/src/utils.py:173
Functionsave
(model, optimizer, scheduler, step, opt, dir_path, name)
retrieval_lm/src/utils.py:46
Methodsearch_document_demo
(self, query, n_docs=10)
retrieval_lm/passage_retrieval.py:168
Methodsetup_retriever_demo
(self, model_name_or_path, passages, passages_embeddings, n_docs=5, save_or_load_index=False)
retrieval_lm/passage_retrieval.py:178
Functionsig_handler
(signum, frame)
retrieval_lm/src/slurm.py:18
Functionsplit_sentences
(paragraph)
data_creation/generator/postprocess_data.py:58
Functionsplit_sentences
(paragraph)
data_creation/generator/create_prompt_data.py:20
Methodstats
(self)
retrieval_lm/src/utils.py:166
Functionterm_handler
(signum, frame)
retrieval_lm/src/slurm.py:30
Methodtuple_stats
(self)
retrieval_lm/src/utils.py:170
Functionvarsize_gather_nograd
gather tensors of different sizes along the first dimension
retrieval_lm/src/dist_utils.py:41
Functionweighted_average
(x, count)
retrieval_lm/src/dist_utils.py:119
← previous201–256 of 256, ranked by callers