MCPcopy Create free account

hub / github.com/OpenBMB/CPM-Live / functions

Functions302 in github.com/OpenBMB/CPM-Live

↓ 14 callersFunction_dataset_identity
(c: _MixedDatasetConfig)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:248
↓ 11 callersMethodread
(self, size=-1)
cpm-live/cpm_live/dataset/distributed_dataset.py:143
↓ 9 callersMethoddumps
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:76
↓ 9 callersMethodload
(self, fp: io.TextIOWrapper)
cpm-live/cpm_live/dataset/distributed_dataset.py:87
↓ 9 callersMethodwrite
( self, time: float, iteration: int, loss: float, lr: float, l
cpm-live/cpm_live/utils/log.py:59
↓ 8 callersMethodloads
(self, data: str)
cpm-live/cpm_live/dataset/distributed_dataset.py:79
↓ 8 callersMethodprojection
Projection based on embedding's weight. For example, embedding map vocab_size to embed_size, than projection map embed_size back to vocab_siz
cpm-live/cpm_live/layers/embedding.py:55
↓ 7 callersFunction_read_info_list
(meta_path: str)
cpm-live/cpm_live/dataset/distributed_dataset.py:92
↓ 7 callersMethodget
(self)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:1026
↓ 6 callersMethodinference
( self, input: torch.Tensor, # (batch, seqlen) length: torch.Tensor, # (batch)
cpm-live/cpm_live/models/ant.py:156
↓ 5 callersFunction_convert_to_abs_path
(transform_path: str)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:757
↓ 5 callersMethod_update_states
(self, fast_skip: bool = True)
cpm-live/cpm_live/dataset/distributed_dataset.py:263
↓ 5 callersMethodadd
Add a new hypothesis to the list.
cpm-live/cpm_live/generation/generation_utils.py:87
↓ 5 callersFunctionadd_mem_time
(info, mem_usage, tim_usage)
cpm-live/finetune_cpm_bee.py:108
↓ 5 callersMethodclose
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:179
↓ 5 callersMethodgenerate
(self, text_list, **kwargs)
cpm-live/cpm_live/generation/ant.py:41
↓ 5 callersMethodtell
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:155
↓ 5 callersMethodwrite
Write a piece of data into dataset. Args: data (Any): Serialization will be done using pickle. Example: >>>
cpm-live/cpm_live/dataset/distributed_dataset.py:632
↓ 4 callersMethod_get_block_file
(self, block_id: int)
cpm-live/cpm_live/dataset/distributed_dataset.py:369
↓ 4 callersFunction_write_info_list
(meta_path: str, info: List[FileInfo])
cpm-live/cpm_live/dataset/distributed_dataset.py:108
↓ 4 callersFunctionadd_mem_time
(info, mem_usage, tim_usage)
cpm-live/pretrain_cpm_bee.py:109
↓ 4 callersMethodescape
(text: str)
cpm-live/cpm_live/tokenizers/bee.py:179
↓ 4 callersMethodfrom_json_file
(cls, json_file: Union[str, os.PathLike], **args)
cpm-live/cpm_live/utils/config.py:54
↓ 4 callersMethodload_state_dict
(self, d)
cpm-live/cpm_live/dataset/distributed_dataset.py:67
↓ 4 callersMethodopen_file
(self, fname, mode, offset=None)
cpm-live/cpm_live/dataset/distributed_dataset.py:187
↓ 3 callersFunction_build_sample_weights
(config: List[_MixedDatasetConfig])
cpm-live/cpm_live/training_tasks/bee/pretrain.py:763
↓ 3 callersFunction_filtered_range
( begin: int, end: int, rank: int, world_size: int, filter_set: Optional[Set[int]] = None )
cpm-live/cpm_live/dataset/distributed_dataset.py:123
↓ 3 callersFunction_random_string
()
cpm-live/cpm_live/dataset/distributed_dataset.py:30
↓ 3 callersFunctionapply_repetition_penalty
( logits, batch_size, num_beams, prev_output_tokens, repetition_penalty, start_idx=Non
cpm-live/cpm_live/generation/generation_utils.py:34
↓ 3 callersFunctionconvert_data_to_id
( tokenizer: CPMBeeTokenizer, data: Any, prev_ext_states: Optional[_PrevExtTableStates] = None,
cpm-live/cpm_live/training_tasks/bee/pretrain.py:101
↓ 3 callersMethodencode
Encode a string into ids.
cpm-live/cpm_live/tokenizers/ant.py:142
↓ 3 callersFunctiongradient_shrink
(x: torch.Tensor, alpha: float = 0.1)
cpm-live/cpm_live/utils/gradient_shrink.py:15
↓ 2 callersMethod__init__
( self, dim_model: int, dim_ff: int, dtype=torch.half, eps: float = 1e
cpm-live/cpm_live/layers/blocks.py:113
↓ 2 callersMethod__init__
( self, dim, base=10000, distance_scale: Union[int, float] = 1, dtype:
cpm-live/cpm_live/layers/position_embedding.py:220
↓ 2 callersMethod__init__
( self, dim_model: int, dim_ff: int, dtype=torch.half, eps: float = 1e
cpm-live/cpm_live/native_layers/blocks.py:112
↓ 2 callersMethod__init__
( self, dim, base=10000, distance_scale: Union[int, float] = 1, dtype:
cpm-live/cpm_live/native_layers/position_embedding.py:213
↓ 2 callersMethod_auto_cut
(self, text: str)
cpm-live/cpmbee_translator.py:36
↓ 2 callersMethod_open_file
(self, fname, offset)
cpm-live/cpm_live/dataset/distributed_dataset.py:536
↓ 2 callersMethod_replace_entity
(self, text: str, table: Dict[str, str])
cpm-live/cpmbee_translator.py:80
↓ 2 callersMethodadd_data
(self, config: _MixedDatasetConfig)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:625
↓ 2 callersFunctionallgather_objects
(obj)
cpm-live/cpm_live/utils/object.py:6
↓ 2 callersMethodapply_transform
( self, data: CPMBeeInputType, transform: Union[Dict[str, Any], Callable[[CPMBeeInputT
cpm-live/cpm_live/training_tasks/bee/pretrain.py:281
↓ 2 callersMethodbackward
(ctx, grad_output)
cpm-live/cpm_live/utils/gradient_shrink.py:11
↓ 2 callersFunctionbuild_dataset
Open the dataset in write mode and returns a writer. Args: path (str): Path to dataset. dbname (str): The name of the file to whi
cpm-live/cpm_live/dataset/distributed_dataset.py:750
↓ 2 callersMethodchanged
(self)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:706
↓ 2 callersMethodclose
(self)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:971
↓ 2 callersMethoddata_to_id
( self, data: Any, prev_ext_states: Optional[_PrevExtTableStates] = None, shuf
cpm-live/cpm_live/training_tasks/bee/pretrain.py:358
↓ 2 callersMethoddecode
Decode ids into a string.
cpm-live/cpm_live/tokenizers/ant.py:146
↓ 2 callersFunctionget_args
(pretrain: bool = False, finetune: bool = False)
cpm-live/cpm_live/arguments.py:138
↓ 2 callersMethodget_config
(self)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:733
↓ 2 callersMethodget_log_name
(self, now: Optional[datetime.datetime] = None)
cpm-live/cpm_live/utils/log.py:54
↓ 2 callersMethodget_log_time
(self)
cpm-live/cpm_live/utils/log.py:51
↓ 2 callersMethodis_done
If there are enough hypotheses and that none of the hypotheses being generated can become better than the worst one in the heap, then
cpm-live/cpm_live/generation/generation_utils.py:102
↓ 2 callersMethodload_state_dict
(self, data: OrderedDict, strict: bool = False)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:1017
↓ 2 callersMethodpack_batch
(self, force: bool = False)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:516
↓ 2 callersFunctionpad
(orig_items, key, padding_value=0, padding_side="left")
cpm-live/cpm_live/utils/data_utils.py:4
↓ 2 callersFunctionrel_to_bucket
(n_up: int, n_down: int, max_depth: int = 8)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:92
↓ 2 callersMethodseek
(self, offset, whence=0)
cpm-live/cpm_live/dataset/distributed_dataset.py:167
↓ 2 callersMethodstart
(self)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:982
↓ 2 callersMethodstate_dict
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:56
↓ 2 callersMethodtokenize
Tokenize a string.
cpm-live/cpm_live/tokenizers/ant.py:135
↓ 2 callersMethodunescape
(text: str)
cpm-live/cpm_live/tokenizers/bee.py:183
↓ 1 callersMethod__batch_iter
(self)
cpm-live/cpm_live/training_tasks/bee/finetune.py:39
↓ 1 callersMethod__get_item_data
(self, raw_data)
cpm-live/cpm_live/training_tasks/ant/pretrain.py:35
↓ 1 callersMethod__init__
( self, file_name: str = "", block_begin: int = 0, block_end: int = 0,
cpm-live/cpm_live/dataset/distributed_dataset.py:38
↓ 1 callersMethod__init__
( self, vocab_size: int, embedding_size: int, dtype: torch.dtype = torch.half,
cpm-live/cpm_live/layers/embedding.py:25
↓ 1 callersMethod__init__
( self, dim_model: int, dim_ff: int, dtype=torch.half, dropout_p: Opti
cpm-live/cpm_live/layers/feedforward.py:79
↓ 1 callersMethod__init__
(self, config: CPMAntConfig)
cpm-live/cpm_live/models/ant.py:66
↓ 1 callersMethod__init__
(self, config: CPMBeeConfig)
cpm-live/cpm_live/models/bee.py:73
↓ 1 callersMethod__init__
( self, vocab_size: int, embedding_size: int, dtype: torch.dtype = torch.half,
cpm-live/cpm_live/native_layers/embedding.py:24
↓ 1 callersMethod__init__
( self, dim_model: int, dim_ff: int, dtype=torch.half, dropout_p: Opti
cpm-live/cpm_live/native_layers/feedforward.py:77
↓ 1 callersMethod__str__
(self)
cpm-live/cpm_live/tokenizers/bee.py:42
↓ 1 callersFunction_build_dict_tree
(data: CPMBeeInputType, depth: int, need_predict: bool)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:118
↓ 1 callersFunction_build_segment_rel
(node: _DictTree)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:158
↓ 1 callersMethod_convert_to_tensors
(self, data: Any, in_context_samples: List[Any] = [])
cpm-live/cpm_live/generation/bee.py:18
↓ 1 callersMethod_decode
(self, model_inputs, **kwargs)
cpm-live/cpm_live/generation/ant.py:47
↓ 1 callersMethod_decode
(self, model_inputs, other_info, **kwargs)
cpm-live/cpm_live/generation/bee.py:244
↓ 1 callersMethod_dict_from_json_file
(cls, json_file: Union[str, os.PathLike], **args)
cpm-live/cpm_live/utils/config.py:59
↓ 1 callersMethod_ensure_transform_function
( self, module_name: str, transform_script_path: str )
cpm-live/cpm_live/training_tasks/bee/pretrain.py:368
↓ 1 callersMethod_get_file_path
(self, fname)
cpm-live/cpm_live/dataset/distributed_dataset.py:533
↓ 1 callersFunction_get_logger
()
cpm-live/cpm_live/utils/log.py:11
↓ 1 callersMethod_get_next_block
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:389
↓ 1 callersMethod_mask_file
(self, f: FileInfo)
cpm-live/cpm_live/dataset/distributed_dataset.py:362
↓ 1 callersMethod_position_bucket
( self, relative_position, bidirectional=True, num_buckets=32, max_distance=128 )
cpm-live/cpm_live/layers/position_embedding.py:104
↓ 1 callersMethod_position_bucket
(self, relative_position, num_buckets=32, max_distance=128)
cpm-live/cpm_live/layers/position_embedding.py:196
↓ 1 callersMethod_position_bucket
( self, relative_position, bidirectional=True, num_buckets=32, max_distance=128 )
cpm-live/cpm_live/native_layers/position_embedding.py:100
↓ 1 callersMethod_position_bucket
(self, relative_position, num_buckets=32, max_distance=128)
cpm-live/cpm_live/native_layers/position_embedding.py:189
↓ 1 callersMethod_prepare_new_epoch
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:374
↓ 1 callersMethod_process_list
(self, data_list: List[Any])
cpm-live/cpm_live/generation/bee.py:124
↓ 1 callersMethod_process_texts
(self, text_list)
cpm-live/cpm_live/generation/ant.py:33
↓ 1 callersMethod_remove_entity
(self, nlp: spacy.language.Language, text: str)
cpm-live/cpmbee_translator.py:57
↓ 1 callersMethod_segment_relative_position_bucket
(self, query_segment, key_segment)
cpm-live/cpm_live/layers/position_embedding.py:101
↓ 1 callersMethod_segment_relative_position_bucket
(self, query_segment, key_segment)
cpm-live/cpm_live/native_layers/position_embedding.py:97
↓ 1 callersMethod_state_dict
(self)
cpm-live/cpm_live/dataset/distributed_dataset.py:399
↓ 1 callersMethod_write_log
(self, iteration: int, data: Any, result: List[Dict[str, Any]])
cpm-live/pretrain_cpm_bee.py:144
↓ 1 callersFunctionadd_finetune_args
(parser: argparse.ArgumentParser)
cpm-live/cpm_live/arguments.py:116
↓ 1 callersFunctionadd_model_config_args
Model arguments
cpm-live/cpm_live/arguments.py:19
↓ 1 callersFunctionadd_pretrain_args
(parser: argparse.ArgumentParser)
cpm-live/cpm_live/arguments.py:98
↓ 1 callersFunctionadd_training_args
Training arguments.
cpm-live/cpm_live/arguments.py:27
↓ 1 callersMethodbuild_instance
(self, config: _MixedDatasetConfig)
cpm-live/cpm_live/training_tasks/bee/pretrain.py:403
next →1–100 of 302, ranked by callers