Functions — github.com/CStanKonrad/long

hub / github.com/CStanKonrad/long_llama / functions

Functions296 in github.com/CStanKonrad/long_llama

⨍ Functions 296◇ Types & classes 52 ↳ Endpoints 1

↓ 1 callersFunctionload_and_convert_checkpoint

(path)

fot_continued_pretraining/EasyLM/models/llama/convert_easylm_to_hf.py:103

↓ 1 callersMethodload_config

(cls, path)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:359

↓ 1 callersMethodload_flax_checkpoint

Load a standard flax checkpoint that's not saved with the msgpack streaming format.

fot_continued_pretraining/EasyLM/checkpoint.py:135

↓ 1 callersMethodload_state_dict

(self, state_dict)

fot_continued_pretraining/EasyLM/data.py:364

↓ 1 callersFunctionmain

()

instruction_fine_tuning/fine_tuning.py:14

↓ 1 callersFunctionmain

(args)

fot_continued_pretraining/EasyLM/models/llama/convert_hf_to_easylm.py:75

↓ 1 callersFunctionmake_shard_and_gather_fns

Create pytree of sharding and gathering functions from pytree of partition specs.

fot_continued_pretraining/EasyLM/jax_utils.py:92

↓ 1 callersFunctionmatch_keywords

(string, positives, negatives)

fot_continued_pretraining/EasyLM/models/llama/convert_easylm_to_hf.py:93

↓ 1 callersFunctionnames_in_current_mesh

Check if current mesh axes contain these names.

fot_continued_pretraining/EasyLM/jax_utils.py:176

↓ 1 callersFunctionoverride_flags

(overrides: Dict[str, Any])

fot_continued_pretraining/running_utils/runner_utils.py:19

↓ 1 callersMethodparse_json

(self, line)

fot_continued_pretraining/EasyLM/data.py:271

↓ 1 callersFunctionpasskey_retrieval_test

(n_garbage=60000, seed=555)

examples/passkey.py:20

↓ 1 callersFunctionpost_override_callback

()

fot_continued_pretraining/running_utils/runner.py:18

↓ 1 callersFunctionprecompute_freqs_cis

(dim: int, end: int, theta: float=10000.0, dtype: jnp.dtype=jnp.float32)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:401

↓ 1 callersFunctionprepare_for_run

(config_dict: Dict[str, Any])

fot_continued_pretraining/running_utils/runner_utils.py:30

↓ 1 callersFunctionprepare_input_text

( pre_prompt_text: str, prompt_field: Optional[str], post_prompt_text: str, pr

instruction_fine_tuning/data_processing.py:93

↓ 1 callersFunctionrecursive_check_match

(regex, v: Union[Dict[str, Any], str], field_nesting: List[str])

instruction_fine_tuning/data_processing.py:447

↓ 1 callersFunctionreference_attn

(query, key, value, causal, dtype)

fot_continued_pretraining/EasyLM/bpt.py:199

↓ 1 callersFunctionrotate_half

Rotates half the hidden dims of the input.

src/modeling_longllama.py:157

↓ 1 callersFunctionrun_from_dict

(main_fn, config_dict: Dict[str, Any], post_override_callback)

fot_continued_pretraining/running_utils/runner_utils.py:49

↓ 1 callersMethodsave_all

(self, train_state, gather_fns, metadata=None, dataset=None, milestone=False)

fot_continued_pretraining/EasyLM/checkpoint.py:69

↓ 1 callersFunctionscan_attention

(args)

fot_continued_pretraining/EasyLM/bpt.py:100

↓ 1 callersFunctionseparate_data_args

Given the data_args creates a separate instance for each dataset.

instruction_fine_tuning/data_processing.py:280

↓ 1 callersFunctionset_random_seed

(seed)

fot_continued_pretraining/EasyLM/jax_utils.py:141

↓ 1 callersFunctionshow_data_stats

(data_stats: List[DatasetProcessingStats])

instruction_fine_tuning/data_processing.py:621

↓ 1 callersFunctiontokenize_data

( input_response: List[Tuple[str, str]], tokenizer: PreTrainedTokenizer, tokenization_

instruction_fine_tuning/data_processing.py:128

↓ 1 callersFunctiontokenize_one

(input_text: str, response_text: str)

instruction_fine_tuning/data_processing.py:133

↓ 1 callersFunctiontokenize_portion

(input_response_portion: List[Tuple[str, str]])

instruction_fine_tuning/data_processing.py:164

↓ 1 callersFunctiontree_apply

Apply a pytree of functions to the pytree.

fot_continued_pretraining/EasyLM/jax_utils.py:399

↓ 1 callersFunctionwrite_model

(loaded, model_path, model_size)

fot_continued_pretraining/EasyLM/models/llama/convert_easylm_to_hf.py:126

↓ 1 callersFunctionwrite_tokenizer

(tokenizer_path, input_tokenizer_path)

fot_continued_pretraining/EasyLM/models/llama/convert_easylm_to_hf.py:215

Method__call__

(self, inputs)

instruction_fine_tuning/data_processing.py:720

Method__call__

(self, example, has_aux=False)

fot_continued_pretraining/EasyLM/data.py:76

Method__call__

(self, keys=None)

fot_continued_pretraining/EasyLM/jax_utils.py:36

Method__call__

(self, input_ids, scores, cur_len)

fot_continued_pretraining/EasyLM/jax_utils.py:88

Method__call__

(self, x: jnp.ndarray)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:395

Method__call__

( self, hidden_states, attention_mask, position_ids, deterministic: bo

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:550

Method__call__

(self, x: jnp.ndarray, deterministic: bool = True)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:727

Method__call__

( self, hidden_states, attention_mask=None, position_ids=None, determi

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:784

Method__call__

( self, input_ids, attention_mask=None, position_ids=None, params: dic

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:904

Method__call__

( self, hidden_states, attention_mask=None, position_ids=None, determi

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:998

Method__call__

( self, input_ids, attention_mask, position_ids, deterministic=True,

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1073

Method__call__

( self, input_ids, attention_mask=None, position_ids=None, determinist

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1146

Method__getitem__

(self, i)

instruction_fine_tuning/data_processing.py:573

Method__getitem__

(self, i)

instruction_fine_tuning/data_processing.py:700

Method__init__

( self, vocab_size=32000, hidden_size=4096, intermediate_size=11008, n

src/configuration_longllama.py:102

Method__init__

LongLlamaRMSNorm is equivalent to T5LayerNorm

src/modeling_longllama.py:104

Method__init__

(self, dim, max_position_embeddings=2048, base=10000, device=None)

src/modeling_longllama.py:122

Method__init__

(self, config: LongLlamaConfig, mem_config: Optional[LongLlamaMemConfig] = None)

src/modeling_longllama.py:206

Method__init__

(self, config: LongLlamaConfig, mem_config: Optional[LongLlamaMemConfig] = None)

src/modeling_longllama.py:484

Method__init__

(self, config: LongLlamaConfig)

src/modeling_longllama.py:716

Method__init__

(self, config)

src/modeling_longllama.py:1159

Method__init__

(self, config)

src/modeling_longllama.py:1355

Method__init__

( self, data_args: DataArgs, tokenizer: PreTrainedTokenizer, tokenization_args

instruction_fine_tuning/data_processing.py:539

Method__init__

( self, data_args: DataArgs, tokenizer: PreTrainedTokenizer, tokenization_args

instruction_fine_tuning/data_processing.py:645

Method__init__

(self, config, checkpoint_dir, enable=True)

fot_continued_pretraining/EasyLM/checkpoint.py:33

Method__init__

(self)

fot_continued_pretraining/EasyLM/data.py:48

Method__init__

(self, config, tokenizer)

fot_continued_pretraining/EasyLM/data.py:69

Method__init__

(self, config, tokenizer, text_processor)

fot_continued_pretraining/EasyLM/data.py:158

Method__init__

(self, config, tokenizer, text_processor)

fot_continued_pretraining/EasyLM/data.py:258

Method__init__

(self, rng)

fot_continued_pretraining/EasyLM/jax_utils.py:33

Method__init__

(self, temperature)

fot_continued_pretraining/EasyLM/jax_utils.py:85

Method__init__

(self)

fot_continued_pretraining/EasyLM/optimizers.py:23

Method__init__

(self)

fot_continued_pretraining/EasyLM/optimizers.py:65

Method__init__

(self)

fot_continued_pretraining/EasyLM/optimizers.py:121

Method__init__

( self, keep_last=-1, provide_mean=True, provide_latest=True, reset_on

fot_continued_pretraining/EasyLM/logging_utils.py:26

Method__init__

( self, config: LLaMAConfig, input_shape: Tuple = (1, 1), seed: int = 0,

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:837

Method__init__

( self, vocab_file, unk_token="<unk>", bos_token="<s>", eos_token="</s

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1247

Method__init__

( self, token_source: Iterable[Tuple[List[int], List[float], str]], batch_size: int, seq_len: int

fot_continued_pretraining/FoT/data_pipeline.py:29

Method__init__

( self, token_source: Iterable[Tuple[List[int], List[float], str]], batch_size: int, seq_len: int

fot_continued_pretraining/FoT/data_pipeline.py:45

Method__init__

( self, token_source: Iterable[Tuple[List[int], List[float], str]], batch_size: int,

fot_continued_pretraining/FoT/data_pipeline.py:97

Method__init__

( self, token_source: Iterable[Tuple[List[int], List[float], str]], batch_size: int,

fot_continued_pretraining/FoT/data_pipeline.py:207

Method__init__

( self, token_source: Callable[[], Generator[Tuple[List[int], List[float], str, int, int], Non

fot_continued_pretraining/FoT/data_pipeline.py:255

Method__iter__

(self)

fot_continued_pretraining/EasyLM/data.py:173

Method__iter__

(self)

fot_continued_pretraining/EasyLM/data.py:334

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:36

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:50

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:107

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:227

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:243

Method__iter__

(self)

fot_continued_pretraining/FoT/data_pipeline.py:277

Method__len__

(self)

instruction_fine_tuning/data_processing.py:570

Method__len__

(self)

instruction_fine_tuning/data_processing.py:697

Function_chunk_attention_bias

(query_chunk_size, key_chunk_size, bias, deterministic, attn_dropout, attn_pdrop, causal,

fot_continued_pretraining/EasyLM/bpt.py:162

Method_convert_id_to_token

Converts an index (integer) in a token (str) using the vocab.

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1306

Method_convert_token_to_id

Converts a token (str) in an id using the vocab.

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1302

Method_init_weights

(self, module)

src/modeling_longllama.py:592

Method_reorder_cache

(past_key_values, beam_idx)

src/modeling_longllama.py:1326

Method_set_gradient_checkpointing

(self, module, value=False)

src/modeling_longllama.py:603

Method_shape

(self, tensor: torch.Tensor, seq_len: int, bsz: int)

src/modeling_longllama.py:236

Method_tokenize

Returns a tokenized string.

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1298

Functionapply_rotary_emb_as_fst

( xk: jnp.ndarray, freqs_cis: jnp.ndarray, dtype: jnp.dtype = jnp.float32, )

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:434

Functionaverage_metrics

(metrics)

fot_continued_pretraining/EasyLM/jax_utils.py:285

Methodbos_token_id

(self)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1285

Methodbuild_inputs_with_special_tokens

(self, token_ids_0, token_ids_1=None)

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1355

Functionchat_tuning_data_processor

( data_args: DataArgs, data, tokenizer: PreTrainedTokenizer, tokenization_args: TokenizationArgs )

instruction_fine_tuning/data_processing.py:192

Methodcontinuous_data_source

()

fot_continued_pretraining/EasyLM/data.py:179

Methodconvert_to_numpy

(x)

instruction_fine_tuning/data_processing.py:601

Methodconvert_tokens_to_string

Converts a sequence of tokens (string) in a single string.

fot_continued_pretraining/EasyLM/models/llama/llama_model.py:1311

Methodcreate_custom_forward

(module)

src/modeling_longllama.py:868

← previous next →101–200 of 296, ranked by callers