Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/JonasGeiping/cramming
/ functions
Functions
411 in github.com/JonasGeiping/cramming
⨍
Functions
411
◇
Types & classes
86
Method
__init__
(self, param: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:645
Method
__init__
( self, param: Tensor, epsilon: float, beta2: float = 1.0, use_bias_co
cramming/backend/optimizers/shampoo/shampoo_utils.py:659
Method
__init__
(self, param, epsilon: float, beta2: float)
cramming/backend/optimizers/shampoo/shampoo_utils.py:684
Method
__init__
(self, param, epsilon: float, beta2: float)
cramming/backend/optimizers/shampoo/shampoo_utils.py:689
Method
__init__
( self, params, lr: float = 1e-2, betas: Tuple[float, float] = (0.9, 1.0),
cramming/backend/optimizers/shampoo/shampoo.py:63
Method
__init__
(self, sentences, model, mask_id, device, sweep_order)
cramming/data/generation_gibbs.py:12
Method
__init__
Initialize with a given pytorch dataset.
cramming/data/lmdb_datasets.py:29
Method
__init__
Initialize with a given pytorch dataset. The setup dictionary determines cache location and storage type.
cramming/data/cached_datasets.py:30
Method
__init__
(self, hidden_size: int, cfg_attention, use_bias: bool = True, length_factor: float = 1.0)
cramming/architectures/funnel_transformers.py:27
Method
__init__
(self, cfg_arch)
cramming/architectures/funnel_transformers.py:92
Method
__init__
(self, ignore_index=-100, z_loss_factor=1e-4)
cramming/architectures/losses.py:12
Method
__init__
Parameters as in Hui&Belkin, 2021, but k=1, and M=sqrt(C) (so maybe not really Hui&Belkin?)
cramming/architectures/losses.py:34
Method
__init__
Parameters as in Hui&Belkin, 2021, but k=1, and M=sqrt(C) (so maybe not really Hui&Belkin?)
cramming/architectures/losses.py:60
Method
__init__
Overrelax parameter is quite a bit speculative...
cramming/architectures/losses.py:111
Method
__init__
(self, gamma: float = 5.0, size_average: bool = True, ignore_index: int = -100)
cramming/architectures/losses.py:159
Method
__init__
(self, cfg_arch_container: dict = {}, **kwargs)
cramming/architectures/scriptable_bert.py:41
Method
__init__
(self, config)
cramming/architectures/scriptable_bert.py:143
Method
__init__
(self, config)
cramming/architectures/scriptable_bert.py:227
Method
__init__
(self, config)
cramming/architectures/scriptable_bert.py:305
Method
__init__
(self, hidden_size)
cramming/architectures/attention.py:56
Method
__init__
(self, hidden_size, cfg_attention)
cramming/architectures/attention.py:69
Method
__init__
(self, hidden_size, cfg_attention)
cramming/architectures/attention.py:90
Method
__init__
(self, hidden_size: int, cfg_attention)
cramming/architectures/attention.py:126
Method
__init__
(self, hidden_size, cfg_attention)
cramming/architectures/attention.py:274
Method
__init__
(self, hidden_size, cfg_attention, length_factor=1.0)
cramming/architectures/attention.py:334
Method
__init__
(self, hidden_size, cfg_attention)
cramming/architectures/attention.py:400
Method
__init__
( self, hidden_size, cfg_attention, )
cramming/architectures/attention.py:461
Method
__init__
( self, hidden_size, cfg_attention, )
cramming/architectures/attention.py:499
Method
__init__
(self, hidden_size, cfg_attention)
cramming/architectures/attention.py:536
Method
__init__
( self, hidden_size, cfg_attention, )
cramming/architectures/attention.py:582
Method
__init__
(self, seq_op_in_fp32=False)
cramming/architectures/attention.py:680
Method
__init__
Normalized attention pooling as described in Richter&Wattenhofer, 2020.
cramming/architectures/attention.py:699
Method
__init__
(self, seq_op_in_fp32)
cramming/architectures/attention.py:725
Method
__init__
(self, seq_op_in_fp32)
cramming/architectures/attention.py:738
Method
__init__
(self, seq_op_in_fp32)
cramming/architectures/attention.py:751
Method
__init__
(self, width, vocab_size)
cramming/architectures/sanity_check.py:9
Method
__init__
(self, idx, cfg_arch)
cramming/architectures/recurrent_transformers.py:43
Method
__init__
(self, cfg_arch)
cramming/architectures/recurrent_transformers.py:94
Method
__init__
(self, list_of_modules)
cramming/architectures/recurrent_transformers.py:281
Method
__init__
(self, embedding_dim, max_seq_length=5000)
cramming/architectures/embeddings.py:16
Method
__init__
(self, embedding_dim, max_seq_length)
cramming/architectures/embeddings.py:44
Method
__init__
(self, embedding_dim, max_seq_length=1024)
cramming/architectures/embeddings.py:64
Method
__init__
(self, dim, base=10000, def_seq_length=128, seq_dim: int = 0)
cramming/architectures/embeddings.py:135
Method
__init__
(self, dim_model: int, *_, **__)
cramming/architectures/embeddings.py:199
Method
__init__
(self, idx, cfg_arch)
cramming/architectures/fixed_cramlm.py:33
Method
__init__
(self, encoder, cfg_arch)
cramming/architectures/fixed_cramlm.py:87
Method
__init__
(self, encoder, cfg_arch)
cramming/architectures/fixed_cramlm.py:148
Method
__init__
(self, cfg_embedding, norm, norm_eps)
cramming/architectures/components.py:16
Method
__init__
(self, idx, hidden_size, cfg_attention, use_bias=True)
cramming/architectures/components.py:49
Method
__init__
(self, hidden_size, intermed_size, nonlin_fn=torch.nn.GELU, use_bias=True)
cramming/architectures/components.py:72
Method
__init__
(self, idx, cfg_arch)
cramming/architectures/components.py:89
Method
__init__
(self, idx, cfg_arch)
cramming/architectures/components.py:160
Method
__init__
(self, cfg_head, main_model_hidden_size)
cramming/architectures/components.py:177
Method
__init__
(self, cfg_arch)
cramming/architectures/components.py:202
Method
__init__
(self, hidden_size: int, eps: float = 1e-5)
cramming/architectures/components.py:295
Method
__init__
(self, hidden_size: int, eps: float = 1e-8)
cramming/architectures/components.py:308
Method
__init__
(self, list_of_modules)
cramming/architectures/components.py:321
Method
__len__
Draw length from target dataset.
cramming/data/lmdb_datasets.py:119
Method
__len__
Length is length of self.dataset.
cramming/data/cached_datasets.py:69
Method
__len__
(self)
cramming/architectures/recurrent_transformers.py:292
Method
__next__
(self)
cramming/backend/utils.py:265
Method
__repr__
(self)
cramming/backend/optimizers/optimizer_modifiers.py:27
Method
__setstate__
(self, state)
cramming/backend/optimizers/optimizer_modifiers.py:24
Method
__setstate__
(self, state)
cramming/data/lmdb_datasets.py:92
Method
_forward_deepthinking
Requires the encoder to be a ScriptableRecurrentLM. Requires labels!
cramming/architectures/recurrent_transformers.py:207
Method
_forward_fixed
Requires the encoder to be a ScriptableRecurrentLM. Requires labels!
cramming/architectures/recurrent_transformers.py:237
Method
_forward_token_exit
Requires the encoder to be a ScriptableRecurrentLM. Requires labels!
cramming/architectures/recurrent_transformers.py:188
Function
avg_n_dicts
https://github.com/wronnyhuang/metapoison/blob/master/utils.py.
cramming/utils.py:401
Method
broadcast
(self, src_rank: int)
cramming/backend/optimizers/shampoo/shampoo_utils.py:474
Method
broadcast
(self, src_rank: int)
cramming/backend/optimizers/shampoo/shampoo_utils.py:611
Method
compute_norm
(self, grad: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:214
Method
compute_norm
(self, grad: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:471
Method
compute_norm
(self, grad: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:608
Method
compute_root_inverse
(self)
cramming/backend/optimizers/shampoo/shampoo_utils.py:595
Function
construct_model
(cfg_arch, vocab_size, downstream_classes=None)
cramming/architectures/construction.py:16
Function
count_token
(examples)
cramming/data/curriculum_sorting.py:66
Function
count_unigrams
(examples)
cramming/data/curriculum_sorting.py:22
Function
count_word_lengths
(examples)
cramming/data/curriculum_sorting.py:105
Method
direction_norm
(self, grad: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:651
Method
direction_norm
(self, grad: Tensor)
cramming/backend/optimizers/shampoo/shampoo_utils.py:676
Function
dump_metrics
Simple yaml dump of metric values.
cramming/utils.py:421
Function
filter_fn
Assume a metadata key 'meta' is present
cramming/data/pretraining_preparation.py:141
Function
filtering_rule
(examples)
cramming/data/pretraining_preparation.py:397
Function
find_pretrained_checkpoint
Load a checkpoint either locally or from the internet.
cramming/utils.py:172
Method
forward
(self, input_ids, labels)
cramming/backend/torch_default.py:435
Method
forward
(self, hidden_states, attention_mask: Optional[torch.Tensor] = None)
cramming/architectures/funnel_transformers.py:40
Method
forward
(self, states, attention_mask: Optional[torch.Tensor] = None)
cramming/architectures/funnel_transformers.py:65
Method
forward
(self, input_ids, attention_mask: Optional[torch.Tensor] = None, labels: Optional[torch.Tensor] = None)
cramming/architectures/funnel_transformers.py:124
Method
forward
Is this is the optimal implementation? Is this even what is meant? I wish there were more answers or code for PaLM This implementatio
cramming/architectures/losses.py:18
Method
forward
Is this is the optimal implementation? Could also do an index_select variation...
cramming/architectures/losses.py:39
Method
forward
Is this is the optimal implementation? This at least circumvents literal 1-hot labels
cramming/architectures/losses.py:65
Method
forward
Optimal scaling is less clear for L1
cramming/architectures/losses.py:89
Method
forward
This really just does L2(DNN(embed(x[:,:-1]), 2.0 * stop_gradient(embed(x[:,1:]))) as quoted above
cramming/architectures/losses.py:118
Method
forward
(self, input: torch.Tensor, target: torch.Tensor)
cramming/architectures/losses.py:165
Method
forward
(self, input: torch.Tensor, target: torch.Tensor)
cramming/architectures/losses.py:179
Method
forward
(self, input_ids, attention_mask: Optional[torch.Tensor] = None, labels: Optional[torch.Tensor] = None)
cramming/architectures/scriptable_bert.py:95
Method
forward
( self, input_ids, attention_mask: Optional[torch.Tensor] = None, labels: Opti
cramming/architectures/scriptable_bert.py:182
Method
forward
( self, input_ids, attention_mask: Optional[torch.Tensor] = None, labels: Opti
cramming/architectures/scriptable_bert.py:261
Method
forward
( self, input_ids, attention_mask: Optional[torch.Tensor] = None, labels: Opti
cramming/architectures/scriptable_bert.py:338
Method
forward
(self, hidden_states, attention_mask: Optional[torch.Tensor] = None)
cramming/architectures/attention.py:60
← previous
next →
201–300 of 411, ranked by callers