MCPcopy Create free account

hub / github.com/VITA-Group/WeLore / types & classes

Types & classes21 in github.com/VITA-Group/WeLore

⨍ Functions 139◇ Types & classes 21↳ Endpoints 1
↓ 6 callersClassLowRankLayer
given a linear layer find low rank decomposition
lib/LowRankLayer.py:4
↓ 4 callersClassGaLoreProjector
galore_torch/galore_projector.py:7
↓ 4 callersClassLowRankLayerEval
given a linear layer find low rank decomposition
lib/LowRankLayer.py:34
↓ 3 callersClassLlamaRMSNorm
peft_pretraining/modeling_llama.py:74
↓ 2 callersClassLlamaModel
Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`] Args: config: LlamaConfig
peft_pretraining/modeling_llama.py:423
↓ 2 callersClassTokenizerWrapper
lib/data_utils.py:14
↓ 1 callersClassCatcher
lib/rank_utils.py:55
↓ 1 callersClassLlamaAttention
Multi-headed attention from 'Attention Is All You Need' paper
peft_pretraining/modeling_llama.py:161
↓ 1 callersClassLlamaDecoderLayer
peft_pretraining/modeling_llama.py:243
↓ 1 callersClassLlamaMLP
peft_pretraining/modeling_llama.py:144
↓ 1 callersClassLlamaRotaryEmbedding
peft_pretraining/modeling_llama.py:94
↓ 1 callersClassMonitor
welore_downstream_finetune.py:49
↓ 1 callersClassMyDataset
lib/downstream_utils.py:14
↓ 1 callersClassPreprocessedIterableDataset
peft_pretraining/dataloader.py:7
ClassAdafactor
AdaFactor pytorch implementation can be used as a drop in replacement for Adam original fairseq code: https://github.com/pytorch/fairseq/blob
galore_torch/adafactor.py:14
ClassAdamW
Implements Adam algorithm with weight decay fix as introduced in [Decoupled Weight Decay Regularization](https://arxiv.org/abs/1711.05101).
galore_torch/adamw.py:15
ClassAdamW8bit
galore_torch/q_galore_adamw8bit.py:53
ClassAdamW8bit
galore_torch/adamw8bit.py:8
ClassLlamaForCausalLM
peft_pretraining/modeling_llama.py:603
ClassLlamaForSequenceClassification
peft_pretraining/modeling_llama.py:775
ClassLlamaPreTrainedModel
peft_pretraining/modeling_llama.py:332