Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/KellerJordan/modded-nanogpt
/ functions
Functions
2,320 in github.com/KellerJordan/modded-nanogpt
⨍
Functions
2,320
◇
Types & classes
557
↳
Endpoints
41
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:561
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:499
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260609_soap_f1_auxb2_clean/train_gpt_clean_SOTA.py:351
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260619_cwd_rowfloor_tailema/train_gpt_cwd_SOTA.py:379
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:705
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:325
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:498
↓ 1 callers
Function
rescale_to_radius
(param: Tensor, target_norm: Tensor, eps: float = 1e-8)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:481
↓ 1 callers
Method
reset
(self, state=None)
train_gpt_medium.py:1590
↓ 1 callers
Method
reset
(self, state=None)
train_gpt.py:1961
↓ 1 callers
Method
reset_parameters
(self)
train_gpt.py:966
↓ 1 callers
Function
restore_muon_weight_scale
(applied)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:1388
↓ 1 callers
Function
restore_proj_weight_scale
(applied)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:1406
↓ 1 callers
Function
restore_ref_extrap_weights
(applied)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:761
↓ 1 callers
Function
round_up_to_multiple_of
(x: int, y: int)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/shampoo_utils.py:353
↓ 1 callers
Function
run_macro_cem
()
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:2067
↓ 1 callers
Function
save_checkpoint
(step: int)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:1434
↓ 1 callers
Function
save_screening_checkpoint
(path: str, step: int)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:1035
↓ 1 callers
Function
save_screening_checkpoint
(path: str, step: int)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:1034
↓ 1 callers
Function
save_screening_checkpoint
(path: str, step: int)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:1009
↓ 1 callers
Function
scale_invariant_update_
Hyperball-constrained step: take a preconditioned update of size lr * ||param||, then renormalise back onto the Frobenius sphere of the parameter'
records/track_3_optimization/results/20260504_muloco_normuonh/train_gpt_simple_muloco_normuonh.py:217
↓ 1 callers
Function
scale_invariant_update_
Hyperball-constrained step: take a Muon-orthogonalised update of size lr * ||param||, then renormalise back onto the Frobenius sphere of the param
records/track_3_optimization/results/20260430_muonh/train_gpt_simple_muonh.py:197
↓ 1 callers
Function
scale_invariant_update_
(param: Tensor, update: Tensor, lr: float, eps: float = 1e-10)
records/track_3_optimization/results/20260511_klsoap_h_lr_power_decay/train_gpt_simple_klsoap_h_lr_power_decay.py:164
↓ 1 callers
Function
scale_invariant_update_
Hyperball-constrained step: take a Muon-orthogonalised update of size lr * ||param||, then renormalise back onto the Frobenius sphere of the param
records/track_3_optimization/results/20260611_muonh_tuned_aux_3250/train_gpt_simple_muonh_tuned_aux_3250.py:197
↓ 1 callers
Function
scale_invariant_update_
(param: Tensor, update: Tensor, lr: float, eps: float = 1e-10)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:164
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:536
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:474
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260609_soap_f1_auxb2_clean/train_gpt_clean_SOTA.py:326
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260619_cwd_rowfloor_tailema/train_gpt_cwd_SOTA.py:354
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, step: int, target_uw: float = 0.0, eps: float = 1e-12)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:650
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, step: int, eps: float = 1e-12)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:299
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:473
↓ 1 callers
Function
scale_radial_update
(update: Tensor, param: Tensor, eps: float = 1e-12)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:456
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:398
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260506_trustlight/train_gpt_simple_trustlight.py:248
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:453
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260501_contra_muon/train_gpt_simple_contra_muon_2.py:207
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:540
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260509_contra_muon_split_cooldown/train_gpt_simple_contra_muon_split_cooldown.py:207
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260504_contra_muon_mlp_soapish/train_gpt_contra_normuon_soapish_mlp.py:208
↓ 1 callers
Function
scale_to_unit_operator_norm
(G: Tensor, eps: float = 1e-10)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:283
↓ 1 callers
Function
score_hellaswag
(model, schedule_cfg, seq_len: int, get_bigram_hash: Callable)
evals/hellaswag.py:152
↓ 1 callers
Function
score_sequence
(model, schedule_cfg, sequence: PackedHellaswagSequence, get_bigram_hash: Callable)
evals/hellaswag.py:124
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/train_gpt_simple.py:301
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:1489
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:1022
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260519_dynmuon/train_gpt_simple_dynmuon.py:475
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260501_muonsq/train_gpt_simple_muonsq.py:313
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260506_trustlight/train_gpt_simple_trustlight.py:583
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260609_soap_f1_auxb2_clean/train_gpt_clean_SOTA.py:830
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:2203
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260501_contra_muon/train_gpt_simple_contra_muon_2.py:370
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260504_muloco_normuonh/train_gpt_simple_muloco_normuonh.py:435
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260430_muonh/train_gpt_simple_muonh.py:336
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260511_klsoap_h_lr_power_decay/train_gpt_simple_klsoap_h_lr_power_decay.py:422
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260619_cwd_rowfloor_tailema/train_gpt_cwd_SOTA.py:874
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260611_muonh_tuned_aux_3250/train_gpt_simple_muonh_tuned_aux_3250.py:340
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260513_shampoo_1_4_power/train_gpt_shampoo.py:280
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:1556
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260505_newton_muon/train_gpt_simple_newton_muon.py:468
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260509_contra_muon_split_cooldown/train_gpt_simple_contra_muon_split_cooldown.py:374
↓ 1 callers
Function
set_hparams
(step, cooldown_frac=0.7)
records/track_3_optimization/results/20260504_contra_muon_mlp_soapish/train_gpt_contra_normuon_soapish_mlp.py:438
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:712
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:528
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:1021
↓ 1 callers
Function
set_hparams
(step)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:996
↓ 1 callers
Function
shampoo_distributed_config
()
records/track_3_optimization/results/20260513_shampoo_1_4_power/train_gpt_shampoo.py:171
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:573
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:511
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260506_trustlight/train_gpt_simple_trustlight.py:335
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260609_soap_f1_auxb2_clean/train_gpt_clean_SOTA.py:363
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:600
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260619_cwd_rowfloor_tailema/train_gpt_cwd_SOTA.py:391
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:717
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260504_contra_muon_mlp_soapish/train_gpt_contra_normuon_soapish_mlp.py:230
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:337
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:510
↓ 1 callers
Function
soap_basis_qr
(row_gg, col_gg, q_row, q_col, exp_avg_sq)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:493
↓ 1 callers
Function
soap_precondition_momentum
(update, state, beta2=SOAP_BETA2, eps=1e-8)
records/track_3_optimization/results/20260504_contra_muon_mlp_soapish/train_gpt_contra_normuon_soapish_mlp.py:245
↓ 1 callers
Function
soap_update
( grad: Tensor, momentum_buffer: Tensor, square_buffer: Tensor, left_buffer: Tensor, right
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:187
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260611_tailema_2720_submission/train_gpt_tailema_2720.py:537
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260609_soap_f1_auxb2_clean/train_gpt_clean_SOTA.py:389
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260619_cwd_rowfloor_tailema/train_gpt_cwd_SOTA.py:417
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=10)
records/track_3_optimization/results/20260504_contra_muon_mlp_soapish/train_gpt_contra_normuon_soapish_mlp.py:257
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:369
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:536
↓ 1 callers
Function
soap_update_preconditioner
(grad, state, shampoo_beta=SOAP_BETA2, precondition_frequency=SOAP_PRECONDITION_FREQUENCY)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:519
↓ 1 callers
Function
soft_blend_for_step
(step: int)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:627
↓ 1 callers
Function
soft_blend_for_step
(step: int)
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:654
↓ 1 callers
Function
soft_blend_for_step
(step: int)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:890
↓ 1 callers
Function
soft_via_newtonschulz5
(G: Tensor, p: float, scale_mode: str, input_norm: str)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:354
↓ 1 callers
Function
soft_via_newtonschulz5
(G: Tensor, p: float, scale_mode: str, input_norm: str)
records/track_3_optimization/results/20260529_tail_phase_readout_2850/train_gpt_tail_phase_readout_2850.py:409
↓ 1 callers
Function
soft_via_newtonschulz5
(G: Tensor, p: float, scale_mode: str, input_norm: str)
records/track_3_optimization/results/20260520_tail_refinterp_2900/train_gpt_tail_refinterp_2900.py:496
↓ 1 callers
Function
sparse_comms_merge_gradients
(grad, recv_idx, recv_vals, rank, world)
train_gpt.py:332
↓ 1 callers
Function
sparse_comms_share_gradients
(grad, idxes, send_counts, recv_counts)
train_gpt.py:310
↓ 1 callers
Function
sparse_comms_share_indexes
(send_idxes, send_counts, recv_counts)
train_gpt.py:285
↓ 1 callers
Function
sparse_comms_start
(idxes_np, N, rank, world, send_idxes_buffer)
train_gpt.py:257
↓ 1 callers
Method
step
(self)
train_gpt_medium.py:546
↓ 1 callers
Method
step
Combined optimizer step with explicit ordering. Args: do_adam: If True, update Adam params. NorMuon params always update
train_gpt.py:717
↓ 1 callers
Method
step
(self)
records/track_3_optimization/train_gpt_simple.py:198
← previous
next →
901–1,000 of 2,320, ranked by callers