Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/KellerJordan/modded-nanogpt
/ functions
Functions
2,320 in github.com/KellerJordan/modded-nanogpt
⨍
Functions
2,320
◇
Types & classes
557
↳
Endpoints
41
Method
forward
(self, x)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:140
Method
forward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:150
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:171
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:191
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:205
Method
forward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:219
Method
forward
(self, x)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:64
Method
forward
(self, x)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:71
Method
forward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:81
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:102
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:122
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:136
Method
forward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:150
Method
forward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:226
Method
forward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:233
Method
forward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:243
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:264
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:284
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:298
Method
forward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:312
Method
forward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:209
Method
forward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:216
Method
forward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:226
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:247
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:267
Method
forward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:281
Method
forward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:295
Method
from_block
Creates a RootInvShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Information about
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:354
Method
from_block
Creates an EigendecomposedShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Informati
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:555
Method
from_block
Creates an EigenvalueCorrectedShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Infor
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:800
Method
from_kronecker_factors_state
Constructs an EigendecomposedShampooKroneckerFactorsUnwrapped object from the given Kronecker factors state. This method converts th
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:646
Method
from_kronecker_factors_state
Constructs an EigenvalueCorrectedShampooKroneckerFactorsUnwrapped object from the given Kronecker factors state. This method convert
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:892
Function
get_lr
(it)
records/track_2_medium/2024-12-31_Target350M/train_gpt.py:466
Function
get_lr
(it)
records/track_1_short/2024-10-09_SOAP/train_gpt2.py:726
Function
get_lr
(it)
records/track_1_short/2024-12-04_ValueEmbed/train_gpt2.py:459
Function
get_lr
(it)
records/track_1_short/2024-10-14_ModernArch/train_gpt2.py:384
Function
get_lr
(it)
records/track_1_short/2024-10-10_Muon/train_gpt2.py:393
Function
grid
(META)
triton_kernels.py:511
Method
l1_norm_scale_fn
(grad: torch.Tensor)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/shampoo_types_test.py:291
Function
linear_relu_square_kernel
(a_desc, b_desc, c_desc, aux_desc, dequant_scale_ptr,
triton_kernels.py:403
Method
load_from_new_state_to_old_state
( old_state: StateDict, new_state: StateDict )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:186
Method
local_block_info_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:99
Method
local_blocked_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:91
Method
local_grad_selector
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:87
Method
local_masked_blocked_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:95
Method
local_tensor_getter
(t: torch.Tensor | DTensor)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:638
Function
matrix_inverse_root_eigen
Compute matrix inverse root using eigendecomposition of symmetric positive (semi-)definite matrix. A^{-1/r} = Q L^{-1/r} Q^T
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:296
Function
matrix_inverse_root_higher_order
Compute matrix inverse root using coupled iterations, similar to above but generalized to support higher order. Rough sketch (at order =
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:429
Function
matrix_inverse_root_newton
Compute matrix inverse root using coupled inverse Newton iteration. alpha <- -1 / p X <- 1/c * I M <- 1/c^p * A
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:358
Method
merge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_ddp_distributor.py:515
Method
merge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hsdp_distributor.py:612
Method
merge_and_block_gradients
( self, )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:234
Method
merge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hybrid_shard_distributor.py:535
Function
mm_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt_medium.py:68
Function
mm_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:72
Function
mm_op
(x: Tensor, w: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt_medium.py:41
Function
mm_op
(x: Tensor, w: Tensor, x_s: float, w_s: float, grad_s: float)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:45
Function
mm_t_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt.py:99
Function
mm_t_op
Computes y = x @ w with F8 weights stored as (in_features, out_features).
train_gpt.py:64
Function
mpe_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:730
Function
newton_schulz
Perform quintic Newton-Schulz iteration to compute the semi-orthogonalization of a matrix A. This iteratively performs the iteration
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:937
Method
num_bytes_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:95
Method
numel_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:87
Method
only_first_call_runtime_error
( *args: object, **kwargs: object )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/matrix_functions_test.py:549
Method
param_group_hyperparameter_check
(param_group: dict[str, Any])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributed_shampoo.py:396
Function
precond_stat_hook
(module, inputs, output)
records/track_3_optimization/results/20260505_newton_muon/train_gpt_simple_newton_muon.py:189
Method
precondition
Preconditions a list of gradients using the Shampoo preconditioner that rely on ClassicShampooPreconditionerConfig. Args:
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:1498
Method
precondition
(self, masked_grad_list: tuple[Tensor, ...])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sgd_preconditioner_list.py:36
Method
precondition
(self, masked_grad_list: tuple[Tensor, ...])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sign_descent_preconditioner_list.py:46
Method
precondition
Preconditions the gradient list using the AdaGrad preconditioner. Args: masked_grad_list (tuple[Tensor, ...]): A tuple o
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/adagrad_preconditioner_list.py:144
Function
profile_decorator
Decorator that profiles the execution of a class method. This decorator wraps a class method with PyTorch's profiler.record_function to track
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:23
Function
qr_algorithm
Approximately compute the eigendecomposition of a symmetric matrix by performing the QR algorithm. Given an initial estimate of the eigenvect
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:780
Method
quantized_value
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/shampoo_quantization.py:233
Method
raise_preconditioner_value_error
( factor_matrix_index: str = factor_matrix_index, bias_correct
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:302
Method
refresh_assigned_full_params
Refresh the cached full parameter tensors from current model parameters. This method should be called after loading a checkpoint to ensure th
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/_shampoo_hybrid_shard_lossless_distributor.py:235
Method
remove_empty_entry
(d: dict[str, Any], key: str)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:67
Method
requires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:42
Method
requires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:133
Method
requires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:191
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:38
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:77
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:103
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:129
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:166
Method
requires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:187
Method
reset
(self)
train_gpt_medium.py:501
Method
reset
Reset NorMuon momentum buffers and split_embed state (called on training reset).
train_gpt.py:640
Method
reset
(self)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:511
Method
reset_parameters
(self)
train_gpt_medium.py:850
Method
reset_parameters
(self)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:834
Function
rre_mpe_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:931
Function
rre_quadratic_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:887
Function
rre_tsvd_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:690
Function
rre_velocity_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:962
Function
rre_velocity_residual_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:992
Method
save_to_state_dict
r"""Saves module state to `destination` dictionary, containing a state of the module, but not its descendants. This is called on every
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:71
Method
setUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_state_dict_utils_test.py:70
Method
setUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_quantization_test.py:30
Method
setUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_quantization_test.py:146
Method
setUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_distributor_test.py:27
← previous
next →
1,801–1,900 of 2,320, ranked by callers