MCPcopy Create free account

hub / github.com/KellerJordan/modded-nanogpt / functions

Functions2,320 in github.com/KellerJordan/modded-nanogpt

Methodforward
(self, x)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:140
Methodforward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:150
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:171
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:191
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:205
Methodforward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260525_aurora_ema_ref/train_gpt_simple_aurora_ema_ref.py:219
Methodforward
(self, x)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:64
Methodforward
(self, x)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:71
Methodforward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:81
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:102
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:122
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:136
Methodforward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260518_soaph/train_gpt_simple_soaph.py:150
Methodforward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:226
Methodforward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:233
Methodforward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:243
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:264
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:284
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:298
Methodforward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/train_gpt_tailema_2730.py:312
Methodforward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:209
Methodforward
(self, x)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:216
Methodforward
(self, x_BTHD: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:226
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:247
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:267
Methodforward
(self, x: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:281
Methodforward
(self, inputs: Tensor, targets: Tensor)
records/track_3_optimization/results/20260611_tailema_2730_submission/ablation/combo_2740/train_gpt_combo_2740.py:295
Methodfrom_block
Creates a RootInvShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Information about
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:354
Methodfrom_block
Creates an EigendecomposedShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Informati
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:555
Methodfrom_block
Creates an EigenvalueCorrectedShampooKroneckerFactorsState object for a given block. Args: block_info (BlockInfo): Infor
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:800
Methodfrom_kronecker_factors_state
Constructs an EigendecomposedShampooKroneckerFactorsUnwrapped object from the given Kronecker factors state. This method converts th
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:646
Methodfrom_kronecker_factors_state
Constructs an EigenvalueCorrectedShampooKroneckerFactorsUnwrapped object from the given Kronecker factors state. This method convert
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:892
Functionget_lr
(it)
records/track_2_medium/2024-12-31_Target350M/train_gpt.py:466
Functionget_lr
(it)
records/track_1_short/2024-10-09_SOAP/train_gpt2.py:726
Functionget_lr
(it)
records/track_1_short/2024-12-04_ValueEmbed/train_gpt2.py:459
Functionget_lr
(it)
records/track_1_short/2024-10-14_ModernArch/train_gpt2.py:384
Functionget_lr
(it)
records/track_1_short/2024-10-10_Muon/train_gpt2.py:393
Functiongrid
(META)
triton_kernels.py:511
Methodl1_norm_scale_fn
(grad: torch.Tensor)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/shampoo_types_test.py:291
Functionlinear_relu_square_kernel
(a_desc, b_desc, c_desc, aux_desc, dequant_scale_ptr,
triton_kernels.py:403
Methodload_from_new_state_to_old_state
( old_state: StateDict, new_state: StateDict )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:186
Methodlocal_block_info_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:99
Methodlocal_blocked_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:91
Methodlocal_grad_selector
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:87
Methodlocal_masked_blocked_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:95
Methodlocal_tensor_getter
(t: torch.Tensor | DTensor)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:638
Functionmatrix_inverse_root_eigen
Compute matrix inverse root using eigendecomposition of symmetric positive (semi-)definite matrix. A^{-1/r} = Q L^{-1/r} Q^T
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:296
Functionmatrix_inverse_root_higher_order
Compute matrix inverse root using coupled iterations, similar to above but generalized to support higher order. Rough sketch (at order =
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:429
Functionmatrix_inverse_root_newton
Compute matrix inverse root using coupled inverse Newton iteration. alpha <- -1 / p X <- 1/c * I M <- 1/c^p * A
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:358
Methodmerge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_ddp_distributor.py:515
Methodmerge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hsdp_distributor.py:612
Methodmerge_and_block_gradients
( self, )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:234
Methodmerge_and_block_gradients
Merge and block gradients. NOTE: This function MUST be called in the step function of the optimizer after the gradient has been updat
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hybrid_shard_distributor.py:535
Functionmm_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt_medium.py:68
Functionmm_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:72
Functionmm_op
(x: Tensor, w: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt_medium.py:41
Functionmm_op
(x: Tensor, w: Tensor, x_s: float, w_s: float, grad_s: float)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:45
Functionmm_t_backward_op
(g: Tensor, x_f8: Tensor, w_f8: Tensor, x_s: float, w_s: float, grad_s: float)
train_gpt.py:99
Functionmm_t_op
Computes y = x @ w with F8 weights stored as (in_features, out_features).
train_gpt.py:64
Functionmpe_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:730
Functionnewton_schulz
Perform quintic Newton-Schulz iteration to compute the semi-orthogonalization of a matrix A. This iteratively performs the iteration
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:937
Methodnum_bytes_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:95
Methodnumel_list
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:87
Methodonly_first_call_runtime_error
( *args: object, **kwargs: object )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/matrix_functions_test.py:549
Methodparam_group_hyperparameter_check
(param_group: dict[str, Any])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributed_shampoo.py:396
Functionprecond_stat_hook
(module, inputs, output)
records/track_3_optimization/results/20260505_newton_muon/train_gpt_simple_newton_muon.py:189
Methodprecondition
Preconditions a list of gradients using the Shampoo preconditioner that rely on ClassicShampooPreconditionerConfig. Args:
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:1498
Methodprecondition
(self, masked_grad_list: tuple[Tensor, ...])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sgd_preconditioner_list.py:36
Methodprecondition
(self, masked_grad_list: tuple[Tensor, ...])
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sign_descent_preconditioner_list.py:46
Methodprecondition
Preconditions the gradient list using the AdaGrad preconditioner. Args: masked_grad_list (tuple[Tensor, ...]): A tuple o
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/adagrad_preconditioner_list.py:144
Functionprofile_decorator
Decorator that profiles the execution of a class method. This decorator wraps a class method with PyTorch's profiler.record_function to track
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/preconditioner_list.py:23
Functionqr_algorithm
Approximately compute the eigendecomposition of a symmetric matrix by performing the QR algorithm. Given an initial estimate of the eigenvect
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/matrix_functions.py:780
Methodquantized_value
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/shampoo_quantization.py:233
Methodraise_preconditioner_value_error
( factor_matrix_index: str = factor_matrix_index, bias_correct
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:302
Methodrefresh_assigned_full_params
Refresh the cached full parameter tensors from current model parameters. This method should be called after loading a checkpoint to ensure th
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/_shampoo_hybrid_shard_lossless_distributor.py:235
Methodremove_empty_entry
(d: dict[str, Any], key: str)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:67
Methodrequires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:42
Methodrequires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:133
Methodrequires_device_mesh
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:191
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:38
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:77
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:103
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:129
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:166
Methodrequires_distributed
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/parallelism.py:187
Methodreset
(self)
train_gpt_medium.py:501
Methodreset
Reset NorMuon momentum buffers and split_embed state (called on training reset).
train_gpt.py:640
Methodreset
(self)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:511
Methodreset_parameters
(self)
train_gpt_medium.py:850
Methodreset_parameters
(self)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:834
Functionrre_mpe_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:931
Functionrre_quadratic_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:887
Functionrre_tsvd_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:690
Functionrre_velocity_blend_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:962
Functionrre_velocity_residual_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:992
Methodsave_to_state_dict
r"""Saves module state to `destination` dictionary, containing a state of the module, but not its descendants. This is called on every
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/optimizer_modules.py:71
MethodsetUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_state_dict_utils_test.py:70
MethodsetUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_quantization_test.py:30
MethodsetUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_quantization_test.py:146
MethodsetUp
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_distributor_test.py:27
← previousnext →1,801–1,900 of 2,320, ranked by callers