MCPcopy Create free account

hub / github.com/KellerJordan/modded-nanogpt / functions

Functions2,320 in github.com/KellerJordan/modded-nanogpt

Methodtest_root_with_big_numerator_denominator
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/matrix_functions_test.py:667
Methodtest_set_seed_different_seeds_produce_different_results
Test that different seeds produce different random results.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:87
Methodtest_set_seed_enables_deterministic_algorithms
Test that set_seed enables deterministic algorithms.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:82
Methodtest_set_seed_reproducibility
Test that set_seed produces reproducible random numbers.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:66
Methodtest_sets_cublas_workspace_config
Test that setup_environment sets CUBLAS_WORKSPACE_CONFIG.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:37
Methodtest_setstate_call
Test that __setstate__ is properly called during load_state_dict operation.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:355
Methodtest_setup_distribution_cpu
Test setup_distribution on CPU.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:191
Methodtest_setup_distribution_cuda
Test setup_distribution on CUDA.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:205
Methodtest_setup_per_rank_logging_configures_root_logger
Test that setup_per_rank_logging configures the root logger.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:245
Methodtest_setup_per_rank_logging_verbose_sets_debug
Test that verbose=True sets the root logger to DEBUG level.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/utils_test.py:257
Methodtest_sgd_grafting
( self, weight_decay: float, device: torch.device, preconditioner_config: Prec
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/gpu_tests/shampoo_grafting_test.py:227
Methodtest_some_ranks_with_no_grads_due_to_dead_layers
( self, communicate_params: bool )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:703
Methodtest_split_size_is_inf
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_utils_test.py:180
Methodtest_split_tensor_block_recovery_for_one_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_fsdp_distributor_test.py:61
Methodtest_split_tensor_block_recovery_for_one_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_hsdp_distributor_test.py:59
Methodtest_split_tensor_block_recovery_for_three_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_fsdp_distributor_test.py:110
Methodtest_split_tensor_block_recovery_for_three_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_hsdp_distributor_test.py:108
Methodtest_split_tensor_block_recovery_for_two_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_fsdp_distributor_test.py:81
Methodtest_split_tensor_block_recovery_for_two_dim
( self, start_idx: int, end_idx: int, expected_split_tensors: list[Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_hsdp_distributor_test.py:79
Methodtest_state_dict
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/optimizer_modules_test.py:42
Methodtest_state_dict
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:259
Methodtest_state_dict
Test that the state dict is correct by comparing optimizer.state_dict() and the reference state dict.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:380
Methodtest_state_dict
Test that the state dict contains weight_buffer when iterate averaging is enabled.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1320
Methodtest_state_dict_warning
When Shampoo's `post_state_dict_hook` is fired during `state_dict()` call, it should issue a warning if a lambda function is detected
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1027
Methodtest_state_dict_warning
When Shampoo's `post_state_dict_hook` is fired during `state_dict()` call, it should issue a warning if a lambda function is detected
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1065
Methodtest_state_dict_without_non_tensor_objects
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/optimizer_modules_test.py:62
Methodtest_step_with_closure
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:272
Methodtest_step_with_consistent_grads
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1622
Methodtest_step_with_none_grads
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1633
Methodtest_target_tensor_dimensionality_is_inf
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_utils_test.py:97
Methodtest_tensorboard_global_metrics
Test global metrics tensorboard logging.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/loss_metrics_test.py:141
Methodtest_tensorboard_logging
Test that tensorboard logging works when metrics_dir is provided.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/loss_metrics_test.py:47
Methodtest_test_perturbation_before_computation_with_disportionate_epsilon
( self, perturb_before_computation: bool )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/matrix_functions_test.py:156
Methodtest_torch_dtype_resolver_via_omegaconf
Test torch_dtype resolver works via OmegaConf interpolation.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/resolvers_test.py:108
Methodtest_train_eval_mode_switching
Test that train() and eval() mode switching works correctly with iterate averaging. This verifies that the mode switching updates the
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1404
Methodtest_train_eval_mode_without_iterate_averaging
Test that train() and eval() are no-ops when iterate_averaging_config is None. This verifies that calling these methods doesn't raise
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1562
Methodtest_uneven_batching
Test batching an iterable with size not divisible by batch size.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/commons_test.py:31
Methodtest_update_param_state_dict_object
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_state_dict_utils_test.py:115
Methodtest_update_param_state_dict_object_with_missing_key
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_state_dict_utils_test.py:145
Methodtest_update_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_distributor_test.py:41
Methodtest_update_params
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/tests/shampoo_distributor_test.py:230
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:809
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_fully_shard_distributor_test.py:317
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_hybrid_shard_lossless_distributor_test.py:434
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_hybrid_shard_distributor_test.py:590
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_fsdp_distributor_test.py:314
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_fully_shard_lossless_distributor_test.py:366
Methodtest_update_params
(self, use_masked_tensors: bool)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_hsdp_distributor_test.py:424
Methodtest_update_preconditioners_and_precondition
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/sgd_preconditioner_list_test.py:34
Methodtest_update_preconditioners_and_precondition
We provide examples where we update the preconditioners twice using specially chosen gradients such that we get a scalar * id
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/shampoo_preconditioner_list_test.py:627
Methodtest_update_preconditioners_and_precondition
We provide examples where we update the preconditioners twice using specially chosen gradients such that we get a scalar * identity m
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/shampoo_preconditioner_list_test.py:1161
Methodtest_update_preconditioners_and_precondition
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/spectral_descent_preconditioner_list_test.py:57
Methodtest_update_preconditioners_and_precondition
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/adagrad_preconditioner_list_test.py:78
Methodtest_update_preconditioners_and_precondition
( self, scale_fn: Callable[[Tensor], float | Tensor] )
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/sign_descent_preconditioner_list_test.py:61
Methodtest_update_preconditioners_and_precondition_with_dims_ignored
(1) Tensor of Size 2 G1 = [4, 0]^T G2 = [0, 4]^T L = G1 * G1^T + G2 * G2^T = [[4*4, 0],
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/shampoo_preconditioner_list_test.py:928
Methodtest_update_preconditioners_and_precondition_with_epsilon
We provide examples where we deliberately choose a large epsilon. This is to ensure that the matrix inverse computation behav
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/shampoo_preconditioner_list_test.py:834
Methodtest_update_preconditioners_and_precondition_with_epsilon
We provide examples where we deliberately choose a large epsilon. This is to ensure that the matrix inverse computation behaves as ex
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/tests/shampoo_preconditioner_list_test.py:1447
Methodtest_valid_config
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/shampoo_types_test.py:334
Methodtest_valid_config
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/shampoo_types_test.py:345
Methodtest_var
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/tests/shampoo_utils_test.py:270
Methodtest_weight_buffer_in_state
Test that weight_buffer is present in each block's state.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/tests/distributed_shampoo_test.py:1351
Methodtest_weight_decay_type_resolver_via_omegaconf
Test weight_decay_type resolver works via OmegaConf interpolation.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/resolvers_test.py:120
Methodtest_window_size_behavior
Test that window size affects logging behavior correctly.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/loss_metrics_test.py:196
Methodtest_wrap_model_asserts_device_mesh_not_none
Test that wrap_model raises assertion error if device_mesh is None.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:212
Methodtest_wrap_model_asserts_device_mesh_not_none
Test that wrap_model raises assertion error if device_mesh is None.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:271
Methodtest_wrap_model_calls_fully_shard
Test that wrap_model calls fully_shard.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:233
Methodtest_wrap_model_gloo_backend
Test that wrap_model does not use device_ids for gloo backend.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:94
Methodtest_wrap_model_nccl_backend
Test that wrap_model uses device_ids for nccl backend.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:81
Methodtest_wrap_model_passes_mesh
Test that wrap_model passes device mesh to fully_shard.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:258
Methodtest_wrap_model_returns_unwrapped_model
Test that wrap_model returns the model unchanged.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:57
Methodtest_wrap_model_uses_hybrid_shard_strategy
Test that FSDP is called with HYBRID_SHARD strategy.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:190
Methodtest_wrap_model_uses_orig_params
Test that FSDP is called with use_orig_params=True.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:137
Methodtest_wrap_model_with_distributed_config
Test that wrap_model calls distributed_config partial.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:107
Methodtest_wrap_model_with_distributed_config
Test that distributed_config receives param_to_metadata.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:156
Methodtest_wrap_model_with_distributed_config
Test that distributed_config receives device_mesh.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:280
Methodtest_wrapped_model_with_distributed_config
Test WrappedModel with both model and distributed_config.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:36
Methodtest_wrapped_model_with_model_only
Test WrappedModel with only model, no distributed_config.
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/examples/tests/parallelism_test.py:28
Functiontokenize
(doc)
data/fineweb.py:79
Functiontrace_handler
(prof: torch.profiler.profile)
records/track_1_short/2025-12-11_NorMuonOptimsAndFixes/profiler-example-traces/train_gpt-profiler-example.py:1590
Methodupdate_params
Update params stored inside this distributor according to the input search directions argument. Args: blocked_search_directions (
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_ddp_distributor.py:271
Methodupdate_params
Update params stored inside this distributor according to the input search directions argument. Args: blocked_search_directions (
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/_shampoo_hybrid_shard_lossless_distributor.py:163
Methodupdate_params
Update params stored inside this distributor according to the input search directions argument. Args: blocked_search_directions (
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hsdp_distributor.py:228
Methodupdate_params
Update params stored inside this distributor according to the input search directions argument. Args: blocked_search_directions (
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/_shampoo_fully_shard_lossless_distributor.py:164
Methodupdate_params
( self, blocked_search_directions: tuple[Tensor, ...], use_masked_tensors: bool = True
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_distributor.py:80
Methodupdate_params
Update params stored inside this distributor according to the input search directions argument. Args: blocked_search_directions (
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/shampoo_hybrid_shard_distributor.py:250
Methodupdate_preconditioners
Updates the preconditioners. Args: masked_grad_list (tuple[Tensor, ...]): A list of gradients with their corresponding m
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/shampoo_preconditioner_list.py:1693
Methodupdate_preconditioners
( self, masked_grad_list: tuple[Tensor, ...], step: Tensor, perform_amortized_
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/spectral_descent_preconditioner_list.py:46
Methodupdate_preconditioners
( self, masked_grad_list: tuple[Tensor, ...], step: Tensor, perform_amortized_
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sgd_preconditioner_list.py:28
Methodupdate_preconditioners
( self, masked_grad_list: tuple[Tensor, ...], step: Tensor, perform_amortized_
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/sign_descent_preconditioner_list.py:37
Methodupdate_preconditioners
( self, masked_grad_list: tuple[Tensor, ...], step: Tensor, perform_amortized_
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/preconditioner/adagrad_preconditioner_list.py:123
Methodupdate_prev
()
records/track_2_medium/2024-12-31_Target350M/train_gpt.py:99
Functionvea_extrapolate
(iterates, regularization=1e-6, damping=1.0)
records/track_3_optimization/results/20260520_rre_extrapolation_pr300_2925/train_gpt_simple_rre_pr300_2925.py:815
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/utils/gpu_tests/shampoo_utils_test.py:47
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:88
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_ddp_distributor_test.py:761
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_fully_shard_distributor_test.py:62
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_fully_shard_distributor_test.py:262
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_hybrid_shard_lossless_distributor_test.py:81
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_hybrid_shard_lossless_distributor_test.py:371
Methodworld_size
(self)
records/track_3_optimization/results/20260513_shampoo_1_4_power/distributed_shampoo/distributor/gpu_tests/shampoo_checkpoint_test.py:257
← previousnext →2,201–2,300 of 2,320, ranked by callers