Build an ErrorBuffer from an OmegaConf/dict config node. When ``num_blocks > 0`` the buffer becomes 2D (position × timestep), enabling teacher-forcing-aware position-dependent error injection. Pass ``global_block_offset`` so logs can identify which absolute slice of the full sequenc
(config, num_blocks=0, global_block_offset=0,
shard_rank=0, shard_size=1)
| 294 | |
| 295 | |
| 296 | def build_error_buffer(config, num_blocks=0, global_block_offset=0, |
| 297 | shard_rank=0, shard_size=1): |
| 298 | """Build an ErrorBuffer from an OmegaConf/dict config node. |
| 299 | |
| 300 | When ``num_blocks > 0`` the buffer becomes 2D (position × timestep), |
| 301 | enabling teacher-forcing-aware position-dependent error injection. |
| 302 | Pass ``global_block_offset`` so logs can identify which absolute slice |
| 303 | of the full sequence this rank's buffer covers (e.g. the last SP rank |
| 304 | is responsible for the most error-accumulated tail blocks). |
| 305 | |
| 306 | ``shard_rank`` / ``shard_size`` shard timestep buckets: each rank only |
| 307 | allocates the buckets it owns, reducing per-rank CPU memory by |
| 308 | ~``shard_size`` times. Typically set to ``(sp_rank, sp_size)``. |
| 309 | """ |
| 310 | cfg = config if isinstance(config, dict) else dict(config) |
| 311 | return ErrorBuffer( |
| 312 | num_buckets=cfg.get("num_buckets", 40), |
| 313 | max_size_per_bucket=cfg.get("buffer_size_per_bucket", 50), |
| 314 | num_train_timesteps=cfg.get("num_train_timesteps", 1000), |
| 315 | modulate_factor=cfg.get("modulate_factor", 0.3), |
| 316 | replacement_strategy=cfg.get("replacement_strategy", "random"), |
| 317 | num_blocks=num_blocks, |
| 318 | global_block_offset=global_block_offset, |
| 319 | shard_rank=shard_rank, |
| 320 | shard_size=shard_size, |
| 321 | ) |