Return the global rank of a process via its coordinates. Coordinates are specified as kwargs. For example: >>> X = ProcessTopolo

If distributed is initialized print only on rank 0.

Return arguments.

Retrieves a single item from the dataset with the option to only return a portion of the item. get(idx) is the same as [idx] but get

Update the tuner with what configurations have been evaluated and their performance results

Start the timer.

The group of ranks within the same stage of all pipelines.

Retrieves a single item from the dataset with the option to only return a portion of the item. get(idx) is the same as [idx] but get

Which pipeline this rank resides in.

Stop the timer.

Write timers to a tensorboard writer

Drop any empty strings from the list of compile and link flags

Get the tensor model parallel group the caller rank belongs to.

Track the rng state.

Calculate the elapsed time.

Helper method to get the model chunk ID given the iteration number.

check if the filepath for a text tokenizer exists

Return world size for the tensor model parallel group.

Return tokenizer.

The number of pipelines.