Mindspore's fast gelu implementation.

Mindspore's fast gelu implementation.

Mindspore's fast gelu implementation.

Mindspore's fast gelu implementation.

Run forward and backward passes with no pipeline parallelism (no inter-stage communication). Returns dictionary with losses.

Run interleaved 1F1B schedule (model split into model chunks), with communication between pipeline stages as needed. Returns dictionary with

Run non-interleaved 1F1B schedule, with communication between pipeline stages. Returns dictionary with losses if the last stage, empty dict o

Forward step.

Forward step.

Text generation Inputs: model: the model for inferencing origin_inputs: the original inputs based on which the model will co

Text generation for incremental inference Inputs: model: the model for inferencing origin_inputs: the original inputs based

Text generation for incremental inference Inputs: model: the model for inferencing origin_inputs: the original inputs based

Text generation for incremental inference Inputs: model: the model for inferencing origin_inputs: the original inputs based

Beam search. Note that this function does not support model parallel!

Retrieves a single item from the dataset with the option to only return a portion of the item. get(idx) is the same as [idx] but get(

Divide sample into a and b segments.

Modification of `get_batch` to work on `next(data_iterator)` instead of `data_iterator`

Modification of `get_batch` to work on `next(data_iterator)` instead of `data_iterator`

Provide extra arguments required for merging.

Provide extra arguments required for merging.