Set the rng states. For efficiency purposes, we do not check the size of seed for compatibility.

Setup model and optimizer.

Shift input ids one token to the right.

Shift input ids one token to the right.

Shift input ids one token to the right.

Decorator to make all processes in distributed training wait for each local_master to do something.

Single training step.

Log training information such as losses, timing, ....

Calculate user AUC

Union vocab in all workers.

Runs basic whitespace cleaning and splitting on a piece of text.

Runs basic whitespace cleaning and splitting on a piece of text.

Returns a list of the text of each token Args: uncased: lower cases text

Initialize tokenizer.

Build train, valid, and test datasets.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Flattens predictions in the batch (binary case) Remove labels equal to 'ignore'

Flattens predictions in the batch (binary case) Remove labels equal to 'ignore'

Flattens predictions in the batch (binary case) Remove labels equal to 'ignore'

Flattens predictions in the batch

Flattens predictions in the batch

Flattens predictions in the batch

:param example: :param unique_id: :return: input_ids = (C, seq_len) token_type_ids = (C, seq_len) = s

Compute gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper

Compute gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper

Compute gradient of the Lovasz extension w.r.t sorted errors See Alg. 1 in paper

Binary Lovasz hinge loss Args: logits: [P] Logits at each prediction (between -infinity and +infinity) labels: [P] Tensor, binary

Binary Lovasz hinge loss Args: logits: [P] Logits at each prediction (between -infinity and +infinity) labels: [P] Tensor, binary

Binary Lovasz hinge loss Args: logits: [P] Logits at each prediction (between -infinity and +infinity) labels: [P] Tensor, binary

Multi-class Lovasz-Softmax loss Args: @param probas: [P, C] Class probabilities at each prediction (between 0 and 1) @param labels

Multi-class Lovasz-Softmax loss Args: @param probas: [P, C] Class probabilities at each prediction (between 0 and 1) @param labels

Multi-class Lovasz-Softmax loss Args: @param probas: [P, C] Class probabilities at each prediction (between 0 and 1) @param labels

Use multi-tensor-applier to copy values from one list to another. We don't have a blfoat16 implementation so for now if the overflow_buf is no

Provide function that calculates accuracies.

source - https://gist.github.com/simon-weber/7853144 A context manager that will prevent any logging messages triggered during the body from bein

Allocate a memory buffer.

Helper method to run backward step with model split into chunks (run set_virtual_pipeline_model_parallel_rank() before calling backwar

Returns the intersection over box2 area given box1, box2. Boxes are x1y1x2y2 box1: np.array of shape(4) box2: np.array of shape(n

Return intersection-over-union (Jaccard index) of boxes. Both sets of boxes are expected to be in (x1, y1, x2, y2) format. Arguments:

Convert to numpy and return a sample consumed by the batch producer.

Build token types and paddings, trim if needed, and pad if needed.

XXX

Calculate l2 norm of parameters

Check if the dataset format is compatible and upload it as W&B artifact arguments: opt (namespace)-- Commandline arguments f