Gets metadata for the batch.

Runs predictor. Args: fetch_keys: dict_keys object or a list of keys in the fetch dictionary to fetch. validate_fetches: if T

Write to borglet status.

Converts to a possibly partially specified tf.TensorShape.

Adjusts split annotation according to model_dim_reshape_segments.

Returns a DropoutLayer Params.

Einsum with adjusted equation according to model_dim_reshape_segments. It changes each dimension named 'M' in the equation into two dimensions 'N

Returns the dimensions that M is reshaped into.

Deletes multiple parameters. Dots in names indicate navigation into nested Params objects. We do not allow navigation into lists or dicts, an

Returns a copy with entries where fn(entry) is True.

Rescales the output corresponding to AQT quantized matmuls. Refer to quantizable_layer.FromAqtOutput. Args: w_name: weight name.

scale * tf.random.normal(0, 1 / sqrt(dim0)).

Syntactic sugar for asserting that tensor has the expected rank.

Total number of classes for the dataset.

Default parameters for Programs.

Quantizes weights and activations for (act * w) matmul AQT style. Refer to quantizable_layer.ToAqtInputs. Args: w_name: Previously cre

Static shape must be set correctly for RNN beam search compatibility.

Linear layer. y = matmul([..., idims], [idims, odims]).

Update the status message for this task.

Reads the variable and converts it to manually sharded mode. Returns: The return value has the shape of the individual elements of the stac

If in eager mode, watch the provided tensors on the GradientTape.

Add a new preprocessor before an existing preprocessor. Args: input_p: The input params. name: A string with the name of the new preprocess

Accumulates ngram statistics for the given ref and hyp string pair.

Computes overall BLEU score from the statistics accumulated so far.

r"""Applies a function (p.fn) on args. Args: theta: Unused. *args: A tuple of Tensors (one or more). Returns: fn(\*args).

Returns a descendant layer given the path. Args: path: a dot separated string denoting a descendant of this layer. Returns: The

Initialize 'states' with .key, .value, and .key_dists.

Does not modify the external_inputs parameter. This parameter, if provided, is assumed to be a vector that should be concatenated with the ot

Restore variables from a checkpoint. Args: sess: A session with tf.Graph under which this object is constructed. path: If not None, r

Logs the debug string at the level.

Returns the zero_state for the RNN cell. Args: theta: Variables used by the RNNCell. prepared_inputs: unused. batch_size: An in

Returns a copy of this `.NestedMap` filtered by fn. If fn(key, entry) is True, the entry is copied into the returned NestedMap. Otherwise, it

Merges params specified in 'text' into 'params'. 'text' follows the simple text format as produced by ToText. For a param specified in both '

Returns a stack of sequential layers.

Runs `all_runners` in parallel threads. Returns when all of them finish. Args: all_runners: a list of `.BaseRunner`. Returns:

Returns theta with state.

Apply fn over the input tuple.

Returns a tuple of (neg, pos) for clipped neg/pos values of v.

Get 'field_name' from a global configuration object. Currently the global configuration object used is FLAGS, but this may change to Cluster() or

Strides the input sequence. Args: name: name of this layer. stride: To use every k-th token, set the stride to k. When stride == 0,

Logs `text` and saves it under `logdir/filename`.

Returns tensor's rank as an int if it's available, otherwise a Tensor. Args: tensor: The input tensor. Returns: Either an int or a Tenso

Gets a state tensor that can be used to calculate clipping. The state will be a float32 tensor that is safe to pass to TF functions. Args:

FProp and returns the logits for the whole sequence.

Params for this MergerLayer class.

r"""Sets the value for a nested key. There is limited support for indexing lists when square bracket indexing is used, e.g., key[0], key[1],

Returns a stack of num_layers self-attention layers.

Activation layer.

Bias layer. The bias is added to the last dimension of the input.

Computes the attention. Each query selects 'p.attention_window' number of keys to attend to. First we find the closest centroid to that query

Creates a layer of variables potentially sharded in a device mesh. Args: name: name of the layer. weights: list of (name, gshard_laye

Apply ZoneOut regularlization to cur_v. Implements ZoneOut regularization as described in https://arxiv.org/abs/1606.01305 Args: p

Applies the gradients.

Computes attention probs in a non vectorized way. Args: key: A numpy ndarray of shape [batch, seqlen, heads, dim]. query: A numpy nda

Clean DecoderLayerStack, similar to EncoderLayerStack.

r"""Depthwise convolution for autoregressive models. Same implementation as mesh_tensorflow/ transformer/transformer.sublayer_depthwise_conv_

Dictionary of difficulty level strings to int32 indices.

The width of each bin for Euclidean distance in world coordinates.

Computes the value vector given the query of the current step. This function is used by autoregressive decoding. Used for self-attention (hen