Insert a sequence of --flag=value into argv, ensuring they are inserted before `--`.

Returns True iff proc exited with returncode.

Returns whether each input sequence passes validity check. Note that `optax.ctc_loss` returns -logeps (default to 1e5) if the input length is

Lists all builds for a given combination of region, project, image name, and tags. Args: project_id: The GCP project ID. image_na

Loads runtime option(s) from file, or returns {} on failure.

Removes blanks, paddings, and repeats from the input sequences, as used in CTC or RNN-T. Note that unless pad_id is the same as blank_id, pad_id

Returns a proxy which cannot be copied.

Reduces loss tensor. Args: loss: A [...] float tensor of arbitrary shape. reduction: The reduction method. sample_weight:

Loss that encourages router logits to remain small and improves stability. Reference: https://github.com/tensorflow/mesh/blob/fbf7b1e547e8b8c

Computes scores from logits using configured score_fn or default softmax.

Convert a list of keys and values to a static lookup table.

Test beam search and sample decoding from a randomly initialized decoder.

Shared logit sink test: compares TPU splash attention against reference.

Equivalent to `x.update(updates)` but raises upon key conflicts.

Validate worker role for operation. Args: required_role: Required role ('replica_manager' or 'global_manager') operat

Computes area. Args: boxes: An N-d float tensor of shape [..., N, 4] with boxes in corner representation [ymin, xmin, ymax, x

Computes Average Precision@K (AP@K) metrics. AP@K = sum_{k=1}^{K}(Precision@k * rel(k)) / min(K, total_num_relevant_items) where rel(k) = 1 i

Builds configs for BERT model. Defaults are from the BERT-BASE model. Args: vocab_size: Vocab size. hidden_dim: Hidden dim.

Converts a bool mask tensor to a bias mask tensor. Maps: 0 -> -NEG_INF 1 -> 0.

Returns the jobset dependsOn entry dict.

Checks that the supplied arguments are valid JAX types and raise ValueError if not. Args: args: Positional arguments of a function call t

See `BaseCheckpointer.checkpointer_paths`.

Terminate the upload process.

Closes the RabbitMQ connection.

Adapted from flatten_axes(), but with a simplified API and more error logging and messages. Original: https://github.com/google/jax/blob/cdf4

Computes the merge matrix. Args: state: As returned by `init_state` or `update`. Each tensor must be either a scalar or

Configures the logging level and adds FilterDiscoveryLogging. Args: level: Logging verbosity.

Delete node pools. See also: https://googleapis.github.io/google-api-python-client/docs/dyn/container_v1.projects.locations.clusters.nodePool

Downloads a batch of jobs. Args: spec_dir: Directory to look for job specs. state_dir: Directory to look for job states.

Create config for EventQueue. Args: flag_values: Flag configurations defined in gcp_settings. Returns: A configured `RabbitM

Create a fake libtpu server for testing.

Returns a dictionary with keys 'fan_in', 'fan_out', and 'fan_avg' containing the fan values for this parameter. The calculation is co

Flattens the first two dimensions of a non-scalar array.

Returns a human-readable per-resource utilization string. Args: utilization: Mapping from resource type to ProjectResourceUtilization.

Computes projects for the given query. Uses {k,v}_proj from `kv_state`. See parent class for full docstring.

The forward function of DiTBlock. Args: input: input tensor with shape [batch_size, num_length, input_dim]. condition

Restores the state of the given target using a state dict. Equivalent to `flax.serialization.from_state_dict`. Args: target: The obj

Returns a list of XLA devices for the given topology. Args: topology: A string representing the TPU topology, e.g., "v4-8". Must be a key

Get the suitable DotAlgorithmPreset for the given dtype.

Creates or retrieves the GCS client.

Build replica manager hostname for given replica_id. Args: replica_id: The replica ID to build hostname for Returns: Hostnam

Gets the shared module and state with the given name from a nearest ancestor. Shared modules should be registered via `_share_with_descendant

Get stochastic depth rate for the ith stage. Reference: Equation (4) in paper: https://arxiv.org/pdf/1603.09382.pdf Args: peak_r

Infer the number of TPU cores from the TPU type. Args: tpu_type: A string of the format {version}-{cores}[-{variant}]. -variant i

Infer the number of worker processes for the given TPU type. Args: tpu_type: A string of the format {version}-{cores}[-{variant}].

Initializes prefix merger state. Args: tokens: The initial live sequences, of shape [batch_size, num_decodes, max_decode_len].

Check if the TPU device on the host are idle.

Returns the most recent (highest step count) complete checkpoint under base dir. Args: base_dir: Path to checkpoints dir.

Return a copy of this instance where the values of the field have been masked using `optax.MaskedNode()` according to the leaves of `keep(sel

Pads query, key, value, and segment_id tensors to align with block_size requirements. This function ensures that the sequence length dimension of

Measures the time it takes to execute the function on the GPU. This function is modified from https://github.com/jax-ml/jax/blob/978d35f69704

Create an updates object with various semi-reasonable values.

Compute the negative cross similarity loss between predictions and targets. Args: predictions: A float Tensor of shape [..., dim].

PAD token ID.

Computes logits. Args: input_batch: A dict containing: inputs: A Tensor of shape [batch_size, num_frames, dim].

Returns a shallow copy of the input tree with empty subtrees pruned. If a tree would be made empty by removal of its subtrees, it will also be pr

Register a type for serialization. Equivalent to `flax.serialization.from_state_dict`. Args: ty: The type to be registered.

Reshape device list based on mesh_shape. Args: devices: A list of devices. mesh_shape: Mesh shape of the devices. Missing specifi

See `BaseCheckpointer.restore` docstring for details. A complete checkpoint is one with an "index" file, which is only written after the enti

Sets checkpoint state. The input is a `msgpack`-deserialized object corresponding to restored checkpoint state.

Changes into a temp CWD only within the context.

Returns a dictionary with the state of the given target. Equivalent to `flax.serialization.to_state_dict`. Args: target: The target

Tokenizes features. Args: ds: A Dataset. vocab: A vocab or a mapping from field to vocab. If a mapping is provided, f