Collate the batch elements into batched numpy arrays.

Initialize the weights

Background task for periodic subtask refresh

Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_l

Tell JAX inside the worker process not to preallocate the GPU memory.

Converts a PyTree of arrays into a human-readable string for logging.

Get the assets directory for this config.

Execute batch inference requests sequentially.

Checks that two PyTrees have the same structure and optionally checks shapes and dtypes. Creates a much nicer error message than if `jax.tree.map`

Get the checkpoint directory for this config.