At least `x` of `a` and `b` `Tensors` are equal.

Reorder the data on single controller such that each dp rank gets similar total tokens

Bind the worker method to the WorkerGroup

Binds the methods of each worker to the WorkerDict. Note that we only bind public methods that are decorated by register

Check if the resource pool can be satisfied in this ray cluster.

This function should only be called inside by WorkerGroup

Create a random mask given input_ids. Support left padding and right padding. Process: - Sample valid token length - Sample left_padding

Patches flash attention forward to handle 3D position ids in mrope. (3, batch_size, seq_length)

Make a random crop of image_size.

Decode block definition string Gets a list of block arg (dicts) through a string notation of arguments. E.g. ir_r2_k3_s2_e1_i32_o16_se0.25_n

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Calls the forward method of Flash Attention - if the input hidden states contain at least one padding token first unpad the input, th

Random horizontal image flip.

FBNet-C Paper: https://arxiv.org/abs/1812.03443 Ref Impl: https://github.com/facebookresearch/maskrcnn-benchmark/blob/master/maskrcn

Creates a mnasnet-b1 model. Ref impl: https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet Paper: https://arxiv.org/pdf/1807

Creates a MobileNet-V3 model. Ref impl: ? Paper: https://arxiv.org/abs/1905.02244 Args: channel_multiplier: multiplier to number o

Creates the Single-Path NAS model from search targeted for Pixel1 phone. Paper: https://arxiv.org/abs/1904.02877 Args: channel_multipl

Method to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83 Returns:

Method to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83

Method to obtain re-parameterized kernel and bias. Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83

Compute the model size.

Returns potential safetensors alternatives for a given filename. Use case: When downloading a model from the Huggingface Hub, we first lo