Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expand attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Args: mask (torch.Tensor): The attention ma

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expand attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Args: mask (torch.Tensor): The attention mask t

Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`.

Expand attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Args: mask (torch.Tensor): The attention mask t

Expand attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Args: mask (torch.Tensor): The attention mask t

Make causal mask used for bi-directional self-attention.

Make causal mask used for bi-directional self-attention.

Create a causal mask for bi-directional self-attention. Args: input_ids_shape (torch.Size): The shape of input_ids tensor, typic

Make causal mask used for bi-directional self-attention.

Create a causal mask for bi-directional self-attention. Args: input_ids_shape (torch.Size): The shape of input_ids tensor, typically

Make causal mask used for bi-directional self-attention.

Create a causal mask for bi-directional self-attention. Args: input_ids_shape (torch.Size): The shape of input_ids tensor, typically

Create a causal mask for bi-directional self-attention. Args: input_ids_shape (torch.Size): The shape of input_ids tensor, typically

Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_l

Validate the `rope_scaling` configuration.

Validate the `rope_scaling` configuration.

Validate the `rope_scaling` configuration.

Validate the `rope_scaling` configuration.

Apply rotary position embeddings to query and key tensors. Args: q (torch.Tensor): Query tensor. k (torch.Tensor): Key

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Apply rotary position embeddings to query and key tensors. Args: q (torch.Tensor): Query tensor. k (torch.Tensor): Key tenso

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`

Applies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): T

Evaluate the posterior probabilities of the candidates based on the provided logits and choose the best candidate. Depending on the tempe

r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://arxiv.org/abs/

合并多个 Counter 字典

Pad the given path list with a specific value up to a specified length. Parameters: - path (list): The original list that needs padd

Pad the given path list with a specific value up to a specified length. Parameters: - path (list): The original list that needs padding.

Pad the given path list with a specific value up to a specified length. Parameters: - path (list): The original list that needs padding.

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication

Sort by question id and de-duplication