↓ 1 callersFunctionprint_attention_diff(actual, expected, *, dtype, atol, rtol, causal, softmax_scale)
tests/test_attention_correctness.py:112
Method__call__ Standard attention forward pass. Args: q: (batch, seqlen, nheads, headdim) k: (batch, seqlen, nheads_k, headd
rl_engine/kernels/ops/rocm/attention/flash_attn.py:43
Method__call__(
self,
x: torch.Tensor,
weight: torch.Tensor,
*,
eps: float = 1e-6,
rl_engine/kernels/ops/pytorch/norm/rms_norm.py:18
Method__call__ Standard attention forward pass. Args: q: (batch, seqlen, nheads, headdim) k: (batch, seqlen, nheads_k, headd
rl_engine/kernels/ops/cuda/attention/flash_attn.py:36
Method__call__(self, hidden, lm_head_weight, target_ids, bias=None, **kwargs)
tests/test_deepspeed_training_worker.py:122
Method__init__(
self,
vocab_size: int,
hidden_dim: int,
*,
bias: bool = True,
rl_engine/executors/deepspeed_trainer.py:55