Standard cross entropy loss.
(logits: torch.Tensor, targets: torch.Tensor)
| 56 | |
| 57 | # Cross Entropy Loss |
| 58 | def cross_entropy_ref(logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor: |
| 59 | """Standard cross entropy loss.""" |
| 60 | return F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1)) |
| 61 | |
| 62 | # Rotary Position Embedding |
| 63 | def rotary_embedding_ref(x: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor) -> torch.Tensor: |
nothing calls this directly
no outgoing calls
no test coverage detected