(q: Tensor)
| 103 | |
| 104 | def forward(self, q, kv): |
| 105 | def self_attn_residual_func(q: Tensor) -> Tensor: |
| 106 | return self.ls1(self.self_attn(self.norm1(q))) |
| 107 | |
| 108 | def cross_attn_residual_func(q: Tensor, kv: Tensor) -> Tensor: |
| 109 | return self.ls2(self.cross_attn(self.q_norm2(q), self.kv_norm2(kv))) |
nothing calls this directly
no outgoing calls
no test coverage detected