Method
__init__
(self, embedding_dim: int, norm_eps: float, norm_elementwise_affine: bool)
Source from the content-addressed store, hash-verified
| 211 | """ |
| 212 | |
| 213 | def __init__(self, embedding_dim: int, norm_eps: float, norm_elementwise_affine: bool): |
| 214 | super().__init__() |
| 215 | self.silu = nn.SiLU() |
| 216 | self.linear = nn.Linear( |
| 217 | min(embedding_dim, 1024), |
| 218 | 4 * embedding_dim, |
| 219 | bias=True, |
| 220 | ) |
| 221 | self.norm = RMSNorm(embedding_dim, eps=norm_eps) |
| 222 | |
| 223 | def forward( |
| 224 | self, |
Callers
nothing calls this directly
Tested by
no test coverage detected