Applies dropout to `self`. NOTE: dropout is only applied when `Tensor.training` is `True`. - Paper: https://jmlr.org/papers/v15/srivastava14a.html ```python exec="true" source="above" session="tensor" result="python" Tensor.manual_seed(42) t = Tensor.randn(2, 2) with
(self, p=0.5)
| 1332 | # ***** functional nn ops ***** |
| 1333 | |
| 1334 | def dropout(self, p=0.5) -> Tensor: |
| 1335 | """ |
| 1336 | Applies dropout to `self`. |
| 1337 | |
| 1338 | NOTE: dropout is only applied when `Tensor.training` is `True`. |
| 1339 | |
| 1340 | - Paper: https://jmlr.org/papers/v15/srivastava14a.html |
| 1341 | |
| 1342 | ```python exec="true" source="above" session="tensor" result="python" |
| 1343 | Tensor.manual_seed(42) |
| 1344 | t = Tensor.randn(2, 2) |
| 1345 | with Tensor.train(): |
| 1346 | print(t.dropout().numpy()) |
| 1347 | ``` |
| 1348 | """ |
| 1349 | if not 0 <= p <= 1: raise ValueError(f"{p=} is out of range [0, 1]") |
| 1350 | if not Tensor.training or p == 0: return self |
| 1351 | if p == 1: return self.const_like(0) |
| 1352 | return (Tensor.rand_like(self, dtype=dtypes.default_float, contiguous=False) >= p).contiguous().where(self, 0) / (1.0 - p) |
| 1353 | |
| 1354 | def scaled_dot_product_attention(self, key:Tensor, value:Tensor, attn_mask:Tensor|None=None, dropout_p:float=0.0, |
| 1355 | is_causal:bool=False, enable_gqa:bool=False) -> Tensor: |