Gaussian Error Linear Unit. Arguments: x: Input tensor. Returns: The gaussian error linear activation: `0.5 * x * (1 + tanh(sqrt(2 / pi) * (x + 0.044715 * x^3)))` if `approximate` is `True` or `x * P(X <= x) = 0.5 * x * (1 + erf(x / sqrt(2)))`, where P(X) ~ N(0, 1
(x, approximate=False)
| 74 | |
| 75 | @keras_export('keras.activations.gelu') |
| 76 | def gelu(x, approximate=False): |
| 77 | """Gaussian Error Linear Unit. |
| 78 | |
| 79 | Arguments: |
| 80 | x: Input tensor. |
| 81 | |
| 82 | Returns: |
| 83 | The gaussian error linear activation: |
| 84 | `0.5 * x * (1 + tanh(sqrt(2 / pi) * (x + 0.044715 * x^3)))` |
| 85 | if `approximate` is `True` or |
| 86 | `x * P(X <= x) = 0.5 * x * (1 + erf(x / sqrt(2)))`, where P(X) ~ N(0, 1), |
| 87 | if `approximate` is `False`. |
| 88 | |
| 89 | Reference: |
| 90 | - [Gaussian Error Linear Units (GELUs)](https://arxiv.org/abs/1606.08415) |
| 91 | """ |
| 92 | return nn.gelu(x, approximate) |
| 93 | |
| 94 | |
| 95 | @keras_export('keras.activations.elu') |