r"""Applies the element-wise function: .. math:: \text{softplus}(x) = \log(1 + \exp(x)) softplus is a smooth approximation to the ReLU function and can be used to constrain the output to be always positive. For numerical stability the implementation follows this transf
(inp: Tensor)
| 901 | |
| 902 | |
| 903 | def softplus(inp: Tensor) -> Tensor: |
| 904 | r"""Applies the element-wise function: |
| 905 | |
| 906 | .. math:: |
| 907 | \text{softplus}(x) = \log(1 + \exp(x)) |
| 908 | |
| 909 | softplus is a smooth approximation to the ReLU function and can be used |
| 910 | to constrain the output to be always positive. |
| 911 | For numerical stability the implementation follows this transformation: |
| 912 | |
| 913 | .. math:: |
| 914 | \text{softplus}(x) = \log(1 + \exp(x)) |
| 915 | = \log(1 + \exp(-\text{abs}(x))) + \max(x, 0) |
| 916 | = \log1p(\exp(-\text{abs}(x))) + \text{relu}(x) |
| 917 | |
| 918 | Examples: |
| 919 | >>> import numpy as np |
| 920 | >>> x = Tensor(np.arange(-3, 3, dtype=np.float32)) |
| 921 | >>> y = F.softplus(x) |
| 922 | >>> y.numpy().round(decimals=4) |
| 923 | array([0.0486, 0.1269, 0.3133, 0.6931, 1.3133, 2.1269], dtype=float32) |
| 924 | """ |
| 925 | return _elwise(inp, mode=Elemwise.Mode.SOFTPLUS) |
| 926 | |
| 927 | |
| 928 | def logsoftmax(inp: Tensor, axis: Union[int, Sequence[int]]) -> Tensor: |