r"""Applies the :math:`\log(\text{softmax}(x))` function to an n-dimensional input tensor. The :math:`\text{logsoftmax}(x)` formulation can be simplified as: .. math:: \text{logsoftmax}(x_{i}) = \log(\frac{\exp(x_i) }{ \sum_j \exp(x_j)} ) For numerical stability the implementat
(inp: Tensor, axis: Union[int, Sequence[int]])
| 926 | |
| 927 | |
| 928 | def logsoftmax(inp: Tensor, axis: Union[int, Sequence[int]]) -> Tensor: |
| 929 | r"""Applies the :math:`\log(\text{softmax}(x))` function to an n-dimensional |
| 930 | input tensor. The :math:`\text{logsoftmax}(x)` formulation can be simplified as: |
| 931 | |
| 932 | .. math:: |
| 933 | \text{logsoftmax}(x_{i}) = \log(\frac{\exp(x_i) }{ \sum_j \exp(x_j)} ) |
| 934 | |
| 935 | For numerical stability the implementation follows this transformation: |
| 936 | |
| 937 | .. math:: |
| 938 | \text{logsoftmax}(x) |
| 939 | = \log (\frac{\exp (x)}{\sum_{i}(\exp (x_{i}))}) |
| 940 | = x - \log (\sum_{i}(\exp (x_{i}))) |
| 941 | = x - \text{logsumexp}(x) |
| 942 | |
| 943 | Examples: |
| 944 | >>> import numpy as np |
| 945 | >>> x = Tensor(np.arange(-5, 5, dtype=np.float32)).reshape(2,5) |
| 946 | >>> y = F.logsoftmax(x, axis=1) |
| 947 | >>> y.numpy().round(decimals=4) |
| 948 | array([[-4.4519, -3.4519, -2.4519, -1.4519, -0.4519], |
| 949 | [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519]], dtype=float32) |
| 950 | """ |
| 951 | return inp - logsumexp(inp, axis, keepdims=True) |
| 952 | |
| 953 | |
| 954 | def logsigmoid(inp: Tensor) -> Tensor: |
nothing calls this directly
no test coverage detected