Function cross_entropy

imperative/python/megengine/functional/loss.py:139–210

r"""Computes the multi-class cross entropy loss (using logits by default). When using label smoothing, the label distribution is as follows: .. math:: y^{LS}_{k}=y_{k}\left(1-\alpha\right)+\alpha/K where :math:`y^{LS}` and :math:`y` are new label distribution and origin label distribu

(
    pred: Tensor,
    label: Tensor,
    axis: int = 1,
    with_logits: bool = True,
    label_smooth: float = 0,
    reduction: str = "mean",
)

@_reduce_output
def cross_entropy(
    pred: Tensor,
    label: Tensor,
    axis: int = 1,
    with_logits: bool = True,
    label_smooth: float = 0,
    reduction: str = "mean",
) -> Tensor:
    r"""Computes the multi-class cross entropy loss (using logits by default).

    When using label smoothing, the label distribution is as follows:

    .. math:: y^{LS}_{k}=y_{k}\left(1-\alpha\right)+\alpha/K

    where :math:`y^{LS}` and :math:`y` are new label distribution and origin label distribution respectively.
    k is the index of label distribution. :math:`\alpha` is ``label_smooth`` and :math:`K` is the number of classes.

    Args:
        pred: input tensor representing the predicted value.
        label: input tensor representing the classification label.
        axis: an axis along which softmax will be applied. Default: 1
        with_logits: whether to apply softmax first. Default: True
        label_smooth: a label smoothing of parameter that can re-distribute target distribution. Default: 0
        reduction: the reduction to apply to the output: 'none' | 'mean' | 'sum'.

    Returns:
        loss value.

    Examples:

        By default(``with_logitis`` is True), ``pred`` is assumed to be logits,
        class probabilities are given by softmax.
        It has better numerical stability compared with sequential calls to 
        :func:`~.softmax` and :func:`~.cross_entropy`.

        >>> pred = Tensor([[0., 1.], [0.3, 0.7], [0.7, 0.3]])
        >>> label = Tensor([1., 1., 1.])
        >>> F.nn.cross_entropy(pred, label)  # doctest: +SKIP
        Tensor(0.57976407, device=xpux:0)
        >>> F.nn.cross_entropy(pred, label, reduction="none")
        Tensor([0.3133 0.513  0.913 ], device=xpux:0)

        If the ``pred`` value has been probabilities, set ``with_logits`` to False:

        >>> pred = Tensor([[0., 1.], [0.3, 0.7], [0.7, 0.3]])
        >>> label = Tensor([1., 1., 1.])
        >>> F.nn.cross_entropy(pred, label, with_logits=False)  # doctest: +SKIP
        Tensor(0.5202159, device=xpux:0)
        >>> F.nn.cross_entropy(pred, label, with_logits=False, reduction="none")
        Tensor([0.     0.3567 1.204 ], device=xpux:0)

    """
    n0 = pred.ndim
    n1 = label.ndim
    assert n0 == n1 + 1, (
        "target ndim must be one less than input ndim; input_ndim={} "
        "target_ndim={}".format(n0, n1)
    )