Function compute_metrics

deeplabcut/core/metrics/api.py:20–136

Computes pose estimation performance metrics. Given ground truth pose labels and predictions on a dataset, computes RMSE and pose mAP/mAR using OKS. The image paths in the ground_truth dict must be the same as the ones in the predictions dict. Single animal RMSE is computed by

(
    ground_truth: dict[str, np.ndarray],
    predictions: dict[str, np.ndarray],
    single_animal: bool = False,
    unique_bodypart_gt: dict[str, np.ndarray] | None = None,
    unique_bodypart_poses: dict[str, np.ndarray] | None = None,
    pcutoff: float = -1,
    oks_bbox_margin: int = 0,
    oks_sigma: float | np.ndarray = 0.1,
    per_keypoint_rmse: bool = False,
    compute_detection_rmse: bool = True,
)

def compute_metrics(
    ground_truth: dict[str, np.ndarray],
    predictions: dict[str, np.ndarray],
    single_animal: bool = False,
    unique_bodypart_gt: dict[str, np.ndarray] | None = None,
    unique_bodypart_poses: dict[str, np.ndarray] | None = None,
    pcutoff: float = -1,
    oks_bbox_margin: int = 0,
    oks_sigma: float | np.ndarray = 0.1,
    per_keypoint_rmse: bool = False,
    compute_detection_rmse: bool = True,
) -> dict:
    """Computes pose estimation performance metrics.

    Given ground truth pose labels and predictions on a dataset, computes RMSE and pose
    mAP/mAR using OKS.

    The image paths in the ground_truth dict must be the same as the ones in the
    predictions dict.

    Single animal RMSE is computed by simply calculating the Euclidean distance between
    each ground truth keypoint and the corresponding prediction.

    Multi-animal RMSE is computed differently: predictions are first matched to ground
    truth individuals using greedy OKS matching. OKS (or object keypoint similarity) is
    a similarity metric for keypoints (you can read more about it and its definition
    here: https://cocodataset.org/#keypoints-eval). RMSE is then computed only between
    predictions and the ground truth pose they are matched to, only when the OKS is
    greater than a small threshold. Predictions that cannot be matched to any ground
    truth with non-zero OKS are not used to compute RMSE.

    Args:
        ground_truth: The ground truth pose for which to compute metrics in the dataset.
            This should be a dictionary mapping strings (image UIDs, such as image
            paths) to ground truth pose for the image. The pose arrays should be
            in the format (num_individuals, num_bodyparts, 3), where the 3 values are
            x, y and visibility. The ``num_individuals`` corresponds to the number of
            individuals labeled in each image.
        predictions: The predicted poses for which to compute metrics in the dataset.
            This should be a dictionary mapping strings (image UIDs, such as image
            paths) to pose predictions for the image. The pose arrays should be
            in the format (num_predictions, num_bodyparts, 3), where the 3 values are
            x, y and score. The number of predictions can be different to the number of
            ground truth individuals labeled for an image.
        single_animal: Whether the metrics are being computed on a single-animal or
            multi-animal dataset. This has an impact on RMSE computation.
        unique_bodypart_gt: If unique bodyparts are defined for the dataset, they should
            be contained in this dict in the same format as the ``ground_truth`` dict.
        unique_bodypart_poses: If unique bodyparts are defined for the dataset, the
            predictions should be contained in this dict in the same format as the
            ``predictions`` dict.
        pcutoff: The threshold to compute the "rmse_cutoff" score (RMSE of all
            predictions with score above the cutoff).
        oks_bbox_margin: The margin to add around keypoints to compute the area for OKS
            computation.
        oks_sigma: The OKS sigma to use to compute pose.
        per_keypoint_rmse: Compute per-keypoint RMSE values.
        compute_detection_rmse: Computes detection RMSE (without animal assembly) if the