RF-DETR: Real-Time SOTA Object Detection, Instance Segmentation, and Keypoint Detection

RF-DETR is a real-time transformer architecture for object detection, instance segmentation, and keypoint detection (preview) developed by Roboflow. Built on a DINOv2 vision transformer backbone, RF-DETR delivers state-of-the-art accuracy and latency trade-offs on Microsoft COCO and RF100-VL.

RF-DETR uses a DINOv2 vision transformer backbone and supports object detection, instance segmentation, and keypoint detection (preview) in a single, consistent API. The open-source rfdetr package and Apache-designated models are released under Apache 2.0, while Plus components (rfdetr_plus, including RF-DETR-XL/2XL detection models) are licensed under PML 1.0.

https://github.com/user-attachments/assets/add23fd1-266f-4538-8809-d7dd5767e8e6

Install

To install RF-DETR, install the rfdetr package in a Python>=3.10 environment with pip.

pip install rfdetr

Install from source

By installing RF-DETR from source, you can explore the most recent features and enhancements that have not yet been officially released. Please note that these updates are still in development and may not be as stable as the latest published release.

pip install https://github.com/roboflow/rf-detr/archive/refs/heads/develop.zip

Benchmarks

RF-DETR achieves state-of-the-art results in both object detection and instance segmentation, with benchmarks reported on Microsoft COCO and RF100-VL (RF100-VL for detection only). The charts and tables below compare RF-DETR against other top real-time models across accuracy and latency for detection and segmentation. All latency numbers were measured on an NVIDIA T4 using TensorRT, FP16, and batch size 1. For full benchmarking methodology and reproducibility details, see roboflow/sab.

Detection

See object detection benchmark numbers

Segmentation

See instance segmentation benchmark numbers

Keypoints

See keypoint detection benchmark numbers

Keypoint benchmarks report AP_50:95 (OKS-based); this is the standard COCO keypoint comparison metric.

Run Models

Detection

RF-DETR provides multiple model sizes, ranging from Nano to 2XLarge. To use a different model size, replace the class name in the code snippet below with another class from the table.

```python import supervision as sv from rfdetr import RFDETRMedium from rfdetr.assets.coco_classes import COCO_CLASSES

model = RFDETRMedium()

detections = model.predict("https://media.