FineSports: A Multi-person Hierarchical Sports Video Dataset for Fine-grained Action Understanding

Created by Jinglin Xu, Guohao Zhao, Sibo Yin, Wenhao Zhou, Yuxin Peng

This repository contains the PyTorch implementation for FineSports (CVPR 2024).

[Project Page] [Paper]

Overview

<img style="border-radius: 0.3125em;
box-shadow: 0 2px 4px 0 rgba(34,36,38,.12),0 2px 10px 0 rgba(34,36,38,.08);" 
src="https://github.com/PKU-ICST-MIPL/FineSports_CVPR2024/raw/main/images/poster.png" width = "100%" alt=""/>

Requirements

Make sure the following dependencies installed (python):

• pytorch >= 1.10.1
• matplotlib=3.2.2
• einops
• timm
• tensorboardX

pip install -r requirements.txt

Dataset & Annotations

FineSports Download

To download the FineSports dataset, please sign the Release Agreement and send it to send it to Jinglin Xu (xujinglinlove@gmail.com). By sending the application, you are agreeing and acknowledging that you have read and understand the notice. We will reply with the file and the corresponding guidelines right after we receive your request!

<img style="border-radius: 0.3125em;
box-shadow: 0 2px 4px 0 rgba(34,36,38,.12),0 2px 10px 0 rgba(34,36,38,.08);" 
src="https://github.com/PKU-ICST-MIPL/FineSports_CVPR2024/raw/main/images/finesports.png" width = "100%" alt=""/>

Data Structure

$DATASET_ROOT
├── FineSports
|  ├── BallDelivered
|     ├── 00002_1
|        ├── 00001.jpg
|        ...
|        └── 00005.jpg
|     ...
|     └── 00012_1
|        ├── 00001.jpg
|        ...
|        └── 00005.jpg
|  ...
|  └── ToShootersRight
|     ├── 00095_0
|     ...
|     └── 09955_0

$ANNOTATIONS_ROOT
|  ├── FineSports-GT.pkl

Training

Training on 2*NVIDIA RTX A40. Results may slightly vary due to non-fixed random seeds

To download the pre-trained feature backbone and transformer weights, please follow CSN152, DETR, BLIP and set PRETRAIN_BACKBONE_DIR, PRETRAIN_TRANSFORMER_DIR, PRETRAIN_BLIP in configuration respectively.

Train and Validation:

python train_postal_basketball.py

Acknowledgments

Thanks for the TubeR library, which helps us to quickly implement our ideas.