ECG-FM-Benchmarking

The official repository for the paper Benchmarking ECG FMs: A Reality Check Across Clinical Tasks

🎉 Our paper has been accepted at the Fourteenth International Conference on Learning Representations (ICLR 2026). OpenReview: https://openreview.net/forum?id=xXRqWpt3Xr

Here, we benchmark ECG foundation models across 12 public datasets and 26 clinically relevant tasks encompassing 1,650 regression and classification targets. We also proposed ECG-CPC, a new and outperforming ECG foundational model. We provide scripts, configurations, and checkpoints to evaluate models efficiently and reproducibly.

📂 Datasets

You can download the datasets from the following sources:

• PTB-XL
• PTB
• SPH
• EchoNext
• ZZU pECG
• CODE-15%
• Chapman
• CPSC2018, CPSC-Extra, Georgia, Ningbo (Please include Label mappings 2021.xlsx in the respective dataset folder. The original file is linked on the CinC21 challenge website. )
• MIMIC-IV-ECG (save under data/ the following files from physionet: records_w_diag_icd10.csv (MIMIC-IV-ECG-ICD), mds_ed.csv (MDS-ED), machine_measurements.csv (MIMIC-IV-ECG), omr.csv.gz (MIMIC-IV), vitalsign.csv.gz (MIMIC-IV), d_labitems.csv.gz (MIMIC-IV), labevents.csv.gz (MIMIC-IV), d_items.csv.gz (MIMIC-IV), chartevents.csv.gz (MIMIC-IV) )

📦 Checkpoints

Download pretrained checkpoints for evaluation:

• ECG-CPC
• ECGFounder
• ECG-JEPA
• ST-MEM
• MERL
• ECGFM-KED
• HuBERT-ECG
• ECG-FM

⚙️ Installation

Set up the Python environment using the provided YAML files:

# General environment
conda env create -f env.yaml

# For ECG-FM evaluation
conda env create -f ecg_fm_env.yaml

🚀 Quick Start

Follow these steps to set up and run the benchmark:

1. Data Preprocessing

First, preprocess all datasets using the provided preprocess_ecg_dataset.ipynb Jupyter notebook.

2. Configuration Setup

Before running the benchmark, configure the necessary paths:

Edit run.sh file

Open run.sh in your preferred text editor and update the following variables with your local paths:

# Set these paths according to your system
BASE_DIR="/path/to/your/fm-benchmarking"
CHECKPOINTS_DIR="/path/to/your/checkpoints"
DATASET_DIR="/path/to/your/datasets"

Update dataset path

Modify the dataset path in the fm-benchmarking/code/clinical_ts/models/conf/data/ecg_ptbxl.yaml

Run run.sh file

sbatch run.sh

Please cite our publication if you found our research to be helpful.

@inproceedings{
      al-masud2026benchmarking,
      title={Benchmarking {ECG} Foundational Models: A Reality Check Across Clinical Tasks},
      author={M A Al-Masud and Juan Lopez Alcaraz and Nils Strodthoff},
      booktitle={The Fourteenth International Conference on Learning Representations},
      year={2026},
      url={https://openreview.net/forum?id=xXRqWpt3Xr}
}