Source code related to the research paper entitled RVENet: A Large Echocardiographic Dataset for the Deep Learning-Based Assessment of Right Ventricular Function
The purpose of this repository is to enable the application of our deep learning pipeline that was developed for estimating right ventricular ejection fraction (RVEF) from 2D apical 4-chamber view echocardiographic videos. For detailed information about our model and the entire data analysis pipeline, please refer to the following papers:
RVENet: A Large Echocardiographic Dataset for the Deep Learning-Based Assessment of Right Ventricular Function
Bálint Magyar*, Márton Tokodi*, András Soós, Máté Tolvaj, Bálint K. Lakatos, Alexandra Fábián, Elena Surkova, Béla Merkely, Attila Kovács and András Horváth
ECCV (2022)
Deep Learning-Based Prediction of Right Ventricular Ejection Fraction Using 2D Echocardiograms
Márton Tokodi*, Bálint Magyar*, András Soós, Masaaki Takeuchi, Máté Tolvaj, Bálint K. Lakatos, Tetsuji Kitano, Yosuke Nabeshima, Alexandra Fábián, Mark B. Szigeti, András Horváth, Béla Merkely, and Attila Kovács
JACC: Cardiovascular Imaging (2023)
Two-dimensional echocardiography is the most frequently performed imaging test to assess RV function. However, conventional 2D parameters are unable to reliably capture RV dysfunction across the entire spectrum of cardiac diseases. Three-dimensional echocardiography-derived RVEF – a sensitive and reproducible parameter that has been validated against cardiac magnetic resonance imaging – can bypass most of their limitations. Nonetheless, 3D echocardiography has limited availability, is more time-consuming, and requires significant human expertise. Therefore, novel automated tools that utilize readily available and routinely acquired 2D echocardiographic recordings to predict RVEF and detect RV dysfunction reliably would be highly desirable.
Our deep learning pipeline was designed to analyze a DICOM file containing a 2D apical 4-chamber view echocardiographic video to predict 3D echocardiography-derived RVEF as either a continuous variable or a class. Following a multi-step preprocessing, the preprocessed frames of the video and a binary mask are passed to a deep learning model that consists of three key components:
- a feature extractor (different off-the-shelf backbones such as ResNext or ShuffleNet) that derives unique features from each frame,
- a temporal convolutional layer that combines the extracted features to evaluate temporal changes,
- and a regression head or a classification head (comprising two fully connected layers), which returns a single predicted RVEF value or a class (e.g. normal or reduced RVEF).
Mean absolute error or binary cross-entropy was used as the loss function (depending on the task). As the model was trained to analyze one cardiac cycle at a time, the average of the per-cardiac cycle predictions can be calculated to get a single predicted RVEF value for a given video.
A large single-center dataset – comprising 3,583 echocardiographic videos of 831 subjects (in DICOM format) and the corresponding labels (e.g., 3D echocardiography-derived RVEF) – was used for the training and internal validation of the models provided in this repository. To encourage collaborations and promote further innovation, we made this dataset publicly available (RVENet dataset – https://rvenet.github.io/dataset/).
RVENet- the RVENet codebase containing training and evaluation scriptsRVENet_dataset_preprocessing- scripts required for preprocessing the RVENet datasetrun_training.py- the single script that is required to start a trainingrequirements.txt- the list of the required Python packagessetup.py- installation scriptLICENSE.md- details of the license (MIT License)README.md- text file that introduces the project and explains the content of this repository
Run the follwoing script to install all requirements and the RVENet python package:
pip install .
The RVENet dataset can be preprocessed by following the steps below
- Download the dataset (further information at https://rvenet.github.io/dataset/)
- Decompress the downloaded zip files (both training and validation) into a single folder (use the 7z package under Ubuntu)
- Run the
RVENet_dataset_preprocessing/process_dicoms.pyscript with the corresponding parameters to generate images from the dicom files - Run the
RVENet_dataset_preprocessing/create_annotation.pyscript with the corresponding parameters to generate the training and validation json files
Models can be trained by following the steps below
- Edit the parameters of the RVEnet\parameters_classification_baseline.json or the RVEnet\parameters_regression_baseline.json files depending on you target task (regression or classificaiton). The single parameter that must be changed is the DICOM_database_path (path to the folder containing preprocessed frames and annotation json files),
- Run the the run_training.py script
python run_training.py parameters_regression_baseline.json
For inquiries related to the content of this repository, contact Bálint Magyar (magyar.balint[at]itk.ppke.hu)