The page below details:
- The problem,
- Aims of this resource,
- How to use this resource,
- Details of the CMR studies.
The problem
Machine learning researchers Currently, automated analyses techniques are tested by direct comparison with human expert observers. However this ignores sources of human error and means that automated techniques are unable to demonstrate superiority over human techniques. Any analysis technique also needs to be generalisable to multiple institutions before transitioning to a real-world application.
Clinicians Left ventricular ejection fraction (LVEF) and mass (LVM) are key imaging biomarkers in Cardiology. They are used daily for clinical decision-making and as clinical trial outcome measures. Whilst cardiovascular magnetic resonance (CMR) imaging to measure LVEF is performed at high resolution, clinician analysis is remarkably variable. Because measurement is variable, it means clinicians have less confidence in absolute cut-offs (that can trigger surgical intervention or defibrillator implantation). When running clinical trials, it means more patients need to be recruited to detect a difference.
Aims of this resource
This dataset is a real-world, ‘precision’ resource. This is unique because it uses test-retest methodology – patients have been scanned twice in an identical fashion. Scans have also been acquired to reflect patient, disease and scanner variability. Measuring precision using this methodology permits:
- Determining the minimal detectable difference in left ventricular ejection fraction or mass. This tells clinicians what is the smallest change that they can be confident at detecting at follow-up for an individual patient.
- Determining samples sizes for clinical trials.
- Generalizability of automated analysis techniques to ‘real-world’ studies.
- Benchmarking of techniques. Using precision, techniques may show superiority over human analysis. This would have impact on:
- Patients:
- Greater confidence in detecting abnormalities.
- More effective decision making for deciding therapies.
- Doctors:
- Smaller, less costly, clinical trials.
- Significant time-saving through automation.
- Patients:
How to use this resource
Download the dataset after registering. 50 test-retest scans are available after registration. The other 60 test-retest scans can be made in collaboration by contacting us. These anonymized datasets have random IDs. This makes sure scan and rescan analysis is blinded. When you upload data to analyse, paired scans will be automatically matched together. The same paired scans will be presented for comparison observers (an expert, trainee and automated techniques). Share your results to upload by contacting us, so others can compare them.
Machine learning benchmarking:
Download the initial 50 datasets, we leave it up to you whether to split into test/validation cohorts. Analyse your scans, and enter results in the template provided (download here). Analysis can then be uploaded to check accuracy and precision compared to other observers (experts, trainees and automated techniques). If you wish to use the remaining 60 test-retest scans, please contact us via email (contact@thevolumesresource.com) .
Clinician training/benchmarking/core-lab standardisation:
Download the initial 50 datasets. These are split into three batches of n=15, 15 and 20 test-retest scans. The first batch of n=15 can be analysed prior to starting a training programme, and the second batch of n=15 after completion of training to assess for an improvement. The remaining 30 scans can be used for further training as required. Analyse your scans, and enter results in the template provided (download here). Analysis can then be uploaded to check accuracy and precision compared to other observers (experts, trainees and other automated techniques).
Statistical methods used to perform the comparisons on-line are detailed here:
Bhuva AN et al Circ Cardiovasc Imaging. 2019;12:e009214. DOI:10.1161/CIRCIMAGING.119.009214
Details of the CMR studies
Motivation
True assessment of the precision of the measures should incorporate not only both inter and intra-operator variability, but also capture normal physiological variability and technical differences during scan acquisition. This requires analysis of repeat scans acquired in the same individual.
Inclusion criteria
- Patients over age 18 years
- CMR with balanced steady-state free precession pulse (SSFP) cine imaging on two occasions
- Within a timeframe where biological change was not anticipated.
Both scans were acquired either before or after gadolinium based contrast administration, using the same protocol. Scans acquired on the same day involved removing the patient from the table and performing a repeat isocenter.
Exclusion criteria
Patients with a cardiac implantable electronic device, significant arrhythmia (atrial fibrillation or ectopy during the scan), claustrophobia or inability to breath-hold.
Institutions
Five institutions in the United Kingdom:
- Barts Health NHS Trust
- Leeds Teaching Hospitals NHS Trust
- University College London Hospital NHS Trust
- University Hospitals Birmingham NHS Trust
- University Hospitals Bristol NHS Trust
Sequence parameters
Typical sequence parameters | Center 1 | Center 2 | Center 3 | Center 4 | Center 5 | |
n | 24 | 35 | 31 | 15 | 5 | |
Manufacturer | Siemens Healthcare | Siemens Healthcare | Siemens Healthcare | Phillips Medical | Siemens Healthcare | |
Model | Avanto | Aera | Avanto | Achieva | Avanto | |
Strength/ Tesla | 1.5 | 1.5 | 1.5 | 3.0 | 1.5 | |
Slice thickness/ mm | 7 | 8 | 8 | 10 | 7 | |
Gap/ mm | 3 | 2 | 0 | 0 | 3 | |
Cardiac phases | 25 | 25 | 25 | 30 | 25 | |
Field of view/ cm | 270 x 360 | 340 x 420 | 308 x 380 | 288 x 288 | 270 x 340 | |
Acquisition matrix | 320 x 168 | 208 x 153 | 256 x 208 | 153 x 158 | 208 x 170 | |
Echo time/ ms | 1.43 | 1.07 | 1.21 | 1.28 | 1.17 | |
Temporal resolution/ ms | 50.3 | 33.2 | 51.7 | 36.6 | 46.6 | |
Flip angle/ degrees | 80 | 74 | 78 | 40 | 67 | |
Bandwidth/ Hz/pixel | 919 | 925 | 930 | 2569 | 962 |
Publications
Bhuva AN et al Circ Cardiovasc Imaging. 2019;12:e009214. DOI:10.1161/CIRCIMAGING.119.009214