Resource information

The page below details:

  • The problem,
  • Aims of this resource,
  • How to use this resource,
  • Details of the CMR studies.

The problem

Machine learning researchers Currently, automated analyses techniques are tested by direct comparison with human expert observers. However this ignores sources of human error and means that automated techniques are unable to demonstrate superiority over human techniques. Any analysis technique also needs to be generalisable to multiple institutions before transitioning to a real-world application.

Clinicians Left ventricular ejection fraction (LVEF) and mass (LVM) are key imaging biomarkers in Cardiology. They are used daily for clinical decision-making and as clinical trial outcome measures. Whilst cardiovascular magnetic resonance (CMR) imaging to measure LVEF is performed at high resolution, clinician analysis is remarkably variable. Because measurement is variable, it means clinicians have less confidence in absolute cut-offs (that can trigger surgical intervention or defibrillator implantation). When running clinical trials, it means more patients need to be recruited to detect a difference.

Aims of this resource

This dataset is a real-world, ‘precision’ resource. This is unique because it uses test-retest methodology – patients have been scanned twice in an identical fashion. Scans have also been acquired to reflect patient, disease and scanner variability. Measuring precision using this methodology permits:

  1. Determining the minimal detectable difference in left ventricular ejection fraction or mass. This tells clinicians what is the smallest change that they can be confident at detecting at follow-up for an individual patient.
  2. Determining samples sizes for clinical trials. 
  3. Generalizability of automated analysis techniques to ‘real-world’ studies.
  4. Benchmarking of techniques. Using precision, techniques may show superiority over human analysis. This would have impact on:
    1. Patients:
      1. Greater confidence in detecting abnormalities.
      2. More effective decision making for deciding therapies.
    2. Doctors:
      1. Smaller, less costly, clinical trials.
      2. Significant time-saving through automation.

How to use this resource

Download the dataset after registering. 50 test-retest scans are available after registration. The other 60 test-retest scans can be made in collaboration by contacting us. These anonymized datasets have random IDs. This makes sure scan and rescan analysis is blinded. When you upload data to analyse, paired scans will be automatically matched together. The same paired scans will be presented for comparison observers (an expert, trainee and automated techniques). Share your results to upload by contacting us, so others can compare them.

Machine learning benchmarking:

Download the initial 50 datasets, we leave it up to you whether to split into test/validation cohorts.  Analyse your scans, and enter results in the template provided (download here). Analysis can then be uploaded to check accuracy and precision compared to other observers (experts, trainees and automated techniques). If you wish to use the remaining 60 test-retest scans, please contact us via email (contact@thevolumesresource.com) .

Clinician training/benchmarking/core-lab standardisation:

Download the initial 50 datasets. These are split into three batches of n=15, 15 and 20 test-retest scans. The first batch of n=15  can be analysed prior to starting a training programme, and the second batch of n=15 after completion of training to assess for an improvement. The remaining 30 scans can be used for further training as required. Analyse your scans, and enter results in the template provided (download here). Analysis can then be uploaded to check accuracy and precision compared to other observers (experts, trainees and other automated techniques).  

Statistical methods used to perform the comparisons on-line are detailed here:

Bhuva AN et al Circ Cardiovasc Imaging. 2019;12:e009214. DOI:10.1161/CIRCIMAGING.119.009214

Details of the CMR studies

Motivation

True assessment of the precision of the measures should incorporate not only both inter and intra-operator variability, but also capture normal physiological variability and technical differences during scan acquisition. This requires analysis of repeat scans acquired in the same individual.

Inclusion criteria

  • Patients over age 18 years
  • CMR with balanced steady-state free precession pulse (SSFP) cine imaging on two occasions
  • Within a timeframe where biological change was not anticipated.

Both scans were acquired either before or after gadolinium based contrast administration, using the same protocol. Scans acquired on the same day involved removing the patient from the table and performing a repeat isocenter.

Exclusion criteria

Patients with a cardiac implantable electronic device, significant arrhythmia (atrial fibrillation or ectopy during the scan), claustrophobia or inability to breath-hold.

Institutions

Five institutions in the United Kingdom:

  • Barts Health NHS Trust
  • Leeds Teaching Hospitals NHS Trust
  • University College London Hospital NHS Trust
  • University Hospitals Birmingham NHS Trust
  • University Hospitals Bristol NHS Trust

Sequence parameters

Typical sequence parameters    Center 1 Center 2 Center 3 Center 4 Center 5
n    24  35  31  15  5
Manufacturer   Siemens Healthcare Siemens Healthcare Siemens Healthcare Phillips Medical Siemens Healthcare
Model   Avanto Aera Avanto Achieva Avanto
Strength/ Tesla   1.5 1.5 1.5 3.0 1.5
             
Slice thickness/ mm   7 8 8 10 7
Gap/ mm   3 2 0 0 3
Cardiac phases   25 25 25 30 25
             
Field of view/ cm   270 x 360 340 x 420 308 x 380 288 x 288 270 x 340
Acquisition matrix   320 x 168 208 x 153 256 x 208 153 x 158 208 x 170
Echo time/ ms   1.43 1.07 1.21 1.28 1.17
Temporal resolution/ ms   50.3 33.2 51.7 36.6 46.6
Flip angle/ degrees   80 74 78 40 67
Bandwidth/ Hz/pixel   919 925 930 2569 962

Publications

Bhuva AN et al Circ Cardiovasc Imaging. 2019;12:e009214. DOI:10.1161/CIRCIMAGING.119.009214