Clinicians often use imaging to evaluate both the severity and progression of disease, in many cases by assigning severity to one of several categories based on the imaging findings and seeing whether and how the classification changes on follow-up.
This approach can have its limits, though. Because diseases do not always fall cleanly into the human-constructed categories, there is inevitably variability in clinicians’ interpretations of the diseases’ severity.
To address these concerns, the Quantitative Translational Imaging in Medicine (QTIM) Lab at the MGH Martinos Center for Biomedical Imaging has developed an automatic algorithm for disease severity evaluation and change detection on a continuous spectrum. Based on the Siamese neural network, a type of deep learning architecture originally deployed in the 1990s for verification of credit card signatures, the algorithm takes two medical images as inputs and outputs a quantitative measure of difference in disease severity between the two images.
In a study reported last week in npj Digital Medicine, the QTIM researchers demonstrated the efficacy of the approach for two diseases: retinopathy of prematurity and knee osteoarthritis.
Matthew Li, a radiology resident at MGH, a member of the QTIM lab and first author of the paper, explains that the algorithm facilitates a more granular and potentially more reproducible assessment than existing standardized grading systems which rely on ordinal categories. “For example, instead of reporting that the severity of knee osteoarthritis increased from mild to moderate, our algorithm can calculate a quantitative measure of severity relative to normal and how that changes over time. We hope that, with prospective testing, such metrics can eventually be incorporated into clinical research and diagnostic workflows.” Importantly, the algorithm can quantify change in cases where there is a slight alteration in disease severity that is still within the same ordinal disease category (e.g., moderate disease that is trending toward increasing severity).
Of course, retinopathy of prematurity and knee osteoarthritis are only two of the many possible applications of the algorithm and future work will extend the Siamese network approach for other medical applications.
“Taken together, our results demonstrate that Siamese neural networks are potentially useful for evaluating the continuous spectrum of disease severity and change in medical imaging,” said Jayashree Kalpathy-Cramer, the principal investigator of QTIM lab. “This architecture could be incorporated into any algorithm developed for a workflow where the clinician might ask: ‘How bad does the disease look in this image, and has it changed?’”
In the image above: Demonstrative examples of longitudinal tracking of osteoarthritis severity using Siamese network output on knee radiographs. In each image, the top right inset number is the measure of severity relative to the baseline image and the bottom right inset number is the severity relative to a pool of random “normal” images. Observe how these numbers correlated with the KL grade for osteoarthritis (Kellgren-Lawrence classification scale from 0 to 4, where 0 is normal and 4 is most severe). These numbers can be used to measure disease severity and detect change on a continuous spectrum.