Let's agree to disagree: learning highly debatable multirater labelling
Authors:
Carole H. Sudre,
Beatriz Gomez Anson,
Silvia Ingala,
Chris D. Lane,
Daniel Jimenez,
Lukas Haider,
Thomas Varsavsky,
Ryutaro Tanno,
Lorna Smith,
Sébastien Ourselin,
Rolf H. Jäger,
M. Jorge Cardoso
Abstract:
Classification and differentiation of small pathological objects may greatly vary among human raters due to differences in training, expertise and their consistency over time. In a radiological setting, objects commonly have high within-class appearance variability whilst sharing certain characteristics across different classes, making their distinction even more difficult. As an example, markers…
▽ More
Classification and differentiation of small pathological objects may greatly vary among human raters due to differences in training, expertise and their consistency over time. In a radiological setting, objects commonly have high within-class appearance variability whilst sharing certain characteristics across different classes, making their distinction even more difficult. As an example, markers of cerebral small vessel disease, such as enlarged perivascular spaces (EPVS) and lacunes, can be very varied in their appearance while exhibiting high inter-class similarity, making this task highly challenging for human raters. In this work, we investigate joint models of individual rater behaviour and multirater consensus in a deep learning setting, and apply it to a brain lesion object-detection task. Results show that jointly modelling both individual and consensus estimates leads to significant improvements in performance when compared to directly predicting consensus labels, while also allowing the characterization of human-rater consistency.
△ Less
Submitted 4 September, 2019;
originally announced September 2019.
3D multirater RCNN for multimodal multiclass detection and characterisation of extremely small objects
Authors:
Carole H. Sudre,
Beatriz Gomez Anson,
Silvia Ingala,
Chris D. Lane,
Daniel Jimenez,
Lukas Haider,
Thomas Varsavsky,
Lorna Smith,
H. Rolf Jäger,
M. Jorge Cardoso
Abstract:
Extremely small objects (ESO) have become observable on clinical routine magnetic resonance imaging acquisitions, thanks to a reduction in acquisition time at higher resolution. Despite their small size (usually $<$10 voxels per object for an image of more than $10^6$ voxels), these markers reflect tissue damage and need to be accounted for to investigate the complete phenotype of complex patholog…
▽ More
Extremely small objects (ESO) have become observable on clinical routine magnetic resonance imaging acquisitions, thanks to a reduction in acquisition time at higher resolution. Despite their small size (usually $<$10 voxels per object for an image of more than $10^6$ voxels), these markers reflect tissue damage and need to be accounted for to investigate the complete phenotype of complex pathological pathways. In addition to their very small size, variability in shape and appearance leads to high labelling variability across human raters, resulting in a very noisy gold standard. Such objects are notably present in the context of cerebral small vessel disease where enlarged perivascular spaces and lacunes, commonly observed in the ageing population, are thought to be associated with acceleration of cognitive decline and risk of dementia onset. In this work, we redesign the RCNN model to scale to 3D data, and to jointly detect and characterise these important markers of age-related neurovascular changes. We also propose training strategies enforcing the detection of extremely small objects, ensuring a tractable and stable training process.
△ Less
Submitted 21 December, 2018;
originally announced December 2018.
