-
GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Texts
Authors:
Da Wu,
Jingye Yang,
Cong Liu,
Tzung-Chien Hsieh,
Elaine Marchi,
Justin Blair,
Peter Krawitz,
Chunhua Weng,
Wendy Chung,
Gholson J. Lyon,
Ian D. Krantz,
Jennifer M. Kalish,
Kai Wang
Abstract:
Individuals with suspected rare genetic disorders often undergo multiple clinical evaluations, imaging studies, laboratory tests and genetic tests, to find a possible answer over a prolonged period of time. Addressing this "diagnostic odyssey" thus has substantial clinical, psychosocial, and economic benefits. Many rare genetic diseases have distinctive facial features, which can be used by artifi…
▽ More
Individuals with suspected rare genetic disorders often undergo multiple clinical evaluations, imaging studies, laboratory tests and genetic tests, to find a possible answer over a prolonged period of time. Addressing this "diagnostic odyssey" thus has substantial clinical, psychosocial, and economic benefits. Many rare genetic diseases have distinctive facial features, which can be used by artificial intelligence algorithms to facilitate clinical diagnosis, in prioritizing candidate diseases to be further examined by lab tests or genetic assays, or in helping the phenotype-driven reinterpretation of genome/exome sequencing data. Existing methods using frontal facial photos were built on conventional Convolutional Neural Networks (CNNs), rely exclusively on facial images, and cannot capture non-facial phenotypic traits and demographic information essential for guiding accurate diagnoses. Here we introduce GestaltMML, a multimodal machine learning (MML) approach solely based on the Transformer architecture. It integrates facial images, demographic information (age, sex, ethnicity), and clinical notes (optionally, a list of Human Phenotype Ontology terms) to improve prediction accuracy. Furthermore, we also evaluated GestaltMML on a diverse range of datasets, including 528 diseases from the GestaltMatcher Database, several in-house datasets of Beckwith-Wiedemann syndrome (BWS, over-growth syndrome with distinct facial features), Sotos syndrome (overgrowth syndrome with overlapping features with BWS), NAA10-related neurodevelopmental syndrome, Cornelia de Lange syndrome (multiple malformation syndrome), and KBG syndrome (multiple malformation syndrome). Our results suggest that GestaltMML effectively incorporates multiple modalities of data, greatly narrowing candidate genetic diagnoses of rare diseases and may facilitate the reinterpretation of genome/exome sequencing data.
△ Less
Submitted 21 April, 2024; v1 submitted 23 December, 2023;
originally announced December 2023.
-
Improving Deep Facial Phenotyping for Ultra-rare Disorder Verification Using Model Ensembles
Authors:
Alexander Hustinx,
Fabio Hellmann,
Ömer Sümer,
Behnam Javanmardi,
Elisabeth André,
Peter Krawitz,
Tzung-Chien Hsieh
Abstract:
Rare genetic disorders affect more than 6% of the global population. Reaching a diagnosis is challenging because rare disorders are very diverse. Many disorders have recognizable facial features that are hints for clinicians to diagnose patients. Previous work, such as GestaltMatcher, utilized representation vectors produced by a DCNN similar to AlexNet to match patients in high-dimensional featur…
▽ More
Rare genetic disorders affect more than 6% of the global population. Reaching a diagnosis is challenging because rare disorders are very diverse. Many disorders have recognizable facial features that are hints for clinicians to diagnose patients. Previous work, such as GestaltMatcher, utilized representation vectors produced by a DCNN similar to AlexNet to match patients in high-dimensional feature space to support "unseen" ultra-rare disorders. However, the architecture and dataset used for transfer learning in GestaltMatcher have become outdated. Moreover, a way to train the model for generating better representation vectors for unseen ultra-rare disorders has not yet been studied. Because of the overall scarcity of patients with ultra-rare disorders, it is infeasible to directly train a model on them. Therefore, we first analyzed the influence of replacing GestaltMatcher DCNN with a state-of-the-art face recognition approach, iResNet with ArcFace. Additionally, we experimented with different face recognition datasets for transfer learning. Furthermore, we proposed test-time augmentation, and model ensembles that mix general face verification models and models specific for verifying disorders to improve the disorder verification accuracy of unseen ultra-rare disorders. Our proposed ensemble model achieves state-of-the-art performance on both seen and unseen disorders.
△ Less
Submitted 12 November, 2022;
originally announced November 2022.
-
Ecological collapse and the emergence of traveling waves at the onset of shear turbulence
Authors:
Hong-Yan Shih,
Tsung-Lin Hsieh,
Nigel Goldenfeld
Abstract:
The transition to turbulence exhibits remarkable spatio-temporal behavior that continues to defy detailed understanding. Near the onset to turbulence in pipes, transient turbulent regions decay either directly or, at higher Reynolds numbers through splitting, with characteristic time-scales that exhibit a super-exponential dependence on Reynolds number. Here we report numerical simulations of tran…
▽ More
The transition to turbulence exhibits remarkable spatio-temporal behavior that continues to defy detailed understanding. Near the onset to turbulence in pipes, transient turbulent regions decay either directly or, at higher Reynolds numbers through splitting, with characteristic time-scales that exhibit a super-exponential dependence on Reynolds number. Here we report numerical simulations of transitional pipe flow, showing that a zonal flow emerges at large scales, activated by anisotropic turbulent fluctuations; in turn, the zonal flow suppresses the small-scale turbulence leading to stochastic predator-prey dynamics. We show that this "ecological" model of transitional turbulence reproduces the super-exponential lifetime statistics and phenomenology of pipe flow experiments. Our work demonstrates that a fluid on the edge of turbulence is mathematically analogous to an ecosystem on the edge of extinction, and provides an unbroken link between the equations of fluid dynamics and the directed percolation universality class.
△ Less
Submitted 30 May, 2015; v1 submitted 11 May, 2015;
originally announced May 2015.
-
Multi-functional foot use during running in the zebra-tailed lizard (Callisaurus draconoides)
Authors:
Chen Li,
S. Tonia Hsieh,
Daniel I. Goldman
Abstract:
A diversity of animals that run on solid, level, flat, non-slip surfaces appear to bounce on their legs; elastic elements in the limbs can store and return energy during each step. The mechanics and energetics of running in natural terrain, particularly on surfaces that can yield and flow under stress, is less understood. The zebra-tailed lizard (Callisaurus draconoides), a small desert generalist…
▽ More
A diversity of animals that run on solid, level, flat, non-slip surfaces appear to bounce on their legs; elastic elements in the limbs can store and return energy during each step. The mechanics and energetics of running in natural terrain, particularly on surfaces that can yield and flow under stress, is less understood. The zebra-tailed lizard (Callisaurus draconoides), a small desert generalist with a large, elongate, tendinous hind foot, runs rapidly across a variety of natural substrates. We use high speed video to obtain detailed three-dimensional running kinematics on solid and granular surfaces to reveal how leg, foot, and substrate mechanics contribute to its high locomotor performance. Running at ~10 body length/s (~1 m/s), the center of mass oscillates like a spring-mass system on both substrates, with only 15% reduction in stride length on the granular surface. On the solid surface, a strut-spring model of the hind limb reveals that the hind foot saves about 40% of the mechanical work needed per step, significant for the lizard's small size. On the granular surface, a penetration force model and hypothesized subsurface foot rotation indicates that the hind foot paddles through fluidized granular medium, and that the energy lost per step during irreversible deformation of the substrate does not differ from the reduction in the mechanical energy of the center of mass. The upper hind leg muscles must perform three times as much mechanical work on the granular surface as on the solid surface to compensate for the greater energy lost within the foot and to the substrate.
△ Less
Submitted 29 March, 2013;
originally announced March 2013.
