-
Structure-Preserving Numerical Methods for Fokker-Planck Equations
Authors:
Hanna Bartel,
Joshua Lampert,
Hendrik Ranocha
Abstract:
A common way to numerically solve Fokker-Planck equations is the Chang-Cooper method in space combined with one of the Euler methods in time. However, the explicit Euler method is only conditionally positive, leading to severe restrictions on the time step to ensure positivity. On the other hand, the implicit Euler method is robust but nonlinearly implicit. Instead, we propose to combine the Chang…
▽ More
A common way to numerically solve Fokker-Planck equations is the Chang-Cooper method in space combined with one of the Euler methods in time. However, the explicit Euler method is only conditionally positive, leading to severe restrictions on the time step to ensure positivity. On the other hand, the implicit Euler method is robust but nonlinearly implicit. Instead, we propose to combine the Chang-Cooper method with unconditionally positive Patankar-type time integration methods, since they are unconditionally positive, robust for stiff problems, only linearly implicit, and also higher-order accurate. We describe the combined approach, analyse it, and present a relevant numerical example demonstrating advantages compared to schemes proposed in the literature.
△ Less
Submitted 12 April, 2024; v1 submitted 11 April, 2024;
originally announced April 2024.
-
Structure-Preserving Numerical Methods for Two Nonlinear Systems of Dispersive Wave Equations
Authors:
Joshua Lampert,
Hendrik Ranocha
Abstract:
We use the general framework of summation by parts operators to construct conservative, entropy-stable and well-balanced semidiscretizations of two different nonlinear systems of dispersive shallow water equations with varying bathymetry: (i) a variant of the coupled Benjamin-Bona-Mahony (BBM) equations and (ii) a recently proposed model by Svärd and Kalisch (2023) with enhanced dispersive behavio…
▽ More
We use the general framework of summation by parts operators to construct conservative, entropy-stable and well-balanced semidiscretizations of two different nonlinear systems of dispersive shallow water equations with varying bathymetry: (i) a variant of the coupled Benjamin-Bona-Mahony (BBM) equations and (ii) a recently proposed model by Svärd and Kalisch (2023) with enhanced dispersive behavior. Both models share the property of being conservative in terms of a nonlinear invariant, often interpreted as entropy function. This property is preserved exactly in our novel semidiscretizations. To obtain fully-discrete entropy-stable schemes, we employ the relaxation method. We present improved numerical properties of our schemes in some test cases.
△ Less
Submitted 26 February, 2024;
originally announced February 2024.
-
MobilityDL: A Review of Deep Learning From Trajectory Data
Authors:
Anita Graser,
Anahid Jalali,
Jasmin Lampert,
Axel Weißenfeld,
Krzysztof Janowicz
Abstract:
Trajectory data combines the complexities of time series, spatial data, and (sometimes irrational) movement behavior. As data availability and computing power have increased, so has the popularity of deep learning from trajectory data. This review paper provides the first comprehensive overview of deep learning approaches for trajectory data. We have identified eight specific mobility use cases wh…
▽ More
Trajectory data combines the complexities of time series, spatial data, and (sometimes irrational) movement behavior. As data availability and computing power have increased, so has the popularity of deep learning from trajectory data. This review paper provides the first comprehensive overview of deep learning approaches for trajectory data. We have identified eight specific mobility use cases which we analyze with regards to the deep learning models and the training data used. Besides a comprehensive quantitative review of the literature since 2018, the main contribution of our work is the data-centric analysis of recent work in this field, placing it along the mobility data continuum which ranges from detailed dense trajectories of individual movers (quasi-continuous tracking data), to sparse trajectories (such as check-in data), and aggregated trajectories (crowd information).
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
Generative Large Language Models are autonomous practitioners of evidence-based medicine
Authors:
Akhil Vaid,
Joshua Lampert,
Juhee Lee,
Ashwin Sawant,
Donald Apakama,
Ankit Sakhuja,
Ali Soroush,
Denise Lee,
Isotta Landi,
Nicole Bussola,
Ismail Nabeel,
Robbie Freeman,
Patricia Kovatch,
Brendan Carr,
Benjamin Glicksberg,
Edgar Argulian,
Stamatios Lerakis,
Monica Kraft,
Alexander Charney,
Girish Nadkarni
Abstract:
Background: Evidence-based medicine (EBM) is fundamental to modern clinical practice, requiring clinicians to continually update their knowledge and apply the best clinical evidence in patient care. The practice of EBM faces challenges due to rapid advancements in medical research, leading to information overload for clinicians. The integration of artificial intelligence (AI), specifically Generat…
▽ More
Background: Evidence-based medicine (EBM) is fundamental to modern clinical practice, requiring clinicians to continually update their knowledge and apply the best clinical evidence in patient care. The practice of EBM faces challenges due to rapid advancements in medical research, leading to information overload for clinicians. The integration of artificial intelligence (AI), specifically Generative Large Language Models (LLMs), offers a promising solution towards managing this complexity.
Methods: This study involved the curation of real-world clinical cases across various specialties, converting them into .json files for analysis. LLMs, including proprietary models like ChatGPT 3.5 and 4, Gemini Pro, and open-source models like LLaMA v2 and Mixtral-8x7B, were employed. These models were equipped with tools to retrieve information from case files and make clinical decisions similar to how clinicians must operate in the real world. Model performance was evaluated based on correctness of final answer, judicious use of tools, conformity to guidelines, and resistance to hallucinations.
Results: GPT-4 was most capable of autonomous operation in a clinical setting, being generally more effective in ordering relevant investigations and conforming to clinical guidelines. Limitations were observed in terms of model ability to handle complex guidelines and diagnostic nuances. Retrieval Augmented Generation made recommendations more tailored to patients and healthcare systems.
Conclusions: LLMs can be made to function as autonomous practitioners of evidence-based medicine. Their ability to utilize tooling can be harnessed to interact with the infrastructure of a real-world healthcare system and perform the tasks of patient management in a guideline directed manner. Prompt engineering may help to further enhance this potential and transform healthcare for the clinician and the patient.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
-
Landslide Detection and Segmentation Using Remote Sensing Images and Deep Neural Network
Authors:
Cam Le,
Lam Pham,
Jasmin Lampert,
Matthias Schlögl,
Alexander Schindler
Abstract:
Knowledge about historic landslide event occurrence is important for supporting disaster risk reduction strategies. Building upon findings from 2022 Landslide4Sense Competition, we propose a deep neural network based system for landslide detection and segmentation from multisource remote sensing image input. We use a U-Net trained with Cross Entropy loss as baseline model. We then improve the U-Ne…
▽ More
Knowledge about historic landslide event occurrence is important for supporting disaster risk reduction strategies. Building upon findings from 2022 Landslide4Sense Competition, we propose a deep neural network based system for landslide detection and segmentation from multisource remote sensing image input. We use a U-Net trained with Cross Entropy loss as baseline model. We then improve the U-Net baseline model by leveraging a wide range of deep learning techniques. In particular, we conduct feature engineering by generating new band data from the original bands, which helps to enhance the quality of remote sensing image input. Regarding the network architecture, we replace traditional convolutional layers in the U-Net baseline by a residual-convolutional layer. We also propose an attention layer which leverages the multi-head attention scheme. Additionally, we generate multiple output masks with three different resolutions, which creates an ensemble of three outputs in the inference process to enhance the performance. Finally, we propose a combined loss function which leverages Focal loss and IoU loss to train the network. Our experiments on the development set of the Landslide4Sense challenge achieve an F1 score and an mIoU score of 84.07 and 76.07, respectively. Our best model setup outperforms the challenge baseline and the proposed U-Net baseline, improving the F1 score/mIoU score by 6.8/7.4 and 10.5/8.8, respectively.
△ Less
Submitted 27 December, 2023;
originally announced December 2023.
-
A Light-weight Deep Learning Model for Remote Sensing Image Classification
Authors:
Lam Pham,
Cam Le,
Dat Ngo,
Anh Nguyen,
Jasmin Lampert,
Alexander Schindler,
Ian McLoughlin
Abstract:
In this paper, we present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC), the task of identifying the aerial scene of a remote sensing image. To this end, we first valuate various benchmark convolutional neural network (CNN) architectures: MobileNet V1/V2, ResNet 50/151V2, InceptionV3/InceptionResNetV2, EfficientNet B0/B7, DenseNet 121/201, C…
▽ More
In this paper, we present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC), the task of identifying the aerial scene of a remote sensing image. To this end, we first valuate various benchmark convolutional neural network (CNN) architectures: MobileNet V1/V2, ResNet 50/151V2, InceptionV3/InceptionResNetV2, EfficientNet B0/B7, DenseNet 121/201, ConNeXt Tiny/Large. Then, the best performing models are selected to train a compact model in a teacher-student arrangement. The knowledge distillation from the teacher aims to achieve high performance with significantly reduced complexity. By conducting extensive experiments on the NWPU-RESISC45 benchmark, our proposed teacher-student models outperforms the state-of-the-art systems, and has potential to be applied on a wide rage of edge devices.
△ Less
Submitted 25 February, 2023;
originally announced February 2023.
-
HeartBEiT: Vision Transformer for Electrocardiogram Data Improves Diagnostic Performance at Low Sample Sizes
Authors:
Akhil Vaid,
Joy Jiang,
Ashwin Sawant,
Stamatios Lerakis,
Edgar Argulian,
Yuri Ahuja,
Joshua Lampert,
Alexander Charney,
Hayit Greenspan,
Benjamin Glicksberg,
Jagat Narula,
Girish Nadkarni
Abstract:
The electrocardiogram (ECG) is a ubiquitous diagnostic modality. Convolutional neural networks (CNNs) applied towards ECG analysis require large sample sizes, and transfer learning approaches result in suboptimal performance when pre-training is done on natural images. We leveraged masked image modeling to create the first vision-based transformer model, HeartBEiT, for electrocardiogram waveform a…
▽ More
The electrocardiogram (ECG) is a ubiquitous diagnostic modality. Convolutional neural networks (CNNs) applied towards ECG analysis require large sample sizes, and transfer learning approaches result in suboptimal performance when pre-training is done on natural images. We leveraged masked image modeling to create the first vision-based transformer model, HeartBEiT, for electrocardiogram waveform analysis. We pre-trained this model on 8.5 million ECGs and then compared performance vs. standard CNN architectures for diagnosis of hypertrophic cardiomyopathy, low left ventricular ejection fraction and ST elevation myocardial infarction using differing training sample sizes and independent validation datasets. We show that HeartBEiT has significantly higher performance at lower sample sizes compared to other models. Finally, we also show that HeartBEiT improves explainability of diagnosis by highlighting biologically relevant regions of the EKG vs. standard CNNs. Thus, we present the first vision-based waveform transformer that can be used to develop specialized models for ECG analysis especially at low sample sizes.
△ Less
Submitted 13 December, 2022;
originally announced December 2022.
-
Remote Sensing Image Classification using Transfer Learning and Attention Based Deep Neural Network
Authors:
Lam Pham,
Khoa Tran,
Dat Ngo,
Jasmin Lampert,
Alexander Schindler
Abstract:
The task of remote sensing image scene classification (RSISC), which aims at classifying remote sensing images into groups of semantic categories based on their contents, has taken the important role in a wide range of applications such as urban planning, natural hazards detection, environment monitoring,vegetation mapping, or geospatial object detection. During the past years, research community…
▽ More
The task of remote sensing image scene classification (RSISC), which aims at classifying remote sensing images into groups of semantic categories based on their contents, has taken the important role in a wide range of applications such as urban planning, natural hazards detection, environment monitoring,vegetation mapping, or geospatial object detection. During the past years, research community focusing on RSISC task has shown significant effort to publish diverse datasets as well as propose different approaches to deal with the RSISC challenges. Recently, almost proposed RSISC systems base on deep learning models which prove powerful and outperform traditional approaches using image processing and machine learning. In this paper, we also leverage the power of deep learning technology, evaluate a variety of deep neural network architectures, indicate main factors affecting the performance of a RSISC system. Given the comprehensive analysis, we propose a deep learning based framework for RSISC, which makes use of the transfer learning technique and multihead attention scheme. The proposed deep learning framework is evaluated on the benchmark NWPU-RESISC45 dataset and achieves the best classification accuracy of 94.7% which shows competitive to the state-of-the-art systems and potential for real-life applications.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Deep Learning Frameworks Applied For Audio-Visual Scene Classification
Authors:
Lam Pham,
Alexander Schindler,
Mina Schütz,
Jasmin Lampert,
Sven Schlarb,
Ross King
Abstract:
In this paper, we present deep learning frameworks for audio-visual scene classification (SC) and indicate how individual visual and audio features as well as their combination affect SC performance. Our extensive experiments, which are conducted on DCASE (IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events) Task 1B development dataset, achieve the best classification…
▽ More
In this paper, we present deep learning frameworks for audio-visual scene classification (SC) and indicate how individual visual and audio features as well as their combination affect SC performance. Our extensive experiments, which are conducted on DCASE (IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events) Task 1B development dataset, achieve the best classification accuracy of 82.2%, 91.1%, and 93.9% with audio input only, visual input only, and both audio-visual input, respectively. The highest classification accuracy of 93.9%, obtained from an ensemble of audio-based and visual-based frameworks, shows an improvement of 16.5% compared with DCASE baseline.
△ Less
Submitted 12 June, 2021;
originally announced June 2021.
