-
Towards the Visualization of Aggregated Class Activation Maps to Analyse the Global Contribution of Class Features
Authors:
Igor Cherepanov,
David Sessler,
Alex Ulmer,
Hendrik Lücke-Tieke,
Jörn Kohlhammer
Abstract:
Deep learning (DL) models achieve remarkable performance in classification tasks. However, models with high complexity can not be used in many risk-sensitive applications unless a comprehensible explanation is presented. Explainable artificial intelligence (xAI) focuses on the research to explain the decision-making of AI systems like DL. We extend a recent method of Class Activation Maps (CAMs) w…
▽ More
Deep learning (DL) models achieve remarkable performance in classification tasks. However, models with high complexity can not be used in many risk-sensitive applications unless a comprehensible explanation is presented. Explainable artificial intelligence (xAI) focuses on the research to explain the decision-making of AI systems like DL. We extend a recent method of Class Activation Maps (CAMs) which visualizes the importance of each feature of a data sample contributing to the classification. In this paper, we aggregate CAMs from multiple samples to show a global explanation of the classification for semantically structured data. The aggregation allows the analyst to make sophisticated assumptions and analyze them with further drill-down visualizations. Our visual representation for the global CAM illustrates the impact of each feature with a square glyph containing two indicators. The color of the square indicates the classification impact of this feature. The size of the filled square describes the variability of the impact between single samples. For interesting features that require further analysis, a detailed view is necessary that provides the distribution of these values. We propose an interactive histogram to filter samples and refine the CAM to show relevant samples only. Our approach allows an analyst to detect important features of high-dimensional data and derive adjustments to the AI model based on our global explanation visualization.
△ Less
Submitted 29 July, 2023;
originally announced August 2023.
-
Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures
Authors:
Igor Cherepanov,
Alex Ulmer,
Jonathan Geraldi Joewono,
Jörn Kohlhammer
Abstract:
The classification of internet traffic has become increasingly important due to the rapid growth of today's networks and applications. The number of connections and the addition of new applications in our networks causes a vast amount of log data and complicates the search for common patterns by experts. Finding such patterns among specific classes of applications is necessary to fulfill various r…
▽ More
The classification of internet traffic has become increasingly important due to the rapid growth of today's networks and applications. The number of connections and the addition of new applications in our networks causes a vast amount of log data and complicates the search for common patterns by experts. Finding such patterns among specific classes of applications is necessary to fulfill various requirements in network analytics. Deep learning methods provide both feature extraction and classification from data in a single system. However, these networks are very complex and are used as black-box models, which weakens the experts' trust in the classifications. Moreover, by using them as a black-box, new knowledge cannot be obtained from the model predictions despite their excellent performance. Therefore, the explainability of the classifications is crucial. Besides increasing trust, the explanation can be used for model evaluation gaining new insights from the data and improving the model. In this paper, we present a visual interactive tool that combines the classification of network data with an explanation technique to form an interface between experts, algorithms, and data.
△ Less
Submitted 22 November, 2022; v1 submitted 5 September, 2022;
originally announced September 2022.
-
Three-Dimensional Coherent Diffractive Imaging of Isolated Faceted Nanostructures
Authors:
Alessandro Colombo,
Simon Dold,
Patrice Kolb,
Nils Bernhardt,
Patrick Behrens,
Jonathan Correa,
Stefan Düsterer,
Benjamin Erk,
Linos Hecht,
Andrea Heilrath,
Robert Irsig,
Norman Iwe,
Jakob Jordan,
Björn Kruse,
Bruno Langbehn,
Bastian Manschwetus,
Franklin Martinez,
Karl-Heinz Meiwes-Broer,
Kevin Oldenburg,
Christopher Passow,
Christian Peltz,
Mario Sauppe,
Fabian Seel,
Rico Mayro P. Tanyag,
Rolf Treusch
, et al. (7 additional authors not shown)
Abstract:
The structure and dynamics of isolated nanosamples in free flight can be directly visualized via single-shot coherent diffractive imaging using the intense and short pulses of X-ray free-electron lasers. Wide-angle scattering images even encode three-dimensional morphological information of the samples, but the retrieval of this information remains a challenge. Up to now, effective three-dimension…
▽ More
The structure and dynamics of isolated nanosamples in free flight can be directly visualized via single-shot coherent diffractive imaging using the intense and short pulses of X-ray free-electron lasers. Wide-angle scattering images even encode three-dimensional morphological information of the samples, but the retrieval of this information remains a challenge. Up to now, effective three-dimensional morphology reconstructions from single shots were only achieved via fitting with highly constrained models, requiring a priori knowledge about possible geometrical shapes. Here we present a much more generic imaging approach. Relying on a model that allows for any sample morphology described by a convex polyhedron, we reconstruct wide-angle diffraction patterns from individual silver nanoparticles. In addition to known structural motives with high symmetries, we retrieve imperfect shapes and agglomerates which were not accessible previously. Our results open new routes towards true 3D structure determination of single nanoparticles and, ultimately, 3D movies of ultrafast nanoscale dynamics.
△ Less
Submitted 8 August, 2022;
originally announced August 2022.
-
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
Authors:
Jennifer J. Sun,
Markus Marks,
Andrew Ulmer,
Dipam Chakraborty,
Brian Geuther,
Edward Hayes,
Heng Jia,
Vivek Kumar,
Sebastian Oleszko,
Zachary Partridge,
Milan Peelman,
Alice Robie,
Catherine E. Schretter,
Keith Sheppard,
Chao Sun,
Param Uttarwar,
Julian M. Wagner,
Eric Werner,
Joseph Parker,
Pietro Perona,
Yisong Yue,
Kristin Branson,
Ann Kennedy
Abstract:
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of…
▽ More
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of interacting flies (4.4 million frames of pose tracking data). Accompanying these data, we introduce a panel of real-life downstream analysis tasks to assess the quality of learned representations by evaluating how well they preserve information about the experimental conditions (e.g. strain, time of day, optogenetic stimulation) and animal behavior. We test multiple state-of-the-art self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark, revealing that methods developed using human action datasets do not fully translate to animal datasets. We hope that our benchmark and dataset encourage a broader exploration of behavior representation learning methods across species and settings.
△ Less
Submitted 30 June, 2023; v1 submitted 21 July, 2022;
originally announced July 2022.
-
Erdős-Pósa from ball packing
Authors:
Wouter Cames van Batenburg,
Gwenaël Joret,
Arthur Ulmer
Abstract:
A classic theorem of Erdős and Pósa (1965) states that every graph has either $k$ vertex-disjoint cycles or a set of $O(k \log k)$ vertices meeting all its cycles. While the standard proof revolves around finding a large `frame' in the graph (a subdivision of a large cubic graph), an alternative way of proving this theorem is to use a ball packing argument of Kühn and Osthus (2003) and Diestel and…
▽ More
A classic theorem of Erdős and Pósa (1965) states that every graph has either $k$ vertex-disjoint cycles or a set of $O(k \log k)$ vertices meeting all its cycles. While the standard proof revolves around finding a large `frame' in the graph (a subdivision of a large cubic graph), an alternative way of proving this theorem is to use a ball packing argument of Kühn and Osthus (2003) and Diestel and Rempel (2005). In this paper, we argue that the latter approach is particularly well suited for studying edge variants of the Erdős-Pósa theorem.
As an illustration, we give a short proof of a theorem of Bruhn, Heinlein, and Joos (2019), that cycles of length at least $\ell$ have the so-called edge-Erdős-Pósa property. More precisely, we show that every graph $G$ either contains $k$ edge-disjoint cycles of length at least $\ell$ or an edge set $F$ of size $O(k\ell \cdot \log (k\ell))$ such that $G-F$ has no cycle of length at least $\ell$. For fixed $\ell$, this improves on the previously best known bound of $O(k^2 \log k +k\ell)$.
△ Less
Submitted 8 June, 2020; v1 submitted 17 December, 2019;
originally announced December 2019.
-
Cloud Resource Optimization for Processing Multiple Streams of Visual Data
Authors:
Zohar Kapach,
Andrew Ulmer,
Daniel Merrick,
Arshad Alikhan,
Yung-Hsiang Lu,
Anup Mohan,
Ahmed S. Kaseb,
George K. Thiruvathukal
Abstract:
Hundreds of millions of network cameras have been installed throughout the world. Each is capable of providing a vast amount of real-time data. Analyzing the massive data generated by these cameras requires significant computational resources and the demands may vary over time. Cloud computing shows the most promise to provide the needed resources on demand. In this article, we investigate how to…
▽ More
Hundreds of millions of network cameras have been installed throughout the world. Each is capable of providing a vast amount of real-time data. Analyzing the massive data generated by these cameras requires significant computational resources and the demands may vary over time. Cloud computing shows the most promise to provide the needed resources on demand. In this article, we investigate how to allocate cloud resources when analyzing real-time data streams from network cameras. A resource manager considers many factors that affect its decisions, including the types of analysis, the number of data streams, and the locations of the cameras. The manager then selects the most cost-efficient types of cloud instances (e.g. CPU vs. GPGPU) to meet the computational demands for analyzing streams. We evaluate the effectiveness of our approach using Amazon Web Services. Experiments demonstrate more than 50% cost reduction for real workloads.
△ Less
Submitted 18 January, 2019;
originally announced January 2019.
