Showing 1–26 of 26 results for author: Schmitt, J

Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression

Authors: Jonas Schmitt, Ruiping Liu, Junwei Zheng, Jiaming Zhang, Rainer Stiefelhagen

Abstract: Lightweight and effective models are essential for devices with limited resources, such as intelligent vehicles. Structured pruning offers a promising approach to model compression and efficiency enhancement. However, existing methods often tie pruning techniques to specific model architectures or vision tasks. To address this limitation, we propose a novel unified pruning framework Comb, Prune, D… ▽ More Lightweight and effective models are essential for devices with limited resources, such as intelligent vehicles. Structured pruning offers a promising approach to model compression and efficiency enhancement. However, existing methods often tie pruning techniques to specific model architectures or vision tasks. To address this limitation, we propose a novel unified pruning framework Comb, Prune, Distill (CPD), which addresses both model-agnostic and task-agnostic concerns simultaneously. Our framework employs a combing step to resolve hierarchical layer-wise dependency issues, enabling architecture independence. Additionally, the pruning pipeline adaptively remove parameters based on the importance scoring metrics regardless of vision tasks. To support the model in retaining its learned information, we introduce knowledge distillation during the pruning step. Extensive experiments demonstrate the generalizability of our framework, encompassing both convolutional neural network (CNN) and transformer models, as well as image classification and segmentation tasks. In image classification we achieve a speedup of up to x4.3 with a accuracy loss of 1.8% and in semantic segmentation up to x1.89 with a 5.1% loss in mIoU. △ Less

Submitted 6 August, 2024; originally announced August 2024.

Comments: Accepted by ITSC 2024. Code is publicly available at: https://github.com/Cranken/CPD

Extending Network Calculus To Deal With Partially Negative And Decreasing Service Curves

Authors: Anja Hamscher, Vlad-Cristian Constantin, Jens B. Schmitt

Abstract: Network Calculus (NC) is a versatile analytical methodology to efficiently compute performance bounds in networked systems. The arrival and service curve abstractions allow to model diverse and heterogeneous distributed systems. The operations to compute residual service curves and to concatenate sequences of systems enable an efficient and accurate calculation of per-flow timing guarantees. Yet,… ▽ More Network Calculus (NC) is a versatile analytical methodology to efficiently compute performance bounds in networked systems. The arrival and service curve abstractions allow to model diverse and heterogeneous distributed systems. The operations to compute residual service curves and to concatenate sequences of systems enable an efficient and accurate calculation of per-flow timing guarantees. Yet, in some scenarios involving multiple concurrent flows at a system, the central notion of so-called min-plus service curves is too weak to still be able to compute a meaningful residual service curve. In these cases, one usually resorts to so-called strict service curves that enable the computation of per-flow bounds. However, strict service curves are restrictive: (1) there are service elements for which only min-plus service curves can be provided but not strict ones and (2) strict service curves generally have no concatenation property, i.e., a sequence of two strict systems does not yield a strict service curve. In this report, we extend NC to deal with systems only offering aggregate min-plus service curves to multiple flows. The key to this extension is the exploitation of minimal arrival curves, i.e., lower bounds on the arrival process. Technically speaking, we provide basic performance bounds (backlog and delay) for the case of negative service curves. We also discuss their accuracy and show them to be tight. In order to illustrate their usefulness we also present patterns of application of these new results for: (1) heterogeneous systems involving computation and communication resources and (2) finite buffers that are shared between multiple flows. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: To be published in part in RTAS 2024

SPINEPS -- Automatic Whole Spine Segmentation of T2-weighted MR images using a Two-Phase Approach to Multi-class Semantic and Instance Segmentation

Authors: Hendrik Möller, Robert Graf, Joachim Schmitt, Benjamin Keinert, Matan Atad, Anjany Sekuboyina, Felix Streckenbach, Hanna Schön, Florian Kofler, Thomas Kroencke, Stefanie Bette, Stefan Willich, Thomas Keil, Thoralf Niendorf, Tobias Pischon, Beate Endemann, Bjoern Menze, Daniel Rueckert, Jan S. Kirschke

Abstract: Purpose. To present SPINEPS, an open-source deep learning approach for semantic and instance segmentation of 14 spinal structures (ten vertebra substructures, intervertebral discs, spinal cord, spinal canal, and sacrum) in whole body T2w MRI. Methods. During this HIPPA-compliant, retrospective study, we utilized the public SPIDER dataset (218 subjects, 63% female) and a subset of the German Nati… ▽ More Purpose. To present SPINEPS, an open-source deep learning approach for semantic and instance segmentation of 14 spinal structures (ten vertebra substructures, intervertebral discs, spinal cord, spinal canal, and sacrum) in whole body T2w MRI. Methods. During this HIPPA-compliant, retrospective study, we utilized the public SPIDER dataset (218 subjects, 63% female) and a subset of the German National Cohort (1423 subjects, mean age 53, 49% female) for training and evaluation. We combined CT and T2w segmentations to train models that segment 14 spinal structures in T2w sagittal scans both semantically and instance-wise. Performance evaluation metrics included Dice similarity coefficient, average symmetrical surface distance, panoptic quality, segmentation quality, and recognition quality. Statistical significance was assessed using the Wilcoxon signed-rank test. An in-house dataset was used to qualitatively evaluate out-of-distribution samples. Results. On the public dataset, our approach outperformed the baseline (instance-wise vertebra dice score 0.929 vs. 0.907, p-value<0.001). Training on auto-generated annotations and evaluating on manually corrected test data from the GNC yielded global dice scores of 0.900 for vertebrae, 0.960 for intervertebral discs, and 0.947 for the spinal canal. Incorporating the SPIDER dataset during training increased these scores to 0.920, 0.967, 0.958, respectively. Conclusions. The proposed segmentation approach offers robust segmentation of 14 spinal structures in T2w sagittal images, including the spinal cord, spinal canal, intervertebral discs, endplate, sacrum, and vertebrae. The approach yields both a semantic and instance mask as output, thus being easy to utilize. This marks the first publicly available algorithm for whole spine segmentation in sagittal T2w MR imaging. △ Less

Submitted 22 April, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

Comments: https://github.com/Hendrik-code/spineps

Automating the Design of Multigrid Methods with Evolutionary Program Synthesis

Authors: Jonas Schmitt

Abstract: Many of the most fundamental laws of nature can be formulated as partial differential equations (PDEs). Understanding these equations is, therefore, of exceptional importance for many branches of modern science and engineering. However, since the general solution of many PDEs is unknown, the efficient approximate solution of these equations is one of humanity's greatest challenges. While multigrid… ▽ More Many of the most fundamental laws of nature can be formulated as partial differential equations (PDEs). Understanding these equations is, therefore, of exceptional importance for many branches of modern science and engineering. However, since the general solution of many PDEs is unknown, the efficient approximate solution of these equations is one of humanity's greatest challenges. While multigrid represents one of the most effective methods for solving PDEs numerically, in many cases, the design of an efficient or at least working multigrid solver is an open problem. This thesis demonstrates that grammar-guided genetic programming, an evolutionary program synthesis technique, can discover multigrid methods of unprecedented structure that achieve a high degree of efficiency and generalization. For this purpose, we develop a novel context-free grammar that enables the automated generation of multigrid methods in a symbolically-manipulable formal language, based on which we can apply the same multigrid-based solver to problems of different sizes without having to adapt its internal structure. Treating the automated design of an efficient multigrid method as a program synthesis task allows us to find novel sequences of multigrid operations, including the combination of different smoothing and coarse-grid correction steps on each level of the discretization hierarchy. To prove the feasibility of this approach, we present its implementation in the form of the Python framework EvoStencils, which is freely available as open-source software. This implementation comprises all steps from representing the algorithmic sequence of a multigrid method in the form of a directed acyclic graph of Python objects to its automatic generation and optimization using the capabilities of the code generation framework ExaStencils and the evolutionary computation library DEAP. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Denoising diffusion-based MRI to CT image translation enables automated spinal segmentation

Authors: Robert Graf, Joachim Schmitt, Sarah Schlaeger, Hendrik Kristian Möller, Vasiliki Sideri-Lampretsa, Anjany Sekuboyina, Sandro Manuel Krieg, Benedikt Wiestler, Bjoern Menze, Daniel Rueckert, Jan Stefan Kirschke

Abstract: Background: Automated segmentation of spinal MR images plays a vital role both scientifically and clinically. However, accurately delineating posterior spine structures presents challenges. Methods: This retrospective study, approved by the ethical committee, involved translating T1w and T2w MR image series into CT images in a total of n=263 pairs of CT/MR series. Landmark-based registration was… ▽ More Background: Automated segmentation of spinal MR images plays a vital role both scientifically and clinically. However, accurately delineating posterior spine structures presents challenges. Methods: This retrospective study, approved by the ethical committee, involved translating T1w and T2w MR image series into CT images in a total of n=263 pairs of CT/MR series. Landmark-based registration was performed to align image pairs. We compared 2D paired (Pix2Pix, denoising diffusion implicit models (DDIM) image mode, DDIM noise mode) and unpaired (contrastive unpaired translation, SynDiff) image-to-image translation using "peak signal to noise ratio" (PSNR) as quality measure. A publicly available segmentation network segmented the synthesized CT datasets, and Dice scores were evaluated on in-house test sets and the "MRSpineSeg Challenge" volumes. The 2D findings were extended to 3D Pix2Pix and DDIM. Results: 2D paired methods and SynDiff exhibited similar translation performance and Dice scores on paired data. DDIM image mode achieved the highest image quality. SynDiff, Pix2Pix, and DDIM image mode demonstrated similar Dice scores (0.77). For craniocaudal axis rotations, at least two landmarks per vertebra were required for registration. The 3D translation outperformed the 2D approach, resulting in improved Dice scores (0.80) and anatomically accurate segmentations in a higher resolution than the original MR image. Conclusion: Two landmarks per vertebra registration enabled paired image-to-image translation from MR to CT and outperformed all unpaired approaches. The 3D techniques provided anatomically correct segmentations, avoiding underprediction of small structures like the spinous process. △ Less

Submitted 14 November, 2023; v1 submitted 18 August, 2023; originally announced August 2023.

Comments: 35 pages, 7 figures, Code and a model weights available https://doi.org/10.5281/zenodo.8221159 and https://doi.org/10.5281/zenodo.8198697

MSC Class: 68T99 68U10 ACM Class: I.2.1

Journal ref: Eur Radiol Exp 7, 70 (2023)

Evolving Generalizable Multigrid-Based Helmholtz Preconditioners with Grammar-Guided Genetic Programming

Authors: Jonas Schmitt, Harald Köstler

Abstract: Solving the indefinite Helmholtz equation is not only crucial for the understanding of many physical phenomena but also represents an outstandingly-difficult benchmark problem for the successful application of numerical methods. Here we introduce a new approach for evolving efficient preconditioned iterative solvers for Helmholtz problems with multi-objective grammar-guided genetic programming. Ou… ▽ More Solving the indefinite Helmholtz equation is not only crucial for the understanding of many physical phenomena but also represents an outstandingly-difficult benchmark problem for the successful application of numerical methods. Here we introduce a new approach for evolving efficient preconditioned iterative solvers for Helmholtz problems with multi-objective grammar-guided genetic programming. Our approach is based on a novel context-free grammar, which enables the construction of multigrid preconditioners that employ a tailored sequence of operations on each discretization level. To find solvers that generalize well over the given domain, we propose a custom method of successive problem difficulty adaption, in which we evaluate a preconditioner's efficiency on increasingly ill-conditioned problem instances. We demonstrate our approach's effectiveness by evolving multigrid-based preconditioners for a two-dimensional indefinite Helmholtz problem that outperform several human-designed methods for different wavenumbers up to systems of linear equations with more than a million unknowns. △ Less

Submitted 28 April, 2022; v1 submitted 27 April, 2022; originally announced April 2022.

Journal ref: Proceedings of the 2022 Genetic and Evolutionary Computation Conference (Boston, USA) (GECCO '22)

Improving Performance Bounds for Weighted Round-Robin Schedulers under Constrained Cross-Traffic

Authors: Vlad-Cristian Constantin, Paul Nikolaus, Jens Schmitt

Abstract: Weighted round robin (WRR) is an effective, yet particularly easy-to-implement packet scheduler. A slight modification in the implementation of WRR, interleaved weighted round robin, has been proposed as an enhancement of the initial version and has been recently investigated. Network calculus is a versatile framework to model and analyze such network schedulers. By means of this, one can derive t… ▽ More Weighted round robin (WRR) is an effective, yet particularly easy-to-implement packet scheduler. A slight modification in the implementation of WRR, interleaved weighted round robin, has been proposed as an enhancement of the initial version and has been recently investigated. Network calculus is a versatile framework to model and analyze such network schedulers. By means of this, one can derive theoretical upper bounds on network performance metrics, such as delay or backlog. In our previous work, we derive performance bounds by showing that both round-robin variants belong to a class called bandwidth-sharing policy; however, the proofs are incomplete and thus, we cannot conclude that the round-robin schedulers are bandwidth-sharing policies (under variable packet sizes).To that end, in the subsequent erratum, we introduce so-called resource-segregating policies and show the round-robin schedulers to be members of this class. We first present our original work, as published in [CNS22-1], and then the erratum correcting the previously mentioned shortcoming. In our erratum, we provide slightly worse delay bounds compared to [CNS22-1]; yet, across all our experiments, they significantly outperform the state of the art. △ Less

Submitted 12 December, 2022; v1 submitted 16 February, 2022; originally announced February 2022.

Comments: Published at IFIP Networking 2022

Parameterized (Modular) Counting and Cayley Graph Expanders

Authors: Norbert Peyerimhoff, Marc Roth, Johannes Schmitt, Jakob Stix, Alina Vdovina

Abstract: We study the problem $\#\mathrm{EdgeSub}(Φ)$ of counting $k$-edge subgraphs satisfying a given graph property $Φ$ in a large host graph $G$. Building upon the breakthrough result of Curticapean, Dell and Marx (STOC 17), we express the number of such subgraphs as a finite linear combination of graph homomorphism counts and derive the complexity of computing this number by studying its coefficients.… ▽ More We study the problem $\#\mathrm{EdgeSub}(Φ)$ of counting $k$-edge subgraphs satisfying a given graph property $Φ$ in a large host graph $G$. Building upon the breakthrough result of Curticapean, Dell and Marx (STOC 17), we express the number of such subgraphs as a finite linear combination of graph homomorphism counts and derive the complexity of computing this number by studying its coefficients. Our approach relies on novel constructions of low-degree Cayley graph expanders of $p$-groups, which might be of independent interest. The properties of those expanders allow us to analyse the coefficients in the aforementioned linear combinations over the field $\mathbb{F}_p$ which gives us significantly more control over the cancellation behaviour of the coefficients. Our main result is an exhaustive and fine-grained complexity classification of $\#\mathrm{EdgeSub}(Φ)$ for minor-closed properties $Φ$, closing the missing gap in previous work by Roth, Schmitt and Wellnitz (ICALP 21). Additionally, we observe that our methods also apply to modular counting. Among others, we investigate the problems of modular counting of paths, cycles, forests and matroid bases. In the course of our investigations we also provide an exhaustive parameterized complexity classification for the problem of counting graph homomorphisms modulo a prime $p$. △ Less

Submitted 29 April, 2021; originally announced April 2021.

Comments: 49 pages, 4 figures

Report number: MPIM-Bonn-2021

Unleashing the Power of Paying Multiplexing Only Once in Stochastic Network Calculus

Authors: Anne Bouillard, Paul Nikolaus, Jens Schmitt

Abstract: The stochastic network calculus (SNC) holds promise as a versatile and uniform framework to calculate probabilistic performance bounds in networks of queues. A great challenge to accurate bounds and efficient calculations are stochastic dependencies between flows due to resource sharing inside the network. However, by carefully utilizing the basic SNC concepts in the network analysis the necessity… ▽ More The stochastic network calculus (SNC) holds promise as a versatile and uniform framework to calculate probabilistic performance bounds in networks of queues. A great challenge to accurate bounds and efficient calculations are stochastic dependencies between flows due to resource sharing inside the network. However, by carefully utilizing the basic SNC concepts in the network analysis the necessity of taking these dependencies into account can be minimized. To that end, we unleash the power of the pay multiplexing only once principle (PMOO, known from the deterministic network calculus) in the SNC analysis. We choose an analytic combinatorics presentation of the results in order to ease complex calculations. In tree-reducible networks, a subclass of general feedforward networks, we obtain an effective analysis in terms of avoiding the need to take internal flow dependencies into account. In a comprehensive numerical evaluation, we demonstrate how this unleashed PMOO analysis can reduce the known gap between simulations and SNC calculations significantly, and how it favourably compares to state-of-the art SNC calculations in terms of accuracy and computational effort. Motivated by these promising results, we also consider general feedforward networks, when some flow dependencies have to be taken into account. To that end, the unleashed PMOO analysis is extended to the partially dependent case and a case study of a canonical example topology, known as the diamond network, is provided, again displaying favourable results over the state of the art. △ Less

Submitted 19 June, 2022; v1 submitted 29 April, 2021; originally announced April 2021.

Comments: Accepted at ACM SIGMETRICS 2022

ACM Class: C.4

Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise

Authors: Jannik Schmitt, Stefan Roth

Abstract: To adopt neural networks in safety critical domains, knowing whether we can trust their predictions is crucial. Bayesian neural networks (BNNs) provide uncertainty estimates by averaging predictions with respect to the posterior weight distribution. Variational inference methods for BNNs approximate the intractable weight posterior with a tractable distribution, yet mostly rely on sampling from th… ▽ More To adopt neural networks in safety critical domains, knowing whether we can trust their predictions is crucial. Bayesian neural networks (BNNs) provide uncertainty estimates by averaging predictions with respect to the posterior weight distribution. Variational inference methods for BNNs approximate the intractable weight posterior with a tractable distribution, yet mostly rely on sampling from the variational distribution during training and inference. Recent sampling-free approaches offer an alternative, but incur a significant parameter overhead. We here propose a more efficient parameterization of the posterior approximation for sampling-free variational inference that relies on the distribution induced by multiplicative Gaussian activation noise. This allows us to combine parameter efficiency with the benefits of sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks including ImageNet. △ Less

Submitted 16 March, 2021; v1 submitted 15 March, 2021; originally announced March 2021.

Detecting and Counting Small Subgraphs, and Evaluating a Parameterized Tutte Polynomial: Lower Bounds via Toroidal Grids and Cayley Graph Expanders

Authors: Marc Roth, Johannes Schmitt, Philip Wellnitz

Abstract: Given a graph property $Φ$, we consider the problem $\mathtt{EdgeSub}(Φ)$, where the input is a pair of a graph $G$ and a positive integer $k$, and the task is to decide whether $G$ contains a $k$-edge subgraph that satisfies $Φ$. Specifically, we study the parameterized complexity of $\mathtt{EdgeSub}(Φ)$ and of its counting problem $\#\mathtt{EdgeSub}(Φ)$ with respect to both approximate and exa… ▽ More Given a graph property $Φ$, we consider the problem $\mathtt{EdgeSub}(Φ)$, where the input is a pair of a graph $G$ and a positive integer $k$, and the task is to decide whether $G$ contains a $k$-edge subgraph that satisfies $Φ$. Specifically, we study the parameterized complexity of $\mathtt{EdgeSub}(Φ)$ and of its counting problem $\#\mathtt{EdgeSub}(Φ)$ with respect to both approximate and exact counting. We obtain a complete picture for minor-closed properties $Φ$: the decision problem $\mathtt{EdgeSub}(Φ)$ always admits an FPT algorithm and the counting problem $\#\mathtt{EdgeSub}(Φ)$ always admits an FPTRAS. For exact counting, we present an exhaustive and explicit criterion on the property $Φ$ which, if satisfied, yields fixed-parameter tractability and otherwise $\#\mathsf{W[1]}$-hardness. Additionally, most of our hardness results come with an almost tight conditional lower bound under the so-called Exponential Time Hypothesis, ruling out algorithms for $\#\mathtt{EdgeSub}(Φ)$ that run in time $f(k)\cdot|G|^{o(k/\log k)}$ for any computable function $f$. As a main technical result, we gain a complete understanding of the coefficients of toroidal grids and selected Cayley graph expanders in the homomorphism basis of $\#\mathtt{EdgeSub}(Φ)$. This allows us to establish hardness of exact counting using the Complexity Monotonicity framework due to Curticapean, Dell and Marx (STOC'17). Our methods can also be applied to a parameterized variant of the Tutte Polynomial $T^k_G$ of a graph $G$, to which many known combinatorial interpretations of values of the (classical) Tutte Polynomial can be extended. As an example, $T^k_G(2,1)$ corresponds to the number of $k$-forests in the graph $G$. Our techniques allow us to completely understand the parametrized complexity of computing the evaluation of $T^k_G$ at every pair of rational coordinates $(x,y)$. △ Less

Submitted 29 April, 2021; v1 submitted 6 November, 2020; originally announced November 2020.

Comments: 59 pages, 5 figures

Report number: MPIM-Bonn-2020

tinyMD: A Portable and Scalable Implementation for Pairwise Interactions Simulations

Authors: Rafael Ravedutti L. Machado, Jonas Schmitt, Sebastian Eibl, Jan Eitzinger, Roland Leißa, Sebastian Hack, Arsène Pérard-Gayot, Richard Membarth, Harald Köstler

Abstract: This paper investigates the suitability of the AnyDSL partial evaluation framework to implement tinyMD: an efficient, scalable, and portable simulation of pairwise interactions among particles. We compare tinyMD with the miniMD proxy application that scales very well on parallel supercomputers. We discuss the differences between both implementations and contrast miniMD's performance for single-nod… ▽ More This paper investigates the suitability of the AnyDSL partial evaluation framework to implement tinyMD: an efficient, scalable, and portable simulation of pairwise interactions among particles. We compare tinyMD with the miniMD proxy application that scales very well on parallel supercomputers. We discuss the differences between both implementations and contrast miniMD's performance for single-node CPU and GPU targets, as well as its scalability on SuperMUC-NG and Piz Daint supercomputers. Additionaly, we demonstrate tinyMD's flexibility by coupling it with the waLBerla multi-physics framework. This allow us to execute tinyMD simulations using the load-balancing mechanism implemented in waLBerla. △ Less

Submitted 15 September, 2020; originally announced September 2020.

Comments: 35 pages, 8 figures, submitted to Journal of Computational Science

MSC Class: B.8.2; D.1.3; D.3.3; J.2

Neural Network Virtual Sensors for Fuel Injection Quantities with Provable Performance Specifications

Authors: Eric Wong, Tim Schneider, Joerg Schmitt, Frank R. Schmidt, J. Zico Kolter

Abstract: Recent work has shown that it is possible to learn neural networks with provable guarantees on the output of the model when subject to input perturbations, however these works have focused primarily on defending against adversarial examples for image classifiers. In this paper, we study how these provable guarantees can be naturally applied to other real world settings, namely getting performance… ▽ More Recent work has shown that it is possible to learn neural networks with provable guarantees on the output of the model when subject to input perturbations, however these works have focused primarily on defending against adversarial examples for image classifiers. In this paper, we study how these provable guarantees can be naturally applied to other real world settings, namely getting performance specifications for robust virtual sensors measuring fuel injection quantities within an engine. We first demonstrate that, in this setting, even simple neural network models are highly susceptible to reasonable levels of adversarial sensor noise, which are capable of increasing the mean relative error of a standard neural network from 6.6% to 43.8%. We then leverage methods for learning provably robust networks and verifying robustness properties, resulting in a robust model which we can provably guarantee has at most 16.5% mean relative error under any sensor noise. Additionally, we show how specific intervals of fuel injection quantities can be targeted to maximize robustness for certain ranges, allowing us to train a virtual sensor for fuel injection which is provably guaranteed to have at most 10.69% relative error under noise while maintaining 3% relative error on non-adversarial data within normalized fuel injection ranges of 0.6 to 1.0. △ Less

Submitted 30 June, 2020; originally announced July 2020.

Counting Small Induced Subgraphs Satisfying Monotone Properties

Authors: Marc Roth, Johannes Schmitt, Philip Wellnitz

Abstract: Given a graph property $Φ$, the problem $\#\mathsf{IndSub}(Φ)$ asks, on input a graph $G$ and a positive integer $k$, to compute the number of induced subgraphs of size $k$ in $G$ that satisfy $Φ$. The search for explicit criteria on $Φ$ ensuring that $\#\mathsf{IndSub}(Φ)$ is hard was initiated by Jerrum and Meeks [J. Comput. Syst. Sci. 15] and is part of the major line of research on counting sm… ▽ More Given a graph property $Φ$, the problem $\#\mathsf{IndSub}(Φ)$ asks, on input a graph $G$ and a positive integer $k$, to compute the number of induced subgraphs of size $k$ in $G$ that satisfy $Φ$. The search for explicit criteria on $Φ$ ensuring that $\#\mathsf{IndSub}(Φ)$ is hard was initiated by Jerrum and Meeks [J. Comput. Syst. Sci. 15] and is part of the major line of research on counting small patterns in graphs. However, apart from an implicit result due to Curticapean, Dell and Marx [STOC 17] proving that a full classification into "easy" and "hard" properties is possible and some partial results on edge-monotone properties due to Meeks [Discret. Appl. Math. 16] and Dörfler et al. [MFCS 19], not much is known. In this work, we fully answer and explicitly classify the case of monotone, that is subgraph-closed, properties: We show that for any non-trivial monotone property $Φ$, the problem $\#\mathsf{IndSub}(Φ)$ cannot be solved in time $f(k)\cdot |V(G)|^{o(k/ {\log^{1/2}(k)})}$ for any function $f$, unless the Exponential Time Hypothesis fails. By this, we establish that any significant improvement over the brute-force approach is unlikely; in the language of parameterized complexity, we also obtain a $\#\mathsf{W}[1]$-completeness result. △ Less

Submitted 14 April, 2020; originally announced April 2020.

Comments: 33 pages, 2 figures

Report number: MPIM-Bonn-2020

Optimizing Geometric Multigrid Methods with Evolutionary Computation

Authors: Jonas Schmitt, Sebastian Kuckuk, Harald Köstler

Abstract: For many linear and nonlinear systems that arise from the discretization of partial differential equations the construction of an efficient multigrid solver is a challenging task. Here we present a novel approach for the optimization of geometric multigrid methods that is based on evolutionary computation, a generic program optimization technique inspired by the principle of natural evolution. A m… ▽ More For many linear and nonlinear systems that arise from the discretization of partial differential equations the construction of an efficient multigrid solver is a challenging task. Here we present a novel approach for the optimization of geometric multigrid methods that is based on evolutionary computation, a generic program optimization technique inspired by the principle of natural evolution. A multigrid solver is represented as a tree of mathematical expressions which we generate based on a tailored grammar. The quality of each solver is evaluated in terms of convergence and compute performance using automated local Fourier analysis (LFA) and roofline performance modeling, respectively. Based on these objectives a multi-objective optimization is performed using strongly typed genetic programming with a non-dominated sorting based selection. To evaluate the model-based prediction and to target concrete applications, scalable implementations of an evolved solver can be automatically generated with the ExaStencils framework. We demonstrate our approach by constructing multigrid solvers for the steady-state heat equation with constant and variable coefficients that consistently perform better than common V- and W-cycles. △ Less

Submitted 8 October, 2019; v1 submitted 7 October, 2019; originally announced October 2019.

Counting Induced Subgraphs: An Algebraic Approach to #W[1]-hardness

Authors: Julian Dörfler, Marc Roth, Johannes Schmitt, Philip Wellnitz

Abstract: We study the problem #IndSub(P) of counting all induced subgraphs of size k in a graph G that satisfy the property P. This problem was introduced by Jerrum and Meeks and shown to be #W[1]-hard when parameterized by k for some families of properties P including, among others, connectivity [JCSS 15] and even- or oddness of the number of edges [Combinatorica 17]. Very recently [IPEC 18], two of the a… ▽ More We study the problem #IndSub(P) of counting all induced subgraphs of size k in a graph G that satisfy the property P. This problem was introduced by Jerrum and Meeks and shown to be #W[1]-hard when parameterized by k for some families of properties P including, among others, connectivity [JCSS 15] and even- or oddness of the number of edges [Combinatorica 17]. Very recently [IPEC 18], two of the authors introduced a novel technique for the complexity analysis of #IndSub(P), inspired by the "topological approach to evasiveness" of Kahn, Saks and Sturtevant [FOCS 83] and the framework of graph motif parameters due to Curticapean, Dell and Marx [STOC 17], allowing them to prove hardness of a wide range of properties P. In this work, we refine this technique for graph properties that are non-trivial on edge-transitive graphs with a prime power number of edges. In particular, we fully classify the case of monotone bipartite graph properties: It is shown that, given any graph property P that is closed under the removal of vertices and edges, and that is non-trivial for bipartite graphs, the problem #IndSub(P) is #W[1]-hard and cannot be solved in time f(k)*n^{o(k)} for any computable function f, unless the Exponential Time Hypothesis fails. This holds true even if the input graph is restricted to be bipartite and counting is done modulo a fixed prime. A similar result is shown for properties that are closed under the removal of edges only. △ Less

Submitted 23 April, 2019; originally announced April 2019.

Comments: 21 pages

Counting Induced Subgraphs: A Topological Approach to #W[1]-hardness

Authors: Marc Roth, Johannes Schmitt

Abstract: We investigate the problem $\#\mathsf{IndSub}(Φ)$ of counting all induced subgraphs of size $k$ in a graph $G$ that satisfy a given property $Φ$. This continues the work of Jerrum and Meeks who proved the problem to be $\#\mathrm{W[1]}$-hard for some families of properties which include, among others, (dis)connectedness [JCSS 15] and even- or oddness of the number of edges [Combinatorica 17]. Usin… ▽ More We investigate the problem $\#\mathsf{IndSub}(Φ)$ of counting all induced subgraphs of size $k$ in a graph $G$ that satisfy a given property $Φ$. This continues the work of Jerrum and Meeks who proved the problem to be $\#\mathrm{W[1]}$-hard for some families of properties which include, among others, (dis)connectedness [JCSS 15] and even- or oddness of the number of edges [Combinatorica 17]. Using the recent framework of graph motif parameters due to Curticapean, Dell and Marx [STOC 17], we discover that for monotone properties $Φ$, the problem $\#\mathsf{IndSub}(Φ)$ is hard for $\#\mathrm{W[1]}$ if the reduced Euler characteristic of the associated simplicial (graph) complex of $Φ$ is non-zero. This observation links $\#\mathsf{IndSub}(Φ)$ to Karp's famous Evasiveness Conjecture, as every graph complex with non-vanishing reduced Euler characteristic is known to be evasive. Applying tools from the "topological approach to evasiveness" which was introduced in the seminal paper of Khan, Saks and Sturtevant [FOCS 83], we prove that $\#\mathsf{IndSub}(Φ)$ is $\#\mathrm{W[1]}$-hard for every monotone property $Φ$ that does not hold on the Hamilton cycle as well as for some monotone properties that hold on the Hamilton cycle such as being triangle-free or not $k$-edge-connected for $k > 2$. Moreover, we show that for those properties $\#\mathsf{IndSub}(Φ)$ can not be solved in time $f(k)\cdot n^{o(k)}$ for any computable function $f$ unless the Exponential Time Hypothesis (ETH) fails. In the final part of the paper, we investigate non-monotone properties and prove that $\#\mathsf{IndSub}(Φ)$ is $\#\mathrm{W[1]}$-hard if $Φ$ is any non-trivial modularity constraint on the number of edges with respect to some prime $q$ or if $Φ$ enforces the presence of a fixed isolated subgraph. △ Less

Submitted 5 July, 2018; originally announced July 2018.

Comments: 20 pages, 2 figures, IPEC 2018

Quality and Cost of Deterministic Network Calculus - Design and Evaluation of an Accurate and Fast Analysis

Authors: Steffen Bondorf, Paul Nikolaus, Jens B. Schmitt

Abstract: Networks are integral parts of modern safety-critical systems and certification demands the provision of guarantees for data transmissions. Deterministic Network Calculus (DNC) can compute a worst-case bound on a data flow's end-to-end delay. Accuracy of DNC results has been improved steadily, resulting in two DNC branches: the classical algebraic analysis and the more recent optimization-based an… ▽ More Networks are integral parts of modern safety-critical systems and certification demands the provision of guarantees for data transmissions. Deterministic Network Calculus (DNC) can compute a worst-case bound on a data flow's end-to-end delay. Accuracy of DNC results has been improved steadily, resulting in two DNC branches: the classical algebraic analysis and the more recent optimization-based analysis. The optimization-based branch provides a theoretical solution for tight bounds. Its computational cost grows, however, (possibly super-)exponentially with the network size. Consequently, a heuristic optimization formulation trading accuracy against computational costs was proposed. In this paper, we challenge optimization-based DNC with a new algebraic DNC algorithm. We show that: (i) no current optimization formulation scales well with the network size and (ii) algebraic DNC can be considerably improved in both aspects, accuracy and computational cost. To that end, we contribute a novel DNC algorithm that transfers the optimization's search for best attainable delay bounds to algebraic DNC. It achieves a high degree of accuracy and our novel efficiency improvements reduce the cost of the analysis dramatically. In extensive numerical experiments, we observe that our delay bounds deviate from the optimization-based ones by only 1.142% on average while computation times simultaneously decrease by several orders of magnitude. △ Less

Submitted 16 May, 2017; v1 submitted 7 March, 2016; originally announced March 2016.

Comments: Accepted at ACM SIGMETRICS 2017

Microdatabases for the Industrial Internet

Authors: K. Eric Harper, Thijmen de Gooijer, Johannes O. Schmitt, David Cox

Abstract: The Industrial Internet market is targeted to grow by trillions of US dollars by the year 2030, driven by adoption, deployment and integration of billions of intelligent devices and their associated data. This digital expansion faces a number of significant challenges, including reliable data management, security and privacy. Realizing the benefits from this evolution is made more difficult becaus… ▽ More The Industrial Internet market is targeted to grow by trillions of US dollars by the year 2030, driven by adoption, deployment and integration of billions of intelligent devices and their associated data. This digital expansion faces a number of significant challenges, including reliable data management, security and privacy. Realizing the benefits from this evolution is made more difficult because a typical industrial plant includes multiple vendors and legacy technology stacks. Aggregating all the raw data to a single data center before performing analysis increases response times, raising performance concerns in traditional markets and requiring a compromise between data duplication and data access performance. Similar to the way microservices can integrate disparate information technologies without imposing monolithic cross-cutting architecture impacts, we propose microdatabases to manage the data heterogeneity of the Industrial Internet while allowing records to be captured and secured close to the industrial processes, but also be made available near the applications that can benefit from the data. A microdatabase is an abstraction of a data store that standardizes and protects the interactions between distributed data sources, providers and consumers. It integrates an information model with discoverable object types that can be browsed interactively and programmatically, and supports repository instances that evolve with their own lifecycles. The microdatabase abstraction is independent of technology choice and was designed based on solicitation and review of industry stakeholder concerns. △ Less

Submitted 15 January, 2016; originally announced January 2016.

Comments: 5 pages, 2 figures, pending submission

Semantic Degrees for Industrie 4.0

Authors: Chih-Hong Cheng, Tuncay Guelfirat, Christian Messinger, Johannes Schmitt, Matthias Schnelte, Peter Weber

Abstract: Under the context of Industrie 4.0 (I4.0), future production systems provide balanced operations between manufacturing flexibility and efficiency, realized in an autonomous, horizontal, and decentralized item-level production control framework. Structured interoperability via precise formulations on an appropriate degree is crucial to achieve engineering efficiency in the system life cycle. Howeve… ▽ More Under the context of Industrie 4.0 (I4.0), future production systems provide balanced operations between manufacturing flexibility and efficiency, realized in an autonomous, horizontal, and decentralized item-level production control framework. Structured interoperability via precise formulations on an appropriate degree is crucial to achieve engineering efficiency in the system life cycle. However, selecting the degree of formalization can be challenging, as it crucially depends on the desired common understanding (semantic degree) between multiple parties. In this paper, we categorize different semantic degrees and map a set of technologies in industrial automation to their associated degrees. Furthermore, we created guidelines to assist engineers selecting appropriate semantic degrees in their design. We applied these guidelines on publically available scenarios to examine the validity of the approach, and identified semantic elements over internally developed use cases targeting semantically-enabled plug-and-produce. △ Less

Submitted 22 May, 2015; v1 submitted 21 May, 2015; originally announced May 2015.

Comments: Timestamp of work-in-progress; the paper has been circulated within standardization units

An Analytical Model of Packet Collisions in IEEE 802.15.4 Wireless Networks

Authors: Matthias Wilhelm, Vincent Lenders, Jens B. Schmitt

Abstract: Numerous studies showed that concurrent transmissions can boost wireless network performance despite collisions. While these works provide empirical evidence that concurrent transmissions may be received reliably, existing signal capture models only partially explain the root causes of this phenomenon. We present a comprehensive mathematical model that reveals the reasons and provides insights on… ▽ More Numerous studies showed that concurrent transmissions can boost wireless network performance despite collisions. While these works provide empirical evidence that concurrent transmissions may be received reliably, existing signal capture models only partially explain the root causes of this phenomenon. We present a comprehensive mathematical model that reveals the reasons and provides insights on the key parameters affecting the performance of MSK-modulated transmissions. A major contribution is a closed-form derivation of the receiver bit decision variable for arbitrary numbers of colliding signals and constellations of power ratios, timing offsets, and carrier phase offsets. We systematically explore the root causes for successful packet delivery under concurrent transmissions across the whole parameter space of the model. We confirm the capture threshold behavior observed in previous studies but also reveal new insights relevant for the design of optimal protocols: We identify capture zones depending not only on the signal power ratio but also on time and phase offsets. △ Less

Submitted 18 August, 2014; v1 submitted 19 September, 2013; originally announced September 2013.

Comments: Accepted for publication in the IEEE Transactions on Wireless Communications under the title "On the Reception of Concurrent Transmissions in Wireless Sensor Networks."

Measurement and Prediction of Centrical/Peripheral Network Properties based on Regression Analysis - A Parametric Foundation for Performance Self-Management in WSNs

Authors: Adam Bachorek, Bagavathiannan Palanisamy, Jens B. Schmitt

Abstract: Predicting performance-related behavior of the underlying network structure becomes more and more indispensable in terms of the aspired application outcome quality. However, the reliable forecast of QoS metrics like packet transfer delay in wireless network systems is still a challenging task. Even though existing approaches are technically capable of determining such network properties under cert… ▽ More Predicting performance-related behavior of the underlying network structure becomes more and more indispensable in terms of the aspired application outcome quality. However, the reliable forecast of QoS metrics like packet transfer delay in wireless network systems is still a challenging task. Even though existing approaches are technically capable of determining such network properties under certain assumptions, they mostly abstract away from primal aspects that inherently have an essential impact on temporal network performance dynamics. Also, they usually require auxiliary resources to be implemented and deployed along with the actual network components. In the course of developing a lightweight measurement-based alternative for the self-inspection and prediction of volatile performance characteristics in environments of any kind, we selectively investigate the duration of message delivery and packet loss rate against various parameters peculiar to common radio network technologies like Wireless Sensor Networks (WSNs). Our hands-on experiments reveal the relations between the oftentimes underestimated medium access delay and a variety of main influencing factors including packet size, backoff period, and number of neighbor nodes contending for the communication medium. A closed formulation of selected weighted drivers facilitates the average-case prediction of inter-node packet transfer delays for arbitrary configurations of given network parameters even on resource-scarce WSN devices. We validate our prediction method against basic multi-hop networking scenarios. Yield field test results proof the basic feasibility and high precision of our approach to network property estimation in virtue of self-governed local measurements and regression-based calculations paving the way for a prospective self-management of network properties based upon autonomous distributed coordination. △ Less

Submitted 18 August, 2013; originally announced August 2013.

On the Catalyzing Effect of Randomness on the Per-Flow Throughput in Wireless Networks

Authors: Florin Ciucu, Jens Schmitt

Abstract: This paper investigates the throughput capacity of a flow crossing a multi-hop wireless network, whose geometry is characterized by general randomness laws including Uniform, Poisson, Heavy-Tailed distributions for both the nodes' densities and the number of hops. The key contribution is to demonstrate \textit{how} the \textit{per-flow throughput} depends on the distribution of 1) the number of no… ▽ More This paper investigates the throughput capacity of a flow crossing a multi-hop wireless network, whose geometry is characterized by general randomness laws including Uniform, Poisson, Heavy-Tailed distributions for both the nodes' densities and the number of hops. The key contribution is to demonstrate \textit{how} the \textit{per-flow throughput} depends on the distribution of 1) the number of nodes $N_j$ inside hops' interference sets, 2) the number of hops $K$, and 3) the degree of spatial correlations. The randomness in both $N_j$'s and $K$ is advantageous, i.e., it can yield larger scalings (as large as $Θ(n)$) than in non-random settings. An interesting consequence is that the per-flow capacity can exhibit the opposite behavior to the network capacity, which was shown to suffer from a logarithmic decrease in the presence of randomness. In turn, spatial correlations along the end-to-end path are detrimental by a logarithmic term. △ Less

Submitted 27 July, 2013; originally announced July 2013.

Air Dominance in Sensor Networks: Guarding Sensor Motes using Selective Interference

Authors: Matthias Wilhelm, Ivan Martinovic, Jens B. Schmitt, Vincent Lenders

Abstract: Securing wireless sensor networks (WSNs) is a hard problem. In particular, network access control is notoriously difficult to achieve due to the inherent broadcast characteristics of wireless communications: an attacker can easily target any node in its transmission range and affect large parts of a sensor network simultaneously. In this paper, we therefore propose a distributed guardian system to… ▽ More Securing wireless sensor networks (WSNs) is a hard problem. In particular, network access control is notoriously difficult to achieve due to the inherent broadcast characteristics of wireless communications: an attacker can easily target any node in its transmission range and affect large parts of a sensor network simultaneously. In this paper, we therefore propose a distributed guardian system to protect a WSN based on physically regulating channel access by means of selective interference. The guardians are deployed alongside a sensor network, inspecting all local traffic, classifying packets based on their content, and destroying any malicious packet while still on the air. In that sense, the system tries to gain "air dominance" over attackers. A key challenge in implementing the guardian system is the resulting real-time requirement in order to classify and destroy packets during transmission. We present a USRP2 software radio based guardian implementation for IEEE 802.15.4 that meets this challenge; using an FPGA-based design we can even check for the content of the very last payload byte of a packet and still prevent its reception by a potential victim mote. Our evaluation shows that the guardians effectively block 99.9% of unauthorized traffic in 802.15.4 networks in our experiments, without disturbing the legitimate operations of the WSN. △ Less

Submitted 17 May, 2013; originally announced May 2013.

Comments: 16 pages, 13 figures

Sharp Bounds in Stochastic Network Calculus

Authors: Florin Ciucu, Felix Poloczek, Jens Schmitt

Abstract: The practicality of the stochastic network calculus (SNC) is often questioned on grounds of potential looseness of its performance bounds. In this paper it is uncovered that for bursty arrival processes (specifically Markov-Modulated On-Off (MMOO)), whose amenability to \textit{per-flow} analysis is typically proclaimed as a highlight of SNC, the bounds can unfortunately indeed be very loose (e.g.… ▽ More The practicality of the stochastic network calculus (SNC) is often questioned on grounds of potential looseness of its performance bounds. In this paper it is uncovered that for bursty arrival processes (specifically Markov-Modulated On-Off (MMOO)), whose amenability to \textit{per-flow} analysis is typically proclaimed as a highlight of SNC, the bounds can unfortunately indeed be very loose (e.g., by several orders of magnitude off). In response to this uncovered weakness of SNC, the (Standard) per-flow bounds are herein improved by deriving a general sample-path bound, using martingale based techniques, which accommodates FIFO, SP, EDF, and GPS scheduling. The obtained (Martingale) bounds gain an exponential decay factor of ${\mathcal{O}}(e^{-αn})$ in the number of flows $n$. Moreover, numerical comparisons against simulations show that the Martingale bounds are remarkably accurate for FIFO, SP, and EDF scheduling; for GPS scheduling, although the Martingale bounds substantially improve the Standard bounds, they are numerically loose, demanding for improvements in the core SNC analysis of GPS. △ Less

Submitted 22 July, 2013; v1 submitted 17 March, 2013; originally announced March 2013.

Key Generation in Wireless Sensor Networks Based on Frequency-selective Channels - Design, Implementation, and Analysis

Authors: Matthias Wilhelm, Ivan Martinovic, Jens B. Schmitt

Abstract: Key management in wireless sensor networks faces several new challenges. The scale, resource limitations, and new threats such as node capture necessitate the use of an on-line key generation by the nodes themselves. However, the cost of such schemes is high since their secrecy is based on computational complexity. Recently, several research contributions justified that the wireless channel itself… ▽ More Key management in wireless sensor networks faces several new challenges. The scale, resource limitations, and new threats such as node capture necessitate the use of an on-line key generation by the nodes themselves. However, the cost of such schemes is high since their secrecy is based on computational complexity. Recently, several research contributions justified that the wireless channel itself can be used to generate information-theoretic secure keys. By exchanging sampling messages during movement, a bit string can be derived that is only known to the involved entities. Yet, movement is not the only possibility to generate randomness. The channel response is also strongly dependent on the frequency of the transmitted signal. In our work, we introduce a protocol for key generation based on the frequency-selectivity of channel fading. The practical advantage of this approach is that we do not require node movement. Thus, the frequent case of a sensor network with static motes is supported. Furthermore, the error correction property of the protocol mitigates the effects of measurement errors and other temporal effects, giving rise to an agreement rate of over 97%. We show the applicability of our protocol by implementing it on MICAz motes, and evaluate its robustness and secrecy through experiments and analysis. △ Less

Submitted 5 May, 2010; originally announced May 2010.

Comments: Submitted to IEEE Transactions on Dependable and Secure Computing