DBSCAN is a popular approach for density-based clustering. In this short “work in progress” paper... more DBSCAN is a popular approach for density-based clustering. In this short “work in progress” paper, we want to present an interpretation of DBSCAN as a matrix factorization problem, which introduces a theoretical connection (but not an equivalence) between DBSCAN and Spectral Clustering (SC). While this does not yield a faster algorithm for DBSCAN, establishing this relationship is a step towards a more unified view of clustering, by identifying further relationships between some of the most popular clustering algorithms.
Ensembles are among the state-of-the-art in many machine learning applications. With the ongoing ... more Ensembles are among the state-of-the-art in many machine learning applications. With the ongoing integration of ML models into everyday life, e.g., in the form of the Internet of Things, the deployment and continuous application of models become more and more an important issue. Therefore, small models that offer good predictive performance and use small amounts of memory are required. Ensemble pruning is a standard technique for removing unnecessary classifiers from a large ensemble that reduces the overall resource consumption and sometimes improves the performance of the original ensemble. Similarly, leaf-refinement is a technique that improves the performance of a tree ensemble by jointly re-learning the probability estimates in the leaf nodes of the trees, thereby allowing for smaller ensembles while preserving their predictive performance. In this paper, we develop a new method that combines both approaches into a single algorithm. To do so, we introduce $$L_1$$ L 1 regulariza...
In ever more disciplines, science is driven by data, which leads to data analytics becoming a pri... more In ever more disciplines, science is driven by data, which leads to data analytics becoming a primary skill for researchers. This includes the complete process from data acquisition at sensors, over pre-processing and feature extraction to the use and application of machine learning. Sensors here often produce a plethora of data that needs to be dealt with in near-realtime, which requires a combined effort of implementations at the hardware level to high-level design of data flows. In this paper we outline two use-cases of this wide span of data analysis for science in a real-world example in astroparticle physics. We outline a high-level design approach which is capable of defining the complete data flow from sensor hardware to final analysis.
More and more data is created by humans and cyber-physical systems having sensing, acting and net... more More and more data is created by humans and cyber-physical systems having sensing, acting and networking capabilities. Together, these systems form the Internet of Things (IoT). The realtime analysis of its data may provide us with valuable insights about the complex inner processes of the IoT. Moreover, these insights offer new opportunities ranging from sensor monitoring to actor control. The volume and velocity of the data at the distributed nodes challenge human as well as machine monitoring of the IoT. Broadcasting all measurements to a central node might exceed the network capacity as well as the resources at the central node or the human attention span. Hence, data should be reduced already at the local nodes such that the submitted information can be used for efficient monitoring. There are several methods that aim at data summarization ranging from clustering, aggregation to compression. Where most of the approaches transform the representation, we want to select unchanged ...
Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '15, 2015
The panel session 'Data Driven Science' discusses application and use of knowledge discov... more The panel session 'Data Driven Science' discusses application and use of knowledge discovery, machine learning and data analytics in science disciplines; in natural, physical, medical and social science; from physics to geology, and from neuroscience to population health. Knowledge discovery methods are finding broad application in all areas of scientific endeavor, to explore experimental data, to discover new models, to propose new scientific theories and ideas. In addition, the availability of ever larger scientific data sets is driving a new data-driven paradigm for modeling of complex phenomena in physical, natural and social sciences. The purpose of this panel is to bring together users of knowledge discovery, machine learning and data analytics methods across the science disciplines, to understand what tools and methods are proving effective in areas such as data exploration and modeling, to uncover common problems that can be addressed in the KDD community, and to explore the emerging data-driven paradigm in science.
One of the major goals in designing learning agents is to let these agents develop useful skills ... more One of the major goals in designing learning agents is to let these agents develop useful skills over time. Webster's dictionary [Webster, 1993] defines "skill" as the ability to use one's knowledge effectively and readily in execution 01' performance and a learned power of doing a thing competently. In the case of robots, these skills are not only related to physical actions of the robot, but also to the coordination of activities, communication with humans, and active sensing. Within this chapter, we discuss issues that are related to learning elementary skills. As elementary skills we define skills that map robot perceptions directly onto robot actions by means of a strategy (a control function or policy, resp.) that is goal-oriented. Examples of such elementary skills are force control, peg-into-hole insertion, navigation through a door,targeted parameterization of an active sensor system, et cetera.
Simulations have recently been considered as data generators for machine learning. However, the h... more Simulations have recently been considered as data generators for machine learning. However, the high computational cost associated with them requires a smart sampling of what to simulate. We distinguish between two scenarios of simulation data mining, which can be optimized with active learning and active class selection.
2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), 2021
Both complex and evolving nature of time series structure make forecasting among one of the most ... more Both complex and evolving nature of time series structure make forecasting among one of the most important and challenging tasks in time series analysis. Typical methods for forecasting are designed to model time-evolving dependencies between data observations. However, it is generally accepted that none of them is universally valid for every application. Therefore, methods for learning heterogeneous ensembles by combining a diverse set of forecasts together appear as a promising solution to tackle this task. Several approaches, ranging from simple and enhanced averaging tactics to applying meta-learning methods, have been proposed to learn how to combine individual models in an ensemble. However, finding the optimal strategy for ensemble aggregation remains an open research question, particularly, when the ensemble needs to be adapted in real time. In this paper, we leverage a deep reinforcement learning framework for learning linearly weighted ensembles as a meta-learning method. In this framework, the combination policy in ensembles is modelled as a sequential decision making process, and an actor-critic model aims at learning the optimal weights in a continuous action space. The policy is updated following a drift detection mechanism for tracking performance shifts of the ensemble model. An extensive empirical study on many real-world datasets demonstrates that our method achieves excellent or on par results in comparison to the state-of-the-art approaches as well as several baselines.
Machine learning applications have become ubiquitous. Their applications range from embedded cont... more Machine learning applications have become ubiquitous. Their applications range from embedded control in production machines over process optimization in diverse areas (e.g., traffic, finance, sciences) to direct user interactions like advertising and recommendations. This has led to an increased effort of making machine learning trustworthy. Explainable and fair AI have already matured. They address the knowledgeable user and the application engineer. However, there are users that want to deploy a learned model in a similar way as their washing machine. These stakeholders do not want to spend time in understanding the model, but want to rely on guaranteed properties. What are the relevant properties? How can they be expressed to the stake- holder without presupposing machine learning knowledge? How can they be guaranteed for a certain implementation of a machine learning model? These questions move far beyond the current state of the art and we want to address them here. We propose ...
DBSCAN is a popular approach for density-based clustering. In this short “work in progress” paper... more DBSCAN is a popular approach for density-based clustering. In this short “work in progress” paper, we want to present an interpretation of DBSCAN as a matrix factorization problem, which introduces a theoretical connection (but not an equivalence) between DBSCAN and Spectral Clustering (SC). While this does not yield a faster algorithm for DBSCAN, establishing this relationship is a step towards a more unified view of clustering, by identifying further relationships between some of the most popular clustering algorithms.
Ensembles are among the state-of-the-art in many machine learning applications. With the ongoing ... more Ensembles are among the state-of-the-art in many machine learning applications. With the ongoing integration of ML models into everyday life, e.g., in the form of the Internet of Things, the deployment and continuous application of models become more and more an important issue. Therefore, small models that offer good predictive performance and use small amounts of memory are required. Ensemble pruning is a standard technique for removing unnecessary classifiers from a large ensemble that reduces the overall resource consumption and sometimes improves the performance of the original ensemble. Similarly, leaf-refinement is a technique that improves the performance of a tree ensemble by jointly re-learning the probability estimates in the leaf nodes of the trees, thereby allowing for smaller ensembles while preserving their predictive performance. In this paper, we develop a new method that combines both approaches into a single algorithm. To do so, we introduce $$L_1$$ L 1 regulariza...
In ever more disciplines, science is driven by data, which leads to data analytics becoming a pri... more In ever more disciplines, science is driven by data, which leads to data analytics becoming a primary skill for researchers. This includes the complete process from data acquisition at sensors, over pre-processing and feature extraction to the use and application of machine learning. Sensors here often produce a plethora of data that needs to be dealt with in near-realtime, which requires a combined effort of implementations at the hardware level to high-level design of data flows. In this paper we outline two use-cases of this wide span of data analysis for science in a real-world example in astroparticle physics. We outline a high-level design approach which is capable of defining the complete data flow from sensor hardware to final analysis.
More and more data is created by humans and cyber-physical systems having sensing, acting and net... more More and more data is created by humans and cyber-physical systems having sensing, acting and networking capabilities. Together, these systems form the Internet of Things (IoT). The realtime analysis of its data may provide us with valuable insights about the complex inner processes of the IoT. Moreover, these insights offer new opportunities ranging from sensor monitoring to actor control. The volume and velocity of the data at the distributed nodes challenge human as well as machine monitoring of the IoT. Broadcasting all measurements to a central node might exceed the network capacity as well as the resources at the central node or the human attention span. Hence, data should be reduced already at the local nodes such that the submitted information can be used for efficient monitoring. There are several methods that aim at data summarization ranging from clustering, aggregation to compression. Where most of the approaches transform the representation, we want to select unchanged ...
Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '15, 2015
The panel session 'Data Driven Science' discusses application and use of knowledge discov... more The panel session 'Data Driven Science' discusses application and use of knowledge discovery, machine learning and data analytics in science disciplines; in natural, physical, medical and social science; from physics to geology, and from neuroscience to population health. Knowledge discovery methods are finding broad application in all areas of scientific endeavor, to explore experimental data, to discover new models, to propose new scientific theories and ideas. In addition, the availability of ever larger scientific data sets is driving a new data-driven paradigm for modeling of complex phenomena in physical, natural and social sciences. The purpose of this panel is to bring together users of knowledge discovery, machine learning and data analytics methods across the science disciplines, to understand what tools and methods are proving effective in areas such as data exploration and modeling, to uncover common problems that can be addressed in the KDD community, and to explore the emerging data-driven paradigm in science.
One of the major goals in designing learning agents is to let these agents develop useful skills ... more One of the major goals in designing learning agents is to let these agents develop useful skills over time. Webster's dictionary [Webster, 1993] defines "skill" as the ability to use one's knowledge effectively and readily in execution 01' performance and a learned power of doing a thing competently. In the case of robots, these skills are not only related to physical actions of the robot, but also to the coordination of activities, communication with humans, and active sensing. Within this chapter, we discuss issues that are related to learning elementary skills. As elementary skills we define skills that map robot perceptions directly onto robot actions by means of a strategy (a control function or policy, resp.) that is goal-oriented. Examples of such elementary skills are force control, peg-into-hole insertion, navigation through a door,targeted parameterization of an active sensor system, et cetera.
Simulations have recently been considered as data generators for machine learning. However, the h... more Simulations have recently been considered as data generators for machine learning. However, the high computational cost associated with them requires a smart sampling of what to simulate. We distinguish between two scenarios of simulation data mining, which can be optimized with active learning and active class selection.
2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), 2021
Both complex and evolving nature of time series structure make forecasting among one of the most ... more Both complex and evolving nature of time series structure make forecasting among one of the most important and challenging tasks in time series analysis. Typical methods for forecasting are designed to model time-evolving dependencies between data observations. However, it is generally accepted that none of them is universally valid for every application. Therefore, methods for learning heterogeneous ensembles by combining a diverse set of forecasts together appear as a promising solution to tackle this task. Several approaches, ranging from simple and enhanced averaging tactics to applying meta-learning methods, have been proposed to learn how to combine individual models in an ensemble. However, finding the optimal strategy for ensemble aggregation remains an open research question, particularly, when the ensemble needs to be adapted in real time. In this paper, we leverage a deep reinforcement learning framework for learning linearly weighted ensembles as a meta-learning method. In this framework, the combination policy in ensembles is modelled as a sequential decision making process, and an actor-critic model aims at learning the optimal weights in a continuous action space. The policy is updated following a drift detection mechanism for tracking performance shifts of the ensemble model. An extensive empirical study on many real-world datasets demonstrates that our method achieves excellent or on par results in comparison to the state-of-the-art approaches as well as several baselines.
Machine learning applications have become ubiquitous. Their applications range from embedded cont... more Machine learning applications have become ubiquitous. Their applications range from embedded control in production machines over process optimization in diverse areas (e.g., traffic, finance, sciences) to direct user interactions like advertising and recommendations. This has led to an increased effort of making machine learning trustworthy. Explainable and fair AI have already matured. They address the knowledgeable user and the application engineer. However, there are users that want to deploy a learned model in a similar way as their washing machine. These stakeholders do not want to spend time in understanding the model, but want to rely on guaranteed properties. What are the relevant properties? How can they be expressed to the stake- holder without presupposing machine learning knowledge? How can they be guaranteed for a certain implementation of a machine learning model? These questions move far beyond the current state of the art and we want to address them here. We propose ...
Uploads