Causal Discovery

This paper presents a number of new algorithms for discovering the Markov Blanket of a target variable T from training data. The Markov Blanket can be used for variable selection for classification, for causal discovery, and for Bayesian Network learning. We ...

Causal questions are central to many areas of science. Does high salt intake increase the
risk of heart attack? Does hormone replacement therapy reduce breast cancer risk? Would
upping the minimum wage increase unemployment? These questions have in common that
they require us to predict the consequences of an action. Traditionally machine learning solves
problems of the form given some data, infer properties of the distribution that generated that
data. These problems ask us to predict properties of the distribution after taking some action.
Without experimentally testing the intervention or making assumptions that constrain how
the action can change the distribution such inference is impossible.
The current approach to causal inference in elds such as economics and social science,
where the variables of interest are subject to human experience, is to use expert knowledge to
describe qualitatively how variables relate to one another and then t the statistical aspects
of the model. However there are problems, particularly in bioinformatics, where the number
of variables is large, and we do not have sucient direct human experience or prior theory
to specify causal models. In these cases, automated causal structure learning provides the
prospect of signicant improvements relative to purely nding the strongest associations.
We present a survey of recent research on causality, focusing on the problems of inferring
causal eects from observational data and causal structure learning. The rst problem occurs
when the structure of the relationship between variables is assumed to be at least partially
given by theory or prior knowledge and has been addressed in some depth in statistics [5],
social science [1], economics [2] and machine learning [3]. We describe how causal structural
equation models unify the diering approaches to this problem and provide a complete frame-
work to determine if and how causal queries can be inferred from observational data without
parametric assumptions about the relationship between variables [6]. Causal structure learning
attempts to infer causal structure from data based on more general assumptions. In particular,
algorithms have been developed based on the assumption of faithfulness; that the distribution
was generated by a directed acyclic graph over its variables and all the dependencies in the
distribution are re
ected by d-separation in that graph [7], and additive noise; eects are
deterministic functions of their causes plus an additive noise term [4]. We summarise the
application of these techniques on a range of practical problems and discuss their potential
and limitations.

Inferring causal relationships from observational data is rarely straightforward, but the problem is especially difficult in high dimensions. For these applications, causal discovery algorithms typically require parametric restrictions or extreme sparsity constraints. We relax these assumptions and focus on an important but more specialized problem, namely recovering a directed acyclic subgraph of variables known to be causally descended from some (possibly large) set of confounding covariates, i.e. a confounder blanket. This is useful in many settings, for example when studying a dynamic biomolecular subsystem with genetic data providing causally relevant background information. Under a structural assumption that, we argue, must be satisfied in practice if informative answers are to be found, our method accommodates graphs of low or high sparsity while maintaining polynomial time complexity. We derive a sound and complete algorithm for identifying causal relationships under these conditions and implement testing procedures with provable error control for linear and nonlinear systems. We demonstrate our approach on a range of simulation settings.

In this paper I argue that constitutive relevance relations in mechanisms behave like a special kind of causal relation in at least one important respect: Under suitable circumstances constitutive relevance relations produce the Markov factorization. Based on this observation one may wonder whether standard methods for causal discovery could be fruitfully applied to uncover constitutive relevance relations. This paper is intended as a first step into this new area of philosophical research. I investigate to what extent the PC algorithm, originally developed for causal search, can be used for constitutive relevance discovery. I also discuss possible objections and certain limitations of a constitutive relevance discovery procedure based on PC.

Citation information: Gebharter, A. (2017). Uncovering constitutive relevance relations in mechanisms. Philosophical Studies, 174(11), 2645–2666. doi:10.1007/s11098-016-0803-3

In two experiments, we studied the strategies that people use to discover causal relationships. According to inferential approaches to causal discovery, if people attempt to discover the power of a cause, then they should naturally select the most informative and unambiguous context. For generative causes this would be a context with a low base rate of effects generated by other causes and for preventive causes a context with a high base rate. In the following experiments, we used probabilistic and/or deterministic target causes and contexts. In each experiment, participants observed several contexts in which the effect occurred with different probabilities. After this training, the participants were presented with different target causes whose causal status was unknown. In order to discover the influence of each cause, participants were allowed, on each trial, to choose the context in which the cause would be tested. As expected by inferential theories, the participants preferred t...

A Random Graph is a random object which take its values in the space of graphs. We take advantage of the expressibility of graphs in order to model the uncertainty about the existence of causal relationships within a given set of variables. We adopt a Bayesian point of view in order to capture a causal structure via interaction and learning with a causal environment. We test our method over two different scenarios, and the experiments mainly confirm that our technique can learn a causal structure. Furthermore, the experiments and results presented for the first test scenario demonstrate the usefulness of our method to learn a causal structure as well as the optimal action. On the other hand the second experiment, shows that our proposal manages to learn the underlying causal structure of several tasks with different sizes and different causal structures.

Objective—This study aims at developing and introducing a new algorithm, called direct causal learner (DCL), for learning the direct causal influences of a single target. We applied it to both simulated and real clinical and genome wide association study (GWAS) datasets and compared its performance to classic causal learning algorithms. Method—The DCL algorithm learns the causes of a single target from passive data using Bayesian-scoring, instead of using independence checks, and a novel deletion algorithm. We generate 14400 simulated datasets and measure the number of datasets for which DCL correctly and partially predicts the direct causes. We then compare its performance with the constraint-based path consistency (PC) and conservative PC (CPC) algorithms, the Bayesian-score based fast greedy search (FGS) algorithm, and the partial ancestral graphs algorithm fast causal inference (FCI). In addition, we extend our comparison of all five algorithms to both a real GWAS dataset and real breast cancer datasets over various time-points in order to observe how effective they are at predicting the causal influences of Alzheimer's disease and breast cancer survival. Results—DCL consistently outperforms FGS, PC, CPC, and FCI in discovering the parents of the target for the datasets simulated using a simple network. Overall, DCL predicts significantly more datasets correctly (McNemar's test significance: p ≪ 0.0001) than any of the other algorithms for these network types. For example, when assessing overall performance (simple and complex network results combined), DCL correctly predicts approximately 1400 more datasets than the top FGS method, 1600 more datasets than the top CPC method, 4500 more datasets than the top PC method, and 5600 more datasets than the top FCI method. Although FGS did correctly predict more datasets than DCL for the complex networks, and DCL correctly predicted only a few more datasets than CPC for these networks, there is no significant difference in performance between these three algorithms for this network type. However, when we use a more continuous measure of accuracy, we find that all the DCL methods are able to better partially predict more direct causes than FGS and CPC for the complex networks. In addition, DCL consistently had faster runtimes than the other algorithms. In the application to the real datasets, DCL identified rs6784615, located on the NISCH gene, and rs10824310, located on the PRKG1 gene, as direct causes of late onset Alzheimer's disease (LOAD) development. In addition, DCL identified ER category as a direct predictor of breast cancer mortality within 5 years, and HER2 status as a direct predictor of 10-year breast cancer mortality. These predictors have been identified in previous studies to have a direct causal relationship with their respective phenotypes, supporting the predictive power of DCL. When the other algorithms discovered predictors from the real datasets, these predictors were either also found by DCL or could not be supported by previous studies. Conclusion—Our results show that DCL outperforms FGS, PC, CPC, and FCI in almost every case, demonstrating its potential to advance causal learning. Furthermore, our DCL algorithm effectively identifies direct causes in the LOAD and Metabric GWAS datasets, which indicates its potential for clinical applications.

The Markov blanket of a target variable is the minimum conditioning set of variables that makes the target independent of all other variables. Markov blankets inform feature selection, aid in causal discovery and serve as a basis for scalable methods of constructing Bayesian networks. We apply decision tree induction to the task of Markov blanket identification. Notably, we compare (a) C5.0, a widely used algorithm for decision rule induction, (b) C5C, which post-processes C5.0 's rule set to retain the most frequently referenced variables and (c) PC, a standard method for Bayesian network induction. C5C performs as well as or better than C5.0 and PC across a number of data sets. Our modest variation of an inexpensive, accurate, off-the-shelf induction engine mitigates the need for specialized procedures, and establishes baseline performance against which specialized algorithms can be compared.

In this paper, I want to substantiate three related claims regarding causal discovery from non-experimental data. Firstly, in scientific practice, the problem of ignorance is ubiquitous, persistent, and far-reaching. Intuitively, the problem of ignorance bears upon the following situation. A set of random variables V is studied but only partly tested for (conditional) independencies; i.e. for some variables A and B it is not known whether they are (conditionally) independent. Secondly, Judea Pearl’s most meritorious and influential algorithm for causal discovery (the IC algorithm) cannot be applied in cases of ignorance. It presupposes that a full list of (conditional) independence relations is on hand and it would lead to unsatisfactory results when applied to partial lists. Finally, the problem of ignorance is successfully treated by means of ALIC, the adaptive logic for causal discovery presented in this paper.

At the heart of causal structure learning from observational data lies a deceivingly simple question: given two statistically dependent random variables, which one has a causal effect on the other? This is impossible to answer using statistical dependence testing alone and requires that we make additional assumptions. We propose fast and simple criteria for distinguishing cause and effect in pairs of discrete or continuous random variables. The intuition behind them is that predicting the effect variable using the cause variable should be 'simpler' than the reverse different notions of 'simplicity' giving rise to different criteria. We demonstrate the accuracy of the criteria on synthetic data generated under a broad family of causal mechanisms and types of noise.

At the heart of causal structure learning from observational data lies a deceivingly simple question: given two statistically dependent random variables, which one has a causal effect on the other? This is impossible to answer using statistical dependence testing alone and requires that we make additional assumptions. We propose several fast and simple criteria for distinguishing cause and effect in pairs of discrete or continuous random variables. The intuition behind them is that predicting the effect variable using the cause variable should be `simpler' than the reverse -- different notions of `simplicity' giving rise to different criteria. We demonstrate the accuracy of the criteria on synthetic data generated under a broad family of causal mechanisms and types of noise.

At the heart of causal structure learning from observational data lies a deceivingly simple question: given two statistically dependent random variables, which one has a causal effect on the other? This is impossible to answer using statistical dependence testing alone and requires that we make additional assumptions. We propose several fast and simple criteria for distinguishing cause and effect in pairs of discrete or continuous random variables. The intuition behind them is that predicting the effect variable using the cause variable should be `simpler' than the reverse -- different notions of `simplicity' giving rise to different criteria. We demonstrate the accuracy of the criteria on synthetic data generated under a broad family of causal mechanisms and types of noise.

At the heart of causal structure learning from observational data lies a deceivingly simple question: given two statistically dependent random variables, which one has a causal effect on the other? This is impossible to answer using statistical dependence testing alone and requires that we make additional assumptions. We propose fast and simple criteria for distinguishing cause and effect in pairs of discrete or continuous random variables. The intuition behind them is that predicting the effect variable using the cause variable should be 'simpler' than the reverse different notions of 'simplicity' giving rise to different criteria. We demonstrate the accuracy of the criteria on synthetic data generated under a broad family of causal mechanisms and types of noise.

The Causality Workbench project provides an environment to test causal discovery algorithms. Via a web portal (http: //clopinet.com/causality), we provide a number of resources, including a repository of datasets, models, and software packages, and a virtual laboratory allowing users to benchmark causal discovery algorithms by performing virtual experiments to study artificial causal systems. We regularly organize competitions. In this paper,

Bayesian Networks (BN) is a knowledge representation formalism that has been proven to be valuable in biomedicine for constructing decision support systems and for generating causal hypotheses from data. Given the emergence of datasets in medicine and biology with thousands of variables and that current algorithms do not scale more than a few hundred variables in practical domains, new efficient and accurate algorithms are needed to learn high quality BNs from data. We present a new algorithm called Max-Min Hill-Climbing (MMHC) that builds upon and improves the Sparse Candidate (SC) algorithm; a state-of-the-art algorithm that scales up to datasets involving hundreds of variables provided the generating networks are sparse. Compared to the SC, on a number of datasets from medicine and biology, (a) MMHC discovers BNs that are structurally closer to the data-generating BN, (b) the discovered networks are more probable given the data, (c) MMHC is computationally more efficient and scal...

Bayesian networks (BN) have been used for prediction or classification tasks in various domains. In the first applications, the BN structure was causally defined by expert knowledge. Then, algorithms were proposed in order to learn the BN structure from observational data. Generally, these algorithms can only find a structure encoding the right conditional independencies but not all the causal relationships. Some new domains appear where the model will only be learnt in order to discover these causal relationships. To this end, we will focus on discovering causal relations in order to get Causal Bayesian Networks (CBN). To learn such models, interventional data (i.e. samples conditioned on the particular values of one or more variables that have been experimentally manipulated) are required. These interventions are usually very expensive to perform, therefore the choice of variables to experiment on can be vital when the number of experimentations is restricted. In many cases, available ontologies provide high level knowledge for the same domain under study. Consequently, using this semantical knowledge can turn out of a big utility to improve causal discovery. This article proposes a new method for learning CBNs from observational data and interventions. We first extend the greedy approach for perfect observational and experimental data proposed in [13], by adding a new step based on the integration of ontological knowledge, which will allow us to choose efficiently the interventions to perform in order to obtain the complete CBN. Then, we propose an enriched visualization for better understanding of the causal graphs.

Learning Causal Bayesian Networks (CBNs) is a new line of research in the machine learning field. Within the existing works in this direction [8, 12, 13], few of them have taken into account the gain that can be expected when integrating additional knowledge during the learning process. In this paper, we present a new serendipitous strategy for learning CBNs using prior knowledge extracted from ontologies. The integration of such domain’s semantic information can be very useful to reveal new causal relations and provide the necessary knowledge to anticipate the optimal choice of experimentations. Our strategy also supports the evolving character of the semantic background by reusing the causal discoveries in order to enrich the domain ontologies.

The standard approach guiding research on the relationship between categories and causality views categories as reflecting causal relations in the world. We provide evidence that the opposite direction also holds: categories that have been acquired in previous learning contexts may influence subsequent causal learning. In three experiments we show that identical causal learning input yields different attributions of causal capacity