Scalable learning of large networks
Sushmita Roy§ , Sergey Plis§ , Margaret Werner-Washburne+ and Terran Lane§
§
Department of Computer Science, University of New Mexico
+
Department of Biology, University of New Mexico
Abstract
dencies among arbitrarily sized groups of nodes. When
Cellular networks inferred from microarray data provide the network is not known, network structure inference
important insight into genetic interactions in cells under algorithms are used to infer the structure. As optimal
is known to be NP-hard,
different conditions. Unfortunately, many network infer- de-novo network reconstruction
k−1
polynomial
time
(O(n
))
algorithms
learning networks
ence algorithms do not scale to whole-genome data with
with
bounded
degree
have
been
developed
[1]. Here n
hundreds of variables. We propose a novel divide and
is
number
of
nodes
and
k
is
the
degree.
Unfortunately,
conquer approach, Cluster and Infer Networks (CIN), for
scalable learning of large networks. CIN learns network when n becomes very large (several hundreds) and k is
structures in two steps: (a) pre-cluster network nodes and of any interesting size (2 ≤ k ≤ 10), as is the case for
learn networks per cluster, (b) revisit the cluster assign- genome-scale networks, these algorithms become expensive in time and memory.
ment of variables with poor neighborhoods.
We present a tractable approach to learn the structure
Results on networks with known topologies suggest
of
cellular networks represented as undirected graphithat CIN has substantial speed benefits, without signifcal
models. Our approach, Cluster and Infer Networks
icant performance loss. We applied our approach to a
(CIN),
clusters the nodes into smaller, possibly overmicroarray compendium of glucose-starved yeast cells.
lapping
groups and learns separate networks per cluster.
Analysis of the inferred structure identified several subBy
partitioning
the nodes into smaller groups, we avoid
networks involved in different metabolic processes. Our
searching
over
the
complete node set, resulting in runtime
results are consistent with existing literature and probenefits.
Because
the initial clustering may not be pervide fine-grained, network-level information of cellular
fect,
we
iteratively
reassign nodes to clusters to improve
response mechanism under starvation conditions.
the
quality
of
the
node
neighborhoods, repeating the proKeywords: Cellular networks, network structure infercedure
until
convergence.
The complete network strucence
ture is obtained by combining the networks inferred per
1 Introduction
cluster.
CIN is a meta-algorithm and can work with any existCellular adaptations essential for survival under changing
algorithm of learning network structure. We used the
ing environmental conditions are driven by a complex,
Markov
blanket search (MBS) algorithm [6], for learning
but coordinated, set of interactions among genes, proteins
the
structure
of undirected graphs (specifically, Markov
and metabolites. Identification of these interactions using
random
fields).
The MBS algorithm uses an information
whole-genome microarray data is crucial for understandtheoretic
score
to
identify the best neighborhood, Markov
ing the functional aspects of these networks and, thereblanket,
for
each
variable.
fore, how cells respond to changing environmental conditions.
We compare CIN with and without cluster reProbabilistic graphical models are well-known frame- assignment, against standard MBS that infers networks
works for modeling cellular networks [5]. Such models over the entire, unpartitioned node set. CIN gives signifare especially suitable for identifying higher-order depen- icant speed improvements, without significant accuracy
1
loss of the inferred structures, when evaluated on simulated data from networks with known structure. Adding
cluster reassignment further improves performance, still
keeping CIN faster than MBS with no clustering.
Nocluster
We evaluated the performance by comparing how well
different sub-graphs in the true networks matched subgraphs in the inferred networks. Specifically, we matched:
(a) edges in the true and inferred networks, (b) edges in
the true network to paths in the inferred network and viceversa, (c) shortest paths in the true and inferred networks,
(d) sub-graphs per vertex generated from its 1-step neighbors (1-N).
We also applied our algorithm to real microarray data
from the yeast compendium [3]. We generated the 1-step
neighborhood subgraphs (1-N) for the inferred network
and obtained gene ontology process annotation for each
subgraph.
20
15
10
5
0
ECOLI‐ALL ECOLI‐TF G75‐ALL
G75‐TF
G50‐ALL
G50‐TF
Datasets
Figure 1: Run time for different algorithms; lower runtimes
are better. Algorithms were compared using three networks of
known structure. Per network, two datasets were generated by
either perturbing all genes (ECOLI-ALL, G75-ALL, G50-ALL)
or only transcription factor genes (ECOLI-TF, G75-TF, G50TF).
3
Experiments
We evaluated the merit of our CIN approach on data generated from networks of known topology. We used three
networks, ECOLI, G75 and G50, with n = 188, 150, and
100 nodes respectively. ECOLI is obtained from the regulatory network of the bacteria E. coli [7], and G75 and
G50 are artificial regulatory networks generated from a
network simulator. These networks are sufficiently large
to require our pre-clustering approach, yet small enough
to enable structure learning via an approach without clustering. The k-means algorithm was used to cluster the
data.
Revisit
25
3.1
2
Norevisit
30
Time(sec)
We also applied CIN to a yeast microarray compendium of glucose-starvation induced stationary phase.
We were able to identify several subgraphs that were enriched in a large number of metabolic processes (e.g. glycolysis, acetyl-CoA metabolic process), characteristic of
yeast cells in starvation conditions. A large proportion
of subgraphs enriched in biological functions (p-value
< 10−5 ) were likely to be true positives (false discovery rate < 0.05). This indicates that our approach accurately and efficiently capture dependencies crucial for
understanding cellular stress response mechanisms.
Run0me
35
Results
CIN has significant speed benefits without substantial performance loss
We analyzed CIN in combination with the Markov blanket search (MBS) algorithm on three networks of known
topology. We considered three approaches to learn networks: (a) CIN with MBS with no cluster reassignment
(Norevisit), (b) CIN with MBS with cluster reassignment
(Revisit), and (c) standard MBS (Nocluster).
The running time of CIN with MBS, No-revisit and Revisit, is significantly smaller than standard MBS (Fig. 1).
The quality of the inferred networks using both CIN approaches are comparable to MBS on the complete variable
set (Fig. 2). We show results only for edge-path match.
Results from edge-edge, path-path and 1-N subgraphs are
similar.
Overall, we found that CIN had significant speed benefits over standard MBS. Revisiting clusters improves results at additional runtime cost, but is still faster than standard MBS.
3.2
CIN identifies subgraphs enriched in meaningful
biological processes from yeast stationary phase
compendium.
We applied CIN with MBS to a recently generated microarray compendium of yeast in stationary phase. As the
true network is not known for these data, we used Gene
Ontology (GO) for validation of the inferred subgraphs
[4]. At a stringent enrichment threshold (p-value <
10−5 ), 89% of the subgraphs, enriched in a GO term, were
Edge‐Path match
0.7
Nocluster
Norevisit
Revisit
0.6
Score
0.5
Isolation of quiescent and nonquiescent cells from
yeast stationary-phase cultures.
J Cell Biol,
174(1):89–100, July 2006.
0.4
[3] Anthony D. Aragon, Angelina L. Rodriguez, Osorio
Meirelles, Sushmita Roy, George S. Davidson, Chris
0.1
Allen, Ray Joe, Phillip Tapia, Don Benn, and Mar0
garet Werner-Washburne. Characterization of differECOLI‐ALL ECOLI‐TF G75‐ALL G75‐TF G50‐ALL G50‐TF
Datasets
entiated quiescent and non-quiescent cells in yeast
stationary-phase cultures. Molecular Biology of the
Figure 2: Match scores for different algorithms; higher scores
Cell, 2008.
are better. Nocluster: MBS algorithm without pre-clustering.
0.3
0.2
Norevisit: CIN with MBS without cluster reassignment. Revisit:
CIN with MBS with cluster reassignment.
[4] M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein,
H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S.
Dwight, J. T. Eppig, M. A. Harris, D. P. Hill, L. Issellikely to be true positives (FDR < 0.05). A fine-grained
Tarver, A. Kasarskis, S. Lewis, J. C. Matese, J. E.
analysis of the subgraphs implicated several metabolic
Richardson, M. Ringwald, G. M. Rubin, and G. Sherpathways (glycolysis, pyruvate metabolism, acetyl-CoA
lock. Gene ontology: tool for the unification of bimetabolic process), which are in agreement with known
ology. The Gene Ontology Consortium. Nat Genet,
literature [2]. We also found one subgraph involved in ag25(1):25–29, May 2000.
ing (SSD1, SNF1, YFL042C, PDR1, SCH9) and another
in cell cycle arrest (YDR196C, HRT3, FAR8, FAR3).
[5] Nir Friedman. Inferring cellular networks using probThese cells have been hypothesized as models for aging
abilistic graphical models. Science, 303:799–805,
studies, and we provide specific candidate networks that
2004.
can provide deeper understanding of aging and other cellular processes involved in proper stress response.
[6] Sushmita Roy, Terran Lane, and Margaret WernerWashburne. Learning structurally consistent undi4 Conclusion
rected probabilistic graphical models. Submitted to
We present here a tractable approach, Cluster and InNIPS, 2008.
fer Networks (CIN), for learning large networks from
genome-scale expression data. Our results on networks [7] Heladia Salgado, Socorro Gama-Castro, Martin
Peralta-Gil, Edgar Diaz-Peredo, Fabiola Sanchezwith known topology indicate that CIN has significant
Solano, Alberto Santos-Zavaleta, Irma Martinezspeed improvements without incurring substantial perforFlores, Veronica Jimenez-Jacinto, Cesar Bonavidesmance loss. Analysis of networks inferred from glucoseMartinez, Juan Segura-Salazar, Agustino Martinezstarvation yeast microarray data identified several biologAntonio, and Julio Collado-Vides. Regulondb (verical meaningful dependencies. We are currently applying
sion 5.0): Escherichia coli K-12 transcriptional reguCIN to microarray data under other environmental conlatory network, operon organization, and growth conditions, to get a better understanding of the conditionditions. Nucleic Acids Research, 34:D394, 2006.
specific rewiring of cellular networks.
References
[1] Pieter Abbeel, Daphne Koller, and Andrew Y. Ng.
Learning factor graphs in polynomial time and sample complexity. JMLR, 7:1743–1788, 2006.
[2] C. Allen, S. Büttner, A. D. Aragon, J. A. Thomas,
O. Meirelles, J. E. Jaetao, D. Benn, S. W. Ruby,
M. Veenhuis, F. Madeo, and M. Werner-Washburne.