Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

A comparative study on swarm intelligence for structure learning of Bayesian networks

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A Bayesian network (BN) is an important probabilistic model in the field of artificial intelligence and a powerful formalism used to describe uncertainty in the real world. As science and technology develop, considerable data on complex systems have been acquired by various means, which presents a significant challenge regarding how to accurately and robustly learn a network structure for a complex system. To address this challenge, many BN structure learning methods based on swarm intelligence have been developed. In this study, we perform a systematic comparison of three typical methods based on ant colony optimization, artificial bee colony algorithm, and bacterial foraging optimization. First, we analyze and summarize their main characteristics from the perspective of stochastic searching. Second, we conduct thorough experimental comparisons to examine the roles of different mechanisms in each method by means of multiaspect metrics, i.e., the K2 score, structural differences, and execution time. Next, we perform further experiments to validate the robustness of different algorithms on some benchmark data sets with noise. Finally, we present the prospects and references for researchers who are engaged in learning BN networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Alcobó JR (2004) Incremental hill-climbing search applied to Bayesian network structure learning. In: Proceedings of the 15th European conference on machine learning, Pisa, Italy

  • Aouay S, Jamoussi S, Ayed YB (2013) Particle swarm optimization based method for Bayesian network structure learning. In: Proceedings of the 5th international conference on modeling, simulation and applied optimization (ICMSAO’13), pp 1–6

  • Bang-Jensen J, Gutin GZ (2008) Digraphs: theory, algorithms and applications. Springer, Berlin

    MATH  Google Scholar 

  • Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF (1989) The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. AIME, Lecture notes in medical informatics, vol 38, pp 247–256

  • Bielza C, Larraňaga P (2014) Bayesian networks in neuroscience: a survey. Front Comput Neurosci 8:131

    Article  MATH  Google Scholar 

  • Buntine W (1996) A guide to the literature on learning probabilistic networks from data. IEEE Trans Knowl Data Eng 8(2):195–210

    Article  Google Scholar 

  • Castro PAD, Von Zuben FJ (2005) An immune-inspired approach to Bayesian networks. In: Proceedings of the 5th international conference on hybrid intelligent systems (HIS’05), IEEE, pp 6–9

  • Cheng J, Bell DA, Liu W (1997) Learning belief networks from data: an information theory based approach. In: Proceedings of the 6th international conference on information and knowledge management (CIKM’97), pp 325–331

  • Chickering DM, Geiger D, Heckerman D (1994) Learning Bayesian networks is NP-Hard. Technical Report MSR-TR-94-17, Microsoft Research

  • Cooper GF, Herskovits E (1992) A Bayesian method for the introduction of probabilistic networks from data. Mach Learn 9(4):309–347

    MATH  Google Scholar 

  • Daly R, Shen Q, Aitken S (2011) Learning Bayesian networks: approaches and issues. Knowl Eng Rev 26(02):99–157

    Article  Google Scholar 

  • Daly R, Shen Q (2009) Learning Bayesian network equivalence classes with ant colony optimization. J Artif Intell Res 35(1):391–447

    MathSciNet  MATH  Google Scholar 

  • De Campos LM, Puerta JM (2001) Stochastic local and distributed search algorithms for learning belief networks. In: Proceedings of third international symposium on adaptive systems: evolutionary computation and probabilistic graphical model, pp 109–115

  • De Campos LM, Fernández-Luna JM, Gámez JA, Puerta JM (2002) Ant colony optimization for learning Bayesian networks. Int J Approx Reason 31(3):291–311

    Article  MathSciNet  MATH  Google Scholar 

  • De Campos LM, Gámez JA, Puerta JM (2008) Learning Bayesian networks by ant colony optimisation: searching in two different spaces. Mathw Soft Comput 9(3):251–268

    MathSciNet  MATH  Google Scholar 

  • De Campos LM, Huete JF (2000) A new approach for learning belief networks using independence criteria. Int J Approx Reason 24(1):11–37

    Article  MathSciNet  MATH  Google Scholar 

  • De CT Gomes L, De Sousa JS, Bezerra GB, De Castro LN, Von Zuben FJ (2003) Copt-aiNet and the gene ordering problem. In: Second Brazilian workshop on bioinformatics

  • Dorigo M, Maniezzo VA, Colorni A (1996) The ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern B 26(1):29–41

    Article  Google Scholar 

  • Eberhart RC, Shi Y, Kennedy J (2001) Swarm intelligence. Morgan Kaufmann, Los Altos

    Google Scholar 

  • Heckerman DE, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Mach Learn 20(3):197–243

    MATH  Google Scholar 

  • Heckerman D (1998) A tutorial on learning Bayesian networks: learning in graphical models. Kluwer, Dordrecht

    Book  MATH  Google Scholar 

  • Heng XC, Qin Z, Tian L, Shao LP (2007) Learning Bayesian network structures with discrete particle swarm optimization algorithm. In: IEEE symposium on foundations of computational intelligence (FOCI’07). IEEE, pp 47–52

  • Henrion M (1986) Propagating uncertainty by logic sampling in Bayes networks. In: Proceedings of the AAAI workshop on uncertainty in aritificial intelligence, Philadelphia

  • Ji JZ, Wei HK, Liu CN, Yin BC (2013) Artificial bee colony algorithm merged with pheromone communication mechanism for the 0–1 multidimensional knapsack problem. Math Probl Eng 2013, Article ID 676275

  • Ji JZ, Zhang HX, Hu RB, Liu CN (2008) A tabu-search based Bayesian network structure learning algorithm. Beijing Gongye Daxue Xuebao 37(8):1274–1280

    Google Scholar 

  • Ji JZ, Zhang HX, Hu RB, Liu CN (2009) A Bayesian network learning algorithm based on independence test and ant colony optimization. Acta Autom Sin 35(3):281–288

    Article  MATH  Google Scholar 

  • Ji JZ, Hu RB, Zhang HX, Liu CN (2011) A hybrid method for learning the Bayesian networks based on ant colony optimization. Appl Soft Comput 11(4):3373–3384

    Article  Google Scholar 

  • Ji JZ, Wei HK, Liu CN (2013) An artificial bee colony algorithm for learning Bayesian networks. Soft Comput 6:983–994

    Article  Google Scholar 

  • Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering Department, Erciyes University, Turkey

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the 1995 IEEE international conference on neural networks, pp 1942–1948

  • Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge

    MATH  Google Scholar 

  • Larrañaga P, Karshenas H, Bielza C, Santana R (2013) A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf Sci 233(1):109–125

    Article  MathSciNet  MATH  Google Scholar 

  • Li XL (2010) A particle swarm optimization and immune theory-based algorithm for structure learning of Bayesian networks. Int J Database Theory Appl 3(2):61–69

    Google Scholar 

  • Martens D, Baesens B, Fawcett T (2011) Editorial survey: swarm intelligence for data mining. Mach Learn 82(1):1–42

    Article  MathSciNet  Google Scholar 

  • Mori M, Tsukiyama M, Fukuda T (1993) Immune algorithm with searching diversity and its application to resource allocation problem. Trans Inst Electron Eng Jpn C 113:872–878

    Google Scholar 

  • Mumford JA, Ramsey JD (2014) Bayesian networks for fMRI: a primer. NeuroImage 86:573–582

    Article  Google Scholar 

  • Needham CJ, Bradford JR, Bulpitt AJ, Westhead DR (2007) A primer on learning in Bayesian networks for computational biology. PLoS Comput Biol 3(8):e129

    Article  Google Scholar 

  • Passino KM (2002) Biomimicry of bacterial foraging for distributed optimization and control. Control Syst 22(3):52–67

    Article  Google Scholar 

  • Pearl J (1988) Reasoning in intelligent systems: networks of plausible inference. Morgan Kaufamann, San Mateo

    MATH  Google Scholar 

  • Pinto PC, Nägele A, Dejori M, Runkler TA, Sousa JMC (2009) Using a local discovery ant algorithm for Bayesian network structure learning. IEEE Trans Evol Comput 13(4):767–779

    Article  Google Scholar 

  • Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471

    Article  MATH  Google Scholar 

  • Robinson RW (1977) Counting unlabeled acyclic digraphs. In: Little CHC (ed) Combinatorial mathematics V. Springer, Berlin

  • Romero C, Ventura S (2010) Educational data mining: a review of the state of the art. IEEE Trans Syst Man Cybern C Appl Rev 40(6):601–618

    Article  Google Scholar 

  • Rubio-Largo Á, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM (2012) A comparative study on multiobjective swarm intelligence for the routing and wavelength assignment problem. IEEE Trans Syst Man Cybern C Appl Rev 42(6):1644–1655

    Article  Google Scholar 

  • Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

    Article  MathSciNet  MATH  Google Scholar 

  • Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Visi Comput 27(6):803–816

    Article  Google Scholar 

  • Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG (1993) Bayesian analysis in expert systems. Stat Sci 8(3):219–247

    Article  MathSciNet  MATH  Google Scholar 

  • Suzuki J (1999) Learning Bayesian belief networks based on the minimum description length principle: basic properties. IEICE Trans Fundam Electron Commun Comput Sci 82(10):2237–2245

    Google Scholar 

  • Tsamardinos I, Brown LE, Aliferis CF (2006) The max–min hill-climbing Bayesian network structure learning algorithm. Mach Learn 65(1):31–78

    Article  Google Scholar 

  • Tsamardinos I, Aliferis CF, Statnikov A (2003) Scaling-up Bayesian network learning to thousands of variables using local learning technique. Technical Report DSL TR-03-02, Department of Biomedical Informatics, Vanderbilt University

  • Wang T, Yang J (2010) A heuristic method for learning Bayesian networks using discrete particle swarm optimization. Knowl Inf Syst 24(2):269–281

    Article  Google Scholar 

  • Weber P, Medina-Oliva G, Simon C, Iung B (2012) Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Eng Appl Artif Intell 25(4):671–682

    Article  Google Scholar 

  • Wong ML, Lam W, Leung KS (1999) Using evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 21(2):174–178

    Article  Google Scholar 

  • Wu Y, McCall J, Corne D (2010) Two novel ant colony optimization approaches for Bayesian network structure learning. In: Proceedings of the 2010 IEEE congress on evolutionary computation (CEC’10), pp 1–7

  • Yang CC, Ji JZ, Liu JM, Liu JD, Yin BC (2016) Structural learning of Bayesian networks by bacterial foraging optimization algorithm. Int J Approx Reason 69:147–167

  • Zhao J, Sun J, Xu W, Zhou D (2009) Structure learning of Bayesian networks based on discrete binary quantum-behaved particle swarm optimization algorithm. In: Proceedings of the 5th international conference on natural computation (ICNC’09), 86-90

Download references

Acknowledgments

We would like to thank the anonymous referees for their many valuable suggestions and comments. We thank the authors of the corresponding algorithms for sharing the binary executable systems. This work is partly supported by the NSFC Research Program (61375059, 61332016), the National “973” Key Basic Research Program of China (2014CB744601), the Specialized Research Fund for the Doctoral Program of Higher Education (20121103110031), and the Beijing Municipal Education Research Plan key Project (Beijing Municipal Fund Class B) (KZ201410005004).

Author information

Authors and Affiliations

  1. Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, College of Computer Science and Technology, Beijing University of Technology, Beijing, China

    Junzhong Ji, Cuicui Yang, Jinduo Liu & Baocai Yin

  2. Department of Computer Science and Technology, Hong Kong Baptist University, Kowloon Tong, Hong Kong

    Jiming Liu

Authors
  1. Junzhong Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cuicui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinduo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Baocai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junzhong Ji.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, J., Yang, C., Liu, J. et al. A comparative study on swarm intelligence for structure learning of Bayesian networks. Soft Comput 21, 6713–6738 (2017). https://doi.org/10.1007/s00500-016-2223-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2223-x

Keywords

Search

Navigation