Abstract
Link mining problems are characterized by high complexity (since linked objects are not statistically independent) and uncertainty (since data is noisy and incomplete). Thus they necessitate a modeling language that is both probabilistic and relational. Markov logic provides this by attaching weights to formulas in first-order logic and viewing them as templates for features of Markov networks. Many link mining problems can be elegantly formulated and efficiently solved using Markov logic. Inference algorithms for Markov logic draw on ideas from satisfiability testing, Markov chain Monte Carlo, belief propagation, and resolution. Learning algorithms are based on convex optimization, pseudo-likelihood, and inductive logic programming. Markov logic has been used successfully in a wide variety of link mining applications and is the basis of the open-source Alchemy system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This conversion includes the removal of existential quantifiers by Skolemization, which is not sound in general. However, in finite domains an existentially quantified formula can simply be replaced by a disjunction of its groundings.
- 2.
For simplicity we assume that all variables have the same domain. The extension to different domains is straightforward.
- 3.
This differs from MaxWalkSAT, which assigns random values to all atoms. However, the LazySAT initialization is a valid MaxWalkSAT initialization, and the two give very similar results empirically. Given the same initialization, the two algorithms will produce exactly the same results.
References
J. Besag. Statistical analysis of non-lattice data. The Statistician, 24:179–195, 1975.
M. Collins. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 1–8, ACL Philadelphia, PA, 2002.
M. Craven and S. Slattery. Relational learning with statistical predicate invention: Better models for hypertext. Machine Learning, 43(1/2):97–119, 2001.
P. Damien, J. Wakefield, and S. Walker. Gibbs sampling for Bayesian non-conjugate and hierarchical models by auxiliary variables. Journal of the Royal Statistical Society, Series B, 61:331–344, 1999.
L. De Raedt and L. Dehaspe. Clausal discovery. Machine Learning, 26:99–146, 1997.
S. Della Pietra, V. Della Pietra, and J. Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19:380–392, 1997.
P. Domingos and D. Lowd. Markov Logic: An Interface Layer for AI. Morgan & Claypool, San Rafael, CA, 2009.
P. Domingos and M. Richardson. Mining the network value of customers. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 57–66, ACM Press, San Francisco, CA, 2001.
M. R. Genesereth and N. J. Nilsson. Logical Foundations of Artificial Intelligence. Morgan Kaufmann, San Mateo, CA, 1987.
W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, editors. Markov Chain Monte Carlo in Practice. Chapman and Hall, London, 1996.
A. Jaimovich, O. Meshi, and N. Friedman. Template based inference in symmetric relational Markov random fields. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence, pages 191–199, AUAI Press, Vancouver, BC, 2007.
H. Kautz, B. Selman, and Y. Jiang. A general stochastic approach to solving problems with hard and soft constraints. In D. Gu, J. Du, and P. Pardalos, editors, The Satisfiability Problem: Theory and Applications, pages 573–586. American Mathematical Society, New York, NY, 1997.
S. Kok and P. Domingos. Learning the structure of Markov logic networks. In Proceedings of the 22nd International Conference on Machine Learning, pages 441–448, ACM Press, Bonn, 2005.
S. Kok and P. Domingos. Statistical predicate invention. In Proceedings of the 24th International Conference on Machine Learning, pages 433–440, ACM Press, Corvallis, OR, 2007.
S. Kok and P. Domingos. Extracting semantic networks from text via relational clustering. In Proceedings of the 19th European Conference on Machine Learning, pages 624–639, Springer, Antwerp, 2008.
S. Kok and P. Domingos. Hypergraph lifting for structure learning in Markov logic networks. In Proceedings of the 26th International Conference on Machine Learning, ACM Press, Montreal, QC, 2009.
S. Kok, M. Sumner, M. Richardson, P. Singla, H. Poon, D. Lowd, and P. Domingos. The Alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington, Seattle, WA, 2007. http://alchemy.cs.washington.edu.
N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chichester, 1994.
D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(3):503–528, 1989.
J. W. Lloyd. Foundations of Logic Programming. Springer, Berlin, 1987.
D. Lowd and P. Domingos. Efficient weight learning for Markov logic networks. In Proceedings of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, pages 200–211, Springer, Warsaw, 2007.
D. J. Lunn, A. Thomas, N. Best, and D. Spiegelhalter. WinBUGS — a Bayesian modeling framework: concepts, structure, and extensibility. Statistics and Computing, 10:325–337, 2000.
L. Mihalkova and R. Mooney. Bottom-up learning of Markov logic network structure. In Proceedings of the 24th International Conference on Machine Learning, pages 625–632, ACM Press, Corvallis, OR, 2007.
M. Møller. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6:525–533, 1993.
A. Nath and P. Domingos. A language for relational decision theory. In Proceedings of the International Workshop on Statistical Relational Learning, Leuven, 2009.
J. Nocedal and S. Wright. Numerical Optimization. Springer, New York, NY, 2006.
J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, CA, 1988.
H. Poon and P. Domingos. Sound and efficient inference with probabilistic and deterministic dependencies. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 458–463, AAAI Press, Boston, MA, 2006.
H. Poon, P. Domingos, and M. Sumner. A general method for reducing the complexity of relational inference and its application to MCMC. In Proceedings of the 23rd National Conference on Artificial Intelligence, pages 1075–1080, AAAI Press, Chicago, IL, 2008.
J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5:239–266, 1990.
M. Richardson and P. Domingos. Mining knowledge-sharing sites for viral marketing. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 61–70, ACM Press, Edmonton, AB, 2002.
M. Richardson and P. Domingos. Markov logic networks. Machine Learning, 62:107–136, 2006.
S. Riedel. Improving the accuracy and efficiency of MAP inference for Markov logic. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pages 468–475, AUAI Press, Helsinki, 2008.
J. A. Robinson. A machine-oriented logic based on the resolution principle. Journal of the ACM, 12:23–41, 1965.
D. Roth. On the hardness of approximate reasoning. Artificial Intelligence, 82:273–302, 1996.
P. Singla and P. Domingos. Discriminative training of Markov logic networks. In Proceedings of the 20th National Conference on Artificial Intelligence, pages 868–873, AAAI Press, Pittsburgh, PA, 2005.
P. Singla and P. Domingos. Entity resolution with Markov logic. In Proceedings of the 6th IEEE International Conference on Data Mining, pages 572–582, IEEE Computer Society Press, Hong Kong, 2006.
P. Singla and P. Domingos. Memory-efficient inference in relational domains. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 488–493, AAAI Press, Boston, MA, 2006.
P. Singla and P. Domingos. Markov logic in infinite domains. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence, pages 368–375, AUAI Press, Vancouver, BC, 2007.
P. Singla and P. Domingos. Lifted first-order belief propagation. In Proceedings of the 23rd National Conference on Artificial Intelligence, pages 1094–1099, AAAI Press, Chicago, IL, 2008.
A. Srinivasan. The Aleph manual. Technical report, Computing Laboratory, Oxford University, 2000.
B. Taskar, P. Abbeel, and D. Koller. Discriminative probabilistic models for relational data. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, pages 485–492, Morgan Kaufmann, Edmonton, AB, 2002.
S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge, 1994.
W. Wei, J. Erenrich, and B. Selman. Towards efficient sampling: Exploiting random walk strategies. In Proceedings of the 19th National Conference on Artificial Intelligence, pages 670–676, AAAI Press, San Jose, CA, 2004.
M. Wellman, J. S. Breese, and R. P. Goldman. From knowledge bases to decision models. Knowledge Engineering Review, 7:35–53, 1992.
J. S. Yedidia, W. T. Freeman, and Y. Weiss. Generalized belief propagation. In T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 689–695. MIT Press, Cambridge, MA, 2001.
Acknowledgments
This research was partly supported by ARO grant W911NF-08-1-0242, DARPA contracts FA8750-05-2-0283, FA8750-07-D-0185, HR0011-06-C-0025, HR0011-07-C-0060 and NBCH-D030010, NSF grants IIS-0534881 and IIS-0803481, ONR grants N-00014-05-1-0313 and N00014-08-1-0670, an NSF CAREER Award (first author), a Sloan Research Fellowship (first author), an NSF Graduate Fellowship (second author), and a Microsoft Research Graduate Fellowship (second author). The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ARO, DARPA, NSF, ONR, or the US Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Domingos, P. et al. (2010). Markov Logic: A Language and Algorithms for Link Mining. In: Yu, P., Han, J., Faloutsos, C. (eds) Link Mining: Models, Algorithms, and Applications. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-6515-8_5
Download citation
DOI: https://doi.org/10.1007/978-1-4419-6515-8_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-6514-1
Online ISBN: 978-1-4419-6515-8
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)