Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.5555/3540261.3540997guideproceedingsArticle/Chapter ViewAbstractPublication PagesnipsConference Proceedingsconference-collections
research-article

Novel upper bounds for the constrained most probable explanation task

Published: 06 December 2021 Publication History

Abstract

We propose several schemes for upper bounding the optimal value of the constrained most probable explanation (CMPE) problem. Given a set of discrete random variables, two probabilistic graphical models defined over them and a real number q, this problem involves finding an assignment of values to all the variables such that the probability of the assignment is maximized according to the first model and is bounded by q w.r.t. the second model. In prior work, it was shown that CMPE is a unifying problem with several applications and special cases including the nearest assignment problem, the decision preserving most probable explanation task and robust estimation. It was also shown that CMPE is NP-hard even on tractable models such as bounded treewidth networks and is hard for integer linear programming methods because it includes a dense global constraint. The main idea in our approach is to simplify the problem via Lagrange relaxation and decomposition to yield either a knapsack problem or the unconstrained most probable explanation (MPE) problem, and then solving the two problems, respectively using specialized knapsack algorithms and mini-buckets based upper bounding schemes. We evaluate our proposed scheme along several dimensions including quality of the bounds and computation time required on various benchmark graphical models and how it can be used to find heuristic, near-optimal feasible solutions in an example application pertaining to robust estimation and adversarial attacks on classifiers.

Supplementary Material

Additional material (3540261.3540997_supp.pdf)
Supplemental material.

References

[1]
Tobias Achterberg. SCIP: solving constraint integer programs. Mathematical Programming Computation, 1(1):1–41, July 2009.
[2]
Tobias Achterberg, Timo Berthold, Thorsten Koch, and Kati Wolter. Constraint integer programming: A new approach to integrate cp and mip. In International Conference on Integration of Artificial Intelligence (AI) and Operations Research (OR) Techniques in Constraint Programming, pages 6–20. Springer, 2008.
[3]
Walid Ben-Ameur, Mohamed-Ahmed Mohamed-Sidi, and José Neto. The k-separator problem. In International Computing and Combinatorics Conference, pages 337–348. Springer, 2013.
[4]
Stefano Bistarelli, Rosella Gennari, and Francesca Rossi. Constraint propagation for soft constraints: Generalization and termination conditions. In International Conference on Principles and Practice of Constraint Programming, pages 83–97. Springer, 2000.
[5]
Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, March 2004.
[6]
Arthur Choi and Adnan Darwiche. Relax, Compensate and Then Recover. In Takashi Onada, Daisuke Bekki, and Eric McCready, editors, New Frontiers in Artificial Intelligence, pages 167–180. Springer Berlin Heidelberg, 2011.
[7]
Arthur Choi, Yexiang Xue, and Adnan Darwiche. Same-decision probability: A confidence measure for threshold-based decisions. International Journal of Approximate Reasoning, 53(9):1415–1428, 2012.
[8]
Adnan Darwiche. A Differential Approach to Inference in Bayesian Networks. Journal of the ACM, 50:280–305, 2003.
[9]
Adnan Darwiche. Modeling and Reasoning with Bayesian Networks. Cambridge University Press, 2009.
[10]
Adnan Darwiche and Auguste Hirth. On the reasons behind decisions. In Twenty Fourth European Conference on Artificial Intelligence, volume 325 of Frontiers in Artificial Intelligence and Applications, pages 712–720. IOS Press, 2020.
[11]
Rina Dechter and Irina Rish. Mini-buckets: A general scheme for bounded inference. Journal of the ACM (JACM), 50(2):107–153, 2003.
[12]
Martin E Dyer. An o (n) algorithm for the multiple-choice knapsack linear program. Mathematical programming, 29(1):57–63, 1984.
[13]
Gal Elidan and Amir Globerson. The 2010 UAI approximate inference challenge. Available online at: http://www.cs.huji.ac.il/project/UAI10/index.php, 2010.
[14]
Amir Globerson and Tommi Jaakkola. Fixing max-product: Convergent message passing algorithms for map lp-relaxations. Advances in neural information processing systems, 20:553–560, 2007.
[15]
Vibhav Gogate. Results of the 2014 UAI competition. http://www.hlt.utdallas.edu/~vgogate/uai14-competition/index.html, 2014.
[16]
David Gunning and David Aha. DARPA's Explainable Artificial Intelligence (XAI) Program. AI Magazine, 40(2):44–58, June 2019.
[17]
LLC Gurobi Optimization. Gurobi Optimizer Reference Manual. 2021.
[18]
Alexander T Ihler, Natalia Flerova, Rina Dechter, and Lars Otten. Join-graph based cost-shifting schemes. arXiv preprint arXiv:1210.4878, 2012.
[19]
Hans Kellerer, Ulrich Pferschy, and David Pisinger. Knapsack problems. Springer, 2004.
[20]
Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.
[21]
Nikos Komodakis, Nikos Paragios, and Georgios Tziritas. Mrf optimization via dual decomposition: Message-passing revisited. In 2007 IEEE 11th International Conference on Computer Vision, pages 1–8. IEEE, 2007.
[22]
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010.
[23]
Daniel Lowd and Jesse Davis. Learning Markov network structure with decision trees. In Proceedings of the 10th International Conference on Data Mining, pages 334–343, 2010.
[24]
Radu Marinescu and Rina Dechter. Best-First AND/OR Search for Most Probable Explanations. CoRR, abs/1206.5268, 2012.
[25]
Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[26]
David Pisinger. Linear time algorithms for knapsack problems with bounded weights. Journal of Algorithms, 33(1):1–14, 1999.
[27]
David Pisinger. Dynamic programming on the word ram. Algorithmica, 35(2):128–145, 2003.
[28]
David Pisinger. Where are the hard knapsack problems? Computers & Operations Research, 32(9):2271–2284, 2005.
[29]
Tahrima Rahman, Shasha Jin, and Vibhav Gogate. Cutset Bayesian Networks: A New Representation for Learning Rao-Blackwellised Graphical Models. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pages 5751–5757, 2019.
[30]
Tahrima Rahman, Prasanna Kothalkar, and Vibhav Gogate. Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD. Proceedings, Part II, pages 630–645, 2014.
[31]
Sara Rouhani, Tahrima Rahman, and Vibhav Gogate. Algorithms for the Nearest Assignment Problem. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pages 5096–5102, 2018.
[32]
Sara Rouhani, Tahrima Rahman, and Vibhav Gogate. A novel approach for constrained optimization in graphical models. In Advances in Neural Information Processing Systems, volume 33, pages 11949–11960, 2020.
[33]
Thomas Schiex. Arc consistency for soft constraints. In International Conference on Principles and Practice of Constraint Programming, pages 411–425. Springer, 2000.
[34]
Martin J Wainwright, Tommi S Jaakkola, and Alan S Willsky. Map estimation via agreement on trees: message-passing and linear programming. IEEE transactions on information theory, 51(11):3697–3717, 2005.
[35]
Laurence A. Wolsey. Integer programming. Wiley-Interscience series in discrete mathematics and optimization. J. Wiley, New York, 1998.
[36]
Eitan Zemel. An O(n) algorithm for the linear multiple choice knapsack problem and related problems. Information Processing Letters, 18(3):123–128, 1984.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems
December 2021
30517 pages

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 06 December 2021

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media