article

Same-decision probability: A confidence measure for threshold-based decisions

Authors:

Adnan DarwicheAuthors Info & Claims

International Journal of Approximate Reasoning, Volume 53, Issue 9

Pages 1415 - 1428

https://doi.org/10.1016/j.ijar.2012.04.005

Published: 01 December 2012 Publication History

Abstract

We consider in this paper the robustness of decisions based on probabilistic thresholds. To this effect, we propose the same-decision probability as a query that can be used as a confidence measure for threshold-based decisions. More specifically, the same-decision probability is the probability that we would have made the same threshold-based decision, had we known the state of some hidden variables pertaining to our decision. We study a number of properties about the same-decision probability. First, we analyze its computational complexity. We then derive a bound on its value, which we can compute using a variable elimination algorithm that we propose. Finally, we consider decisions based on noisy sensors in particular, showing through examples that the same-decision probability can be used to reason about threshold-based decisions in a more refined way.

References

[1]

J.M. Agosta, T. Gardos, M.J. Druzdel, Query-based diagnostics, in: The Fourth European Workshop on Probabilistic Graphical Models (PGM), 2008.

[2]

J.M. Agosta, O.Z. Khan, P. Poupart, Evaluation results for a query-based diagnostics application, in: The Fifth European Workshop on Probabilistic Graphical Models (PGM), 2010.

[3]

Allender, E. and Wagner, K.W., Counting hierarchies: polynomial time and constant depth circuits. Bulletin of the EATCS. v40. 182-194.

[4]

Chan, H., Sensitivity Analysis of Probabilistic Graphical Models: Theoretical Results and Their Applications on Bayesian Network Modeling and Inference. 2009. VDM Verlag.

[5]

T. Charitos, L.C. van der Gaag, Sensitivity analysis for threshold decision making with dynamic networks, in: Proceedings of the 22nd Conference in Uncertainty in Artificial Intelligence (UAI), 2006.

[6]

Darwiche, A., Modeling and Reasoning with Bayesian Networks. 2009. Cambridge University Press.

[7]

R. Dechter, Bucket elimination: a unifying framework for probabilistic inference, in: Proceedings of the Twelfth Annual Conference on Uncertainty in Artificial Intelligence (UAI), 1996, pp. 211--219.

[8]

M.J. Druzdzel, H.A. Simon, Causality in Bayesian belief networks, in: Proceedings of the Ninth Annual Conference on Uncertainty in Artificial Intelligence (UAI), 1993, pp. 3--11.

Digital Library

[9]

Friedman, N., Geiger, D. and Goldszmidt, M., Bayesian network classifiers. Machine Learning. v29 i2--3. 131-163.

[10]

Hamscher, W., Console, L. and de Kleer, J., Readings in Model-based Diagnosis. 1992. Morgan Kaufmann.

[11]

Heckerman, D., Breese, J.S. and Rommelse, K., Decision-theoretic troubleshooting. Communications of the ACM. v38 i3. 49-57.

[12]

Heckerman, D., Geiger, D. and Chickering, D.M., Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning. v20 i3. 197-243.

[13]

Huang, C. and Darwiche, A., Inference in belief networks: a procedural guide. International Journal of Approximate Reasoning. v15 i3. 225-263.

[14]

Jensen, F.V., Lauritzen, S. and Olesen, K., Bayesian updating in recursive graphical models by local computation. Computational Statistics Quarterly. v4. 269-282.

[15]

Krause, A. and Guestrin, C., Optimal value of information in graphical models. Journal of Artificial Intelligence Research. v35. 557-591.

[16]

J. Kwisthout, The computational complexity of probabilistic networks, Ph.D. thesis, University of Utrecht, 2009.

[17]

J. Kwisthout, L.C. van der Gaag, The computational complexity of sensitivity analysis and parameter tuning, in: Proceedings of the 24th Conference in Uncertainty in Artificial Intelligence (UAI), 2008, pp. 349--356.

[18]

Lauritzen, S.L. and Spiegelhalter, D.J., Local computations with probabilities on graphical structures and their application to expert systems. Journal of Royal Statistics Society, Series B. v50 i2. 157-224.

[19]

Littman, M.L., Goldsmith, J. and Mundhenk, M., The computational complexity of probabilistic planning. Journal of Artificial Intelligence Research. v9. 1-36.

[20]

T.-C. Lu, K.W. Przytula, Focusing strategies for multiple fault diagnosis, in: Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS), 2006, pp. 842--847.

[21]

Park, J. and Darwiche, A., Complexity results and approximation strategies for MAP explanations. Journal of Artificial Intelligence Research. v21. 101-133.

[22]

Pauker, S.G. and Kassirer, J.P., The threshold approach to clinical decision making. New England Journal of Medicine. v302 i20. 1109-1117.

[23]

Pearl, J., Causality: Models, Reasoning and Inference. 2009. 2nd ed. Cambridge University Press.

[24]

Toda, S., PP is as hard as the polynomial-time hierarchy. SIAM Journal on Computing. v20 i5. 865-877.

[25]

C. Umans, Approximability and completeness in the polynomial hierarchy, Ph.D. thesis, University of California, Berkeley, 2000.

[26]

van der Gaag, L., Renooij, S. and Coupé, V., Sensitivity analysis of probabilistic networks. In: Lucas, P., Gámez, J., Salmerón, A. (Eds.), Studies in Fuzziness and Soft Computing, vol. 214. Springer, Berlin/Heidelberg. pp. 103-124.

[27]

L.C. van der Gaag, V.M.H. Coupé, Sensitive analysis for threshold decision making with Bayesian belief networks, in: 6th Congress of the Italian Association for Artificial Intelligence (AI*IA), 1999, pp. 37--48.

[28]

Wagner, K.W., The complexity of combinatorial problems with succinct input representation. Acta Informatica. v23 i3. 325-356.

[29]

Zhang, N.L. and Poole, D., Exploiting causal independence in Bayesian network inference. Journal of Artificial Intelligence Research. v5. 301-328.

Cited By

Ji CDarwiche A(2023)A New Class of Explanations for Classifiers with Non-binary FeaturesLogics in Artificial Intelligence10.1007/978-3-031-43619-2_8(106-122)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43619-2_8
Tantau T(2022)On the satisfaction probability of k-CNF formulasProceedings of the 37th Computational Complexity Conference10.4230/LIPIcs.CCC.2022.2(1-27)Online publication date: 20-Jul-2022
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2022.2
Rahman TRouhani SGogate VRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Novel upper bounds for the constrained most probable explanation taskProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540997(9613-9624)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540997
Show More Cited By

Same-decision probability: A confidence measure for threshold-based decisions
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning

Recommendations

Bayesian network inference using marginal trees

Variable elimination (VE) and join tree propagation (JTP) are two alternatives to inference in Bayesian networks (BNs). VE, which can be viewed as one-way propagation in a join tree, answers each query against the BN meaning that computation can be ...
Belief-Informed Robust Decision Making (BIRDM): Assessing changes in decision robustness due to changing distributions of deep uncertainties
Abstract
Robust Decision Making (RDM) is an established framework for decision making under deep uncertainty. RDM relies on the idea of scenario neutrality, namely that decision robustness is not affected by how scenarios are generated if these ...
Highlights
- The Belief-Informed Robust Decision Making (BIRDM) framework integrates probabilistic information into Robust Decision Making.
A join tree probability propagation architecture for semantic modeling

We propose the first join tree (JT) propagation architecture that labels the probability information passed between JT nodes in terms of conditional probability tables (CPTs) rather than potentials. By modeling the task of inference involving evidence, ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Approximate Reasoning

International Journal of Approximate Reasoning Volume 53, Issue 9

December, 2012

127 pages

ISSN:0888-613X

Issue’s Table of Contents

Copyright © Elsevier Inc. © 2012.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 December 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ji CDarwiche A(2023)A New Class of Explanations for Classifiers with Non-binary FeaturesLogics in Artificial Intelligence10.1007/978-3-031-43619-2_8(106-122)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43619-2_8
Tantau T(2022)On the satisfaction probability of k-CNF formulasProceedings of the 37th Computational Complexity Conference10.4230/LIPIcs.CCC.2022.2(1-27)Online publication date: 20-Jul-2022
https://dl.acm.org/doi/10.4230/LIPIcs.CCC.2022.2
Rahman TRouhani SGogate VRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Novel upper bounds for the constrained most probable explanation taskProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3540997(9613-9624)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3540997
Rouhani SRahman TGogate VLarochelle HRanzato MHadsell RBalcan MLin H(2020)A novel approach for constrained optimization in graphical modelsProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496726(11949-11960)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496726
Goldstein OKachuee MKarkkainen KSarrafzadeh M(2020)Target-Focused Feature Selection Using Uncertainty Measurements in Healthcare DataACM Transactions on Computing for Healthcare10.1145/33836851:3(1-17)Online publication date: 30-May-2020
https://dl.acm.org/doi/10.1145/3383685
Darwiche ASuciu DTao YWei Z(2020)Three Modern Roles for Logic in AIProceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3375395.3389131(229-243)Online publication date: 14-Jun-2020
https://dl.acm.org/doi/10.1145/3375395.3389131
Khosravi PChoi YLiang YVergari ABroeck GWallach HLarochelle HBeygelzimer Ad'Alché-Buc FFox E(2019)On tractable computation of expected predictionsProceedings of the 33rd International Conference on Neural Information Processing Systems10.5555/3454287.3455289(11169-11180)Online publication date: 8-Dec-2019
https://dl.acm.org/doi/10.5555/3454287.3455289
Shih AChoi ADarwiche A(2018)A symbolic approach to explaining Bayesian network classifiersProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304652.3304719(5103-5111)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304652.3304719
Choi YVan Den Broeck G(2018)On robust trimming of Bayesian network classifiersProceedings of the 27th International Joint Conference on Artificial Intelligence10.5555/3304652.3304705(5002-5009)Online publication date: 13-Jul-2018
https://dl.acm.org/doi/10.5555/3304652.3304705
Chen SChoi ADarwiche A(2018)Algorithms and applications for the same-decision probabilityJournal of Artificial Intelligence Research10.5555/2655713.265573049:1(601-633)Online publication date: 20-Dec-2018
https://dl.acm.org/doi/10.5555/2655713.2655730
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents