A Method for Debugging Process Discovery Pipelines to Analyze the Consistency of Model Properties
Authors: 
Christopher Klinkmüller, 
Alexander Seeliger, Richard Müller, Luise Pufahl, 
Ingo WeberAuthors Info & Claims
Christopher Klinkmüller, Alexander Seeliger, Richard Müller, Luise Pufahl, Ingo WeberAuthors Info & ClaimsPages 65 - 84
Abstract
Event logs have become a valuable information source for business process management, e.g., when analysts discover process models to inspect the process behavior and to infer actionable insights. To this end, analysts configure discovery pipelines in which logs are filtered, enriched, abstracted, and process models are derived. While pipeline operations are necessary to manage log imperfections and complexity, they might, however, influence the nature of the discovered process model and its properties. Ultimately, not considering this possibility can negatively affect downstream decision making. We hence propose a framework for assessing the consistency of model properties with respect to the pipeline operations and their parameters, and, if inconsistencies are present, for revealing which parameters contribute to them. Following recent literature on software engineering for machine learning, we refer to it as debugging. From evaluating our framework in a real-world analysis scenario based on complex event logs and third-party pipeline configurations, we see strong evidence towards it being a valuable addition to the process mining toolbox.
References
[1]
van der Aalst W Process Mining: Data Science in Action 2016 Heidelberg Springer
[2]
Adriansyah, A., Buijs, J.C.A.M.: Mining process performance from event logs. In: BPM Workshops, pp. 217–218 (2013)
[3]
Amershi, S., et al.: Software engineering for machine learning: a case study. In: ICSE SEIP, pp. 291–300 (2019)
[4]
Arpteg, A., Brinne, B., Crnkovic-Friis, L., Bosch, J.: Software engineering challenges of deep learning. In: SEAA, pp. 50–59 (2018)
[5]
Augusto A, Conforti R, Dumas M, La Rosa M, and Polyvyanyy A Split miner: automated discovery of accurate and simple business process models from event logs Knowl. Inf. Syst. 2019 59 251-284
[6]
Ballambettu, N.P., Suresh, M.A., Bose, R.P.J.C.: Analyzing process variants to understand differences in key performance indices. In: CAISE, pp. 298–313 (2017)
[7]
Bauer, M., Senderovich, A., Gal, A., Grunske, L., Weidlich, M.: How much event data is enough? a statistical framework for process discovery. In: CAISE, pp. 239–256 (2018)
[8]
Bose, R.P.J.C., Mans, R.S.: Van Der Aalst, W.M.P.: Wanna improve process mining results? In: IEEE SSCI, pp. 127–134 (2013)
[9]
Buijs JCAM, van Dongen BF, and van der Aalst WMP Quality dimensions in process discovery: the importance of fitness, precision, generalization and simplicity Int. J. Coop. Inf. Syst. 2014 23 01 1440001
[10]
van Eck, M.L., Lu, X., Leemans, S.J.J., van der Aalst, W.M.P.: PM2: a process mining project methodology. In: CAISE, pp. 297–313 (2015)
[11]
Fani Sani, M., van Zelst, S.J., van der Aalst, W.M.P.: The impact of event log subset selection on the performance of process discovery algorithms. In: ADBIS, pp. 391–404 (2019)
[12]
García-Bañuelos L, van Beest NRTP, Dumas M, Rosa ML, and Mertens W Complete and interpretable conformance checking of business processes IEEE Trans. Softw. Eng. 2018 44 3 262-290
[13]
Homma T and Saltelli A Importance measures in global sensitivity analysis of nonlinear models Reliab. Eng. Syst. Saf. 1996 52 1 1-17
[14]
Jansen MJW Analysis of variance designs for model output Comput. Phys. Commun. 1999 117 1 35-43
[15]
Kalenkova, A., Polyvyanyy, A., La Rosa, M.: A framework for estimating simplicity of automatically discovered process models based on structural and behavioral characteristics. In: BPM, pp. 129–146 (2020)
[16]
Klinkmüller, C., van Beest, N.R.T.P., Weber, I.: Towards reliable predictive process monitoring. In: CAISE Forum, pp. 163–181 (2018)
[17]
Klinkmüller, C., Müller, R., Weber, I.: Mining process mining practices: an exploratory characterization of information needs in process analytics. In: BPM, pp. 322–337 (2019)
[18]
Klinkmüller C and Weber I Every apprentice needs a master: Feedback-based effectiveness improvements for process model matching Inf. Syst. 2021 95 101612
[19]
Leemans, S.J.J., Fahland, D., van der Aalst, W.M.P.: Discovering block-structured process models from event logs - a constructive approach. In: Petri Nets, pp. 311–329 (2013)
[20]
Leemans, S.J.J., Goel, K., Van Zelst, S.J.: Using multi-level information in hierarchical process mining: Balancing behavioural quality and model complexity. In: ICPM, pp. 137–144 (2020)
[21]
Leemans, S.J.J., Shabaninejad, S., Goel, K., Khosravi, H., Sadiq, S., Wynn, M.T.: Identifying cohorts: recommending drill-downs based on differences in behaviour for process mining. In: ER, pp. 92–102 (2020)
[22]
Maggi, F.M., Di Francescomarino, C., Dumas, M., Ghidini, C.: Predictive monitoring of business processes. In: CAISE, pp. 457–472 (2014)
[23]
Mannhardt, F., Blinde, D.: Analyzing the trajectories of patients with sepsis using process mining. In: BPMDS, pp. 72–80 (2017)
[24]
Manousakis, I., Goiri, I.N., Bianchini, R., Rigo, S., Nguyen, T.D.: Uncertainty propagation in data processing systems (2018)
[25]
Mariscal, G., Marbán, S., Fernández, C.: A survey of data mining and knowledge discovery process models and methodologies. Knowl. Eng. Rev. 25(2), 137–166 (2010)
[26]
Pegoraro, M., van der Aalst, W.M.P.: Mining uncertain event data in process mining. In: ICPM, pp. 89–96 (2019)
[27]
Polyvyanyy A, Armas-Cervantes A, Dumas M, and García-Bañuelos L On the expressive power of behavioral profiles Formal Aspects Comput. 2016 28 4 597-613
[28]
Puy A, Lo Piano S, and Saltelli A Is vars more intuitive and efficient than sobol’ indices? Environ. Model Softw. 2021 137 104960
[29]
Razavi S and Gupta HV A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. theory Water Resour. Res. 2016 52 1 423-439
[30]
Rozinat A and van der Aalst WMP Conformance checking of processes based on monitoring real behavior Inf. Syst. 2008 33 1 64-95
[31]
Sacha D, Senaratne H, Kwon BC, Ellis G, and Keim DA The role of uncertainty, awareness, and trust in visual analytics IEEE Trans. Vis. Comput. Graph. 2016 22 1 240-249
[32]
Sacha D, Stoffel A, Stoffel F, Kwon BC, Ellis G, and Keim DA Knowledge generation model for visual analytics IEEE Trans. Vis. Comput. Graph. 2014 20 12 1604-1613
[33]
Saltelli A Making best use of model evaluations to compute sensitivity indices Comput. Phys. Commun. 2002 145 2 280-297
[34]
Saltelli A, Aleksankina K, Becker W, Fennell P, Ferretti F, Holst N, Li S, and Wu Q Why so many published sensitivity analyses are false: a systematic review of sensitivity analysis practices Environ. Model Softw. 2019 114 29-39
[35]
Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, and Tarantola S Variance based sensitivity analysis of model output design and estimator for the total sensitivity index Comput. Phys. Commun. 2010 181 2 259-270
[36]
Saltelli A et al. Global Sensitivity Analysis 2008 Hoboken Wiley
[37]
Sargent RG Verification and validation of simulation models J. Simul. 2013 7 12-24
[38]
Seeliger, A., Sánchez Guinea, A., Nolle, T., Mühlhäuser, M.: Processexplorer: intelligent process mining guidance. In: BPM (2019)
[39]
Sobol IM Uniformly distributed sequences with an additional uniform property USSR Comput. Math. Math. Phys. 1976 16 5 236-242
[40]
Suriadi S, Andrews R, ter Hofstede AHM, and Wynn MT Event log imperfection patterns for process mining: Towards a systematic approach to cleaning event logs Inf. Syst. 2017 64 132-150
[41]
Weidlich M, Mendling J, and Weske M Efficient consistency measurement based on behavioral profiles of process models IEEE Trans. Softw. Eng. 2011 37 3 410-429
[42]
Weidlich, M., Polyvyanyy, A., Mendling, J., Weske, M.: Efficient computation of causal behavioural profiles using structural decomposition. In: Petri Nets, pp. 63–83 (2010)
[43]
Weidlich M, Polyvyanyy A, Mendling J, and Weske M Causal behavioural profiles - efficient computation, applications, and evaluation Fundam. Inf. 2011 113 3–4 399-435
[44]
Wieringa RJ Design Science Methodology for Information Systems and Software Engineering 2014 Heidelberg Springer
[45]
Yang, K., Huang, B., Stoyanovich, J., Schelter, S.: Fairness-aware instrumentation of preprocessing pipelines for machine learning. In: HILDA (2020)
Recommendations
Domain-driven actionable process model discovery
We apply integer linear programming for discovering process model based on user's knowledge.We perform a case study with real event logs in logistics.The proposed method, proximity miner, improves the result of discovered process model. Process ...
Time-interval process model discovery and validation--a genetic process mining approach
A process management technique, called process mining, received much attention recently. Process mining can extract organizational or social structures from event logs recorded in an information system. However, when constructing process models, most ...
A Process Mining Success Factors Model
Business Process ManagementAbstractProcess mining – a suite of techniques for extracting insights from event logs of Information Systems (IS) – is increasingly being used by a wide range of organisations to improve operational efficiency. However, despite extensive studies of ...
Comments
Information & Contributors
Information
Published In
Sep 2021
479 pages
ISBN:978-3-030-85468-3
DOI:10.1007/978-3-030-85469-0
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 06 September 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 22 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in