research-article

Measuring Performance in Knowledge-intensive Processes

Authors:

Bedilia Estrada-Torres,

Pedro Henrique Piccoli Richetti,

Adela Del-Río-Ortega,

Fernanda Araujo Baião,

Manuel Resinas,

Flávia Maria Santoro,

Antonio Ruiz-CortésAuthors Info & Claims

ACM Transactions on Internet Technology (TOIT), Volume 19, Issue 1

Article No.: 15, Pages 1 - 26

https://doi.org/10.1145/3289180

Published: 06 February 2019 Publication History

Abstract

Knowledge-intensive Processes (KIPs) are processes whose execution is heavily dependent on knowledge workers performing various interconnected knowledge-intensive decision-making tasks. Among other characteristics, KIPs are usually non-repeatable, collaboration-oriented, unpredictable, and, in many cases, driven by implicit knowledge, derived from the capabilities and previous experiences of participants. Despite the growing body of research focused on understanding KIPs and on proposing systems to support these KIPs, the research question on how to define performance measures thereon remains open. In this article, we address this issue with a proposal to enable the performance management of KIPs. Our approach comprises an ontology that allows us to define process performance indicators (PPIs) in the context of KIPs, and a methodology that builds on the ontology and the concepts of lead and lag indicators to provide process participants with actionable guidelines that help them conduct the KIP in a way that fulfills a set of performance goals. Both the ontology and the methodology have been applied to a case study of a real organization in Brazil to manage the performance of an Incident Troubleshooting Process within an ICT (Information and Communications Technology) Outsourcing Company.

Supplementary Material

a15-estrada-torres-apndx.pdf (estrada-torres.zip)

Supplemental movie, appendix, image and software files for, Measuring Performance in Knowledge-intensive Processes

Download
4.01 MB

References

[1]

Peter C. Brewer and Thomas W. Speh. 2000. Using the balance scorecard to measure supply chain performance. J. Bus. Logist. 21, 1 (2000), 75--93.

[2]

Joos C. A. M. Buijs, Boudewijn F. van Dongen, and Wil M. P. van der Aalst. 2013. Mining Configurable Process Models from Collections of Event Logs. Springer, Berlin, 33--48.

Digital Library

[3]

Evellin C. S. Cardoso, Paulo Sérgio Santos Jr., João Paulo Andrade Almeida, Renata S. S. Guizzardi, and Giancarlo Guizzardi. 2010. Semantic integration of goal and business process modeling. In Proceedings of the International Conference on Research and Practical Issues Enterprise Information Systems (CONFENIS’10).

[4]

Felix Tung Sun Chan. 2003. Performance measurement in a supply chain. Int. J.Adv. Manufact. Technol. 21, 7 (2003), 534--548.

[5]

Mu-Yen Chen, Mu-Jung Huang, and Yu-Chen Cheng. 2009. Measuring knowledge management performance using a competitive perspective: An empirical study. Expert Syst. Appl. 36, 4 (2009), 8449--8459.

Digital Library

[6]

Claire Costello and Owen Molloy. 2009. Building a process performance model for business activity monitoring. In Information Systems Development, Challenges in Practice, Theory, and Education, Vol. 1. Springer, 237--248.

[7]

Francisco M. del-Rey-Chamorro, Rajkumar Roy, Bert van Wegen, and Andy Steele. 2003. A framework to create key performance indicators for knowledge management. J. Knowl. Manag. 7, 2 (2003), 46--62.

[8]

Adela del Río-Ortega, Félix García, Manuel Resinas, Elmar Weber, Francisco Ruiz, and Antonio Ruiz-Cortés. 2017a. Enriching decision making with data-based thresholds of process-related KPIs. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’17). 193--209.

[9]

Adela del Río-Ortega, Manuel Resinas, Cristina Cabanillas, and Antonio Ruiz-Cortés. 2013. On the definition and design-time analysis of process performance indicators. Info. Syst. 38, 4 (2013), 470--490.

Digital Library

[10]

Adela del Río-Ortega, Manuel Resinas, Amador Durán, Beatriz Bernárdez, Antonio Ruiz-Cortés, and Miguel Toro. 2017b. Visual PPINOT: A graphical notation for process performance indicators. Bus. Info. Syst. Eng. (2017).

[11]

Adela del Río-Ortega, Manuel Resinas, and Antonio Ruiz-Cortés. 2010. Defining process performance indicators: An ontological approach. In Proceedings of the Confederated International Conferences: CoopIS, IS, DOA, and ODBASE (2010). 555--572.

[12]

Claudio Di Ciccio, Andrea Marrella, and Alessandro Russo. 2015. Knowledge-intensive processes: Characteristics, requirements and analysis of contemporary approaches. J. Data Semant. 4, 1 (2015), 29--57.

[13]

Marlon Dumas, Marcello La Rosa, Jan Mendling, and Hajo A. Reijers. 2013. Fundamentals of Business Process Management. Vol. 1. Springer-Verlag, Berlin.

Digital Library

[14]

James D. Evans. 1996. Straightforward Statistics for the Behavioral Sciences. Brooks/Cole Publishing Company.

[15]

Bernhard Ganter and Gerd Stumme. 2003. Creation and merging of ontology top-levels. In Conceptual Structures for Knowledge Creation and Communication. 131--145.

[16]

Oscar González, Rubby Casallas, and Dirk Deridder. 2009. MMC-BPM: A domain-specific language for business processes analysis. In Proceedings of the Business Information Systems Conference (BIS’09), Vol. 21. 157--168.

[17]

Daniela Grigori, Fabio Casati, Malu Castellanos, Umeshwar Dayal, Mehmet Sayal, and Ming-Chien Shan. 2004. Business process intelligence. Comput. Ind. 53, 3 (2004), 321--343.

Digital Library

[18]

Giancarlo Guizzardi. 2005. Ontological foundations for structural conceptual models. Ph.D. Dissertation.

Digital Library

[19]

Giancarlo Guizzardi, Ricardo de Almeida Falbo, and Renata S. S. Guizzardi. 2008. Grounding software domain ontologies in the unified foundational ontology (UFO): The case of the ODE software process ontology. In Proceedings of the Workshop Requirements Engineering and Software Environments (IDEAS’08). 127--140.

[20]

Kaisa Henttonen, Aino Kianto, and Paavo Ritala. 2016. Knowledge sharing and individual work performance: An empirical study of a public sector organisation. J. Knowl. Manag. 20, 4 (2016), 749--768.

[21]

Steffen Herbold, Jens Grabowski, and Stephan Waack. 2011. Calculation and optimization of thresholds for sets of software metrics. Empir. Softw. Eng. 16, 6 (2011), 812--841.

Digital Library

[22]

Richard Hull, Agnes Koschmider, Hajo A. Reijers, and William Wong. 2016. Fresh approaches to business process modeling (Dagstuhl seminar 16191). Dagstuhl Reports 6, 5 (2016), 1--30.

[23]

Öykü Isik, Willem Mertens, and Joachim Van den Bergh. 2013. Practices of knowledge-intensive process management: Quantitative insights. Bus. Process Manag. J. 19, 3 (2013), 515--534.

[24]

Robert S. Kaplan and David P. Norton. 1992. The balanced scorecard: Measures that drive performance. Harvard Bus. Rev. 70, 1 (1992), 71--79.

[25]

Birgit Korherr and Beate List. 2007. Extending the EPC and the BPMN with business process goals and performance measures. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS’07). 287--294.

[26]

Elfriede Krauth, Hans Moonen, Viara Popova, and Martijn C. Schut. 2005. Performance measurement and control in logistics service providing. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS’05). 239--247.

[27]

Peter Kueng. 2000. Process performance measurement system: A tool to support process-based organizations. Total Qual. Manag. 11, 1 (2000), 67--85.

[28]

Kun Chang Lee, Sangjae Lee, and In Won Kang. 2005. KMPI: Measuring knowledge management performance. Info. Manag. 42, 3 (2005), 469--482.

Digital Library

[29]

Richard Lenz and Manfred Reichert. 2007. IT support for healthcare processes—Premises, challenges, perspectives. Data Knowl. Eng. 61, 1 (2007), 39--58.

Digital Library

[30]

Todd A. Little and Amit V. Deokar. 2011. Role of knowledge asset indicators in understanding knowledge-intensive processes. In Proceedings of the Americas Conference on Information Systems (AMCIS’11).

[31]

Todd A. Little and Amit V. Deokar. 2016. Understanding knowledge creation in the context of knowledge-intensive business processes. J. Knowl. Manag. 20, 5 (2016), 858--879.

[32]

Fabrizio M. Maggi, R. P. Jagadeesh Chandra Bose, and Wil M. P. van der Aalst. 2012. Efficient discovery of understandable declarative process models from event logs. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’12). 270--285.

Digital Library

[33]

Olivera Marjanovic and Ronald Freeze. 2011. Knowledge-intensive business processes: Theoretical foundations and research challenges. In Proceedings of the Hawaii International Conference on Systems Science. 1--10.

Digital Library

[34]

Anne P. Massey, Mitzi M. Montoya-Weiss, and Tony M. O’Driscoll. 2002. Performance-centered design of knowledge-intensive processes. J. Manag. Info. Syst. 18, 4 (2002), 37--58.

Digital Library

[35]

Chris McChesney, Sean Covey, and Jim Huling. 2012. The 4 Disciplines of Execution: Achieving Your Wildly Important Goals. Simon and Schuster.

[36]

Christof Momm, Robert Malec, and Sebastian Abeck. 2007. Towards a model-driven development of monitored processes. In Proceedings of the Internationale Tagung Wirtschaftsinformatik (WI’07). 319--336.

[37]

Ednilson Veloso Moura, Flávia Maria Santoro, and Fernanda Araujo Baião. 2015. XCuteKIP: Support for knowledge-intensive process activities. In Proceedings of the International Workshop on Groupware (CRIWG’15). 164--180.

[38]

Andy Neely, Huw Richards, John Mills, Ken Platts, and Mike Bourne. 1997. Designing performance measures: A structured approach. Int. J. Oper. Prod. Manag. 17, 11 (1997), 1131--1152.

[39]

Natalya F. Noy and Deborah L. McGuinness. 2001. Ontology Development 101: A Guide to Creating Your First Ontology. Technical Report.

[40]

Object Management Group (OMG). 2014. Case Management Model and Notation, Version 1.0.

[41]

Oluwafemi Oyemomi, Shaofeng Liu, Irina Neaga, and Ali Alkhuraiji. 2016. How knowledge sharing and business process contribute to organizational performance: Using the fsQCA approach. J. Bus. Res. 69, 11 (2016), 5222--5227.

[42]

Carlos Pedrinaci, Dave Lambert, Branimir Wetzstein, Tammo van Lessen, Luchesar Cekov, and Marin Dimitrov. 2008. SENTINEL: A semantic business process monitoring tool. In Proceedings of International Workshop on Ontology-supported Business Intelligence 2008. 26--30.

Digital Library

[43]

Viara Popova and Alexei Sharpanskykh. 2010. Modeling organizational performance indicators. Info. Syst. 35, 4 (2010), 505--527.

Digital Library

[44]

Alexander Rannacher, Robert Stranzenbach, Flavius Sturm, Susanne Mütze-Niewöhner, and Christopher M. Schlick. 2013. A system dynamics model for the evaluation of the productivity of knowledge-intensive services. iBusiness 5, 3B (2013), 55--58.

[45]

Anand S. Rao and Michael P. Georgeff. 1995. BDI agents: From theory to practice. In Proceedings of International Conference on Multi-Agent Systems (ICMAS’95). 312--319.

[46]

Pierre J. Richard, Timothy M. Devinney, George S. Yip, and Gerry Johnson. 2009. Measuring organizational performance: Towards methodological best practice. J. Manag. 35, 3 (2009), 718--804.

[47]

Cornelia Richter von Hagen, Dietmar Ratz, and Roman Povalej. 2005. Towards self-organizing knowledge-intensive processes. J. Univers. Knowl. Manag. 0, 2 (2005), 148--169.

[48]

Ann Rosenberg, Greg Chase, Rukhshaan Omar, James Taylor, and Mark von Rosing. 2010. Applying Real-World BPM in an SAP Environment. Galileo Press.

Digital Library

[49]

Jorge Saldivar, Carla Vairetti, Carlos Rodríguez, Daniel Florian, Fabio Casati, and Rosa Alarcón. 2016. Analysis and improvement of business process models using spreadsheets. Info. Syst. 57 (2016), 1--19.

Digital Library

[50]

Juliana Baptista dos Santos França, Joanne Manhães Netto, Juliana do E. S.Carvalho, Flávia Maria Santoro, Fernanda Araujo Baião, and Mariano Pimentel. 2015. KIPO: The knowledge-intensive process ontology. Softw. Syst. Model. 14, 3 (2015), 1127--1157.

Digital Library

[51]

Gerd Stumme and Alexander Maedche. 2001. FCA-MERGE: Bottom-up merging of ontologies. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’01).

Digital Library

[52]

Flavius Sturm, Alexander Rannacher, Lesya Straub-Bilan, Robert Stranzenbach, and Susanne Mütze-Niewöhner. 2011. Productivity measurement of knowledge-intensive services--towards adaptable service performance evaluation systems. In Proceedings of International Conference of the European Association for Research on Services (RESER’08).

[53]

Mari Carmen Suárez-Figueroa, Asunción Gómez-Pérez, and Mariano Fernández-López. 2012. The NeOn methodology for ontology engineering. In Proceedings of the Conference on Ontology Engineering in a Networked World. 9--34.

[54]

Ganesh Vaidyanathan. 2005. A framework for evaluating third-party logistics. Commun. ACM 48, 1 (2005), 89--94.

Digital Library

[55]

Amy Van Looy and Aygun Shafagatova. 2016. Business process performance measurement: A structured literature review of indicators, measures and metrics. SpringerPlus 5, 1 (2016), 1797.

[56]

Zhining Wang, Pratyush Nidhi Sharma, and Jinwei Cao. 2016. From knowledge sharing to firm performance: A predictive model comparison. J. Bus. Res. 69, 10 (2016), 4650--4658.

[57]

Branimir Wetzstein, Zhilei Ma, and Frank Leymann. 2008. Towards measuring key performance indicators of semantic business processes. In Proceedings of the International Conference on Business Information Systems (BIS’08). 227--238.

Cited By

Estrada-Torres Bdel-Río-Ortega AResinas M(2023)Defining Process Performance Measures in an Object-Centric ContextBusiness Process Management Workshops10.1007/978-3-031-25383-6_16(210-222)Online publication date: 9-Feb-2023
https://doi.org/10.1007/978-3-031-25383-6_16
Röglinger MPlattfaut RBorghoff VKerpedzhiev GBecker JBeverungen Dvom Brocke JVan Looy Adel-Río-Ortega ARinderle-Ma SRosemann MSantoro FTrkman P(2022)Exogenous Shocks and Business Process ManagementBusiness & Information Systems Engineering10.1007/s12599-021-00740-w64:5(669-687)Online publication date: 10-Feb-2022
https://doi.org/10.1007/s12599-021-00740-w
Engel RFernandez PRuiz-Cortes AMegahed AOjeda-Perez J(2022)SLA-aware operational efficiency in AI-enabled service chains: challenges aheadInformation Systems and e-Business Management10.1007/s10257-022-00551-w20:1(199-221)Online publication date: 28-Jan-2022
https://doi.org/10.1007/s10257-022-00551-w
Show More Cited By

Index Terms

Measuring Performance in Knowledge-intensive Processes
1. Software and its engineering
  1. Software organization and properties

Recommendations

Improving knowledge intensive processes through an enterprise knowledge medium
SIGCPR '99: Proceedings of the 1999 ACM SIGCPR conference on Computer personnel research
Knowledge creating communities in the context of work processes

This article deals with the support potential of knowledge communities for a co-operative knowledge generation and problem solving before the background of the increasing economic importance of knowledge-intensive processes. Therefore, requirements for ...
Knowledge-Intensive Healthcare Processes: Rethinking Business Process Ownership
HICSS '13: Proceedings of the 2013 46th Hawaii International Conference on System Sciences

In very recent times, the pressing need to compete on the basis of process-related knowledge rather than process automation has expanded the field of Business Process Management (BPM) to include knowledge intensive business processes (BP). This paper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 19, Issue 1

Regular Papers, Special Issue on Service Management for IOT and Special Issue on Knowledge-Driven BPM

February 2019

321 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/3283809

Editor:
Ling Liu
Georgia Institute of Technology, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 February 2019

Accepted: 01 October 2018

Revised: 01 October 2018

Received: 01 December 2016

Published in TOIT Volume 19, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

European Commission
Ministerio de Economia y Competitividad
Conselho Nacional de Desenvolvimento Cientifico e Tecnologico
Andalusian R&D&I Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
460
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Estrada-Torres Bdel-Río-Ortega AResinas M(2023)Defining Process Performance Measures in an Object-Centric ContextBusiness Process Management Workshops10.1007/978-3-031-25383-6_16(210-222)Online publication date: 9-Feb-2023
https://doi.org/10.1007/978-3-031-25383-6_16
Röglinger MPlattfaut RBorghoff VKerpedzhiev GBecker JBeverungen Dvom Brocke JVan Looy Adel-Río-Ortega ARinderle-Ma SRosemann MSantoro FTrkman P(2022)Exogenous Shocks and Business Process ManagementBusiness & Information Systems Engineering10.1007/s12599-021-00740-w64:5(669-687)Online publication date: 10-Feb-2022
https://doi.org/10.1007/s12599-021-00740-w
Engel RFernandez PRuiz-Cortes AMegahed AOjeda-Perez J(2022)SLA-aware operational efficiency in AI-enabled service chains: challenges aheadInformation Systems and e-Business Management10.1007/s10257-022-00551-w20:1(199-221)Online publication date: 28-Jan-2022
https://doi.org/10.1007/s10257-022-00551-w
Estrada-Torres BDel-Rio-Ortega AResinas MRuiz-Cortes A(2021)Modeling Variability in the Performance Perspective of Business ProcessesIEEE Access10.1109/ACCESS.2021.31015759(111683-111703)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3101575
Simões RMelo GBrito e Abreu FOliveira T(2021)Towards Understanding Quality-Related Characteristics in Knowledge-Intensive Processes - A Systematic Literature ReviewQuality of Information and Communications Technology10.1007/978-3-030-85347-1_15(197-207)Online publication date: 25-Aug-2021
https://doi.org/10.1007/978-3-030-85347-1_15
Marquez-Chamorro ARevoredo KResinas MDel-Rio-Ortega ASantoro FRuiz-Cortes A(2020)Context-Aware Process Performance Indicator PredictionIEEE Access10.1109/ACCESS.2020.30446708(222050-222063)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3044670
Behan JRomero OZimányi E(2019)Multidimensional Integration of RDF DatasetsBig Data Analytics and Knowledge Discovery10.1007/978-3-030-27520-4_9(119-135)Online publication date: 26-Aug-2019
https://dl.acm.org/doi/10.1007/978-3-030-27520-4_9

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents