Available online at www.sciencedirect.com ScienceDirect IFAC PapersOnLine 52-13 (2019) 1145–1149 Exploring Opportunities for Artificial Emotional Intelligence in Service Production Systems  Marlene Amorim*, Yuval Cohen**, João Reis***,   Mário Rodrigues****  *DEGEIT & GOVCOPP, Universidade de Aveiro, Campus Universitário de Santiago, 3810-193, Aveiro, Portugal (Tel: 351 234370200; e-mail: mamorim@ ua.pt). ** Department of Industrial Engineering, Afeka College of Engineering, Tel Aviv Israel (e-mail: yuvalc@afeka.ac.il) *** Department of Military Science and CINAMIL/CISD, Military Academy, Lisbon, Portugal (email:joao.reis@academiamilitar.pt) ****IEETA & ESTGA, Universidade de Aveiro, Aveiro, Portugal (e-mail: mjfr@ua.pt) Abstract: This paper offers an exploratory view about opportunities for the integration of Artificial Emotional Intelligence (AEI) in service production systems. Service systems are conceptualized as increasingly digitalized and networked co-production environments where employees, customers and technology engage in rich interactions to deploy outputs and create value. The study builds on a systematic literature review to identify three dimensions of application of AEI for improving service efficiency and reliability, namely to augment, to assist and to mimic customer and/or providers production capabilities. The paper offers a contribution to promoting the dialogue between the knowledge domains of service systems and AEI. © 2019, IFAC : (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. : Keywords:: artificial intelligence, service production systems, customer interaction, human machine interaction, : digital services.   : 1. INTRODUCTION    Services account for a major share of economic value in modern societies (Vargo, Maglio and Akaka, 2008). The importance of the service industries for value creation and for employment in economies is estimated, and expanding, beyond 70%, while the diversity of service companies, as well as service functions within manufacturing grows consistently (Gebauer, Fleisch and Friedli, 2005). environments wheresystems customers’ andasknowledge are Service production can bework defined co-production environments where customers’ work and knowledge are ’s resources, implying that environments and are environments where where customers’ customers’ work work and knowledge knowledge are ’s resources, implying that brought together with the company’s resources, implying that that environments where customers’ work and knowledge are ’s resources, implying environments where work andproduction knowledge are customers always playcustomers’ an active role in the of that the ’s resources, implying resources, implying2008, that service outputs (Zeithaml et al.,’s 2006, Pinhanez, Sampson, 2010, Spohrer and Maglio, 2010). Service production systems operate via an orchestrated set of interactions, and exchanges, between the company and the and even the customer (e.g. of in materials, the so called “people customer, involving the self handling information, and even the customer self (e.g. sectors in the such so called “people processing services”, that include as healthcare, and even even the the customer self (e.g. sectors in the the such so called called “people and customer (e.g. in so “people processing services”, thatself include as healthcare, processing services”, that include sectors such as healthcare, and even the customer self (e.g. in the so called “people processing services”, that include sectors such as healthcare, and even the customer (e.g. in the so coordination called “people transportation, education, etc.), as sectors well as such the of processing services”, thatself include as healthcare, processing that include as healthcare, production services”, tasks (Sampson and sectors Spring,such 2012). This coparticipated nature of service production systems, in practice, i.e.taking with the company’s translates into a number of interactions place between i.e. with the company’s customers and the production system, i.e. i.e. with with the the company’s company’s employees or with its mediated or automated interfaces and i.e. with the company’s i.e. with the company’s touch points (Giannakis,2011, Schumann, Wünderlich and Wangenheim, 2012, Larivière, et al. 2017). In this vein, service production systems have been pioneers in the domain of experimentation of human computer interactions, and systems for in effective and efficient of providers’ particularly investigations aimingintegration for the development of systems for effective and efficient integration of providers’ systems for effective and efficient integration of providers’ providers’ systems for effective and efficient integration of resources,fori.e. employees, technology and the customers. A systems effective and efficient integration of providers’ systems forexample effectivehas andbeen efficient providers’ prominent the integration adoption ofofonline and mobile delivery channels, and their integration in cross channel processes, enabling firms to achieve gains in the design of service encounters, to meet customer requirements, notably for access and service convenience at an unprecedented pace (Reis, Amorim and Melão, 2018). Digital technologies find a fertile ground in services, because they create opportunities for continuous innovation in the ways that information - that is a core input for service production systems - can be exchanged, transported and transformed by different users and contexts (Lusch and Vargo, 2014). The recent developments and the adoption of– – advanced information technologies (e.g. Internet-of-Things –– IoT, Cyber-Physical Systems, etc.) are triggering promising– new avenues for service innovation and the emergence of– smart service systems. The expanding links between the physical world and networked technologies, including networked sensors creates a powerful and augmented space for the interactions and collaboration between service providers and customers for value creation. The generalization of the connections between information to people and people to people, is supporting the connection of objects, spaces, things etc. As such the context for service production and interaction between service actors is evolving towards a reality that is intuitive, context-aware, and with 2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.11.350 1146 Marlene Amorim et al. / IFAC PapersOnLine 52-13 (2019) 1145–1149 intelligence capabilities to enable gains in efficiency and value creation. Two additional concurrent facts, enabled by the advanced ICT, are colouring this scenario: the amazing growth in the volume and nature of data that is available, captured by the smart networked systems, and the sophistication of the computational capabilities, to deal with them (Yang et al. 2017). In such contexts, Artificial Intelligence (AI) allows for the delegation of the complex tasks of pattern identification, learning, etc. to the capabilities of computer based processes that can surpass humans in their performance for capturing, structuring and understanding large volumes of data (O’Leary, 2013). One of the emerging areas of research in AI concerns the development of capabilities to collect process and respond to emotional states of individuals. The concept of Emotional Intelligence (EI) refers to the ability of individuals to deal with his and others peoples’ emotions (Saloyey and Mayer, 1990). The nature of EI involves the activation of several concurrent skills concerning: i) the ability to identify emotions; ii) the ability to use emotions; iii) the ability to understand emotions and iv) the ability to manage them. Over the years several research efforts have been devoted to develop methods to support the assessment and measurement of EI, such as the Mayer-Salovey-Caruso EI Test – MSCEIT – that reflect the intrincate nature and complexity of EI (Mayer, Salovey, Caruso, and Sitarenios, 2003). In this paper we offer an exploratory study to gain understanding about AEI applications in service production systems. The study builds on the analysis from a structured literature review, to identify examples of applications and experimentation of AEI in service systems, therefore offering a structured approach to understand the potential of AEI in this domain, and suggesting paths for future research. 2. STUDY DESIGN AND METHODS 2.1 Process and criteria for selection of publications In December 2018, a systematic literature review was carried out resorting to SCOPUS database to search for publications (i.e. journal articles, extended abstracts, and conference proceedings) related with artificial emotional intelligence AIE. While no limitations were applied for date, or type of publication, only articles written in English were taken into account. The publications resulting from the data base query went through a selection process involving 2 researchers and that included the analysis of the title and the abstract, on a first reduction round, followed by the reading of the complete content of the manuscripts, to obtain a final set of publications to be included in the review. The guiding objective of the publications’ search was to identify research work describing or discussing the application of technologies for AIE in the specific context of service business. Given the novelty of the topic, and in order to limit the possible exclusion of relevant publications, in the database search the researchers employed the generic string “artificial emotional intelligence”, without restricting the query to the specific field of service companies, given that service production contexts can be embedded in manufacturing contexts. Likewise in order to cover an ample spectre of publications that could refer to the application of artificial emotional intelligence to the broad field of production systems, the search considered an ample scope for the scientific fields, namely, “Engineering”, “Social Sciences”, and “Business, Management and Accounting”. Employing a stepwise approach the each researcher independently assessed the relevance of the publications, by reading the title and the abstract, using a rating scale, from 0 not relevant – to 2 – very relevant. Publications rated with a 0 were excluded from the sample, and those with at least 2 points were selected. In a following step the full text was read, allowing, if necessary for the revision of the decision about the inclusion of the publication. 2.2 Characterization of selected publications In the first phase of the query, a total of 511 documents were identified, by applying the aforementioned filtering conditions. The researchers’ assessment about the relevance of the publications led to a sample number of 67 publications, that was further restricted to 61 after full text reading. Most of the publications were excluded because they focused in the advancement in the technical development of artificial intelligence functions and capabilities, without analysing a specific industry or service scenario for its application. As expected, it was possible to observe an upward trend in the publication volume (Fig. 1). Regarding the domain and type of publication, a substantial number of the selected articles were obtained from indexed conference proceedings, something that can be explained by the emergent nature of the topic. Predominant domains included artificial intelligence scientific meetings as well as computer science, robotics and automation. Nevertheless, the selected sample suggested that the topic is gaining momentum among industry and production systems academics, as per some of the articles appeared in outlets such as: International Journal of Modern Manufacturing Technologies, International Journal of Production, International Journal of Cognitive Research in Science, Engineering and Education, Computers and Electrical Engineering, Journal of Cognitive Engineering and Decision. Fig. 1. Yearly evolution of relevant publications 3. AIE AND SERVICE PRODUCTION SYSTEMS Marlene Amorim et al. / IFAC PapersOnLine 52-13 (2019) 1145–1149 Articles were read in full, and analysed in order to address three questions: i) who is the subject of emotion identification?; ii) what kind of inputs are being used as sources data to understand the emotions; iii) what is the [production system] purpose of understanding the emotion? The researchers classified the manuscripts content into categories, to address each of the proposed questions independently, and this information was then compiled from the individual reading notes of each researcher. In order to provide a common framework for the classification of the information extracted from the publications, the researchers followed the service interaction classifications prevalent in the literature and that distinguish; face-to-face service interactions and face to screen service interactions. In this service typology proposed by Froehle and Roth (2004), faceto-screen services can be (usefully) subdivided in to technology mediated contact (e.g. phone service interactions, online calls, etc.) and technology generated customer contact (i.e. self-service). In what concerns the subjects considered for emotion identification, it was possible to observe articles focusing on two distinct actors of service systems: service employees, and service customers. The works addressing the emotions of employees, understood as key resources for the performance and quality of the service system outputs, were outnumbered by research work focusing on the emotions of customers. Examples of advanced work addressing the emotions of employees include: the work of Ribak et al. (2017) who propose an “intelligent control room” for monitoring and assessing the condition of employees (e.g. stress) from the collection and interpretation of voice and facial expressions; the work of Eboli, Mazzulla and Pungillo (2017) that explores the relationship between driving risk and the drivers’ physical and emotional conditions; the work of Thatcher and Kilingaru (2012) that build on iformation on eye movement to understand emotions from airline employees and identify abnormal flight conditons; the work of Baowman and Rogers (2016) that explores the influence of emotional issues for the decision making capabilities of direct care workers in ambient assisted living; and, in the same line of concern, the work of Coroiu (2015) that propose an emotional intelligent agent to support, and improve decision making in manufacturing contexts. Overall the extant work is acknowledging that the emotional conditions of employees are determinant for their effectiveness and for the quality of their work, and that their emotions are subject to variations (e.g. derived from personal contexts as well as from workload conditions). The development of AEI models and applications in this domain is advancing in ameliorating the capabilities for emotion recognition and in the development of tools that can assist and improve employee performance, by creating awareness of emotional conditions and providing tools and information to mitigate any undesirable consequences. In the domain of emotions understating for customers, there is a remarkable concentration of efforts in the areas of health and care services, education and online purchasing and service delivery. Illustrative examples include: the work of 1147 Amifra et al. (2018) that integrates emotion recognition – for home patients - into monitoring services for chronic disease; and, in the same vein the work of Myakala et al. (2017) that identify emotions from children cry-detection, in order to assist and inform care personnel. The early work of Marcelino et al. (2015) explored the same domain of service opportunity, i.e. elderly care by incorporating emotion understanding in an eService platform for pervasive case, to support the daily monitoring and detection of danger scenarios. In the domain of education, online learning contexts are a rich field for experimenting with AEI applications, given their ease in capturing and tracking the user information. Subramainan et al. (2017), Xu et al. (2017), among others, are including capabilities for students’ emotion recognition (e.g. anger, anxiety, boredom) into the monitoring capabilities for an online learning system in order to emit alerts and allow educators to adjust their teaching strategies. In the domain of online service, notably in retail and customer assistance, the pioneer examples stand out, such as the work of Povoda et al. (2015) that focused on understanding customer emotions building on data from helpdesk messages to devise priority rules for adequate service responses. The data being used to inform emotion-understanding models, is primarily driven from two types of sources. One domain concerns behavioural sources, such as facial expressions, voice tones, speech characteristics, eye movement, lip movement and so on. Collecting such data requires the adoption of capturing and recording technologies such as, eye tracking devices (Thatcher and Kilingaru, 2012); biometric mouses (Kaklauskas et al., 2008); computer and mobile cameras (e.g. Xu et al. 2017), among others. One of the key advantages of capturing behavioural data that is frequently pointed out is the fact that many of such devices are generalized technology, hold by most of the population and therefore being easily accessible and with a relatively low cost. Other area of data that can allow for the understanding of emotions is performance data, i.e. data that offers evidence on the tasks conducted by employees and customers, and whose characteristics (e.g. duration, delays, interruptions, failures) can allow for inferring about the emotional condition of individuals. This data is extracted from records and outputs of production processes (e.g. text, e-mail messages, log times, etc.). The digitalization of service processes comes in favour for the abundance of such data. With the proliferation of mediated services, self-service systems, online services and alike the traceability of customers and employees interactions and task results is largely increased (as compared to traditional face.to-face services where, most often, the service encounters were not recorded or tracked). The purposes of the reported adoption of AIE in service systems, are related with the objectives of improving the efficiency, the reliability and the overall performance of the production system. This is set to be achieved by the augmenting and/or complementary role that AEI has towards the capabilities of employers, customers and technology. Anchoring the analysis on the typologies for service interaction, that involve the elements of the service 1148 Marlene Amorim et al. / IFAC PapersOnLine 52-13 (2019) 1145–1149 production system – employees, technology and customers – it was possible to observe that the AIE experimentations are contributing to enable service capabilities around three broad forms, as illustrated in Fig.1. infused with some emotion recognition and adequate response capabilities that humans hold. Examples of this stream of work include, Tuyen, Jeong and Chong (2017), that have developed a robot for the delivery of ambient assisted living services, or Loghmani, Rovetta and Venture (2017). 4. CONCLUSIONS Fig. 1 Subjects, Data Sources and Capabilities associated with AEI in service systems The application of AIE described in service systems is allowing for the improvement of service system capabilities in the following manners: i) “augment” the capabilities of employees and customers in face to face service interactions. In such service settings AI algorithms and models that are being advanced are, for instance providing employees with information about their own emotions, in order to allow them to adjust their behaviour (e.g. the aforementioned work of Ribak et al. (2017) that aims to gather knowledge about the emotional conditions of employees in a intelligent control room). In addition to providing such, own “awareness” information, we could also observe examples of the possibilities of AI for providing knowledge to employees about the emotional state of the customers, i.e., providing insights about the emotional sate of the “other” (e.g. the aforementioned work of Povoda et al. (2015) that offered to the company knowledge about customer emotions derived from the content of helpdesk messages). This perspective allows for a tipification of the functional purposes of AEI in face to face service settings as follows: augmenting service capabilities by offering self and/others’ emotion understanding; ii) “assist” the capabilities of the service production system in technology mediated interactions. In such settings some of the richness of the face to face contact between employers and customers is lost (e.g. call center services). As such some of the AEI proposals that are being advanced are supporting the work of the employees offering complementary information that they cannot get by personal contact (e.g. see for example the referenced works o AEI applications in online learning settings, were the system is capturing data to understand the users’ attention, motivation, and allow for pedagogical adjustments - Subramainan et al. (2017), Xu et al. (2017)). iii) “mimic” the capabilities of employees in automated contact, or self service systems. In such service settings the state of the art of the existing AEI proposals concerns the utilization of customers’ data to train and “qualify” automated systems, including robots, so that they can be The purpose of this study is to advance the understanding of the opportunities for the application artificial emotional intelligence in service production systems. This purpose is achieved by analysing and synthesizing the most recent research and applications of artificial emotional intelligence in service contexts through a systematic review of 61 articles on the topic. Given the emergent nature of the topic, to the best of our knowledge no previous attempts have been made to synthesize AEI on this domain. The main contribution of this paper is to provide a structured characterization of existing empirical and conceptual research on AEI in the context of services, structuring the roles of AEI developments for improving the efficiency and the reliability of service systems, by augmenting, assisting and mimicking the roles of employees and customers in production. In particular, this paper brings forth evidence of the existence of an unbalance in the number of examples and contribution in the domains of intervention for AEI in services, therefore calling for future research on and opens areas for the development of applications. Overall the study contributes to enrich the dialogue about opportunities for digital innovation in services by approximating the bodies of knowledge of service systems and AI. REFERENCES Amira T., Dan I., Az-Eddine B., Ngo H.H., Said G., Katarzyna W.-W.,"Monitoring chronic disease at home using connected devices",2018,"2018 13th System of Systems Engineering Conference, SoSE 2018" Bowman, S. E., & Rogers, W. A. (2016). Understanding decision making among direct care workers in assisted living. Journal of Cognitive Engineering and Decision Making, 10(4), 369-390. Coroiu, A. M. (2015). Emotional Intelligent Agent in Decision-Making Process with Implications in Manufacturing. International Journal of Modern Manufacturing Technologies, 7(2), 44. Eboli L., Mazzulla G., Pungillo G.,"The relationship between driving risk levels and drivers’ personality traits, physical and emotional conditions",2017,"Transport Infrastructure and Systems - Proceedings of the AIIT International Congress on Transport Infrastructure and Systems, TIS 2017. Froehle, C. M., and Roth, A. V. (2004). New measurement scales for evaluating perceptions of the technologymediated customer service experience. Journal of Operations Management, 22(1), 1-21. Gebauer, H., Fleisch, E., and Friedli, T. (2005). Overcoming the service paradox in manufacturing companies. European management journal, 23(1), 14-26. Marlene Amorim et al. / IFAC PapersOnLine 52-13 (2019) 1145–1149 Giannakis, M. (2011). Conceptualizing and managing service supply chains. The Service Industries Journal, 31(11), 1809-1823. Kaklauskas, A., Zavadskas, E. K., Seniut, M., Krutinis, M., Dzemyda, G., Ivanikovas, S., and Jaruševičius, A. (2008). Web-based biometric mouse decision support system for user’s emotional and labour productivity analysis. Larivière, B., Bowen, D., Andreassen, T. W., Kunz, W., Sirianni, N. J., Voss, C., & De Keyser, A. (2017). “Service Encounter 2.0”: An investigation into the roles of technology, employees and customers. Journal of Business Research, 79, 238-246. Loghmani, M. R., Rovetta, S., and Venture, G. (2017, May). Emotional intelligence in robots: Recognizing human emotions from daily-life gestures. In Robotics and Automation (ICRA), 2017 IEEE International Conference on (pp. 1677-1684). IEEE. Lusch, R. F., and Vargo, S. L. 2014. Service-Dominant Logic: Premises, Perspectives, Possibilities, Cambridge, UK: Cambridge University Press. Vargo, S. L., Maglio, P. P., Akaka, M. A. (2008). On value and value co-creation: A service systems and service logic perspective. European management journal, 26(3), 145-152. Marcelino, I., Laza, R., Domingues, P., Gómez-Meire, S., and Pereira, A. (2015). eServices-Service Platform for Pervasive Elderly Care. In Ambient IntelligenceSoftware and Applications(pp. 203-211). Springer, Cham. Mayer, J., Salovey, P., Caruso, D. and Sitarenios, G. (2003). Measuring emotional intelligence with the MSCEIT V 2.0. Emotion, 3(1): 97–105. Myakala, P. R., Nalumachu, R., Sharma, S., and Mittal, V. K. (2017, November). A low cost intelligent smart system for real time infant monitoring and cry detection. In Region 10 Conference, TENCON 2017-2017 IEEE (pp. 2795-2800). IEEE. O'Leary, D. E. (2013). Artificial intelligence and big data. IEEE Intelligent Systems, 28(2), 96-99. Pinhanez, C. (2008). Service systems as customer-intensive systems and its implications for service science and engineering. In Hawaii International Conference on System Sciences, Proceedings of the 41st Annual (pp. 117-117). IEEE. Povoda, L., Arora, A., Singh, S., Burget, R., and Dutta, M. K. (2015, October). Emotion recognition from helpdesk messages. In Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), 2015 7th International Congress on (pp. 310-313). IEEE. Reis, J., Amorim, M., and Melão, N. (2018, September). Omni-Channel Service Architectures in a TechnologyBased Business Network: An Empirical Insight. In International Conference on Exploring Service Science (pp. 31-44). Springer, Cham. Rybak, N., Hassall, M., Parsa, K., & Angus, D. J. (2017, October). New real-time methods for operator situational awareness retrieval and higher process safety in the control room. In Systems Engineering Symposium (ISSE), 2017 IEEE International (pp. 1-7). IEEE. 1149 Salovey, P. and Mayer, D. 1990. Emotional intelligence. Imagination, Cognition and Personality, 9(3): 185–211. Sampson, S. E. (2010). The unified service theory. In Handbook of service science (pp. 107-131). Springer, Boston, MA. Sampson, S. E., and Spring, M. (2012). Customer roles in service supply chains and opportunities for innovation. Journal of Supply Chain Management, 48(4), 30-50. Schumann, J. H., Wünderlich, N. V., and Wangenheim, F. (2012). Technology mediation in service delivery: A new typology and an agenda for managers and academics. Technovation, 32(2), 133-143. Spohrer, J. C., & Maglio, P. P. (2010). Toward a science of service systems. In Handbook of service science (pp. 157-194). Springer, Boston, MA. Subramainan, L., Mahmoud, M. A., Ahmad, M. S., & Yusoff, M. Z. M. (2017, June). A Simulator’s Specifications for Studying Students’ Engagement in a Classroom. In International Symposium on Distributed Computing and Artificial Intelligence(pp. 206-214). Springer, Cham. Thatcher, S., and Kilingaru, K. (2012). Intelligent monitoring of flight crew situation awareness. In Advanced Materials Research (Vol. 433, pp. 6693-6701). Trans Tech Publications. Tuyen, N. T. V., Jeong, S., and Chong, N. Y. (2017, June). Learning human behavior for emotional body expression in socially assistive robotics. In Ubiquitous Robots and Ambient Intelligence (URAI), 2017 14th International Conference on (pp. 45-50). IEEE. Wilson, A., Zeithaml, V. A., Bitner, M. J., & Gremler, D. D. (2012). Services marketing: Integrating customer focus across the firm(No. 2nd Eu). McGraw Hill. Yang, C., Huang, Q., Li, Z., Liu, K., & Hu, F. (2017). Big Data and cloud computing: innovation opportunities and challenges. International Journal of Digital Earth, 10(1), 13-53. Xu, J., Huang, Z., Shi, M., & Jiang, M. (2017, September). Emotion Detection in E-learning Using ExpectationMaximization Deep Spatial-Temporal Inference Network. In UK Workshop on Computational Intelligence (pp. 245-252). Springer, Cham.