Cited By
View all- Colabianchi SCostantino FSabetta N(2024)Assessment of a large language model based digital intelligent assistant in assembly manufacturingComputers in Industry10.1016/j.compind.2024.104129162:COnline publication date: 1-Nov-2024
Voice user interfaces in manufacturing logistics: a literature review
Pages 627 - 639
Abstract
Due to the increasing digitalization in manufacturing logistics, devices to integrate the worker into the digital manufacturing system are necessary. A voice user interface (VUI) can be considered suitable for this purpose due to its flexibility and intuitive operability. Despite the popularity and acceptance of VUIs in everyday life, their use in industrial applications, especially in manufacturing logistics, is still rare. While VUIs have been successfully used in order picking for decades, hardly any other industrial fields of application exist. In this paper, we have identified various barriers to the use of VUI in industrial applications. We categorized them and identified four key barriers. We then conducted a systematic literature review to determine and compare already investigated application areas of VUIs, their characteristics, advantages and disadvantages. We found that in particular the operation of machines and industrial robots, as well as general data and information output on machine and system status, maintenance and employee training are frequently investigated. It is noticeable that VUIs are often used in combination with other user interfaces (UIs). Some challenges to VUI usage, such as high ambient noise levels, have already been solved through various approaches, while other challenges remain. Based on the results of the literature review, we put forward a research agenda regarding further suitable industrial application areas as well as general challenges for the use of VUIs in industrial environments.
References
[1]
Abner, B., J. Rabelo, R., Popov Zambiasi, S., & Romero, D. (2020). Production management as-a-service: A softbot approach. In APMS 2020: Advances in production management systems: Towards smart and digital manufacturing (pp. 19–30). Springer.
[2]
Afanasev, M. Y., Fedosov, Y. V., Andreev, Y. S., Krylova, A. A., Shorokhov, S. A., Zimenko, K. V., & Kolesnikov, M. V. (2019). A concept for integration of voice assistant and modular cyber-physical production system. In 2019 IEEE 17th international conference on industrial informatics (INDIN) (pp. 27–32).
[3]
Ajaykumar G, Steele M, and Huang C-M A survey on end-user robot programming ACM Computing Surveys 2021 54 8 1-36
[4]
Angleraud A, Sefat AM, Netzev M, and Pieters R Coordinating shared tasks in human-robot collaboration by commands Frontiers in Robotics and AI 2021
[5]
Badave, A., Kokare, P. S., & Deshmukh, P. (2020). ALEXA technology for industrial automation system. International Journal of Future Generation Communication and Networking, 13(2).
[6]
Birch B, Griffiths C, and Morgan A Environmental effects on reliability and accuracy of MFCC based voice recognition for industrial human-robot-interaction Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture 2021 235 12 1939-1948
[7]
Bohus, D., & Rudnicky, A. I. (2005). LARRI: A language-based maintenance and repair assistant. In W. Minker, D. Bühler, & L. Dybkjær (Eds), Spoken multimodal human-computer dialogue in mobile environments (pp. 203–218). Springer.
[8]
Bommi RM et al. Speech and gesture recognition interactive robot Materials Today: Proceedings. 2021 47 37-40
[9]
Chakraborty, S., Mukherjee, S., Saha, S. K., & Saha, H. N. (2019). Autonomous vehicle for industrial supervision based on google assistant services & IoT analytics. In 2019 IEEE 10th annual information technology, electronics and mobile communication conference (IEMCON) (pp 1061–1070).
[10]
Chan KY, Yiu CKF, Dillon TS, and Nordholm S Enhancement of speech recognitions for control automation using an intelligent particle swarm optimization IEEE Transactions on Industrial Informatics 2012 8 4 869-879
[11]
Costa, D., Pires, F., Rodrigues, N., Barbosa, J., Igrejas, G., & Leitão, P. (2019). Empowering humans in a cyber-physical production system: human-in-the-loop perspective. In 2019 IEEE international conference on industrial cyber physical systems (ICPS) (pp. 139–144).
[12]
de Bem A, Secinaro S, Calandra D, and Lanzalonga F Knowledge management and digital transformation for Industry 4.0: A structured literature review Knowledge Management Research & Practice 2022 20 320-338
[13]
de Vries J, de Koster R, and Stam D Exploring the role of picker personality in predicting picking performance with pick by voice, pick to light and RF-terminal picking International Journal of Production Research 2016 54 8 2260-2274
[14]
Dujmesic N, Bajor I, and Rozic T Warehouse processes improvement by pick by voice technology Tehnički Vjesnik-Technical Gazette (TV-TG) 2018 25 4 1227-1233
[15]
Fischer, J., Pantförder, D., & Vogel-Heuser, B. (2017). Improvement of maintenance through speech interaction in cyber-physical production systems. In 2017 IEEE 15th international conference on industrial informatics (INDIN) (pp. 290–295).
[16]
Fleiner, C., Riedel, T., Beigl, M., & Ruoff, M. (2021). Ensuring a robust multimodal conversational user interface during maintenance work. In MuC ‘21: Mensch und Computer 2021 (pp. 79–91).
[17]
Gärtler, M., & Schmidt, B. (2021). Practical challenges of virtual assistants and voice interfaces in industrial applications. In Proceedings of the 54th Hawaii international conference on system sciences (pp. 4063–4072).
[18]
Gorecky, D., Schmitt, M., Loskyll, M. & Zühlke, D., 2014. Human-machine-interaction in the industry 4.0 era. In 2014 12th IEEE international conference on industrial informatics (INDIN), pp. 289–294.
[19]
Gundecha, T. J., Navale, M., Chatrabhuj, S. A., Solanke, A. V., & Ghorpade, H. B. (2020). Automation of mechanical press machine using revolution Pi and PLC. International Journal of Future Generation Communication and Networking, 13(2), 118–126.
[20]
Gustavsson, P., Syberfeldt, A., Brewster, R., & Wang, L. (2017). Human-robot collaboration demonstrator combining speech recognition and haptic control. Manufacturing Systems 4.0—Proceedings of the 50th CIRP Conference on Manufacturing Systems, 63, 396–401.
[21]
Haslwanter, J. D. H., Heiml, M., & Wolfartsberger, J. (2019). Lost in translation: Machine translation and text-to-speech in industry 4.0. PETRA ‘19: Proceedings of the 12th ACM international conference on pervasive technologies related to assistive environments (pp. 333–342).
[22]
Hüsson, D., & Holland, A. (2019). Intelligent personal assistant and reporting—explaining data to users through speech synthesis A prototype for user voice interaction and descriptive analytics in a web-based ERP-system. In 10th conference professional knowledge management.
[23]
Hüsson D, Holland A, and Sánchez RA Intelligent personal assistant in business-context: key-feature evaluation for user acceptance Business Systems Research 2020 11 3 147-166
[24]
Karomati Baroroh D, Chu C-H, and Wang L Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence Journal of Manufacturing Systems 2021 61 696-711
[25]
Liu H et al. Deep learning-based multimodal control interface for human-robot collaboration Procedia CIRP 2018 72 3-8
[26]
Longo F, Nicoletti L, and Padovano A Ubiquitous knowledge empowers the smart factory: The impacts of a service-oriented digital twin on enterprises’ performance Annual Reviews in Control 2019 47 221-236
[27]
Longo F and Padovano A Voice-enabled assistants of the operator 4.0 in the social smart factory: Prospective role and challenges for an advanced human-machine interaction Manufacturing Letters 2020 26 12-16
[28]
Mallikarjuna, K., Kumar, A. S., Bala Krishna, A, Prasad, P. V. R. D., & Raju, M. S. V. S. B. (2016). Parametric studies on motion intensity factors in a robotic welding using speech recognition. In 2016 IEEE 6th international conference on advanced computing (IACC) (pp. 415–420).
[29]
Menolotto M et al. Motion capture technology in industrial applications: A systematic review Sensors 2020 20 19 5687
[30]
Mentzas, G. (2021). Human-AI collaboration in quality control with augmented manufacturing analytics. In APMS 2021: Advances in production management systems— Artificial Intelligence for sustainable and resilient production systems (pp. 303–310).
[31]
Miller, A. (2004). Order picking for the 21st century. Manufacturing & Logistics IT.
[32]
Nayyar, A., & Kumar, A. (2020). A roadmap to industry 4.0: Smart production, sharp business and sustainable development. Springer.
[33]
Norberto PJ Robot-by-voice: Experiments on commanding an industrial robot using the human voice Industrial Robot: an International Journal. 2004 32 505-511
[34]
Panetto H et al. Challenges for the cyber-physical manufacturing enterprises of the future Annual Reviews in Control 2019 47 200-213
[35]
Pazienza, A., Macchiarulo, N., Vitulano, F., & Fiorentini, A. (2019). A novel integrated industrial approach with cobots in the age of industry 4.0 through conversational interaction and computer vision. In Sixth Italian conference on computational linguistics (CLiC-it2019).
[36]
Pires JN Robot-by-voice: Experiments on commanding an industrial robot using the human voice Industrial Robot 2005 32 6 505-511
[37]
Rabelo, R. J., Romero, D., & Zambiasi, S. P. (2018). Softbots supporting the operator 4.0 at smart factory environments. In Advances in production management systems: Smart manufacturing for industry 4.0 (pp. 456–464).
[38]
Rabelo, R. J., Zambiasi, S. P., & Romero, D. (2019). Collaborative softbots: Enhancing operational excellence in systems of cyber-physical systems. In L. M. Camarinha-Matos, H. Afsarmanesh, & D. Antonelli (Eds.), Collaborative networks and digital transformation. Springer.
[39]
Rogowski, A. (2010). Robotized cell remote control using voice commands in natural language. In 2010 15th international conference on methods and models in automation and robotics (pp. 383–386).
[40]
Rogowski A Industrially oriented voice control system Robotics and Computer-Integrated Manufacturing 2012 28 3 303-315
[41]
Rogowski A Remote programming and control of the flexible machining cell International Journal of Computer Integrated Manufacturing 2013 28 6 650-663
[42]
Rogowski A Web-based remote voice control of robotized cells Robotics and Computer-Integrated Manufacturing 2013 29 4 77-89
[43]
Rogowski A and Skrobek P Object identification for task-oriented communication with industrial robots Sensors (basel, Switzerland) 2020 20 1773
[44]
Romero, D., Stahare, J., Wuest, T., & Noran, O. (2016). Towards an operator 4.0 typology: A human-centric perspective on the fourth industrial revolution technologies. In International conference on computers & industrial engineering (CIE46).
[45]
Schlick C et al. Head-mounted display for supervisory control in autonomous production cells Displays 1997 17 3–4 199-206
[46]
Schmidt, B., Borrison, R., Cohen, A., Dix, M., Gärtler, M., Hollender, M., Klöpper, B., Maczey, S., & Siddharthan, S (2018). Industrial virtual assistants: Challenges and opportunities. In UbiComp ‘18: Proceedings of the 2018 ACM international joint conference and 2018 international symposium on pervasive and ubiquitous computing and wearable computers (pp 794–801).
[47]
Schwartz, T., Zinnikus, I., Krieger, H-U., Bürckert, C., Folz, J., Kiefer, B., Hevesi, P., Lüth, C., Pirkl, G., Spieldenner, T., Schmitz, N., Wirkus, M., & Straube, S. (2016). Hybrid teams: Flexible collaboration between humans, robots and virtual agents. In Multiagent system technologies (pp 131–146).
[48]
Serras, M., García-Sardiña, L., Simões, B., & Alvarez, H. (2020). AREVA: Augmented reality voice assistant for industrial maintenance. Procesamiento de Lenguaje Natural,65, 135–138.
[49]
Sim E-S, Lee H-G, Lee J-C, and Park J-W Efficient work measurement system of manufacturing cells using speech recognition and digital image processing technology International Journal of Advanced Manufacturing Technology. 2006 29 772-785
[50]
Statista Research Department. (2018). Retrieved from https://www.statista.com/statistics/503466/iot-subsystems-revenue-worldwide/
[51]
Statista Research Department. (2021). Operational stock of multipurpose industrial robots worldwide from 2010 to 2020. Retrieved from https://www.statista.com/statistics/281380/estimated-operational-stock-of-industrial-robots-worldwide/
[52]
Statista Research Department. (2022). Number of digital voice assistants in use worldwide from 2019 to 2024 (in billions). Retrieved from https://www.statista.com/statistics/973815/worldwide-digital-voice-assistant-in-use/
[53]
Stocker A, Brandl P, Michalczuk R, and Rosenberger M Mensch-zentrierte IKT-Lösungen in einer smart factory E & I Elektrotechnik und Informationstechnik 2014 10 207-211
[54]
Strandhagen JW, Alfnes E, Strandhagen JO, and Vallandingham LR The fit of Industry 4.0 applications in manufacturing logistics: A multiple case study Advanced Manufacturing 2017 5 344-358
[55]
Tanwar, S., & Lavingia, K. (2020). Augmented reality and industry 4.0. In Advances in science, technology & innovation (ASTI), IEREK interdisciplinary series for sustainable development (pp. 143–155).
[56]
Tsarouchi P, Makris S, and Chryssolouris G Human–robot interaction review and challenges on task planning and programming International Journal of Computer Integrated Manufacturing 2016 29 8 916-931
[57]
Udoka SJ Automated data capture techniques: A prerequisite for effective integrated manufacturing systems Automated Data Capture Techniques: A Prerequisite for Effective Integrated Manufacturing Systems, Computers & Industrial Engineering 1991 21 1–4 217-221
[58]
Vajpai J and Bora A Industrial applications of automatic speech recognition systems International Journal of Engineering Research and Applications 2016 6 3 88-95
[59]
Villani, V., Pini, F., Leali, F., Secchi, C., & Fantuzzi, C. (2018a). Survey on human-robot interaction for robot programming in industrial applications. In 16th IFAC symposium on information control problems in manufacturing (INCOM) (pp. 66–71).
[60]
Villani V et al. A general methodology for adapting industrial HMIs to human operators IEEE Transactions on Automation Science and Engineering 2021 18 1 164-175
[61]
Villani V, Pini F, Leali F, and Secchi C Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications Mechatronics 2018 55 248-266
[62]
Wasfy A, Wasfy T, and Noor A Intelligent virtual environment for process training Advances in Engineering Software 2004 35 6 337-355
[63]
Webster, J., & Watson, R. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2).
[64]
Wei, H., Jiang-Qi, B., & Xiao-Hua, C. (2010). Research and implementation of wireless portable maintenance aid for China-made large aircraft. In International conference on computational problem-solving (pp. 432–435).
[65]
Wei, H., & Xincun, S. (2012). VUI system of the portable maintenance aids based on cloud computing. In 2012 international conference on computational problem-solving (ICCP) (pp. 144–146).
[66]
Wellsandt S, Foosherian M, and Thoben K-D Interacting with a digital twin using Amazon Alexa Procedia Manufacturing 2020 52 4-8
[67]
Wellsandt, S., Rusak, Z., Arenas, S. R., Aschenbrenner, D., Hribernik, K. A., Thoben, K-D. (2020b). Concept of a voice-enabled digital assistant for predictive maintenance in manufacturing. In TESConf 2020b—9th international conference on through-life engineering services.
[68]
Wellsandt, S., Hribernik, K., & Thoben, K.-D. (2021a). Anatomy of a digital assistant. In APMS 2021A: Advances in production management systems—Artificial intelligence for sustainable and resilient production systems (pp. 321–330).
[69]
Wellsandt, S., Klein, K., Hribernik, K., Lewandowski, M., Bousdekis, A., Mentzas, G., Thoben, K-D. (2021b). Towards using digital intelligent assistants to put humans in the loop of predictive maintenance systems. IFAC-PapersOnLine (pp. 49–54).
[70]
Zheng, S., Foucault, C., Silva, P., Dasari, S., Yang, T., & Goose, S. (2015). Eye-wearable technology for machine maintenance: Effects of display position and hands-free operation. In Proceedings of the 33rd annual ACM conference on human factors in computing systems (pp. 2125–2134).
[71]
Zhu, Z., Branzoi, V., Wolverton, M., Murray, G., Vitovitch, N., Yarnall, L., Acharya, G., Samarasekera, S., & Kumar, R. (2014). AR-mentor: Augmented reality based mentoring system. In 2014 IEEE international symposium on mixed and augmented reality (ISMAR) (pp. 17–22).
[72]
Zigart, T., & Schlund, S. (2020). Evaluation of augmented reality technologies in manufacturing: A literature review. In I. L. Nunes (Ed.), Advances in human factors and systems interaction: AHFE 2020—Advances in intelligent systems and computing (pp. 75–83). Springer.
Recommendations
Manufacturing Execution System for a Subsidiary of Aerospace Manufacturing Industry
ICCAE '09: Proceedings of the 2009 International Conference on Computer and Automation EngineeringAbstract—Aerospace manufacturing requires precision tools and control systems. Integrated Manufacturing Execution System (MES) in a subsidiary of aerospace manufacturing industry initiate, guide, respond to and report on manufacturing activities as they ...
Analysis on Linkage Mechanism between Manufacturing and Logistics Industry
ICEE '10: Proceedings of the 2010 International Conference on E-Business and E-GovernmentThe financial crisis is the important historical period of China's economic structural adjustment and industrial restructuring, manufacturers and logistics companies need to focus on the overall situation and work together to overcome the current ...
Assessing the Utility of the System Usability Scale for Evaluating Voice-based User Interfaces
ChineseCHI '18: Proceedings of the Sixth International Symposium of Chinese CHIVoice-based User Interfaces (VUIs) challenge our existing conceptions of usability since the standardized evaluation tools we use were typically developed for interfaces with visual feedback, whereas VUI's have predominantly eyes-free interactions. We ...
Comments
Information & Contributors
Information
Published In
© The Author(s) 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 01 September 2023
Accepted: 03 August 2023
Received: 04 July 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
- AiF Projekt
- Technische Universität Dresden (1019)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025
Other Metrics
Citations
Cited By
View all- Colabianchi SCostantino FSabetta N(2024)Assessment of a large language model based digital intelligent assistant in assembly manufacturingComputers in Industry10.1016/j.compind.2024.104129162:COnline publication date: 1-Nov-2024