Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

A heartbeat-based study of attention in the detection of digital alarms from focused and distributed supervisory control systems

  • Original Article
  • Published:
Cognition, Technology & Work Aims and scope Submit manuscript

Abstract

Connected operators of future factories will interact with different devices to be ready to acknowledge any request to intervene or to take back control. The paper studies this ability by proposing a heartbeat-based alarm monitoring system to study human attention when acknowledging a request for action. Request to intervene is done via alarms from two configurations of a digital supervisory control system. Digital interactions of the focused configuration are performed on one touchscreen, whereas those of the distributed configuration are performed on separate touchscreens. Experiments were carried out with two conditions, with four levels of increasing difficulty on each configuration: one synchronous condition in which the flashing and beeping frequency of two alarms occurred synchronously with the heart rate of the participants and one asynchronous condition in which they were not. Four main significant results are obtained: (1) participants for which the alarms are synchronized with their heartbeats made significantly more errors of detection than the others; (2) participants are not really aware of such degradations; (3) these results are obtained for both configurations; (4) when a secondary task occurs, the alarm area scan rates in the synchronous condition are significantly lower than those in the asynchronous condition. Future research about connected operator will focus on the deep understanding of human abilities when the frequency of signals are synchronized with heart rate by studying different synchronization parameters and interaction modes. They will support the design of shared control process between humans and machines when, for instance, automated supports request a human intervention or when automated actions have to be canceled or validated by humans before a delay.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

Data available on request from the authors.

References

  • Alcácer V, Cruz-Machado V (2019) Scanning the industry 4.0: a literature review on technologies for manufacturing systems. Eng Sci Technol Int J 22:899–919

    Google Scholar 

  • Ansari F, Erol S, Sihna W (2018) Rethinking human-machine learning in industry 4.0: how does the paradigm shift treat the role of human learning? Proc Manuf 23:117–122

    Google Scholar 

  • Ballard JC (1996) Computerized assessment of sustained attention: a review of factors affecting vigilance performance. J Clin Exp Neuropsychol 18(6):843–863

    Article  Google Scholar 

  • Baltzer MCA, López D, Flemisch F (2019) Towards an interaction pattern language for human machine cooperation and cooperative movement. Cog Technol Work 21:593–606

  • Beatty J (1982) Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol Bullet 91(2):276–292

  • Benedetto S, Pedrotti M, Minin L, Baccino T, Re A, Montanari R (2011) Driver workload and eye blink duration. Transp Res F 14(3):199–208

    Article  Google Scholar 

  • Byers JC, Bittner AC, Hill SG (1989) Traditional and raw task load index (TLX) correlations: are paired comparisons necessary? In: Mital A (ed) Advances in industrial ergonomics and safety. Taylor & Francis, London, pp 481–485

    Google Scholar 

  • Chen J, Wang RQ, Lin Z, Guo X (2018) Measuring the cognitive loads of construction safety sign designs during selective and sustained attention. Saf Sci 105:9–21

    Article  Google Scholar 

  • Comstock JL, Arnegard RJ (1992) The multi attribute task battery for human operator workload and strategic behavior research. Technical Report 104174 NASA Langley Research Center, Hampton

    Google Scholar 

  • Crichton N (2001) Visual analogue scale (VAS). J Clin Nurs 10(5):697–706

    Google Scholar 

  • Currie J, Bond RR, McCullagh P, Black P, Finlay DD, Peace A (2018) Eye tracking the visual attention of nurses interpreting simulated vital signs scenarios: mining metrics to discriminate between performance level. IEEE Trans Hum Mach Syst 48(2):113–124

    Article  Google Scholar 

  • Dey A, Chen H, Zhuang C, Billinghurst M, Lindeman RW (2018) Effects of sharing real-time multi-sensory heart rate feedback in different immersive collaborative virtual environments. Proceedings of the 2018 IEEE international symposium on mixed and augmented reality (ISMAR), October 16–20, 2018, Munich, Germany

  • Eriksson A, Stanton NA (2017) Takeover time in highly automated vehicles: noncritical transitions to and from manual control. Hum Factors 59(4):689–705

    Article  Google Scholar 

  • Festinger L (1957) A theory of cognitive dissonance. Stanford University Press, Stanford, CA

    Book  Google Scholar 

  • Findlay JM (2003) Visual selection, covert attention and eye movements? In: Findlay JM, Gilchrist ID (eds) Active vision: the psychology of looking and seeing. Oxford Psychology Series, UK, pp 35–54

    Chapter  Google Scholar 

  • Flemisch F, Heesen M, Hesse T, Kelsch J, Schieben A, Beller J (2012) Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations. Cogn Technol Work 14:3–18

    Article  Google Scholar 

  • Flemisch F, Abbink D, Itoh M, Pacaux-Lemoine M-P (2019) Special issue on shared and cooperative control. Cogn Technol Work 21:553–554

    Article  Google Scholar 

  • Fletcher K, Neal A, Yeo G (2017) The effect of motor task precision on pupil diameter. Appl Ergon 65:309–315

    Article  Google Scholar 

  • Fogarty C, Stern JA (1989) Eye movements and blinks: Their relationship to higher cognitive processes. Int J Psychophysiol 8(1):35–42

    Article  Google Scholar 

  • Frank A, Mitschang B (2002) A customizable shared information space to support concurrent design. Comput Ind 48(1):45–57

    Article  Google Scholar 

  • Galluscio EH, Fjelde K (1993) Eye movement and reaction time measures of the effectiveness of caution signs. Saf Sci 16(5–6):627–635

    Article  Google Scholar 

  • Geisler FCM, Vennewald N, Kubiak T, Weber H (2010) The impact of heart rate variability on subjective well-being is mediated by emotion regulation. Personal Individ Differ 49:723–728

    Article  Google Scholar 

  • Gonzalez AGC, Alves MVS, Viana GS, Carvalho LK, Basilio JC (2018) Supervisory control-based navigation architecture: a new framework for autonomous robots in industry 4.0 environments. IEEE Trans Ind Inform 14(4):1732–1743

    Article  Google Scholar 

  • Gorecky D, Schmitt M, Loskyll M, Zühlke D (2014) Human-machine-interaction in the industry 4.0 Era. 12th IEEE International Conference on industrial informatics (INDIN), Porto Alegre, Brazil, 27–30 July 2014

  • Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol 52:139–183

    Article  Google Scholar 

  • Hidalgo-Muñoz AR, Mouratille D, Matton N, Caussec M, Rouillard Y, El-Yagoubi R (2018) Cardiovascular correlates of emotional state, cognitive workload and timeon-task effect during a realistic flight simulation. Int J Psychophysiol 128:62–69

    Article  Google Scholar 

  • Inagaki T, Sheridan T (2019) A critique of the SAE conditional driving automation definition, and analyses of options for improvement. Cogn Technol Work 21:569–578

    Article  Google Scholar 

  • Itoh M (2009) Individual differences in effects of secondary cognitive activity during driving on temperature at the nose tip. Proceedings of the 2009 International conference on mechatronics and automation, August 9–12, 2009, Changchun, China

  • Jimenez V, Vanderhaegen F (2019) Dissonance oriented stability analysis of Cyber-Physical & Human Systems. IFAC-PapersOnLine 51(34):230–235

  • Jones PM, Jasek CA (1997) Intelligent support for activity management (ISAM): an architecture to support distributed supervisory control. IEEE Trans Syst Man Cybern 27:274–288

    Article  Google Scholar 

  • Kamezaki M, Hayashi H, Manawadu UE, Sugano S (2020) Human-centered intervention based on tactical-level input in unscheduled takeover scenarios for highly-automated vehicles. Int J Intell Transp Syst Res 18:451–460

    Google Scholar 

  • Kelsch J, Temme G, Schindler J (2013) Arbitration based framework for design of holistic multimodal human-machine interaction. Contributions to AAET 2013, 6.-7. Feb. 2013, Braunschweig

    Google Scholar 

  • Kervern G-Y (1995) Eléments fondamentaux des cindyniques (Fondamental elements of cindynics). Economica Editions, Paris

    Google Scholar 

  • Laudante E (2017) Industry 4.0, innovation and design. A new approach for ergonomic analysis in manufacturing system. Design for Next, 12th European Academy of Design Conference, Roma, Italy, April 12–14, 2017

  • Longo F, Nicoletti L, Padovano A (2017) Smart operators in industry 4.0: a human-centered approach to enhance operators’ capabilities and competencies within the new smart factory context. Comput Ind Eng 113:144–159

    Article  Google Scholar 

  • Mach, S., Kastrau, A., Schmalfuß, F. (2018). Information at hand—using wearable devices to display task information in the context of industry 4.0. International conference on human-computer interaction, Las Vegas, USA, July 15–20, 2018, p. 93–100

  • Morignot P, Perez Rastelli J, Nashashibi F (2014) Arbitration for balancing control between the driver and ADAS systems in an automated vehicle: survey and approach. 2014 IEEE intelligent vehicles symposium (IV), June 8–11, Dearborn, Michigan, USA, p 575–580.

  • Nakajima Y, Nozaki T, Ohnishi K (2014) Heartbeat synchronization with haptic feedback for telesurgical robot. IEEE Trans Ind Electron 61(7):3753–3764

    Article  Google Scholar 

  • Oken BS, Salinsky MC, Elsas SM (2006) Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin Neurophysiol 117:1885–1901

    Article  Google Scholar 

  • Or CKL, Duffy VG (2007) Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occup Ergon 7:83–94

    Article  Google Scholar 

  • Pacaux-Lemoine M-P, Debernard S (2002) Common work space for human–machine cooperation in air traffic control. Control Eng Pract 10(5):571–576

    Article  Google Scholar 

  • Pacaux-Lemoine M-P, Flemisch F (2019) Layers of shared and cooperative control, assistance, and automation. Cogn Technol Work 21:579–591

    Article  Google Scholar 

  • Pacaux-Lemoine M-P, Trentesaux D (2019) Ethical risks of human-machine symbiosis in Industry 4.0: insights from the human-machine cooperation approach. IFAC PapersOnLine 52(19):19–24

    Article  MathSciNet  Google Scholar 

  • Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybernet Part A: Syst Human 30(3):286–297

  • Peruzzini M, Grandi F, Pellicciari M (2017) Benchmarking of tools for user experience analysis in industry 4.0. Proc Manuf 11:806–813

    Google Scholar 

  • Pfeiffer S (2016) Robots industry 4.0 and humans or why assembly work is more than routine work. Societies 6:16. https://doi.org/10.3390/soc6020016

    Article  Google Scholar 

  • Pizziol S, Dehais F, Tessier C (2011) Towards human operator state assessment. Proceedings of the 1st international conference on application and theory of automation in command and control systems. IRIT Press, p 99–106

  • Polet P, Vanderhaegen F, Zieba S (2012) Iterative learning control based tools to learn from human error. Eng Appl Artif Intell 25(7):1515–1522

    Article  Google Scholar 

  • Rauch E, Linder C, Dallaseg P (2019) Anthropocentric perspective of production before and within Industry 4.0. Computers Ind Eng. https://doi.org/10.1016/j.cie.2019.01.018 (In Press)

    Article  Google Scholar 

  • Recarte MA, Pérez E, Conchillo Á, Nunes LM (2008) Mental workload and visual impairment: differences between pupil, blink, and subjective rating. Span J Psychol 11(2):374–385

    Article  Google Scholar 

  • Romero D, Stahre J, Wuest T, Noran O (2016) Towards an Operator 4.0 typology: a human-centric perspective on the fourth industrial revolution technologies. International conference on computers and industrial engineering (CIE46), At Tianjin, China

  • Rosch JL, Vogel-Walcutt JJ (2013) A review of eye-tracking applications as tools for training. Cogn Technol Work 15:313–327

    Article  Google Scholar 

  • Ruppert T, Jaskó S, Holczinger T, Abonyi T (2018) Enabling technologies for operator 4.0: a survey. Appl Sci 8(9):1650. https://doi.org/10.3390/app8091650

    Article  Google Scholar 

  • Salomon R, Ronchi R, Dönz J, Bello-Ruiz J, Herbelin B, Martet R, Faivre N, Schaller K, Blanke O (2016) The insula mediates access to awareness of visual stimuli presented synchronously to the heartbeat. J Neurosci 36(18):5115–5127

    Article  Google Scholar 

  • Save L, Feuerberg B (2012) Designing human-automation interaction: a new level of automation taxonomy. In: D. de Waard, K. Brookhuis, F. Dehais, C. Weikert, S. Röttger, D. Manzey, S. Biede, F. Reuzeau, and P. Terrier (Eds.). Human Factors: a view from an integrative perspective. Proceedings HFES Europe chapter conference toulouse. ISBN 978–0–945289–44–9. Available from http://hfes-europe.org. p. 43–55.

  • Segura A, Diez HV, Barandiaran I, Arbelaiz A, Álvarez H, Simões B, Posada J, García-Alonso A, Ugart R (2018) Visual computing technologies to support the operator 4.0. Computers Ind Eng. https://doi.org/10.1016/j.cie.2018.11.060

    Article  Google Scholar 

  • Srinivasan N, Srivastava P, Lohani M, Baijal S (2009) Focused and distributed attention. Progress Brain Res 176:87–100

  • Sheridan TB (1992) Telerobotics, automation, and human supervisory control. MIT Press, USA

    Google Scholar 

  • Taelman J, Vandeput S, Spaepen A, Van Huffel S (2009) Influence of mental stress on heart rate and heart rate variability. In: Vander Sloten J., Verdonck P., Nyssen M., Haueisen J. (eds), 4th European conference of the international federation for medical and biological engineering. IFMBE Proceedings, vol 22. Springer, Berlin, Heidelberg, p. 1366–1369

  • Tsai YF, Viirre E, Strychacz C, Chase B, Jung TP (2007) Task performance and eye activity: predicting behavior relating to cognitive workload. Aviat Space Environ Med 78:176–185

    Google Scholar 

  • Vanderhaegen F (1999) Cooperative system organisation and task allocation: illustration of task allocation in air traffic control. Le Travail Humain 62(3):197–222

    Google Scholar 

  • Vanderhaegen F (2012) Cooperation and learning to increase the autonomy of ADAS. Cogn Technol Work 14(1):61–69

    Article  Google Scholar 

  • Vanderhaegen F (2014) Dissonance engineering: a new challenge to analyse risky knowledge when using a system. Int J Computers Commun Control 9(6):750–759

    Article  Google Scholar 

  • Vanderhaegen F (2016) A rule-based support system for dissonance discovery and control applied to car driving. Expert Syst Appl 65:361–371

    Article  Google Scholar 

  • Vanderhaegen F (2018) Dissonances d’usages, opportunités et menaces : vers une démarche d’ingénierie cognitive de leur analyse. Proceedings of the Ergo’IA 2018 conference, NY : ACM, Bidart-Biarritz, France, October

  • Vanderhaegen F (2019) Pedagogical learning supports based on human–systems inclusion applied to rail flow control. Cogn Technol Work. https://doi.org/10.1007/s10111-019-00602-2 (in press)

    Article  Google Scholar 

  • Vanderhaegen F, Jimenez V (2018) The amazing human factors and their dissonances for autonomous Cyber-Physical&Human Systems. First IEEE conference on industrial cyber-physical systems, Saint-Petersbourg, Russia, 14–18 May, 2018, p 597–602.

  • Vanderhaegen F, Chalmé S, Anceaux F, Millot P (2006) Principles of cooperation and competition—application to car driver behavior analysis. Cogn Technol Work 8:183–192

    Article  Google Scholar 

  • Vanderhaegen F, Wolff M, Ibarboure S, Mollard R (2019) Heart-computer synchronization interface to control human-machine symbiosis: a new human availability support for cooperative systems. Proceedings of the 14th IFAC/IFIP/IFORS/IEA symposium on analysis, design, and evaluation of human-machine systems, Tallinn, Estonia, September 16–19

  • Vanderhaegen F, Wolff M, Mollard R (2020) Non-conscious errors in the control of dynamic events synchronized with heartbeats: a new challenge for human reliability study. Saf Sci. https://doi.org/10.1016/j.ssci.2020.104814 (in press)

    Article  Google Scholar 

  • Vlemincx E, Taelman J, De Peuter S, Van Diest I, Van den Bergh O (2011) Sigh rate and respiratory variability during mental load and sustained attention. Psychophysiology 48(1):117–200

    Article  Google Scholar 

  • Wang L, Wang G (2016) Big data in cyber-physical systems, digital manufacturing and industry 4.0. Int J Eng Manuf 4:1–8. https://doi.org/10.5815/ijem.2016.04.01

    Article  Google Scholar 

  • Zieba S, Polet P, Vanderhaegen F, Debernard S (2010) Principles of adjustable autonomy: a framework for resilient human machine cooperation. Cogn Technol Work 12(3):193–203

    Article  Google Scholar 

  • Zülch G (2014) Evaluating human work in the digital factory—a new German guideline. In IFIP international conference on advances in production management systems (APMS). September, Ajaccio, France

Download references

Acknowledgements

The present research work has been supported by the Scientific Research Network on Integrated Automation and Human–Machine Systems (GRAISyHM), and by the Regional Council of “Hauts-de-France” (Regional Council of Nord—Pas de Calais—Picardie from France), project CONPETISES (Pedagogical control of human driving tasks by automated systems). The author gratefully acknowledges the support of these institutions. They also thank Julien Nelson who helps them to refine the standard use of English of their paper.

Author information

Authors and Affiliations

  1. Univ. Polytechnique Hauts-de-France, LAMIH, CNRS UMR 8201, Le Mont Houy, 59313, Valenciennes Cedex 9, France

    F. Vanderhaegen

  2. INSA Hauts-de-France Le Mont Houy , 59313, CEdex 9, Valenciennces CEdex 9, France

    F. Vanderhaegen

  3. Univ. Paris Descartes, COGNAC-G, CNRS UMR 8257, 45, rue des Saints-Pères, 75006, Paris, France

    M. Wolff & R. Mollard

  4. ESTIA-PEPSS, Technopole Izarbel, 64210, Bidart, France

    M. Wolff & R. Mollard

Authors
  1. F. Vanderhaegen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Wolff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Mollard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Vanderhaegen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vanderhaegen, F., Wolff, M. & Mollard, R. A heartbeat-based study of attention in the detection of digital alarms from focused and distributed supervisory control systems. Cogn Tech Work 25, 119–134 (2023). https://doi.org/10.1007/s10111-022-00720-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10111-022-00720-4

Keywords

Search

Navigation