research-article

The effect of noise on software engineers' performance

Authors:

Giuseppe Scanniello,

Natalia Juristo,

Burak TurhanAuthors Info & Claims

ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Article No.: 9, Pages 1 - 10

https://doi.org/10.1145/3239235.3240496

Published: 11 October 2018 Publication History

Abstract

Background: Noise, defined as an unwanted sound, is one of the commonest factors that could affect people's performance in their daily work activities. The software engineering research community has marginally investigated the effects of noise on software engineers' performance.

Aims: We studied if noise affects software engineers' performance in: (i) comprehending functional requirements and (ii) fixing faults in source code.

Method: We conducted two experiments with final-year undergraduate students in Computer Science. In the first experiment, we asked 55 students to comprehend functional requirements exposing them or not to noise, while in the second experiment 42 students were asked to fix faults in Java code.

Results: The participants in the second experiment, when exposed to noise, had significantly worse performance in fixing faults in source code. On the other hand, we did not observe any statistically significant difference in the first experiment.

Conclusions: Fixing faults in source code seems to be more vulnerable to noise than comprehending functional requirements.

References

[1]

S. Abrahão, C. Gravino, E. Insfran, G. Scanniello, and G. Tortora. 2013. Assessing the Effectiveness of Sequence Diagrams in the Comprehension of Functional Requirements: Results from a Family of Five Experiments. IEEE Trans. on Softw. Eng. 39, 3 (2013), 327--342.

Digital Library

[2]

S. T. Acuna, N. Juristo, A. M. Moreno, and A. Mon. 2010. A Software Process Model Handbook for Incorporating People's Capabilities (1st ed.). Springer Publishing Company, Incorporated.

Digital Library

[3]

European Environment Agency and Europæiske Miljøagentur. 2014. Noise in Europe 2014. European Environment Agency.

[4]

V. Basili, G. Caldiera, and D. H. Rombach. 1994. The Goal Question Metric Paradigm, Encyclopedia of Software Engineering. John Wiley and Sons. 528--532 pages.

[5]

G. R. Bergersen, D. I. K. Sjøberg, and T. Dybå. 2014. Construction and Validation of an Instrument for Measuring Programming Skill. IEEE Trans. on Softw. Eng. 40, 12 (2014), 1163--1184.

[6]

D.E. Broadbent. 1978. The current state of noise research: Reply to Poulton. Psychological Bulletin 85 (1978), 1052--1067.

[7]

N. Cliff. 1996. Ordinal methods for behavioral data analysis. Psychology Press, New-York, USA.

[8]

W. J. Conover. 1998. Practical Nonparametric Statistics (3rd ed.). Wiley.

[9]

L.L. Constantine. 1995. Constantine on Peopleware. Yourdon Press.

[10]

M. Csikszentmihalyi. 2009. Creativity: Flow and the Psychology of Discovery and Invention (Harper Perennial Modern Classics). HarperCollins e-books.

[11]

A. Dean, M. Morris, J. Stufken, and D. Bingham. 2015. Handbook of Design and Analysis of Experiments. Chapman and Hall/CRC.

[12]

T. DeMarco and T. Lister. 1985. Programmer Performance and the Effects of the Workplace. In Proc. of the International Conference on Software Engineering. IEEE, 268--272.

Digital Library

[13]

T. DeMarco and T. Lister. 2013. Peopleware: Productive Projects and Teams (3rd ed.). Addison-Wesley Professional.

Digital Library

[14]

F. Fagerholm and J. Münch. 2012. Developer experience: Concept and definition. In International Conference on Software and System Process. IEEE, 73--77.

Digital Library

[15]

J. I. Gallin and F. P. Ognibene. 2007. Principles and Practice of Clinical Research (2nd ed.). Academic Press.

[16]

S. Gievska, R. Lindeman, and J. Sibert. 2005. Examining the qualitative gains of mediating human interruptions during hci. In Proc. of the International Conference on Human-Computer Interaction.

[17]

D. Graziotin, X. Wang, and P. Abrahamsson. 2014. Software Developers, Moods, Emotions, and Performance. IEEE Software 31, 4 (2014), 24--27.

[18]

P. A. Hancock and J. S. Warm. 1989. A Dynamic Model of Stress and Sustained Attention. Human Factors 31, 5 (1989), 519--537.

Digital Library

[19]

Shamsi T. Iqbal and Brian P. Bailey. 2007. Understanding and Developing Models for Detecting and Differentiating Breakpoints During Interactive Tasks. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 697--706.

Digital Library

[20]

A. Jedlitschka, M. Ciolkowski, and D. Pfahl. 2008. Reporting Experiments in Software Engineering. In Guide to Advanced Empirical Software Engineering. Springer London, 201--228.

[21]

N. Juristo and A.M. Moreno. 2001. Basics of Software Engineering Experimentation. Kluwer Academic Publishers.

Digital Library

[22]

E. Kamsties, A. von Knethen, and R. Reussner. 2003. A controlled experiment to evaluate how styles affect the understandability of requirements specifications. Inf. Softw. Technol. 45, 14 (2003), 955--965.

[23]

K. Kuusinen, H. Petrie, F. Fagerholm, and T. Mikkonen. 2016. Flow, Intrinsic Motivation, and Developer Experience in Software Engineering. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, 104--117.

[24]

C. D. Manning, P. Raghavan, and H. Schtze. 2008. Introduction to Information Retrieval. Cambridge University Press, New York, NY, USA.

Digital Library

[25]

L. E. Maxwell. 2015. Noise in the office workplace. Facility Planning and Management Notes 1, 11 (2015).

[26]

A. N. Meyer, L. E. Barton, G. C. Murphy, T. Zimmermann, and T. Fritz. 2017. The Work Life of Developers: Activities, Switches and Perceived Productivity. IEEE Trans. on Softw. Eng. PP, 99 (2017), 1--1.

Digital Library

[27]

A. N. Meyer, T. Fritz, G. C. Murphy, and T. Zimmermann. 2014. Software Developers' Perceptions of Productivity. In Proc. of the ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 19--29.

Digital Library

[28]

A. Morin, J. D. Runyan, and T. M. Brinthaupt. 2015. Editorial: Inner Experiences: Theory, Measurement, Frequency, Content, and Functions. Frontiers in Psychology 6 (2015), 1758.

[29]

E.C. Poulton. 1979. Composite model for human performance in continuous noise. Psychological Review 86 (1979), 361--375.

[30]

F. Ricca, M. Di Penta, M. Torchiano, P. Tonella, and M. Ceccato. 2010. How Developers' Experience and Ability Influence Web Application Comprehension Tasks Supported by UML Stereotypes: A Series of Four Experiments. IEEE Trans. on Softw. Eng. 36, 1 (2010), 96--118.

Digital Library

[31]

F. Ricca, M. Di Penta, M. Torchiano, P. Tonella, M. Ceccato, and C. A. Visaggio. 2008. Are fit tables really talking?. In Proc. of the ACM/IEEE International Conference on Software Engineering. IEEE, 361--370.

Digital Library

[32]

F. Ricca, G. Scanniello, M. Torchiano, G. Reggio, and E. Astesiano. 2014. Assessing the Effect of Screen Mockups on the Comprehension of Functional Requirements. ACM Trans. Softw. Eng. Methodol. 24, 1, Article 1 (2014), 1:1--1:38 pages.

Digital Library

[33]

J. Romano, J. D. Kromrey, J. Coraggio, and J. Skowronek. 2006. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen'sd for evaluating group differences on the NSSE and other surveys?. In annual meeting of the Florida Association of Institutional Research, February. 1--3.

[34]

G. Scanniello, M. Risi, P. Tramontana, and S. Romano. 2017. Fixing Faults in C and Java Source Code: Abbreviated vs. Full-Word Identifier Names. ACM Trans. Softw. Eng. Methodol. 26, 2, Article 6 (2017), 43 pages.

Digital Library

[35]

S. Shapiro and M. Wilk. 1965. An analysis of variance test for normality. Biometrika 52, 3-4 (1965), 591--611.

[36]

D. I. K. Sjoberg, J. E. Hannay, O. Hansen, V. B. Kampenes, A. Karahasanovic, N. Liborg, and A. C. Rekdal. 2005. A Survey of Controlled Experiments in Software Engineering. IEEE Trans. on Softw. Eng. 31, 9 (2005), 733--753.

Digital Library

[37]

J. L. Szalma and P. A. Hancock. 2011. Noise effects on human performance: a meta-analytic synthesis. Psychological bulletin 137, 4 (2011), 682--707.

[38]

S. Vegas, C. Apa, and N. Juristo. 2016. Crossover Designs in Software Engineering Experiments: Benefits and Perils. IEEE Trans. on Softw. Eng. 42, 2 (2016), 120--135.

Digital Library

[39]

J. Verelst. 2004. The Influence of the Level of Abstraction on the Evolvability of Conceptual Models of Information Systems. In Procedeedings of the International Symposium on Empirical Software Engineering. IEEE Computer Society, 17--26.

Digital Library

[40]

S. Wellek and M. Blettner. 2012. On the Proper Use of the Crossover Design in Clinical Trials. Dtsch Arztebl International 109, 15 (2012), 276--281.

[41]

C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, and A. Wesslén. 2012. Experimentation in Software Engineering. Springer.

Digital Library

[42]

M. Züger, C. Corley, A. N. Meyer, B. Li, T. Fritz, D. Shepherd, V. Augustine, P. Francis, N. Kraft, and W. Snipes. 2017. Reducing Interruptions at Work: A Large-Scale Field Study of FlowLight. In Proc. of the CHI Conference on Human Factors in Computing Systems. ACM, 61--72.

Digital Library

Cited By

Matos JFrança CBaldassarre MPrechelt LRodríguez-Pérez G(2022)Pandemic agilityProceedings of the 15th International Conference on Cooperative and Human Aspects of Software Engineering10.1145/3528579.3529184(66-75)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3528579.3529184
Kitchenham BMadeyski LScanniello GGravino C(2022)The Importance of the Correlation in Crossover ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2021.307048048:8(2802-2813)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TSE.2021.3070480
Johnson BZimmermann TBird C(2021)The Effect of Work Environments on Productivity and Satisfaction of Software EngineersIEEE Transactions on Software Engineering10.1109/TSE.2019.290305347:4(736-757)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TSE.2019.2903053
Show More Cited By

Index Terms

The effect of noise on software engineers' performance
1. Software and its engineering
  1. Software creation and management

Recommendations

The effect of noise on requirements comprehension
ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

We conducted a controlled experiment with 55 final-year undergraduate students in Computer Science. We asked them to comprehend functional requirements exposing them or not to noise. We did not observe any effect of noise on requirements comprehension.
Experimentation in software engineering

Experimentation in software engineering supports the advancement of the field through an iterative learning process. In this paper we present a framework for analyzing most of the experimental work performed in software engineering over the past several ...
A web-based support environment for software engineering experiments

The software engineering communities frequently propose new software engineering technologies, such as new development techniques, programming languages and tools, without rigorous scientific evaluation. One way to evaluate software engineering ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

October 2018

487 pages

ISBN:9781450358231

DOI:10.1145/3239235

General Chair:
Markku Oivo
University of Oulu, Finland
,
Program Chairs:
Daniel Méndez
Technical University of Munich, Germany
,
Audris Mockus
University of Tennessee

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEM '18

Sponsor:

SIGSOFT

ESEM '18: ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

October 11 - 12, 2018

Oulu, Finland

Acceptance Rates

Overall Acceptance Rate 130 of 594 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
295
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matos JFrança CBaldassarre MPrechelt LRodríguez-Pérez G(2022)Pandemic agilityProceedings of the 15th International Conference on Cooperative and Human Aspects of Software Engineering10.1145/3528579.3529184(66-75)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3528579.3529184
Kitchenham BMadeyski LScanniello GGravino C(2022)The Importance of the Correlation in Crossover ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2021.307048048:8(2802-2813)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TSE.2021.3070480
Johnson BZimmermann TBird C(2021)The Effect of Work Environments on Productivity and Satisfaction of Software EngineersIEEE Transactions on Software Engineering10.1109/TSE.2019.290305347:4(736-757)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TSE.2019.2903053
Dunn MMolesworth BKoo TLodewijks G(2021)Measured effects of workload and auditory feedback on remote pilot task performanceErgonomics10.1080/00140139.2021.200387065:6(886-898)Online publication date: 22-Nov-2021
https://doi.org/10.1080/00140139.2021.2003870
Arakawa YZhou BCheng JStrohmeier PIshimaru SKosch T(2020)Augmented WorkplaceProceedings of the Augmented Humans International Conference10.1145/3384657.3385334(1-2)Online publication date: 16-Mar-2020
https://dl.acm.org/doi/10.1145/3384657.3385334
Romano SScanniello GBaldassarre MFucci D(2020)On the Effect of Noise on Software Engineers’ Performance: Results from Two Replicated Experiments2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA51224.2020.00062(334-341)Online publication date: Aug-2020
https://doi.org/10.1109/SEAA51224.2020.00062
Quesada-López CMadrigal-Sánchez DJenkins M(2020)An Empirical Analysis of IFPUG FPA and COSMIC FFP Measurement MethodsInformation Technology and Systems10.1007/978-3-030-40690-5_26(265-274)Online publication date: 31-Jan-2020
https://doi.org/10.1007/978-3-030-40690-5_26
Romano SCapece NErra UScanniello GLanza M(2019)The city metaphor in software visualization: feelings, emotions, and thinkingMultimedia Tools and Applications10.1007/s11042-019-07748-178:23(33113-33149)Online publication date: 28-May-2019
https://doi.org/10.1007/s11042-019-07748-1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents