Journal of

Information Systems Engineering

and Business Intelligence
Vol.7, No.1, April 2021
Available online at: http://e-journal.unair.ac.id/index.php/JISEBI

The Effect of DevOps Implementation on Teamwork Quality

in Software Development
Ady Hermawan1), Lindung Parningotan Manik 2)*
1)2)
University of Nusa Mandiri, Indonesia
Jl. Jatiwaringin No. 2, Jakarta Timur
1)
ady.hermawan31@gmail.com, 2)lindung.lpm@nusamandiri.ac.id

2)Research Center for Informatics, Indonesian Institute of Sciences, Indonesia

Jl. Cisitu, Sangkuriang, Bandung
2)
lind004@lipi.go.id

Abstract

Background: The Agile method, which is claimed to reduce time needed for software development cycle has been widely used.
It addresses communication gaps between customers and developers. Today, the DevOps has been extended as part of the Agile
process to address communication gaps between developer’s team members. Despite the rising popularity, the effect of DevOps
implementation on the teamwork quality in software development is still unknown.
Objective: The objective of this research is to conduct a study on the impact of DevOps on teamwork quality. Two software
houses, PT X and PT Y, are chosen as the case studies.
Methods: This research uses quantitative methods to analyse research data using simple linear regression. The questionnaire
technique is used to retrieve respondent data using 62 questions, consisting of 20 DevOps questions from 4 indicators and 42
teamwork quality questions from 6 indicators.
Results: The results from various quality tests indicate that all instruments are valid and reliable while hypothesis tests showed
that the DevOps implementation variable has an influence on the teamwork quality variable by 75.6%.
Conclusion: It can be concluded that the implementation of the DevOps in software development has a positive correlation with
the teamwork quality.

Keywords: Agile Method, DevOps, Software Development, Teamwork Quality

Article history: Received 11 March 2021, first decision 25 March 2021, accepted 7 April 2021, available online 28 April 2021

I. INTRODUCTION
Nowadays, the number of software houses continues to grow. Although there is no definite data, it is predicted that
the number has reached more than 500 companies in Indonesia only [1]. It shows that many companies or organisations
need more and more information technology support from the software industry in enabling their business activities.
Therefore, there is a need for more convenient methods to increase the effectiveness of the software development
activities [2]. This is not only to maintain the quality of software products [3] but also to increase the quality of
software processes themselves.
Software development methods play an important role in the development cycle. In a conventional development
like the Waterfall model, the communication between customers or stakeholders and the software house tends to be
poor and it leads to a longer cycle [4]. To solve the problems and to accelerate the development speed, a new method
was created, namely Agile which means ‘to move fast and easily’. The Agile method enhances the interaction between
customers or stakeholders and developers. Some of the approaches that implement the Agile method include Scrum,
Extreme Programming (XP), Lean, and Kanban [5]. The agile method encourages a transition from formal
communications to frequent communications. In Scrum, for example, the communication gap is mitigated by
conducting daily scrum (stand up meeting), sprint planning, sprint review, and sprint retrospective [6].
Although the Agile method can reduce the time of a software development cycle, there is still a gap between team
members that must be addressed [7]. The development team focuses mainly on producing new features and ensures

*
Corresponding author

ISSN 2443-2555 (online) 2598-6333 (print) © 2021 The Authors. Published by Universitas Airlangga.
This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
doi: http://dx.doi.org/10.20473/jisebi.7.1.84-90
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

that customers or stakeholders are able to use the software as soon as possible. On the other hand, the operation team
looks an issue entirely from different aspects; whereby they mainly concentrate on maintaining reliable and bug-free
software. This sometimes lead to poor teamwork [8]. To solve this problem, DevOps was introduced as an extension
of the Agile method. DevOps brings the development team and the operation team together. Automation plays an
important role in the DevOps so that the process of code integration, code testing, and deployment in the Agile method
can be done automatically [9].
Many organisations or companies appear to be interested in this new approach to manage their development and
operations, including Flickr, Netflix, and Etsy [10]. This method resolves a dilemma that many teams face when there
are other teams who want to collaborate in the testing and deployment automation but cannot find a way to do so [11].
DevOps is defined as a modern software development method to respond to the dependency between development
and operation by bringing modern methods and tools together to produce a unification between developers and
operators [12]. The goal of DevOps is to reduce the processing time required by software development and operation
without reducing quality [13]. Thus, applying these methods can increase the efficiency and effectiveness of software
development [14].
Apart from the methods applied, the success of an organisation is largely dependent on the team collaboration and
efficient communication between team members [15]. Therefore, teamwork quality also needs to be considered.
Teamwork is defined as the interdependent performance components needed to organise the performance of multiple
people effectively [16]. It means that teamwork is not just work done by team members, but a collective process
wherein each team member collaborates [17]. Teamwork quality should be considered using several indicators from
many viewpoints [18]. Teamwork quality itself is based on an input-process-output model on group behaviour [19].
Previous studies of DevOps have been conducted. Ref [20] presented how DevOps could be adopted by
practitioners in the real world. It is discovered that the DevOps adoption entails a relationship between seven
categories, namely automation, agility, continuous measurement, collaborative culture, transparency, resilience, and
sharing. More detailed DevOps attributes were studied in [21]. There were 18 attributes assessed under four variables
such as automation, source control, continuous delivery, and cohesive teams to help practitioners implement DevOps
in their enterprises. Furthermore, a systematic mapping study of DevOps and software quality has been performed in
[22]. This research investigated how DevOps features, automation, measurement, sharing, and culture impacted the
software quality. However, none of these studies focus on the effect of DevOps on teamwork quality.
This research is performed to study the effect of the DevOps practices on the teamwork quality in software
development to find out as to whether DevOps really addresses the gaps between developer’s team members. This
study was conducted in two IT companies doubling up as software houses, located in the city of Jakarta, Indonesia.
PT X and PT Y were chosen as the research subjects since they have implemented the DevOps practices in their
software development activities.

II. METHODS
It was found in [23] that DevOps had a positive effect on the teamwork quality, agility to adopt new technologies,
and responsiveness to business needs. A positive correlation was also found in [24] between DevOps and software
quality. Therefore, we the hypothesis is:
H1 There is a positive effect of the DevOps implementation on the teamwork quality in software development.
The research model is shown in Fig. 1.
In the DevOps implementation, we determine four indicators as defined in [25] namely automation, collaboration
culture, measurement, and sharing. DevOps plays an important role in controlling automation from the development
stage to production deployment, as well as in saving time. It also prevents defects and creates consistency.
Furthermore, it creates good collaboration culture since DevOps prepares the environment for interaction and allows
the team to accept changes. As DevOps enables continuous delivery and deployment, a continuous improvement is
needed. These things are measured with a set of key performance indicators which should be open, transparent, easily
accessible, able to be processed and visualised. By implementing DevOps, each team member can also share
knowledge, code, and documentation to collaborate more quickly and easily and to reduce repetitive work.

85
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

Teamwork quality

DevOps implementation Communication

Automation Coordination
H1(+) Balance of member
Culture
contribution
Measurement
Mutual support
Sharing
Effort
Cohesion

Fig. 1 Research model

In the teamwork quality, we determine six indicators as defined in [19] namely communication, coordination, even
contribution among members, mutual support, effort, and cohesion. DevOps underscores the importance of openness
and communication in sharing information among members, as well as coordination. Each team member should
contribute relevant knowledge and experience to the work whenever relevant. Mutual support is needed and team
members should be willing to help each other in carrying out their duties in the project. Moreover, each team member
shares the workload and prioritises team tasks over other personal tasks. Thus, uniform effort or shared effort is also
one of the main goals in a high-quality collaboration. Meanwhile, cohesion is to what extent an individual wish to stay
in the team. If team members do not feel a sense of belonging, or if there is low motivation to keep the team running,
then high-quality teamwork is difficult to achieve.
We designed data collection instruments that reflect the indicators as defined above by using questionnaire
technique with Likert scale 1 to 5, ranging from (1) disagree, (2) disagree, (3) neutral, (4) agree, (5) strongly agree, to
retrieve respondent data using 62 questions, comprising 20 DevOps implementation (PD) questions and 42 teamwork
quality (TW) questions. The questionnaires were distributed online to the respondents, who were selected by using
the purposive sampling technique, namely the determination of the sample using certain considerations [26]. Quality
tests like validity tests, reliability tests, normality tests, and others were performed to check whether the questionnaires
are valid or reliable enough. Meanwhile, the hypothesis tests are performed to verify whether the hypothesis is
accepted or not.

III. RESULTS
The descriptive statistics of the respondents in terms of genders, ages, and experiences are shown in Fig. 2, Fig. 3,
and Fig. 4, respectively.
> 56 > 15
2% 2%
6-10
Fema 26-35 12%
le, 12 26%

Male, < 25
45 72% <5
86%

Fig. 2 Respondents’ gender Fig. 3 Respondents’ age Fig. 4 Respondents’ working years

86
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

TABLE 1
THE VALIDITY TEST RESULTS FOR THE TEAMWORK QUALITY INSTRUMENTS
Indicators Question Description Pearson Explanation
Correlation’s
Value
TW1 There is always communication between team members 0.585 Valid
TW2 Team members often communicate both directly during meetings and
0.523 Valid
through other communication media
TW3 There is a mediator (someone outside the team) who mediates the
0.538 Valid
communication between team members
TW4 All team members openly share relevant ideas and information about team
0.639 Valid
performance
Communication TW5 In some cases, critical information is kept from other team members 0.388 Valid
TW6 There is a disagreement within team members about the open flow of
0.523 Valid
information
TW7 Everyone is satisfied with the accuracy of the information from other team
0.711 Valid
members
TW8 Team members love the timeliness where they get information from others 0.721 Valid
TW9 Everyone is satisfied with the usefulness of the information obtained from
0.638 Valid
other team members
TW10 The work that is done in the team is very harmonious 0.577 Valid
TW11 Within the team, there are clear objectives for the subtasks and are fully
0.668 Valid
understood by all members
TW12 There is conflicting interest within team members regarding subtasks 0.488 Valid
Coordination TW13 The division of work at the sub-task level in the team is appropriate with
0.590 Valid
competence, so that it goes well
TW14 There are clear targets to be achieved in the subtasks among team members
0.561 Valid
and all of them understand
TW15 Sub-task targets are approved by all team members 0.685 Valid
TW16 Team members help each other and provide support to one another 0.759 Valid
TW17 In the event if conflict, settlement is effective and immediate 0.722 Valid
TW18 Controversies and discussions are handled constructively 0.671 Valid
TW19 Team members' suggestions, input, and contributions are welcomed and
0.654 Valid
Mutual support greatly appreciated
TW20 Team members' suggestions, input, and contributions are discussed,
0.644 Valid
developed, and acted upon
TW21 The team members can reach agreement on important matters 0.716 Valid
TW22 The team members work well together 0.753 Valid
TW23 Everyone completely encourages teamwork 0.645 Valid
TW24 Everyone prioritises teamwork above all else 0.774 Valid
TW25 Everyone strives to achieve the team objectives 0.718 Valid
Effort
TW26 There is a conflict of effort that team members make to achieve performance 0.649 Valid
TW27 Each team member gives full support to the work given 0.731 Valid
TW28 Each team member devotes all his abilities to the benefit the team 0.799 Valid
TW29 Each team member feels very attached to this job 0.642 Valid
TW30 The job is considered very important for team members 0.764 Valid
TW31 Every team member integrates with and supports each other 0.665 Valid
TW32 There are good personal relationships among team members 0.700 Valid
TW33 There are a lot of personal conflicts in the team 0.297 Valid
Cohesion
TW34 Everyone feels glad to be essential for the team 0.758 Valid
TW35 Everyone sees nothing extraordinary in teamwork 0.312 Valid
TW36 Everyone feels safeguarding information in the team are important 0.714 Valid
TW37 There is a feeling of unity among team members 0.778 Valid
TW38 There is mutual sympathy among team members 0.759 Valid
TW39 Everyone recognises the special characteristics (qualities and shortcomings)
0.541 Valid
of each team member
TW40 Team members contribute to the accomplishment of team objectives in
0.525 Valid
Balance of accordance with their individual abilities
member TW41 The imbalance contributions of team members cause conflicts in the team 0.536 Valid
contribution TW42 There are team members who often underperform their job 0.437 Valid
TW43 The assignment and division of tasks is in accordance with the abilities of
0.681 Valid
each team member
TW44 Team members' contributions have been shared fairly and equitably 0.699 Valid

87
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

TABLE 2
THE VALIDITY TEST RESULTS FOR THE DEVOPS IMPLEMENTATION INSTRUMENTS
Indicators Question Description Pearson Explanation
Code Correlation’s
Value
PD1 Team members have sufficient discretion to directly test and deploy their code
0.318 Valid
to the server at any time
PD2 Software is released through small products on a continuous basis 0.487 Valid
Culture
PD3 Products that have been released are monitored to see user feedback 0.752 Valid
PD4 There is openness, transparency and respect between team members 0.739 Valid
PD5 Continuous evaluation and improvement are performed 0.741 Valid
PD6 There is automation between the development process and the development
0.663 Valid
team
PD7 Continuous Integration (CI) and Continuous Delivery (CD) are conducted 0.474 Valid
Automation
PD8 Source codes are stored in the repository 0.494 Valid
PD9 IT infrastructures are tuned into a set of codes or scripts 0.615 Valid
PD10 Many automation tools are used 0.611 Valid
PD11 In-app on-board guidance service is available 0.353 Valid
PD12 Software response time can be measured 0.623 Valid
Measurement PD13 To all stakeholders, metrics of measurements are kept transparent and open 0.575 Valid
PD14 Measurement targets are aligned with common objectives are set 0.676 Valid
PD15 Unmet measurement targets are followed up and re-established 0.580 Valid
PD16 Obstacles are notified without feeling shy and fear 0.665 Valid
PD17 Problem solving is prioritised rather than blaming 0.687 Valid
Sharing PD18 Workloads are shared with other team members 0.688 Valid
PD19 Knowledge, information, and documentation are exchanged with others 0.576 Valid
PD20 There is a good communication with team members 0.756 Valid

The results of validity tests for the instruments of teamwork quality and DevOps implementation are given in Table
1 and Table 2 respectively. A Pearson correlation’s value higher than 0.25 indicates that the data collection instrument
is valid [27]. Thus, based on the results, all questionnaires are valid. We also performed a reliability test for more
quality tests. The reliability test results are given in Table 3. We use Cronbach’s alpha as the reliability measure, which
is a statistic for internal-consistency reliability. The results show that all values are higher than 0.9, while a Cronbach’s
alpha higher than 0.5 is considered as an adequate value [27].
Furthermore, the test results can also be strengthened by looking at the normality test using the Kolmogorov-
Smirnov [27]. The test result gives a significant value of 0.441. The residual regression model is normally distributed
if the data has a significance value of more than 0.05 [27]. Moreover, the heteroscedasticity test is carried out to
determine whether there is an inequality of a variant from one residual to another in the regression model. The test
result gives a significant value of 0.083. The model is free of heteroscedasticity if the significant value is greater than
0.05 [27].
To determine whether the regression model correlates with the independent variables, the multicollinearity test is
conducted. It is carried out by looking at the variance inflation factor (VIF) value and the tolerance value. The VIF is
calculated by dividing the variance in a model with numerous terms by the variance of a model with only one term
[28]. It evaluates the severity of multicollinearity which provides an index that measures the variance increment of a
regression coefficient due to collinearity. Meanwhile, the tolerance is the inverse of the VIF. Multicollinearity happens
when at least two indicators are correlated in the model and creates redundant information about the response [29].
The data does not contain multicollinearity if it has a tolerance value of more than 0.1 and a VIF value of less than 10
[27] where the test results yield a value of 1.000 for both indicators. The summary of the test results is given in Table
4.
TABLE 3
THE RESULTS OF THE RELIABILITY TEST
Variable Cronbach’s alpha Explanation
DevOps implementation (PD) 0.905 > 0.5 = reliable
Teamwork quality (TW) 0.961 > 0.5 = reliable

TABLE 4
THE RESULTS OF THE NORMALITY, THE HETEROSCEDASTICITY, AND THE MULTICOLLINEARITY TESTS
Test Indicators Value Explanation
Normality Asymp. Sig. (2-tailed) 0.441 Sig. > 0.05 = normal
Heteroscedasticity Sig. 0.083 Sig. > 0.05 = normal
Multicollinearity Tolerance 1.000 Tolerance > 0.1 & VIF < 10 = Non-
VIF 1.000 Multicollinearity

88
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

IV. DISCUSSION
We conducted two hypothesis tests, namely the determination coefficient test and the partial test for analysing the
acquired data. The coefficient of determination test is carried out to determine the size of the independent variable to
explain the dependent variable [28]. The test result gives the R Square value of 0.756. This means that the DevOps
implementation variable has an influence on the teamwork quality variable by 75.6%. The remaining 24.4% is
influenced by other variables outside this regression equation or variables that are not included in this study.
Furthermore, a partial test is performed to find out whether the independent variable can influence the variation of the
dependent variable. The results of this test can be obtained from the significant value and the unstandardized
coefficients B value [29] where the hypothesis is supported if the significant value is less than 0.05 and the
unstandardized coefficients B value is in line with the hypothesis that has been made. Since the results of significant
value is 0.000 and the unstandardized coefficients B value is 2.123, which has a positive sign, then the null hypothesis
is rejected.
The results show that the implementation of DevOps has a positive effect on teamwork quality in software
development. In other words, the more precise the implementation of DevOps on a team, the better the impact it will
have on the quality of teamwork in software development. Since the teamwork quality has a very positive effect on
team performance [30] [31], then it more likely brings success to the software development project. It not only
increases team productivity but also improves coordination and decision making so that it can be positively associated
with team effectiveness [32].
The results of this research are supported in [33], which stated that the use of a collaborative system, like DevOps
in the context of this study, influences teamwork quality and performance for tasks that are supported by the system.
It is also interesting to mention that the project teams which are dominated by younger people could affect the
teamwork quality since they are considered as digital native generations and more familiar with technologies [34],
such as the collaborative DevOps system, in this context. This result also supports previous studies that claimed there
is a positive correlation between DevOps and software quality [22] [24]. Since DevOps has a positive correlation with
teamwork quality, then the teamwork quality could also have a positive correlation with software quality. Although,
this hypothesis should be studied and tested in the future.
Regarding threats to validity, there could be other variables that are more suitable for this study. Moreover, the
hypothesis in this research was also simplified since the study was limited in a such timeframe. As another limitation,
the questionnaire was made by the authors only based on the literature in the references. It could be improved if other
subject matter experts were involved in creating the questionnaire. Also, it should have been tested and validated
before distributing it to the respondents. It is necessary to make sure that all questions in the questionnaire are well-
understood by respondents. Furthermore, only two organisations and 57 respondents were selected as the samples,
which prevent the generalisations of the results of this study.

V. CONCLUSIONS
A study to determine the effect of the DevOps implementation on teamwork quality has been conducted. This
research involved 57 respondents spread across two software houses that have implemented DevOps in their software
development activities. Based on the results, it can be concluded that the implementation of DevOps has a positive
effect on the teamwork quality in software development. Thus, it is highly recommended for organisations to
implement and adapt DevOps in their software development activities so that the teamwork quality can be maximised.
Further research is expected to explore more hypotheses and expand the research objects in order to provide more
accurate results. Moreover, other data collection methods other than questionnaires such as direct interviews should
be performed in the future to increase the objectivity level of the respondents. Furthermore, it is also expected that
future studies are able to increase the size of the respondent samples and obtain more data in order to acquire better
results that are closer to general conclusions.

Author Contributions: Ady Hermawan: Conceptualization, Data Curation, Formal analysis, Investigation,
Methodology, Project administration, Resources, Software, Writing - Original Draft. Lindung Parningotan Manik:
Validation, Visualization, Supervision, Writing - Review & Editing.

Funding: This research received no specific grant from any funding agency.

Conflicts of Interest: The authors declare no conflict of interest.

89
 Hermawan & Manik
Journal of Information Systems Engineering and Business Intelligence, 2021, 7 (1), 84-90

REFERENCES
[1] G. Anggadwita, H. Amani, R. Saragih, and D. T. Alamanda, “Competitive strategy of creative application content in the ASEAN
economic community: Software development using SWOT analysis in Indonesia,” Int. J. Econ. Manag., vol. 10, no. S1, pp. 95–107,
2016.
[2] L. E. Lwakatare et al., “DevOps in practice: A multiple case study of five companies,” Inf. Softw. Technol., vol. 114, pp. 217–230, 2019.
[3] L. P. Manik, “Design Pattern Evaluation on A RESTful API Wrapper: A Case Study of Software Integration with An Internet Payment
Gateway using Model-Driven Architecture,” J. Inf. Technol. Comput. Sci., vol. 4, no. 3, pp. 222–232, 2019.
[4] L. Zhu, L. Bass, and G. Champlin-Scharff, “DevOps and Its Practices,” IEEE Softw., vol. 33, pp. 32–34, 2016.
[5] S. S. Samarawickrama and I. Perera, “Continuous scrum: A framework to enhance scrum with DevOps,” in 2017 Seventeenth
international conference on advances in ICT for emerging regions (ICTer), 2017, pp. 1–7.
[6] A. Khalid, S. A. Butt, T. Jamal, and S. Gochhait, “Agile Scrum Issues at Large-Scale Distributed Projects,” Int. J. Softw. Innov., vol. 8,
no. 2, pp. 85–94, Apr. 2020.
[7] M. Senapathi, J. Buchan, and H. Osman, “DevOps Capabilities, Practices, and Challenges: Insights from a Case Study,” in Proceedings
of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018, 2018, pp. 57–67.
[8] L. Leite, C. Rocha, F. Kon, D. Milojicic, and P. Meirelles, “A Survey of DevOps Concepts and Challenges,” ACM Comput. Surv., vol.
52, no. 6, Nov. 2019.
[9] R. Singh, “DevOPS Now and Then.” MDPI AG, Nov-2020.
[10] A. Dyck, R. Penners, and H. Lichter, “Towards Definitions for Release Engineering and DevOps,” in 2015 IEEE/ACM 3rd International
Workshop on Release Engineering, 2015, p. 3.
[11] M. Callanan and A. Spillane, “DevOps: Making It Easy to Do the Right Thing,” IEEE Softw., vol. 33, no. 3, pp. 53–59, 2016.
[12] R. Jabbari, N. bin Ali, K. Petersen, and B. Tanveer, “What is DevOps? A Systematic Mapping Study on Definitions and Practices,” in
Proceedings of the Scientific Workshop Proceedings of XP2016, 2016.
[13] F. M. A. Erich, C. Amrit, and M. Daneva, “A qualitative study of DevOps usage in practice,” J. Softw. Evol. Process, vol. 29, no. 6, p.
e1885, Jun. 2017.
[14] S. Mohamed, “Software Release Management Evolution-Comparative Analysis across Agile and DevOpsContinuous Delivery,” Int. J.
Emerg. Trends Technol. Comput. Sci., vol. 3, pp. 2349–6495, 2016.
[15] A. Hashmi, S. Ishak, and H. B. Hassan, “Role of team size as a contextual variable for the relationship of transformational leadership and
teamwork quality,” Asian J. Multidiscip. Stud., vol. 6, no. 5, 2018.
[16] E. Salas, N. J. Cooke, and M. A. Rosen, “On Teams, Teamwork, and Team Performance: Discoveries and Developments,” Hum. Factors
J. Hum. Factors Ergon. Soc., vol. 50, no. 3, pp. 540–547, Jun. 2008.
[17] J. Oh, H. Lee, and H. Zo, “The Effect of Leadership and Teamwork on ISD Project Success,” J. Comput. Inf. Syst., vol. 0, no. 0, pp. 1–
11, 2019.
[18] M. Hoegl and H. G. Gemuenden, “Teamwork Quality and the Success of Innovative Projects: A Theoretical Concept and Empirical
Evidence,” Organ. Sci., vol. 12, no. 4, pp. 435–449, Aug. 2001.
[19] Y. Lindsjørn, D. I. K. Sjøberg, T. Dingsøyr, G. R. Bergersen, and T. Dybå, “Teamwork quality and project success in software
development: A survey of agile development teams,” J. Syst. Softw., vol. 122, pp. 274–286, 2016.
[20] W. P. Luz, G. Pinto, and R. Bonifácio, “Adopting DevOps in the real world: A theory, a model, and a case study,” J. Syst. Softw., vol.
157, p. 110384, 2019.
[21] V. Gupta, P. K. Kapur, and D. Kumar, “Modeling and measuring attributes influencing DevOps implementation in an enterprise using
structural equation modeling,” Inf. Softw. Technol., vol. 92, pp. 75–91, 2017.
[22] A. Mishra and Z. Otaiwi, “DevOps and software quality: A systematic mapping,” Comput. Sci. Rev., vol. 38, p. 100308, 2020.
[23] P. Perera, M. Bandara, and I. Perera, “Evaluating the impact of DevOps practice in Sri Lankan software development organizations,” in
2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer), 2016, pp. 281–287.
[24] P. Perera, R. Silva, and I. Perera, “Improve software quality through practicing DevOps,” in 2017 Seventeenth International Conference
on Advances in ICT for Emerging Regions (ICTer), 2017, pp. 1–6.
[25] S. K. Bang, S. Chung, Y. Choh, and M. Dupuis, “A Grounded Theory Analysis of Modern Web Applications: Knowledge, Skills, and
Abilities for DevOps,” in Proceedings of the 2nd Annual Conference on Research in Information Technology, 2013, pp. 61–62.
[26] Sugiyono, Metode penelitian pendidikan:(pendekatan kuantitatif, kualitatif dan R \& D). Alfabeta, 2008.
[27] I. Nazaruddin and A. T. Basuki, “Analisis statistik dengan SPSS,” 2015.
[28] G. James, D. Witten, T. Hastie, and R. Tibshirani, An Introduction to Statistical Learning: With Applications in R. Springer Publishing
Company, Incorporated, 2014.
[29] J. Hair, W. Black, B. Babin, and R. Anderson, “Multivariate data analysis. Prentice Hall,” London, 2009.
[30] Y. Lindsjørn, G. R. Bergersen, T. Dingsøyr, and D. I. K. Sjøberg, “Teamwork Quality and Team Performance: Exploring Differences
Between Small and Large Agile Projects,” in Agile Processes in Software Engineering and Extreme Programming, 2018, pp. 267–274.
[31] E. Weimar, A. Nugroho, J. Visser, A. Plaat, M. Goudbeek, and A. Schouten, “The Influence of Teamwork Quality on Software Team
Performance,” 2017.
[32] J. Lyubovnikova, A. Legood, N. Turner, and A. Mamakouka, “How Authentic Leadership Influences Team Performance: The Mediating
Role of Team Reflexivity,” J. Bus. Ethics, vol. 141, no. 1, pp. 59–70, 2017.
[33] R. F. Easley, S. Devaraj, and J. M. Crant, “Relating Collaborative Technology Use to Teamwork Quality and Performance: An Empirical
Analysis,” J. Manag. Inf. Syst., vol. 19, no. 4, pp. 247–265, 2003.
[34] J. Coleman, “Introduction: Digital technologies in the lives of young people,” Oxford Rev. Educ., vol. 38, no. 1, pp. 1–8, 2012.

90