research-article

Software development with feature toggles: practices used by practitioners

Authors:

Rezvan Mahdavi-Hezaveh,

Laurie WilliamsAuthors Info & Claims

Empirical Software Engineering, Volume 26, Issue 1

https://doi.org/10.1007/s10664-020-09901-z

Published: 01 January 2021 Publication History

Abstract

Background

Using feature toggles is a technique that allows developers to either turn a feature on or off with a variable in a conditional statement. Feature toggles are increasingly used by software companies to facilitate continuous integration and continuous delivery. However, using feature toggles inappropriately may cause problems which can have a severe impact, such as code complexity, dead code, and system failure. For example, the erroneous repurposing of an old feature toggle caused Knight Capital Group, an American global financial services firm, to go bankrupt due to the implications of the resultant incorrect system behavior.

Aim

The goal of this research project is to aid software practitioners in the use of practices to support software development with feature toggles through an empirical study of feature toggle practice usage by practitioners.

Method

We conducted a qualitative analysis of 99 artifacts from the grey literature and 10 peer-reviewed papers about feature toggles. We conducted a survey of practitioners from 38 companies.

Results

We identified 17 practices in 4 categories: Management practices, Initialization practices, Implementation practices, and Clean-up practices. We observed that all of the survey respondents use a dedicated tool to create and manage feature toggles in their code. Documenting feature toggle’s metadata, setting up the default value for feature toggles, and logging the changes made on feature toggles are also frequently-observed practices.

Conclusions

The feature toggle development practices discovered and enumerated in this work can help practitioners more effectively use feature toggles. This work can enable future mining of code repositories to automatically identify feature toggle practices.

References

[1]

Bird J (2014) Feature toggles are one of the worst kinds of technical debt. [Online]. Available: https://dzone.com/articles/feature-toggles-are-one-worst. Accessed 24 April 2019

[2]

Dale NB, Weems C, and Headington MR Introduction to java and software design: swing update 2003 Burlington Jones & Bartlett Learning

[3]

Day B (2020) Get good at devops: Feature flag deployments with asp.net, webapi, & javascript. Available: https://channel9.msdn.com/Events/Visual-Studio/Visual-Studio-Live-Redmond-2016/W07. Accessed 27 January 2020

[4]

featureflags.io (2020) Feature flags, toggles, controls - cleaning up. Available: http://featureflags.io/feature-flags-cleaning-up/. Accessed 13 January 2020

[5]

Feitelson DG, Frachtenberg E, and Beck KL Development and deployment at facebook IEEE Internet Comput 2013 17 4 8-17

[6]

Fowler M (2013) Continuousdelivery. [Online]. Available: https://martinfowler.com/bliki/ContinuousDelivery.html. Accessed 24 April 2019

[7]

Fowler M (2010) Feature toggle. [Online]. Available: https://martinfowler.com/bliki/FeatureToggle.html. Accessed 24 April 2019

[8]

Garousi V, Felderer M, Mäntylä MV (2016) The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: Proceedings of the 20th international conference on evaluation and assessment in software engineering, pp 1–6

[9]

Garousi V, Felderer M, and Mäntylä MV Guidelines for including grey literature and conducting multivocal literature reviews in software engineering Inf Softw Technol 2019 106 101-121

[10]

Garousi V, Felderer M, Mäntylä MV, Rainer A (2019) Benefitting from the grey literature in software engineering research. arXiv:1911.12038

[11]

Google (2019) Chromium flags. Available: https://docs.google.com/spreadsheets/d/1XwGEpBidtPKS_xFzW8bfQ4kbuVe0OuPlpMsSd8wTmmo/edit. Accessed 24 April 2019

[12]

Hammarberg M and Sunden J Kanban in action 2014 Shelter Island Manning Publications Co.

[13]

Harmes R (2009) Flipping out. [Online]. Available: http://code.flickr.net/2009/12/02/flipping-out/. Accessed 24 April 2019

[14]

Hodges B (2017) Progressive experimentation with feature flags. Available: https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/progressive-experimentation-feature-flags. Accessed 24 April 2019

[15]

Hodgson P (2017) Feature toggles (aka feature flags). [Online]. Available: https://martinfowler.com/articles/feature-toggles.html. Accessed 24 April 2019

[16]

Hodgson P (2018) Lean product development: Managing feature flags at scale. [Online]. Available: https://www.youtube.com/watch?v=uFW4SSRtkUU. Accessed 24 April 2019

[17]

Humble J, Farley D (2010) Continuous delivery: reliable software releases through build, test, and deployment automation (Adobe Reader), Pearson Education, Upper Saddle River

[18]

Instagram Engineering (2015) Flexible feature control at instagram. Available: https://instagram-engineering.com/flexible-feature-control-at-instagram-a7d3417658df. Accessed 24 April 2019

[19]

Kieser G (2017) Removing risk from product launches: a webinar with LaunchDarkly, CircleCI and GoPro. Available: https://circleci.com/blog/removing-risk-from-product-launches-a-webinar-with-launchdarkly-circleci-and-gopro/. Accessed 27 January 2020

[20]

Knight Capital Group (2014) Knightmare: A devops cautionary tale. Available: https://dougseven.com/2014/04/17/knightmare-a-devops-cautionary-tale/. Accessed 24 April 2019

[21]

Kohavi R, Longbotham R, Sommerfield D, and Henne RM Controlled experiments on the web: survey and practical guide Data Mining and Knowledge Discovery 2009 18 1 140-181

[22]

Launchdarkly (2018) Best practices. Available: https://github.com/launchdarkly/featureflags/blob/master/5%20-%20Best%20Practices.md. Accessed 24 April 2019

[23]

Leena SN (2017) Merge hells!! feature toggles to the rescue - pipeline conference 2017. [Online]. Available: https://www.youtube.com/watch?v=R9EYY0Uu250. Accessed 24 April 2019

[24]

Lefebvre C, Manheimer E, Glanville J (2008) Searching for studies. Cochrane handbook for systematic reviews of interventions:, Cochrane book series, pp 95–150

[25]

Likert R (1932) A technique for the measurement of attitudes. Arch Psychol

[26]

Meinicke J, Wong C-P, Vasilescu B, Kaestner C (2020) Exploring differences and commonalities between feature flags and configuration options. In: Proceedings of the 42nd international conference on software engineering - software engineering in practice (ICSE-SEIP)

[27]

Meyer M (2014) Using feature flags to ship changes with confidence. [Online]. Available: https://blog.travis-ci.com/2014-03-04-use-feature-flags-to-ship-changes-with-confidence/. Accessed 24 April 2019

[28]

Mclaughlin T (2017) Introducing stormcrow. Available: https://blogs.dropbox.com/tech/2017/03/introducing-stormcrow/. Accessed 24 April 2019

[29]

Moore GA Crossing the chasm: Marketing and selling technology project 2009 New York Harper Collins

[30]

Nadel B (2018) Launchdarkly lunch-and-learn panel discussion: New York City. Available: https://www.bennadel.com/blog/3464-launchdarkly-lunch-and-learn-panel-discussion-new-york-city.htm. Accessed 18 June 2019

[31]

Nadel B (2018) Using the launchdarkly dashboard and json types to create light-weight application administrative features. Available: https://www.bennadel.com/blog/3465-using-the-launchdarkly-dashboard-and-json-types-to-create-light-weight-application-administrative-features.htm. Accessed 27 January 2020

[32]

Neely S, Stolt S (2013) Continuous delivery? easy! just change everything (well, maybe it is not that easy). In: 2013 Agile conference. IEEE, pp 121–128

[33]

Østhus I (2017) Continuous deployment: Unleash your features gradually!. Available: http://bytes.schibsted.com/unleash-features-gradually/. Accessed 24 April 2019

[34]

Parnin C, Helms E, Atlee C, Boughton H, Ghattas M, Glover A, Holman J, Micco J, Murphy B, Savor T, et al. The top 10 adages in continuous deployment IEEE Software 2017 34 3 86-95

[35]

Piessens D (2015) It’s more than feature toggles. Available: https://www.youtube.com/watch?v=TCwMxTuDLwI. Accessed 27 January 2020

[36]

Rahman AAU, Helms E, Williams L (2015) Chris Parnin Synthesizing continuous deployment practices used in software development. In: Proceedings IEEE Agile Conference, pp 1–10

[37]

Rahman MT, Querel L-P, Rigby PC, Adams B (2016) Feature toggles: practitioner practices and a case study. In: Proceedings of the 13th International conference on mining software repositories. ACM, pp 201–211

[38]

Rahman MT, Rigby PC, Shihab E (2018) The modular and feature toggle architectures of google chrome. Empir Softw Eng, 1–28

[39]

Roberts J (2019) Featuretoggle in.net. [Online]. Available: http://jason-roberts.github.io/FeatureToggle.Docs/. Accessed 24 April 2019

[40]

Rodríguez P, Haghighatkhah A, Lwakatare LE, Teppola S, Suomalainen T, Eskeli J, Karvonen T, Kuvaja P, Verner JM, and Oivo M Continuous deployment of software intensive products and services: A systematic mapping study J Syst Softw 2017 123 263-291

[41]

Rollout (2018) Your feature flag management needs to include retirement [Online]. Available: https://rollout.io/blog/feature-flag-retirement/. Accessed 24 April 2019

[42]

Rusovsky E (2017) Feature flag management systems save you from technical debt. Available: https://rollout.io/blog/feature-flag-management-technical-debt/. Accessed 18 June 2019

[43]

Saldaña J (2015) The coding manual for qualitative researchers. Sage

[44]

Sayagh M, Kerzazi N, Adams B, Petrillo F (2018) Software configuration engineering in practice: Interviews, survey, and systematic literature review. IEEE Trans Softw Eng

[45]

Schermann G, Cito J, and Leitner P Continuous experimentation: challenges, implementation techniques, and current research IEEE Softw 2018 35 2 26-31

[46]

Schermann G, Cito Jürgen, Leitner P, Zdun U, and Gall HC We’re doing it live: A multi-method empirical study on continuous experimentation Inf Softw Technol 2018 99 41-57

[47]

Schneid K (2017) Branching strategies for developing new features within the context of continuous delivery. In: CSE@ SE, pp 28–35

[48]

Sowa RLE (2010) Feature bits: Enabling flow within and across teams. [Online]. Available: https://www.infoq.com/presentations/Feature-Bits. Accessed 24 April 2019

[49]

Split (2019) Audit logs. Available: https://docs.split.io/docs/streaming-audit-logs. Accessed 24 April 2019

[50]

Stephens R Beginning software engineering 2015 New York Wiley

[51]

Tang C, Kooburat T, Venkatachalam P, Chander A, Wen Z, Narayanan A, Dowell P, Karl R (2015) Holistic configuration management at facebook. In: Proceedings of the 25th symposium on operating systems principles, pp 328–343

[52]

Tejaswini UL (2019) There is no devops without feature flags! [Online]. Available: https://www.ibm.com/developerworks/community/blogs/c914709e-8097-4537-92ef-8982fc416138/entry/THERE_IS_NO_DEVOPS_WITHOUT_FEATURE_FLAGS?lang=en. Accessed 24 April 2019

[53]

Tiwari A (2013) Decoupling deployment and release- feature toggles. [Online]. Available: https://www.abhishek-tiwari.com/decoupling-deployment-and-release-feature-toggles/. Accessed 24 April 2019

[54]

Tsvetkov A (2017) Feature toggles in .Net: tips and tricks. Available: https://surfingthecode.com/feature-toggles-in-.net-tips-and-tricks/. Accessed 13 January 2020

[55]

Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proceedings of the 18th international conference on evaluation and assessment in software engineering. Citeseer, p 38

Cited By

Mahdavi-Hezaveh RFatima SWilliams L(2024)Paving a Path for a Combined Family of Feature Toggle and Configuration Option ResearchACM Transactions on Software Engineering and Methodology10.1145/367255533:7(1-27)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672555
Rahman TShalabi ISharma T(2024)Exploring Influence of Feature Toggles on Code ComplexityProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661190(363-368)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661190
Fischer SMichelon GAssunção WRamler REgyed A(2023)Designing a Test Model for a Configurable System: An Exploratory Study of Preprocessor Directives and Feature TogglesProceedings of the 17th International Working Conference on Variability Modelling of Software-Intensive Systems10.1145/3571788.3571795(31-39)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1145/3571788.3571795
Show More Cited By

Index Terms

Software development with feature toggles: practices used by practitioners
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Software management
2. Software and its engineering
  1. Software creation and management
  2. Software notations and tools
    1. Software configuration management and version control systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Feature toggles as code: Heuristics and metrics for structuring feature toggles
Abstract Context:
Using feature toggles is a technique to turn a feature either on or off in program code by checking the value of a variable in a conditional statement. This technique is increasingly used by software practitioners ...
On the Removal of Feature Toggles: A Study of Python Projects and Practitioners Motivations
Abstract
Feature Toggling is a technique to control the execution of features in a software project. For example, practitioners using feature toggles can experiment with new features in a production environment by exposing them to a subset of users. Some ...
Synthesizing Continuous Deployment Practices Used in Software Development
AGILE '15: Proceedings of the 2015 Agile Conference

Continuous deployment speeds up the process of existing agile methods, such as Scrum, and Extreme Programming (XP) through the automatic deployment of software changes to end-users upon passing of automated tests. Continuous deployment has become an ...

Comments

Information & Contributors

Information

Published In

cover image Empirical Software Engineering

Empirical Software Engineering Volume 26, Issue 1

Jan 2021

460 pages

ISSN:1382-3256

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 January 2021

Accepted: 23 September 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mahdavi-Hezaveh RFatima SWilliams L(2024)Paving a Path for a Combined Family of Feature Toggle and Configuration Option ResearchACM Transactions on Software Engineering and Methodology10.1145/367255533:7(1-27)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672555
Rahman TShalabi ISharma T(2024)Exploring Influence of Feature Toggles on Code ComplexityProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661190(363-368)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661190
Fischer SMichelon GAssunção WRamler REgyed A(2023)Designing a Test Model for a Configurable System: An Exploratory Study of Preprocessor Directives and Feature TogglesProceedings of the 17th International Working Conference on Variability Modelling of Software-Intensive Systems10.1145/3571788.3571795(31-39)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1145/3571788.3571795
Rezaei Nasab AShahin MHoseyni Raviz SLiang PMashmool ALenarduzzi V(2023)An empirical study of security practices for microservices systemsJournal of Systems and Software10.1016/j.jss.2022.111563198:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.jss.2022.111563
Medeiros RMartinez JDíaz OFalleri J(2023)Visualizations for the evolution of Variant-Rich SystemsInformation and Software Technology10.1016/j.infsof.2022.107084154:COnline publication date: 20-Jan-2023
https://dl.acm.org/doi/10.1016/j.infsof.2022.107084
Jézéquel JKienzle JAcher MFelfernig AFuentes L(2022)From feature models to feature toggles in practiceProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A10.1145/3546932.3547009(234-244)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3546932.3547009
Sugnanam HRahman M(2022)Classifying Toggles-smells and Investigating Development EffortProceedings of the 15th Innovations in Software Engineering Conference10.1145/3511430.3511461(1-2)Online publication date: 24-Feb-2022
https://dl.acm.org/doi/10.1145/3511430.3511461
Mahdavi-Hezaveh RAjmeri NWilliams L(2022)Feature toggles as codeInformation and Software Technology10.1016/j.infsof.2021.106813145:COnline publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.infsof.2021.106813
Izadi MAkbari KHeydarnoori A(2022)Predicting the objective and priority of issue reports in software repositoriesEmpirical Software Engineering10.1007/s10664-021-10085-327:2Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s10664-021-10085-3
Li BWei YSun XBo LChen DTao C(2022)Towards the identification of bug entities and relations in bug reportsAutomated Software Engineering10.1007/s10515-022-00325-129:1Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1007/s10515-022-00325-1

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents