research-article

Estimating Story Points from Issue Reports

Authors:

Alessandro Murgia,

Michele Marchesi,

Roberto TonelliAuthors Info & Claims

PROMISE 2016: Proceedings of the The 12th International Conference on Predictive Models and Data Analytics in Software Engineering

Article No.: 2, Pages 1 - 10

https://doi.org/10.1145/2972958.2972959

Published: 09 September 2016 Publication History

Abstract

Estimating the effort of software engineering tasks is notoriously hard but essential for project planning. The agile community often adopts issue reports to describe tasks, and story points to estimate task effort. In this paper, we propose a machine learning classifier for estimating the story points required to address an issue. Through empirical evaluation on one industrial project and eight open source projects, we demonstrate that such classifier is feasible. We show that ---after an initial training on over 300 issue reports--- the classifier estimates a new issue in less than 15 seconds with a mean magnitude of relative error between 0.16 and 0.61. In addition, issue type, summary, description, and related components prove to be project dependent features pivotal for story point estimation.

References

[1]

T. K. Abdel-Hamid. Investigating the cost/schedule trade-off in software development. Software, IEEE, 7(1):97--105, 1990.

Digital Library

[2]

W. AbdelMoez, M. Kholief, and F. M. Elsalmy. Improving bug fix-time prediction model by filtering out outliers. In Proceedings of the 2013 International Conference on Technological Advances in Electrical, Electronics and Computer Engineering, pages 359--364, May 2013.

[3]

P. Abrahamsson, I. Fronza, R. Moser, J. Vlasenko, and W. Pedrycz. Predicting development effort from user stories. In Proceedings of the 2011 International Symposium on Empirical Software Engineering and Measurement, pages 400--403. IEEE, 2011.

Digital Library

[4]

J. Aranda and S. Easterbrook. Anchoring and adjustment in software estimation. In Proceedings of the 2005 European Software Engineering Conference Held Jointly with the 2005 International Symposium on Foundations of Software Engineering, pages 346--355. ACM, 2005.

Digital Library

[5]

E. Aronson, T. D. Wilson, and R. M. Akert. Social Psychology (3rd Edition). Pearson, 1999.

[6]

R. L. Atkinson, R. C. Atkinson, E. E. Smith, D. J. Bem, and S. Nolen-Hoeksema. Hilgard's Introduction to Psychology (12th Edition). Orlando: Harcourt Brace College Publishers, 1996.

[7]

N. C. Augen. An empirical study of using planning poker for user story estimation. In Agile Conference, 2006, pages 9--pp. IEEE, 2006.

Digital Library

[8]

K. Beck, M. Beedle, A. Van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, et al. Manifesto for agile software development, 2001.

[9]

L. C. Briand. On the many ways software engineering can benefit from knowledge engineering. In Proceedings of the 2002 international conference on Software engineering and knowledge engineering, pages 3--6. ACM, 2002.

Digital Library

[10]

R. Brown. Group Processes: Dynamics Within and Between Groups. Wiley-Blackwell, 2000.

[11]

M. Cohn. It's effort, not complexity. https://goo.gl/nNYlL1, 2010. Accessed: 2016-06-02.

[12]

T. DeMarco. Controlling software projects: Management, measurement, and estimates. Prentice Hall PTR, 1986.

Digital Library

[13]

Why do the cards deviate slightly from the fibonacci sequence? http://goo.gl/xgs7PF. Accessed: 2016-07-13.

[14]

J. Grenning. Planning poker or how to avoid analysis paralysis while release planning. Hawthorn Woods: Renaissance Software Consulting, 3, 2002.

[15]

R. T. Hughes. Expert judgement as an estimating method. Information and Software Technology, 38(2):67--75, 1996.

[16]

Github issues tutorial. https://goo.gl/OR1gwr. Accessed: 2016-07-13.

[17]

What is an issue - atlassian documentation. https://goo.gl/JduWpL. Accessed: 2016-07-13.

[18]

N. Japkowicz and M. Shah. Evaluating learning algorithms: a classification perspective. Cambridge University Press, 2011.

Digital Library

[19]

Jira - pros and cons of cloud vs. server. https://goo.gl/5KF2rI. Accessed: 2016-07-13.

[20]

T. Joachims. Text categorization with support vector machines: Learning with many relevant features. Springer, 1998.

Digital Library

[21]

L. Marks, Y. Zou, and A. E. Hassan. Studying the fix-time for bugs in large open source projects. In Proceedings of the 2011 International Conference on Predictive Models in Software Engineering, page 11. ACM, 2011.

Digital Library

[22]

A. Murgia, G. Concas, R. Tonelli, M. Ortu, S. Demeyer, and M. Marchesi. On the influence of maintenance activity types on the issue resolution time. In Proceedings of the 2014 International Conference on Predictive Models in Software Engineering, pages 12--21. ACM, 2014.

Digital Library

[23]

A. Murgia, D. Janssens, S. Demeyer, and B. Vasilescu. Among the machines: Human-bot interaction on social Q&A websites. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA '16, pages 1272--1279, New York, NY, USA, 2016. ACM.

Digital Library

[24]

A. Murgia, P. Tourani, B. Adams, and M. Ortu. Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In Proceedings of the 11th Working Conference on Mining Software Repositories, MSR 2014, pages 262--271, New York, NY, USA, 2014. ACM.

Digital Library

[25]

J. W. Paulson, G. Succi, and A. Eberlein. An empirical study of open-source and closed-source software products. IEEE Transactions on Software Engineering, 30(4):246--256, 2004.

Digital Library

[26]

Planning poker 3.0. https://goo.gl/Tl5mws. Accessed: 2016-06-16.

[27]

I. Rish. An empirical study of the naive bayes classifier. Technical report, 2001.

[28]

Q. Song, M. Shepperd, M. Cartwright, and C. Mair. Software defect association mining and defect correction effort prediction. IEEE Transactions on Software Engineering, 32(2):69--82, 2006.

Digital Library

[29]

A. Trendowicz, J. Münch, and R. Jeffery. State of the practice in software effort estimation: a survey and literature review. In Software Engineering Techniques, pages 232--245. Springer, 2008.

Digital Library

[30]

M. Usman, E. Mendes, F. Weidt, and R. Britto. Effort estimation in agile software development: A systematic literature review. In Proceedings of the 2014 International Conference on Predictive Models in Software Engineering, pages 82--91. ACM, 2014.

Digital Library

[31]

C. Weiss, R. Premraj, T. Zimmermann, and A. Zeller. How long will it take to fix this bug? In Proceedings of the 2007 International Workshop on Mining Software Repositories, page 1. IEEE Computer Society, 2007.

Digital Library

[32]

What is a story point? https://goo.gl/Vfb9v1, 2007. Accessed: 2016-06-02.

[33]

H. Zeng and D. Rine. Estimation of software defects fix effort using neural networks. In Proceedings of the 2004 Annual International Conference on Computer Software and Applications, volume 2, pages 20--21. IEEE, 2004.

Digital Library

Cited By

Pasuksmit JThongtanunam PKarunasekera S(2024)A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in AgileACM Computing Surveys10.1145/366336556:11(1-37)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3663365
Corbari Dos Santos GSilva DPeres LValentim N(2024)Case Study of a Model that evaluates the Learner Experience with DICTsExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637138(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637138
Li YRen ZWang ZYang LDong LZhong CZhang HRoychoudhury APaiva AAbreu RStorey M(2024)Fine-SE: Integrating Semantic Features and Expert Features for Software Effort EstimationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623349(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623349
Show More Cited By

Recommendations

Using developers' features to estimate story points
ICSSP '18: Proceedings of the 2018 International Conference on Software and System Process

Effort estimation is important to correctly plan the use of resources in a software project. In agile projects, a correct effort estimation helps decide which issues have to be fixed or finished during the next iteration. However, estimating issues can ...
Effort estimation in software development using story point: a machine learning approach

Agile methodologies are besieged with problems and potential solutions around predictive insights on a project. These problems range from estimation, quality, to effort and duration requirements. Despite of having innumerable predictive models not a ...
Comparison of Functional Size Based Estimation and Story Points, Based on Effort Estimation Effectiveness in SCRUM Projects
SEAA '14: Proceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications

In this study, we compared the effectiveness of two approaches to effort estimation for organizations utilizing SCRUM. We compared SCRUM's native effort estimation method Story Points and poker planning, with effort estimation models based on COSMIC ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PROMISE 2016: Proceedings of the The 12th International Conference on Predictive Models and Data Analytics in Software Engineering

September 2016

84 pages

ISBN:9781450347723

DOI:10.1145/2972958

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Institute for the Promotion of Innovation through Science and Technology in Flanders
Sardinia Regional Government

Conference

PROMISE 2016

PROMISE 2016: The 12th International Conference on Predictive Models and Data Analytics in Software Engineering

September 9, 2016

Ciudad Real, Spain

Acceptance Rates

PROMISE 2016 Paper Acceptance Rate 10 of 23 submissions, 43%;

Overall Acceptance Rate 98 of 213 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
705
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)7

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pasuksmit JThongtanunam PKarunasekera S(2024)A Systematic Literature Review on Reasons and Approaches for Accurate Effort Estimations in AgileACM Computing Surveys10.1145/366336556:11(1-37)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3663365
Corbari Dos Santos GSilva DPeres LValentim N(2024)Case Study of a Model that evaluates the Learner Experience with DICTsExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637138(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637138
Li YRen ZWang ZYang LDong LZhong CZhang HRoychoudhury APaiva AAbreu RStorey M(2024)Fine-SE: Integrating Semantic Features and Expert Features for Software Effort EstimationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623349(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623349
Meenakshi Pareek M(2024)Software Effort Estimation Using Deep Learning: A Gentle ReviewArtificial Intelligence and Sustainable Computing10.1007/978-981-97-0327-2_26(351-364)Online publication date: 24-Apr-2024
https://doi.org/10.1007/978-981-97-0327-2_26
Islam MSandborn P(2024)Analyzing the Influence of Processor Speed and Clock Speed on Remaining Useful Life Estimation of Software SystemsIntelligent Computing10.1007/978-3-031-62281-6_34(490-507)Online publication date: 14-Jun-2024
https://doi.org/10.1007/978-3-031-62281-6_34
Raza AEspinosa-Leal L(2024)Predicting the Duration of User Stories in Agile Project ManagementSmart Technologies for a Sustainable Future10.1007/978-3-031-61905-2_31(316-328)Online publication date: 13-Jun-2024
https://doi.org/10.1007/978-3-031-61905-2_31
Amasaki SMcIntosh SChoi EHerbold S(2023)On Effectiveness of Further Pre-training on BERT Models for Story Point EstimationProceedings of the 19th International Conference on Predictive Models and Data Analytics in Software Engineering10.1145/3617555.3617877(49-53)Online publication date: 8-Dec-2023
https://dl.acm.org/doi/10.1145/3617555.3617877
Tawosi VMoussa RSarro F(2023)Agile Effort Estimation: Have We Solved the Problem Yet? Insights From a Replication StudyIEEE Transactions on Software Engineering10.1109/TSE.2022.322873949:4(2677-2697)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TSE.2022.3228739
Fu MTantithamthavorn C(2023)GPT2SP: A Transformer-Based Agile Story Point Estimation ApproachIEEE Transactions on Software Engineering10.1109/TSE.2022.315825249:2(611-625)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TSE.2022.3158252
Arachchi KAmalraj C(2023)An Agile Project Management Supporting Approach for Estimating Story Points in User Stories2023 8th International Conference on Information Technology Research (ICITR)10.1109/ICITR61062.2023.10382930(1-6)Online publication date: 7-Dec-2023
https://doi.org/10.1109/ICITR61062.2023.10382930
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents