research-article

Searching the state space: a qualitative study of API protocol usability

Authors:

Joshua Sushine,

James D. Herbsleb,

Jonathan AldrichAuthors Info & Claims

ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

Pages 82 - 93

Published: 16 May 2015 Publication History

Abstract

Application Programming Interfaces (APIs) often define protocols --- restrictions on the order of client calls to API methods. API protocols are common and difficult to use, which has generated tremendous research effort in alternative specification, implementation, and verification techniques. However, little is understood about the barriers programmers face when using these APIs, and therefore the research effort may be misdirected.

To understand these barriers better, we perform a two-part qualitative study. First, we study developer forums to identify problems that developers have with protocols. Second, we perform a think-aloud observational study, in which we systematically observe professional programmers struggle with these same problems to get more detail on the nature of their struggles and how they use available resources. In our observations, programmer time was spent primarily on four types of searches of the protocol state space. These observations suggest protocol-targeted tools, languages, and verification techniques will be most effective if they enable programmers to efficiently perform state search.

References

[1]

Stephanie Balzer and Thomas R. Gross. Verifying multi-object invariants with relationships. In ECOOP 2011 -- Object-Oriented Programming, pages 358--382. Springer Berlin Heidelberg, 2011.

Digital Library

[2]

Nels E. Beckman, Duri Kim, and Jonathan Aldrich. An empirical study of object protocols in the wild. In ECOOP 2011 -- Object-Oriented Programming, pages 2--26. Springer Berlin Heidelberg, 2011.

Digital Library

[3]

Kevin Bierhoff and Jonathan Aldrich. Lightweight object specification with typestates. In Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, ESEC/FSE-13, pages 217--226, New York, NY, USA, 2005. ACM.

Digital Library

[4]

Kevin Bierhoff, Nels E. Beckman, and Jonathan Aldrich. Practical API protocol checking with access permissions. In Proceedings of the 23rd European Conference on ECOOP 2009 --- Object-Oriented Programming, Genoa, pages 195--219, Berlin, Heidelberg, 2009. Springer-Verlag.

Digital Library

[5]

Joshua Bloch. Effective Java. Addison-Wesley Professional, second edition, 2008.

[6]

John M Daughtry, Umer Farooq, Jeffrey Stylos, and Brad A Myers. API usability: CHI'2009 special interest group meeting. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, pages 2771--2774. ACM, 2009.

Digital Library

[7]

Uri Dekel and James D. Herbsleb. Improving API documentation usability with knowledge pushing. In Proceedings of the 31st International Conference on Software Engineering, ICSE '09, pages 320--330, Washington, DC, USA, 2009. IEEE Computer Society.

Digital Library

[8]

Robert DeLine and Manuel Fähndrich. Typestates for objects. In Proceedings of the 18th European Conference on Object-Oriented Programming, ECOOP '04, pages 465--490, London, UK, 2004. Springer-Verlag.

[9]

Matthew B. Dwyer, Alex Kinneer, and Sebastian Elbaum. Adaptive online program analysis. In Proceedings of the 29th international conference on Software Engineering, ICSE '07, pages 220--229, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[10]

Brian Ellis, Jeffrey Stylos, and Brad Myers. The factory pattern in API design: A usability evaluation. In Proceedings of the 29th international conference on Software Engineering, ICSE '07, pages 302--312, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[11]

Jeffrey S. Foster, Tachio Terauchi, and Alex Aiken. Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, PLDI '02, pages 1--12, New York, NY, USA, 2002. ACM.

Digital Library

[12]

Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design patterns: elements of reusable object-oriented software. Addison-Wesley, 1995.

Digital Library

[13]

Benjamin V. Hanrahan, Gregorio Convertino, and Les Nelson. Modeling problem difficulty and expertise in stackoverflow. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion, CSCW '12, pages 91--94, New York, NY, USA, 2012. ACM.

Digital Library

[14]

Ciera Jaspan and Jonatathan Aldrich. Are object protocols burdensome? an empirical study of developer forums. In Evaluation and Usability of Programming Languages and Tools Workshop (PLATEAU '11), 2011.

Digital Library

[15]

Ciera Jaspan and Jonathan Aldrich. Checking framework interactions with relationships. In ECOOP 2009 -- Object-Oriented Programming, pages 27--51. Springer Berlin Heidelberg, 2009.

Digital Library

[16]

Ciera N. C. Jaspan. Proper Plugin Protocols. PhD thesis, Carnegie Mellon University, December 2011. Technical Report: CMU-ISR-11-116.

Digital Library

[17]

Andrew J. Ko and Brad A. Myers. Finding causes of program output with the Java Whyline. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, pages 1569--1578, New York, NY, USA, 2009. ACM.

Digital Library

[18]

Andrew J. Ko, Robert DeLine, and Gina Venolia. Information needs in collocated software development teams. In Proceedings of the 29th international conference on Software Engineering, ICSE '07, pages 344--353, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[19]

George Kuk. Strategic interaction and knowledge sharing in the kde developer mailing list. Management Science, 52(7):1031--1042, 2006.

Digital Library

[20]

Thomas D. LaToza, David Garlan, James D. Herbsleb, and Brad A. Myers. Program comprehension as fact finding. In Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, ESEC-FSE '07, pages 361--370, New York, NY, USA, 2007. ACM.

Digital Library

[21]

Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann. Design lessons from the fastest Q&A site in the west. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 2857--2866. ACM, 2011.

Digital Library

[22]

Samuel G. McLellan, Alvin W. Roesler, Joseph T. Tempest, and Clay I. Spinuzzi. Building more usable APIs. IEEE Software, 15(3):78--86, 1998.

Digital Library

[23]

Chris Parnin and Christoph Treude. Measuring api documentation on the web. In Proceedings of the 2nd international workshop on Web 2.0 for software engineering, pages 25--30. ACM, 2011.

Digital Library

[24]

Martin P. Robillard and Robert DeLine. A field study of API learning obstacles. Empirical Software Engineering, 16:703--732, 2011.

Digital Library

[25]

Martin P. Robillard, Eric Bodden, David Kawrykow, Mira Mezini, and Tristan Ratchford. Automated api property inference techniques. Software Engineering, IEEE Transactions on, 39(5):613--637, 2013.

Digital Library

[26]

J. Sillito, G. C. Murphy, and K. De Volder. Asking and answering questions during a programming change task. Software Engineering, IEEE Transactions on, 34(4):434--451, 2008.

Digital Library

[27]

Anselm L. Strauss. Qualitative Analysis for Social Scientists. Cambridge University Press, June 1987.

[28]

Robert E. Strom and Shaula Yemini. Typestate: A programming language concept for enhancing software reliability. IEEE Transactions on Software Engineering, 12(1):157--171, January 1986.

Digital Library

[29]

Jeffrey Stylos and Steven Clarke. Usability implications of requiring parameters in objects' constructors. In Proceedings of the 29th international conference on Software Engineering, ICSE '07, pages 529--539, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[30]

Jeffrey Stylos and Brad A. Myers. The implications of method placement on API learnability. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, SIGSOFT '08/FSE-16, pages 105--112, New York, NY, USA, 2008. ACM.

Digital Library

[31]

Joshua Sunshine. Protocol Programmability. PhD thesis, Carnegie Mellon University, December 2013. CMU-ISR-13-117.

Digital Library

[32]

Joshua Sunshine, James D. Herbsleb, and Jonathan Aldrich. Structuring documentation to support state search: A laboratory experiment about protocol programming. In European Conference on Object Oriented Programming (ECOOP), 2014.

Digital Library

[33]

Christoph Treude, Ohad Barzilay, and M-A Storey. How do programmers ask and answer questions on the web?: NIER track. In Software Engineering (ICSE), 2011 33rd International Conference on, pages 804--807. IEEE, 2011.

Digital Library

[34]

Bogdan Vasilescu, Andrea Capiluppi, and Alexander Serebrenik. Gender, representation and online participation: A quantitative study of stackoverflow. In International Conference on Social Informatics. ASE, 2012.

Digital Library

[35]

John Whaley, Michael C. Martin, and Monica S. Lam. Automatic extraction of object-oriented component interfaces. In Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis, ISSTA '02, pages 218--228, New York, NY, USA, 2002. ACM.

Digital Library

[36]

Alexey Zagalsky, Ohad Barzilay, and Amiram Yehudai. Example overflow: Using social media for code recommendation. In Recommendation Systems for Software Engineering (RSSE), 2012 Third International Workshop on, pages 38--42. IEEE, 2012.

Digital Library

Cited By

Amann SNguyen HNadi SNguyen TMezini MStorey MAdams BHaiduc S(2019)Investigating next steps in static API-misuse detectionProceedings of the 16th International Conference on Mining Software Repositories10.1109/MSR.2019.00053(265-275)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MSR.2019.00053
Wen MLiu YWu RXie XCheung SSu ZAtlee JBultan TWhittle J(2019)Exposing library API misuses via mutation analysisProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00093(866-877)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00093
Coblenz MAldrich JMyers BSunshine JBoix EGabriel R(2018)Interdisciplinary programming language designProceedings of the 2018 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3276954.3276965(133-146)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276954.3276965
Show More Cited By

Index Terms

Recommendations

Searching the State Space: A Qualitative Study of API Protocol Usability
ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

Application Programming Interfaces (APIs) often define protocols--restrictions on the order of client calls to API methods. API protocols are common and difficult to use, which has generated tremendous research effort in alternative specification, ...
Searching the library and asking the peers: learning to use Java APIs on demand
PPPJ '07: Proceedings of the 5th international symposium on Principles and practice of programming in Java

The existence of large API libraries contributes significantly to the programming productivity and quality of Java programmers. The vast number of available library APIs, however, presents a learning challenge for Java programmers. Most Java programmers ...
Web searching and navigation: Age, intelligence, and familiarity
Abstract
In using the Internet to solve everyday problems, older adults tend to find fewer correct answers compared to younger adults. Some authors have argued that these differences could be explained by age‐related decline. The present study aimed to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

May 2015

325 pages

General Chair:
Andrea De Lucia
University of Salerno, Italy
,
Program Chairs:
Christian Bird
Microsoft Research
,
Rocco Oliveto
University of Molise, Italy

Sponsors

ACM: Association for Computing Machinery
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS\DATC: IEEE Computer Society
TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

IEEE Press

Publication History

Published: 16 May 2015

Check for updates

Qualifiers

Research-article

Conference

ICSE '15

Sponsor:

ACM
SIGSOFT
IEEE-CS\DATC
TCSE

ICSE '15: 37th International Conference on Software Engineering

May 16 - 24, 2015

Florence, Italy

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
82
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Amann SNguyen HNadi SNguyen TMezini MStorey MAdams BHaiduc S(2019)Investigating next steps in static API-misuse detectionProceedings of the 16th International Conference on Mining Software Repositories10.1109/MSR.2019.00053(265-275)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MSR.2019.00053
Wen MLiu YWu RXie XCheung SSu ZAtlee JBultan TWhittle J(2019)Exposing library API misuses via mutation analysisProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00093(866-877)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00093
Coblenz MAldrich JMyers BSunshine JBoix EGabriel R(2018)Interdisciplinary programming language designProceedings of the 2018 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3276954.3276965(133-146)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276954.3276965
Laghari GDemeyer SHaddad HWainwright RChbeir R(2018)On the use of sequence mining within spectrum based fault localisationProceedings of the 33rd Annual ACM Symposium on Applied Computing10.1145/3167132.3167337(1916-1924)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3167132.3167337
Salvaneschi GAnslow CLaToza TSunshine J(2016)What do we really know about data flow languages?Proceedings of the 7th International Workshop on Evaluation and Usability of Programming Languages and Tools10.1145/3001878.3001884(30-31)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1145/3001878.3001884
Nadi SKrüger SMezini MBodden EDillon LVisser WWilliams L(2016)Jumping through hoopsProceedings of the 38th International Conference on Software Engineering10.1145/2884781.2884790(935-946)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2884781.2884790

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents