Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3491102.3502095acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Open access

Understanding How Programmers Can Use Annotations on Documentation

Published: 28 April 2022 Publication History
  • Get Citation Alerts
  • Abstract

    Modern software development requires developers to find and effectively utilize new APIs and their documentation, but documentation has many well-known issues. Despite this, developers eventually overcome these issues but have no way of sharing what they learned. We investigate sharing this documentation-specific information through annotations, which have advantages over developer forums as the information is contextualized, not disruptive, and is short, thus easy to author. Developers can also author annotations to support their own comprehension. In order to support the documentation usage behaviors we found, we built the Adamite annotation tool, which provides features such as multiple anchors, annotation types, and pinning. In our user study, we found that developers are able to create annotations that are useful to themselves and are able to utilize annotations created by other developers when learning a new API, with readers of the annotations completing 67% more of the task, on average, than the baseline.

    Supplementary Material

    Supplemental Materials (3491102.3502095-supplemental-materials.zip)
    MP4 File (3491102.3502095-talk-video.mp4)
    Talk Video
    MP4 File (3491102.3502095-video-figure.mp4)
    Video Figure
    MP4 File (3491102.3502095-video-preview.mp4)
    Video Preview

    References

    [1]
    Emad Aghajani, Csaba Nagy, Olga Lucero Vega-Márquez, Mario Linares-Vásquez, Laura Moreno, Gabriele Bavota, and Michele Lanza. 2019. Software Documentation Issues Unveiled. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, Montreal, QC, Canada, 1199–1210. https://doi.org/10.1109/ICSE.2019.00122
    [2]
    Maristella Agosti, Giorgetta Bonfiglio-Dosio, and Nicola Ferro. 2007. A historical and contemporary study on annotations to derive key features for systems design. International Journal on Digital Libraries 8 (Feb. 2007), 1–19. https://doi.org/10.1007/s00799-007-0010-0
    [3]
    Mohammad Allahbakhsh, Boualem Benatallah, Aleksandar Ignjatovic, Hamid Reza Motahari-Nezhad, Elisa Bertino, and Schahram Dustdar. 2013. Quality Control in Crowdsourcing Systems: Issues and Directions. IEEE Internet Computing 17 (2013), 76–81. Issue 2. https://doi.org/10.1109/MIC.2013.20
    [4]
    Muhammad Asaduzzaman, Ahmed Shah Mashiyat, Chanchal K. Roy, and Kevin A. Schneider. 2013. Answering Questions about Unanswered Questions of Stack Overflow. In MSR 2013. IEEE, San Francisco, CA, USA, 97–100. https://doi.org/10.1109/MSR.2013.6624015
    [5]
    Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code. In CHI ’09 (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1589–1598. https://doi.org/10.1145/1518701.1518944
    [6]
    Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A Method for Evaluating Software’s Gender Inclusiveness. Interacting with Computers 28, 6 (Nov. 2016), 27 pages. https://doi.org/10.1145/3134737
    [7]
    CodeSandbox BV. 2021. CodeSandbox: Online Code Editor and IDE for Rapid Web Development. CodeSandbox BV. Retrieved September 3, 2021 from https://codesandbox.io/
    [8]
    Paul Chandler and John Sweller. 1991. Cognitive load theory and the format of instruction. Cognition and instruction 8, 4 (1991), 293–332.
    [9]
    Preetha Chatterjee, Manziba Akanda Nishi, Kostadin Damevski, Vinay Augustine, Lori Pollock, and Nicholas A. Kraft. 2017. What information about code snippets is available in different software-related documents? An exploratory study. In SANER 2017. IEEE, New York City, NY, USA, 382–386.
    [10]
    J. C. Chen and S. J. Huang. 2009. An empirical analysis of the impact of software development problem factors on software maintainability. Journal of Systems and Software 82, 6 (2009), 11 pages.
    [11]
    Michelene TH Chi, Miriam Bassok, Matthew W Lewis, Peter Reimann, and Robert Glaser. 1989. Self-explanations: How students study and use examples in learning to solve problems. Cognitive science 13, 2 (1989), 145–182.
    [12]
    Parmit K. Chilana, Amy Ko, and James O. Wobbrock. 2012. LemonAid: selection-based crowdsourced contextual help for web applications. In CHI 2012. ACM, New York City, NY, USA, 1549–1558. https://doi.org/10.1145/2207676.2208620
    [13]
    C.R. Cook, J.C. Scholtz, and J.C. Spohrer. 1993. Empirical Studies of Programmers: Fifth Workshop : Papers Presented at the Fifth Workshop on Empirical Studies of Programmers, December 3-5, 1993, Palo Alto, CA. Ablex Publishing Corporation, Norwood, NJ, USA. https://books.google.com/books?id=rMmxq8q0CGYC
    [14]
    csillag. 2013. Fuzzy Anchoring. Hypothes.is. Retrieved December 26, 2021 from https://web.hypothes.is/blog/fuzzy-anchoring/
    [15]
    Alex Cummaudo, Rajesh Vasa, John Grundy, and Mohamed Abdelrazek. 2020. Requirements of API Documentation: A Case Study Into Computer Vision Services. IEEE Transactions on Software Engineering Early Access (2020), 1–1. https://doi.org/10.1109/TSE.2020.3047088
    [16]
    Uri Dekel and James D. Herbsleb. 2009. Reading the documentation of invoked API functions in program comprehension. In 2009 IEEE 17th International Conference on Program Comprehension. IEEE, New York City, NY, USA, 168–177. https://doi.org/10.1109/ICPC.2009.5090040
    [17]
    Robert Deline, Mary Czerwinski, and George Robertson. 2005. Easing program comprehension by sharing navigation data. In VLHCC 2005. IEEE, New York City, NY, USA, 241–248. https://doi.org/10.1109/VLHCC.2005.32
    [18]
    Google Developers. 2021. Cloud Firestore: Store and sync app data at global scale. Google LLC. Retrieved September 3, 2021 from https://firebase.google.com/products/firestore
    [19]
    Ekwa Duala-Ekoko and Martin P. Robillard. 2012. Asking and answering questions about unfamiliar APIs: An exploratory study. In ICSE 2012. IEEE, New York City, NY, USA, 266–276.
    [20]
    Elastic. 2021. Free and Open Search: Elasticsearch. Elastic. Retrieved September 3, 2021 from https://www.elastic.co/
    [21]
    Stefan Endrikat, Stefan Hanenberg, Romain Robbes, and Andreas Stefik. 2014. How do API documentation and static typing affect API usability?. In ICSE 2014. ACM, New York City, NY, USA, 632–642. https://doi.org/10.1145/2568225.2568299
    [22]
    Sascha Fahl, Marian Harbach, Henning Perl, Markus Koetter, and Matthew Smith. 2013. Rethinking SSL Development in an Appified World. In Proceedings of the 2013 AC SIGSAC Conference on Computer and Communications Security(CCS ’13). ACM, New York City, NY, USA, 49–60. https://doi.org/10.1145/2508859.2516655
    [23]
    Jingchao Fang, Yanhao Wang, Chi-Lan Yang, and Hao-Chuan Wang. 2021. NoteCoStruct: Powering Online Learners with Socially Scaffolded Note Taking and Sharing. Association for Computing Machinery, New York, NY, USA, Chapter Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–5. https://doi.org/10.1145/3411763.3451694
    [24]
    Daniela Fogli, Giuseppe Fresta, and Piero Mussio. 2004. On electronic annotation and its implementation. In AVI 2004. ACM, New York, NY, USA, 98–102. https://doi.org/10.1145/989863.989877
    [25]
    Adam Fourney and Meredith Ringel Morris. 2013. Enhancing Technical Q&A Forums with CiteHistory. In ICWSM 2013. AAAI, Palo Alto, CA, USA, 1–10.
    [26]
    Mitchell Gordon and Philip J. Guo. 2015. Codepourri: Creating visual coding tutorials using a volunteer crowd of learners. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, New York, NY, USA, 13–21. https://doi.org/10.1109/VLHCC.2015.7357193
    [27]
    Mark Guzdial and Jennifer Turns. 2000. Effective Discussion Through a Computer Mediated Anchored Forum. The Journal of the Learning Sciences 9 (2000), 437–469. Issue 4. https://doi.org/10.1207/S15327809JLS0904_3
    [28]
    Lichan Hong and Ed H Chi. 2009. Annotate once, appear anywhere: collective foraging for snippets of interest using paragraph fingerprinting. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1791–1794.
    [29]
    Amber Horvath, Sachin Grover, Sihan Dong, Emily Zhou, Finn Voichick, Mary Beth Kery, Shwetha Shinju, Daye Nam, Mariann Nagy, and Brad Myers. 2019. The Long Tail: Understanding the Discoverability of API Functionality. In VLHCC 2019. IEEE, New York, NY, USA, 157–161. https://doi.org/10.1109/VLHCC.2019.8818681
    [30]
    Amber Horvath, Mariann Nagy, Finn Voichick, Mary Beth Kery, and Brad A Myers. 2019. Methods for investigating mental models for learners of APIs. In CHI LBW ’19. ACM, New York, NY, USA, 1–6.
    [31]
    Hypothes.is. 2012. Hypothes.is: Annotate the web, with anyone, anywhere.Hypothes.is. Retrieved September 3, 2021 from https://web.hypothes.is/
    [32]
    Aniket Kittur, Andrew M. Peters, Abdigani Diriye, Trupti Telang, and Michael R. Bove. 2013. Costs and benefits of structured information foraging. In CHI 2013. ACM, New York, NY, USA, 2989–2998.
    [33]
    Amy Ko and Bob Uttl. 2003. Individual differences in program comprehension strategies in unfamiliar programming systems. In 11th Annual Workshop on Program Comprehension. IEEE, New York, NY, USA, 175–184. https://doi.org/10.1109/WPC.2003.1199201
    [34]
    Thomas D. LaToza, David Garlan, James D. Herbsleb, and Brad A. Myers. 2007. Program comprehension as fact finding. In ESEC-FSE 2007. ACM, New York, NY, USA, 361–270.
    [35]
    Timothy C. Lethbirdge, Janice Singer, and Andrew Forward. 2003. How software engineers use documentation: the state of the practice. IEEE Software 20 (Nov. 2003), 35–39. Issue 6. https://doi.org/10.1109/MS.2003.1241364
    [36]
    Michael Xieyang Liu, Jane Hsieh, Nathan Hahn, Angelina Zhou, Emily Deng, Shaun Burley, Cynthia Taylor, Aniket Kittur, and Brad A. Myers. 2019. Unakite: Scaffolding Developers’ Decision-Making Using the Web. In UIST 2019. ACM, New York, NY, USA, 67–80.
    [37]
    Michael Xieyang Liu, Aniket Kittur, and Brad A. Myers. 2021. To Reuse or Not To Reuse? A Framework and System for Evaluating Summarized Knowledge. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 166 (apr 2021), 35 pages. https://doi.org/10.1145/3449240
    [38]
    Walid Maalej and Hans-Jorg Happel. 2009. From work to word: How do software developers describe their work?. In MSR 2009. IEEE, New York, NY, USA, 121–130. https://doi.org/10.1109/MSR.2009.5069490
    [39]
    Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke. 2014. On the Comprehension of Program Comprehension. Transactions on Software Engineering 23 (2014), 1–37. Issue 4. https://doi.org/10.1145/2622669
    [40]
    Catherine C. Marshall and A. J. Bernheim Brush. 2004. Exploring the relationship between personal and public annotations. In JCDL 2004. ACM, New York, NY, USA, 349–357. https://doi.org/10.1145/3173574.3174182
    [41]
    Michael Meng, Stephanie M Steinhard, and Andreas Schubert. 2019. How developers use API documentation: an observation study. Communication Design Quarterly 7 (2019), 40–49. Issue 2. https://doi.org/10.1145/3358931.3358937
    [42]
    Brad A. Myers and Jeffrey Stylos. 2016. Improving API Usability. Commun. ACM 59, 6 (2016), 62–69. https://doi.org/10.1145/2896587
    [43]
    Seyed Mehdi Nasehi, Jonathan Sillito, Frank Maurer, and Chris Burns. 2012. What makes a good code example?: A study of programming Q&A in StackOverflow. In ICSM 2012. IEEE, New York, NY, USA, 25–34.
    [44]
    Janet Nykaza, Rhonda Messinger, Fran Boehme, Cherie L. Norman, Matthew Mace, and Manuel Gordon. 2002. What programmers really want: results of a needs assessment for SDK documentation. In SIGDOC 2002. ACM, New York, NY, USA, 133–141. https://doi.org/10.1145/584955.584976
    [45]
    Steve Oney, Christopher Brooks, and Paul Resnick. 2018. Creating Guided Code Explanations with Chat.Codes. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 131 (Nov. 2018), 20 pages. https://doi.org/10.1145/3274400
    [46]
    Chris Parnin and Robert DeLine. 2010. Evaluating Cues for Resuming Interrupted Programming Tasks. Association for Computing Machinery, New York, NY, USA, 93–102. https://doi.org/10.1145/1753326.1753342
    [47]
    Piling.js. 2021. The Piling.js Docs. Piling.js. Retrieved September 3, 2021 from https://piling.js.org/docs/
    [48]
    Christi-Anne Postava-Davignon, Candice Kamachi, Cory Clarke, Gregory Kushmerek, Mary Beth Rettger, Pete Monchamp, and Rich Ellis. 2004. Incorporating Usability Testing into the Documentation Process. Technical Communication 51 (Feb. 2004), 36–44. Issue 1.
    [49]
    Mohammad Masudur Rahman, Chanchal K. Roy, and Iman Keivanloo. 2015. Recommending insightful comments for source code using crowdsourced knowledge. In 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation. IEEE, New York, NY, USA, 81–90. https://doi.org/10.1109/SCAM.2015.7335404
    [50]
    Martin P. Robillard. 2009. What Makes APIs Hard to Learn? Answers from Developers. IEEE Software 26 (Oct. 2009), 27–34. Issue 6. https://doi.org/10.1109/MS.2009.193
    [51]
    Martin P. Robillard and Robert DeLine. 2011. A field study of API learning obstacles. Empirical Software Engineering 16 (2011), 703–732. Issue 6.
    [52]
    Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How do API documentation and static typing affect API usability?. In ICSE 2012. ACM, New York, NY, USA, 632–542. https://doi.org/10.1109/ICSE.2012.6227188
    [53]
    Jeff Sandquist. 2016. Recognizing Our Top Community Contributors - Summer 2016. Microsoft. Retrieved September 3, 2021 from https://docs.microsoft.com/en-us/teamblog/recognizing-top-documentation-contributors-summer-2016
    [54]
    Jonathan Sillito and Gail C. Murphy. 2008. Asking and Answering Questions during a Programming Change Task. IEEE Transactions on Software Engineering 34 (2008), 434–451. Issue 4. https://doi.org/10.1109/TSE.2008.26
    [55]
    Gias Uddin and Martin P. Robillard. 2015. How API Documentation Fails. IEEE Software 32 (Aug. 2015), 68–75. Issue 4. https://doi.org/10.1109/MS.2014.80
    [56]
    Laton Vermette, Shruti Dembla, April Y. Wang, Joanna McGrenere, and Parmit K. Chilana. 2017. Social CheatSheet: An Interactive Community-Curated Information Overlay for Web Applications. Proc. ACM Hum.-Comput. Interact. 1, CSCW, Article 102 (Dec. 2017), 19 pages. https://doi.org/10.1145/3134737
    [57]
    Xin Xia, Lingfeng Bao, David Lo, Zhenchang Xing, Ahmed E. Hassan, and Shanping Li. 2018. Measuring Program Comprehension: A Large-Scale Field Study with Professionals. IEEE Transactions on Software Engineering 44 (2018), 951–976. Issue 10. https://doi.org/10.1109/TSE.2017.2734091
    [58]
    Tianyi Zhang, Ganesha Upadhyaya, Anastasia Reinhardt, Hridesh Rajan, and Miryung Kim. 2018. Are Code Examples on an Online Q&A Forum Reliable? A Study of API Misuse on Stack Overflow. In ICSE 2018. ACM, New York, NY, USA, 886–896.
    [59]
    J. Zhi, V. Garousi-Yusifoglu, B. Sun, G. Garousi, S. Shahnewaz, and G. Ruhe. 2015. Cost benefits and quality of software development documentation: a systematic mapping. Journal of Systems and Software 99 (2015), 23 pages.
    [60]
    Sacha Zyto, David Karger, Mark Ackerman, and Sanjoy Mahajan. 2012. Successful classroom deployment of a social document annotation system. In CHI 2012. ACM, New York, NY, USA, 1883–1892. https://doi.org/10.1145/2207676.2208326

    Cited By

    View all
    • (2024)Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud ServicesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642721(1-17)Online publication date: 11-May-2024
    • (2024)Meta-Manager: A Tool for Collecting and Exploring Meta Information about CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642676(1-17)Online publication date: 11-May-2024
    • (2024)Ivie: Lightweight Anchored Explanations of Just-Generated CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642239(1-15)Online publication date: 11-May-2024
    • Show More Cited By

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
    April 2022
    10459 pages
    ISBN:9781450391573
    DOI:10.1145/3491102
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 April 2022

    Check for updates

    Author Tags

    1. Annotations
    2. application programming interfaces (APIs)
    3. documentation
    4. note taking
    5. software engineering

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • NSF

    Conference

    CHI '22
    Sponsor:
    CHI '22: CHI Conference on Human Factors in Computing Systems
    April 29 - May 5, 2022
    LA, New Orleans, USA

    Acceptance Rates

    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)424
    • Downloads (Last 6 weeks)41
    Reflects downloads up to 11 Aug 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud ServicesProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642721(1-17)Online publication date: 11-May-2024
    • (2024)Meta-Manager: A Tool for Collecting and Exploring Meta Information about CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642676(1-17)Online publication date: 11-May-2024
    • (2024)Ivie: Lightweight Anchored Explanations of Just-Generated CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642239(1-15)Online publication date: 11-May-2024
    • (2023)A Field Study of Developer Documentation FormatExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585767(1-7)Online publication date: 19-Apr-2023
    • (2023)“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language ModelsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580817(1-31)Online publication date: 19-Apr-2023
    • (2023)Support for Long-Form Documentation Authoring and Maintenance2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00020(109-114)Online publication date: 3-Oct-2023
    • (2022)Fuse: In-Situ Sensemaking Support in the BrowserProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545693(1-15)Online publication date: 29-Oct-2022
    • (2022)Using Annotations for Sensemaking About CodeProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545667(1-16)Online publication date: 29-Oct-2022
    • (2022)Wigglite: Low-cost Information Collection and TriageProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545661(1-16)Online publication date: 29-Oct-2022
    • (2022)Concept-Annotated Examples for Library ComparisonProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545647(1-16)Online publication date: 29-Oct-2022

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Get Access

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media