Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3526113.3545667acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article
Open access

Using Annotations for Sensemaking About Code

Published: 28 October 2022 Publication History

Abstract

Developers spend significant amounts of time finding, relating, navigating, and, more broadly, making sense of code. While sensemaking, developers must keep track of many pieces of information including the objectives of their task, the code locations of interest, their questions and hypotheses about the behavior of the code, and more. Despite this process being such an integral aspect of software development, there is little tooling support for externalizing and keeping track of developers’ information, which led us to develop Catseye – an annotation tool for lightweight notetaking about code. Catseye has advantages over traditional methods of externalizing code-related information, such as commenting, in that the annotations retain the original context of the code while not actually modifying the underlying source code, they can support richer interactions such as lightweight versioning, and they can be used as navigational aids. In our investigation of developers’ notetaking processes using Catseye, we found developers were able to successfully use annotations to support their code sensemaking when completing a debugging task.

Supplementary Material

Materials used for the lab study (a61-supplements.zip)
MP4 File (uist22a-sub6487-i8.mp4)
Supplemental video

References

[1]
Marjan Adeli, Nicholas Nelson, Souti Chattopadhyay, Hayden Coffey, Austin Henley, and Anita Sarma. 2020. Supporting code comprehension via annotations: Right information at the right time and place. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 1–10.
[2]
Maristella Agosti, Giorgetta Bonfiglio-Dosio, and Nicola Ferro. 2007. A historical and contemporary study on annotations to derive key features for systems design. International Journal on Digital Libraries 8, 1 (2007), 1–19.
[3]
Michael Bernstein, Max Van Kleek, David Karger, and MC Schraefel. 2008. Information scraps: How and why information eludes our personal information management tools. ACM Transactions on Information Systems (TOIS) 26, 4 (2008), 1–46.
[4]
Jürgen Börstler and Barbara Paech. 2016. The role of method chains and comments in software readability and comprehension—An experiment. IEEE Transactions on Software Engineering 42, 9 (2016), 886–898.
[5]
Andrew Bragdon, Robert Zeleznik, Steven P. Reiss, Suman Karumuri, William Cheung, Joshua Kaplan, Christopher Coleman, Ferdi Adeputra, and Joseph J. LaViola. 2010. Code Bubbles: A Working Set-Based Interface for Code Understanding and Maintenance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 2503–2512. https://doi.org/10.1145/1753326.1753706
[6]
Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code. In CHI ’09 (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1589–1598. https://doi.org/10.1145/1518701.1518944
[7]
Jonathan Carter. 2020. CodeTour. Microsoft. Retrieved July 1, 2022 from https://marketplace.visualstudio.com/items?itemName=vsls-contrib.codetour
[8]
Michelene TH Chi, Miriam Bassok, Matthew W Lewis, Peter Reimann, and Robert Glaser. 1989. Self-explanations: How students study and use examples in learning to solve problems. Cognitive science 13, 2 (1989), 145–182.
[9]
Parmit K. Chilana, Amy Ko, and James O. Wobbrock. 2012. LemonAid: selection-based crowdsourced contextual help for web applications. In CHI 2012. ACM, New York City, NY, USA, 1549–1558. https://doi.org/10.1145/2207676.2208620
[10]
Michael J. Coblenz, Amy J. Ko, and Brad A. Myers. 2006. JASPER: An Eclipse Plug-in to Facilitate Software Maintenance Tasks. In Proceedings of the 2006 OOPSLA Workshop on Eclipse Technology EXchange (Portland, Oregon, USA) (eclipse ’06). Association for Computing Machinery, New York, NY, USA, 65–69. https://doi.org/10.1145/1188835.1188849
[11]
Robert Deline, Mary Czerwinski, and George Robertson. 2005. Easing program comprehension by sharing navigation data. In VLHCC 2005. IEEE, New York City, NY, USA, 241–248. https://doi.org/10.1109/VLHCC.2005.32
[12]
Google Developers. 2022. Cloud Firestore: Store and sync app data at global scale. Google LLC. Retrieved March 27, 2022 from https://firebase.google.com/products/firestore
[13]
Facebook. 2022. React - A JavaScript library for building user interfaces. https://reactjs.org/
[14]
Beat Fluri, Michael Wursch, and Harald C Gall. 2007. Do code and comments co-evolve? on the relation between source code and comment changes. In 14th Working Conference on Reverse Engineering (WCRE 2007). IEEE, 70–79.
[15]
James Fogarty, Amy J. Ko, Htet Htet Aung, Elspeth Golden, Karen P. Tang, and Scott E. Hudson. 2005. Examining Task Engagement in Sensor-Based Statistical Models of Human Interruptibility. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA) (CHI ’05). Association for Computing Machinery, New York, NY, USA, 331–340. https://doi.org/10.1145/1054972.1055018
[16]
Ingo Frommholz, Holger Brocks, Ulrich Thiel, Erich Neuhold, Luigi Iannone, Giovanni Semeraro, Margherita Berardi, and Michelangelo Ceci. 2003. Document-centered collaboration for scholars in the humanities–the COLLATE system. In International conference on theory and practice of digital libraries. Springer, 434–445.
[17]
Mitchell Gordon and Philip J. Guo. 2015. Codepourri: Creating visual coding tutorials using a volunteer crowd of learners. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, New York, NY, USA, 13–21. https://doi.org/10.1109/VLHCC.2015.7357193
[18]
Anja Guzzi, Lile Hattori, Michele Lanza, Martin Pinzger, and Arie van Deursen. 2011. Collective Code Bookmarks for Program Comprehension. In 2011 IEEE 19th International Conference on Program Comprehension. 101–110. https://doi.org/10.1109/ICPC.2011.19
[19]
Andrew Head, Fred Hohman, Titus Barik, Steven M. Drucker, and Robert DeLine. 2019. Managing Messes in Computational Notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300500
[20]
Andrew Head, Caitlin Sadowski, Emerson Murphy-Hill, and Andrea Knight. 2018. When not to comment: questions and tradeoffs with api documentation for c++ projects. In Proceedings of the 40th International Conference on Software Engineering. 643–653.
[21]
Austin Henley, Julian Ball, Benjamin Klein, Aiden Rutter, and Dylan Lee. 2021. An inquisitive code editor for addressing novice programmers’ misconceptions of program behavior. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). IEEE, 165–170.
[22]
Amber Horvath, Michael Xieyang Liu, River Hendriksen, Connor Shannon, Emma Paterson, Kazi Jawad, Andrew Macvean, and Brad A. Myers. 2022. Understanding How Programmers Can Use Annotations on Documentation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3491102.3502095
[23]
Hypothes.is. 2012. Hypothes.is: Annotate the web, with anyone, anywhere.Hypothes.is. Retrieved March 30, 2022 from https://web.hypothes.is/
[24]
Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1265–1276. https://doi.org/10.1145/3025453.3025626
[25]
Mary Beth Kery, Bonnie E. John, Patrick O’Flaherty, Amber Horvath, and Brad A. Myers. 2019. Towards Effective Foraging by Data Scientists to Find Past Analysis Choices. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300322
[26]
David Kirsh. 2010. Thinking with external representations. AI & society 25, 4 (2010), 441–454.
[27]
Aniket Kittur, Andrew M. Peters, Abdigani Diriye, Trupti Telang, and Michael R. Bove. 2013. Costs and benefits of structured information foraging. In CHI 2013. ACM, New York, NY, USA, 2989–2998.
[28]
Amy J. Ko, Htet Aung, and Brad A. Myers. 2005. Eliciting Design Requirements for Maintenance-Oriented IDEs: A Detailed Study of Corrective and Perfective Maintenance Tasks. In Proceedings of the 27th International Conference on Software Engineering (St. Louis, MO, USA) (ICSE ’05). Association for Computing Machinery, New York, NY, USA, 126–135. https://doi.org/10.1145/1062455.1062492
[29]
Amy J Ko, Thomas D LaToza, and Margaret M Burnett. 2015. A practical guide to controlled experiments of software engineering tools with human participants. Empirical Software Engineering 20, 1 (2015), 110–141.
[30]
Amy J. Ko, Brad A. Myers, Michael J. Coblenz, and Htet Htet Aung. 2006. An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks. IEEE Transactions on Software Engineering 32, 12 (2006), 971–987. https://doi.org/10.1109/TSE.2006.116
[31]
Amy J. Ko and Bob Uttl. 2003. Individual differences in program comprehension strategies in unfamiliar programming systems. In 11th Annual Workshop on Program Comprehension. IEEE, New York, NY, USA, 175–184. https://doi.org/10.1109/WPC.2003.1199201
[32]
Thomas D. LaToza, David Garlan, James D. Herbsleb, and Brad A. Myers. 2007. Program comprehension as fact finding. In ESEC-FSE 2007. ACM, New York, NY, USA, 361–270.
[33]
Joseph Lawrance, Christopher Bogart, Margaret Burnett, Rachel Bellamy, Kyle Rector, and Scott D. Fleming. 2013. How Programmers Debug, Revisited: An Information Foraging Theory Perspective. IEEE Transactions on Software Engineering 39, 2 (2013), 197–215. https://doi.org/10.1109/TSE.2010.111
[34]
Michael Xieyang Liu, Jane Hsieh, Nathan Hahn, Angelina Zhou, Emily Deng, Shaun Burley, Cynthia Taylor, Aniket Kittur, and Brad A. Myers. 2019. Unakite: Scaffolding Developers’ Decision-Making Using the Web. In UIST 2019. ACM, New York, NY, USA, 67–80.
[35]
Michael Xieyang Liu, Aniket Kittur, and Brad A. Myers. 2022. Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3491102.3501968
[36]
Walid Maalej and Hans-Jorg Happel. 2009. From work to word: How do software developers describe their work?. In 2009 6th IEEE International Working Conference on Mining Software Repositories. 121–130. https://doi.org/10.1109/MSR.2009.5069490
[37]
Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke. 2014. On the Comprehension of Program Comprehension. Transactions on Software Engineering 23 (2014), 1–37. Issue 4. https://doi.org/10.1145/2622669
[38]
Microsoft. 2022. Visual Studio Code. Microsoft. Retrieved March 28, 2022 from https://code.visualstudio.com/
[39]
Gail C Murphy, Mik Kersten, Martin P Robillard, and Davor Čubranić. 2005. The emergent structure of development tasks. In European Conference on Object-Oriented Programming. Springer, 33–48.
[40]
Brad A. Myers and Jeffrey Stylos. 2016. Improving API Usability. Commun. ACM 59, 6 (2016), 62–69. https://doi.org/10.1145/2896587
[41]
Chris Parnin and Robert DeLine. 2010. Evaluating Cues for Resuming Interrupted Programming Tasks. Association for Computing Machinery, New York, NY, USA, 93–102. https://doi.org/10.1145/1753326.1753342
[42]
C. Parnin and C. Gorg. 2006. Building Usage Contexts During Program Comprehension. In 14th IEEE International Conference on Program Comprehension (ICPC’06). 13–22. https://doi.org/10.1109/ICPC.2006.14
[43]
Chris Parnin and Spencer Rugaber. 2011. Resumption strategies for interrupted programming tasks. Software Quality Journal 19, 1 (2011), 5–34.
[44]
Aniket Potdar and Emad Shihab. 2014. An Exploratory Study on Self-Admitted Technical Debt. In 2014 IEEE International Conference on Software Maintenance and Evolution. 91–100. https://doi.org/10.1109/ICSME.2014.31
[45]
Pooja Rani, Mathias Birrer, Sebastiano Panichella, Mohammad Ghafari, and Oscar Nierstrasz. 2021. What do developers discuss about code comments?. In 2021 IEEE 21st International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 153–164.
[46]
Steven P. Reiss. 2008. Tracking Source Locations. In Proceedings of the 30th International Conference on Software Engineering (Leipzig, Germany) (ICSE ’08). Association for Computing Machinery, New York, NY, USA, 11–20. https://doi.org/10.1145/1368088.1368091
[47]
Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How do professional developers comprehend software?. In ICSE 2012. ACM, New York, NY, USA, 632–542. https://doi.org/10.1109/ICSE.2012.6227188
[48]
Yusuke Shinyama, Yoshitaka Arahori, and Katsuhiko Gondow. 2018. Analyzing Code Comments to Boost Program Comprehension. In 2018 25th Asia-Pacific Software Engineering Conference (APSEC). 325–334. https://doi.org/10.1109/APSEC.2018.00047
[49]
Nischal Shrestha, Titus Barik, and Chris Parnin. 2021. Unravel: A Fluent Code Explorer for Data Wrangling. In The 34th Annual ACM Symposium on User Interface Software and Technology. 198–207.
[50]
Jonathan Sillito, Gail C. Murphy, and Kris De Volder. 2008. Asking and Answering Questions during a Programming Change Task. IEEE Transactions on Software Engineering 34, 4 (2008), 434–451. https://doi.org/10.1109/TSE.2008.26
[51]
Janice Singer, Timothy Lethbridge, Norman Vinson, and Nicolas Anquetil. 2010. An examination of software engineering work practices. In CASCON First Decade High Impact Papers. 174–188.
[52]
Igor Steinmacher, Marco Aurelio Graciotto Silva, Marco Aurelio Gerosa, and David F Redmiles. 2015. A systematic literature review on the barriers faced by newcomers to open source software projects. Information and Software Technology 59 (2015), 67–85.
[53]
Margaret-Anne Storey, Jody Ryall, R. Ian Bull, Del Myers, and Janice Singer. 2008. TODO or to bug. In 2008 ACM/IEEE 30th International Conference on Software Engineering. 251–260. https://doi.org/10.1145/1368088.1368123
[54]
Margaret-Anne Storey, Jody Ryall, Janice Singer, Del Myers, Li-Te Cheng, and Michael Muller. 2009. How Software Developers Use Tagging to Support Reminding and Refinding. IEEE Transactions on Software Engineering 35, 4 (2009), 470–483. https://doi.org/10.1109/TSE.2009.15
[55]
Leigh Ann Sudol-DeLyser, Mark Stehlik, and Sharon Carver. 2012. Code Comprehension Problems as Learning Events. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (Haifa, Israel) (ITiCSE ’12). Association for Computing Machinery, New York, NY, USA, 81–86. https://doi.org/10.1145/2325296.2325319
[56]
Lin Tan, Ding Yuan, Gopal Krishna, and Yuanyuan Zhou. 2007. /* iComment: Bugs or bad comments?*. In Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles. 145–158.
[57]
Shin Hwei Tan, Darko Marinov, Lin Tan, and Gary T Leavens. 2012. @ tcomment: Testing javadoc comments to detect comment-code inconsistencies. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. IEEE, 260–269.
[58]
Christoph Treude and Margaret-Anne Storey. 2012. Work Item Tagging: Communicating Concerns in Collaborative Software Development. IEEE Transactions on Software Engineering 38, 1 (2012), 19–34. https://doi.org/10.1109/TSE.2010.91
[59]
Laton Vermette, Shruti Dembla, April Y. Wang, Joanna McGrenere, and Parmit K. Chilana. 2017. Social CheatSheet: An Interactive Community-Curated Information Overlay for Web Applications. Proc. ACM Hum.-Comput. Interact. 1, CSCW, Article 102 (Dec. 2017), 19 pages. https://doi.org/10.1145/3134737
[60]
Anneliese von Mayrhauser and A Marie Vans. 1997. Hypothesis-driven understanding processes during corrective maintenance of large scale software. In 1997 Proceedings International Conference on Software Maintenance. IEEE, 12–20.
[61]
Fengcai Wen, Csaba Nagy, Gabriele Bavota, and Michele Lanza. 2019. A large-scale empirical study on code-comment inconsistencies. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 53–64.
[62]
Young Seok Yoon and Brad A. Myers. 2014. A longitudinal study of programmers’ backtracking. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 101–108. https://doi.org/10.1109/VLHCC.2014.6883030
[63]
Iyad Zayour and Timothy C Lethbridge. 2000. A cognitive and user centric based approach for reverse engineering tool design. In Proceedings of the 2000 conference of the Centre for Advanced Studies on Collaborative research. 16.
[64]
Sacha Zyto, David Karger, Mark Ackerman, and Sanjoy Mahajan. 2012. Successful classroom deployment of a social document annotation system. In CHI 2012. ACM, New York, NY, USA, 1883–1892. https://doi.org/10.1145/2207676.2208326

Cited By

View all
  • (2024)Magic Markup: Maintaining Document-External Markup with an LLMCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660836(22-31)Online publication date: 11-Mar-2024
  • (2024)Patterns of Hypertext-Augmented SensemakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676338(1-17)Online publication date: 13-Oct-2024
  • (2024)Understanding and Supporting Code PerformancesProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3664623(21-24)Online publication date: 23-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
October 2022
1363 pages
ISBN:9781450393201
DOI:10.1145/3526113
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2022

Check for updates

Author Tags

  1. Annotations
  2. code comprehension
  3. notetaking
  4. sensemaking
  5. software engineering

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

UIST '22

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)456
  • Downloads (Last 6 weeks)52
Reflects downloads up to 15 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Magic Markup: Maintaining Document-External Markup with an LLMCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660836(22-31)Online publication date: 11-Mar-2024
  • (2024)Patterns of Hypertext-Augmented SensemakingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676338(1-17)Online publication date: 13-Oct-2024
  • (2024)Understanding and Supporting Code PerformancesProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3664623(21-24)Online publication date: 23-Jun-2024
  • (2024)To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in ProgrammingExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650867(1-8)Online publication date: 11-May-2024
  • (2024)Meta-Manager: A Tool for Collecting and Exploring Meta Information about CodeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642676(1-17)Online publication date: 11-May-2024
  • (2024)Ivie: Lightweight Anchored Explanations of Just-Generated CodeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642239(1-15)Online publication date: 11-May-2024
  • (2024)Daniel Manesh: Supporting Code Performances2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00060(389-391)Online publication date: 2-Sep-2024
  • (2024)P-Inti: Interactive Visual Representation of Programming Concepts for Learning and Instruction2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00030(199-210)Online publication date: 2-Sep-2024
  • (2023)Synergi: A Mixed-Initiative System for Scholarly Synthesis and SensemakingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606759(1-19)Online publication date: 29-Oct-2023
  • (2023)Forking a Sketch: How the OpenProcessing Community Uses Remixing to Collect, Annotate, Tune, and Extend Creative CodeProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595969(326-342)Online publication date: 10-Jul-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media