Odyssey: An Interactive Workbench for Expert-Driven Floating-Point Expression Rewriting
Authors: 
Edward Misback, Caleb C. Chan, Brett Saiki, Eunice Jun, 
Zachary Tatlock, 
Pavel PanchekhaAuthors Info & Claims
Edward Misback, Caleb C. Chan, Brett Saiki, Eunice Jun, Zachary Tatlock, Pavel PanchekhaAuthors Info & ClaimsArticle No.: 77, Pages 1 - 15
Abstract
In recent years, researchers have proposed a number of automated tools to identify and improve floating-point rounding error in mathematical expressions. However, users struggle to effectively apply these tools. In this paper, we work with novices, experts, and tool developers to investigate user needs during the expression rewriting process. We find that users follow an iterative design process. They want to compare expressions on multiple input ranges, integrate and guide various rewriting tools, and understand where errors come from. We organize this investigation’s results into a three-stage workflow and implement that workflow in a new, extensible workbench dubbed Odyssey. Odyssey enables users to: (1) diagnose problems in an expression, (2) generate solutions automatically or by hand, and (3) tune their results. Odyssey tracks a working set of expressions and turns a state-of-the-art automated tool “inside out,” giving the user access to internal heuristics, algorithms, and functionality. In a user study, Odyssey enabled five expert numerical analysts to solve challenging rewriting problems where state-of-the-art automated tools fail. In particular, the experts unanimously praised Odyssey’s novel support for interactive range modification and local error visualization.
Supplemental Material
PDF File
A description of the expert study.
- Download
- 231.49 KB
ZIP File
Supplemental File
- Download
- 24.45 MB
References
[1]
2022. Improving Rust with Herbie. https://pavpanchekha.com/blog/herbie-rust.html.
[2]
2023. VSCode-PRECiSA. https://github.com/nasa/PRECiSA/tree/master/vscode-precisa
[3]
Micah Altman, Jeff Gill, and Michael P. McDonald. 2003. Numerical Issues in Statistical Computing for the Social Scientist. Springer-Verlag. 1–11 pages.
[4]
Micah Altman and Michael P. McDonald. 2003. Replication with attention to numerical accuracy. Political Analysis 11, 3 (2003), 302–307. http://pan.oxfordjournals.org/content/11/3/302.abstract
[5]
Earl T. Barr, Thanh Vo, Vu Le, and Zhendong Su. 2013. Automatic detection of floating-point exceptions(POPL ’13).
[6]
Fabian Beck, Oliver Moseler, Stephan Diehl, and Günter Daniel Rey. 2013. In situ understanding of performance bottlenecks through visually augmented code. In 2013 21st International Conference on Program Comprehension (ICPC). IEEE, 63–72.
[7]
Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems(PLDI ’12). ACM, New York, NY, USA, 453–462. http://doi.acm.org/10.1145/2254064.2254118
[8]
Sylvie Boldo, Jean-Christophe Filliâtre, and Guillaume Melquiond. 2009. Combining Coq and Gappa for certifying floating-point programs. In International Conference on Intelligent Computer Mathematics. Springer, 59–74.
[9]
Brian Burg, Richard Bailey, Amy J Ko, and Michael D Ernst. 2013. Interactive record/replay for web application debugging. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 473–484.
[10]
Wei-Fan Chiang, Ganesh Gopalakrishnan, Zvonimir Rakamaric, and Alexey Solovyev. 2014. Efficient Search for Inputs Causing High Floating-Point Errors. In Proceedings of the 19th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming(PPoPP ’14). Association for Computing Machinery, New York, NY, USA, 43–52. https://doi.org/10.1145/2555243.2555265
[11]
Wei-Fan Chiang, Ganesh Gopalakrishnan, Zvonimir Rakamarić, and Alexey Solovyev. 2014. Efficient Search for Inputs Causing High Floating-point Errors. ACM, 43–52.
[12]
Jürgen Cito, Philipp Leitner, Christian Bosshard, Markus Knecht, Genc Mazlami, and Harald C Gall. 2018. PerformanceHat: augmenting source code with runtime performance traces in the IDE. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings. 41–44.
[13]
Mihai Codoban, Sruti Srinivasa Ragavan, Danny Dig, and Brian Bailey. 2015. Software history under the lens: A study on why and how developers examine it. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 1–10.
[14]
Nasrine Damouche and Matthieu Martel. 2017. Salsa: An automatic tool to improve the numerical accuracy of programs(AFM).
[15]
Eva Darulova and Viktor Kuncak. 2014. Sound Compilation of Reals(POPL). 14 pages. http://doi.acm.org/10.1145/2535838.2535874
[16]
Arnab Das, Ian Briggs, Ganesh Gopalakrishnan, Sriram Krishnamoorthy, and Pavel Panchekha. 2020. Scalable yet Rigorous Floating-Point Error Analysis. In 2020 SC20: International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE Computer Society, Los Alamitos, CA, USA, 1–14. https://doi.org/10.1109/SC41405.2020.00055
[17]
Jos de Jong. 2013. math.js: An extensive math library for JavaScript and Node.js. http://mathjs.org/
[18]
European Commission. 1998. The introduction of the euro and the rounding of currency amounts. European Commission, Directorate General II Economic and Financial Affairs.
[19]
David Goldberg. 1991. What Every Computer Scientist Should Know About Floating-point Arithmetic. Comput. Surveys 23, 1 (March 1991), 5–48. http://doi.acm.org/10.1145/103162.103163
[20]
Eric Goubault and Sylvie Putot. 2011. Static Analysis of Finite Precision Computations(VMCAI’11). 232–247. http://dl.acm.org/citation.cfm?id=1946284.1946301
[21]
Ken Gu, Eunice Jun, and Tim Althoff. 2023. Understanding and Supporting Debugging Workflows in Multiverse Analysis. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19.
[22]
Richard Hamming. 1987. Numerical Methods for Scientists and Engineers (2nd ed.). Dover Publications.
[23]
Andrew Head, Fred Hohman, Titus Barik, Steven M Drucker, and Robert DeLine. 2019. Managing messes in computational notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[24]
Jane Hoffswell, Arvind Satyanarayan, and Jeffrey Heer. 2018. Augmenting code with in situ visualizations to aid program understanding. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[25]
IEEE. 2008. IEEE Standard for Binary Floating-Point Arithmetic. IEEE Std. 754-2008 (2008).
[26]
Yuka Ikarashi, Jonathan Ragan-Kelley, Tsukasa Fukusato, Jun Kato, and Takeo Igarashi. 2021. Guided Optimization for Image Processing Pipelines. In 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 1–5.
[27]
Anastasiia Izycheva and Eva Darulova. 2017. On sound relative error bounds for floating-point arithmetic(FMCAD). 15–22. https://doi.org/10.23919/FMCAD.2017.8102236
[28]
Mary Beth Kery, Amber Horvath, and Brad A Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In CHI, Vol. 10. 3–025.
[29]
Mary Beth Kery and Brad A Myers. 2018. Interactions for untangling messy history in a computational notebook. In 2018 IEEE symposium on visual languages and human-centric computing (VL/HCC). IEEE, 147–155.
[30]
Ronald T. Kneusel. 2017. Numbers and Computers. Springer Cham. https://doi.org/10.1007/978-3-319-50508-4
[31]
Amy J Ko and Brad A Myers. 2004. Designing the whyline: a debugging interface for asking questions about program behavior. In Proceedings of the SIGCHI conference on Human factors in computing systems. 151–158.
[32]
Maria Larsson, Hironori Yoshida, Nobuyuki Umetani, and Takeo Igarashi. 2020. Tsugite: Interactive Design and Fabrication of Wood Joints. In UIST. 317–327.
[33]
Sorin Lerner. 2020. Projection boxes: On-the-fly reconfigurable visualization for live programming. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–7.
[34]
Tom Lieber, Joel R Brandt, and Rob C Miller. 2014. Addressing misconceptions about code with always-on programming visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2481–2490.
[35]
Yang Liu, Alex Kale, Tim Althoff, and Jeffrey Heer. 2020. Boba: Authoring and visualizing multiverse analyses. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1753–1763.
[36]
B. D. McCullough and H. D. Vinod. 1999. The Numerical Reliability of Econometric Software. Journal of Economic Literature 37, 2 (1999), 633–665.
[37]
David Monniaux. 2008. The Pitfalls of Verifying Floating-point Computations. ACM Trans. Program. Lang. Syst. 30, 3, Article 12 (May 2008), 41 pages. http://doi.acm.org/10.1145/1353445.1353446
[38]
Stephen Oney and Brad Myers. 2009. FireCrystal: Understanding interactive behaviors in dynamic web pages. In 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 105–108.
[39]
Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions(PLDI).
[40]
Kevin Quinn. 1983. Ever Had Problems Rounding Off Figures? This Stock Exchange Has. The Wall Street Journal (November 8, 1983), 37.
[41]
Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. Acm Sigplan Notices 48, 6 (2013), 519–530.
[42]
Brett Saiki, Oliver Flatt, Chandrakana Nandi, Pavel Panchekha, and Zachary Tatlock. 2021. Combining Precision Tuning and Rewriting. In 2021 IEEE 28th Symposium on Computer Arithmetic (ARITH).
[43]
Alexey Solovyev, Charlie Jacobsen, Zvonimir Rakamaric, and Ganesh Gopalakrishnan. 2015. Rigorous Estimation of Floating-Point Round-off Errors with Symbolic Taylor Expansions(FM).
[44]
The Herbie Development Team. 2013–. Herbie: Optimize floating-point expressions for accuracy. https://github.com/herbie-fp/herbie
[45]
Laura Titolo, Marco A Feliú, Mariano Moscato, and César A Munoz. 2018. An Abstract Interpretation Framework for the Round-Off Error Analysis of Floating-Point Programs(VMCAI). 516–537.
[46]
U.S. General Accounting Office. 1992. Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia. http://www.gao.gov/products/IMTEC-92-26
[47]
Debora Weber-Wulff. 1992. Rounding error changes Parliament makeup. http://catless.ncl.ac.uk/Risks/13.37.html#subj4
[48]
YoungSeok Yoon and Brad A Myers. 2015. Supporting selective undo in a code editor. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, 223–233.
Index Terms
- Odyssey: An Interactive Workbench for Expert-Driven Floating-Point Expression Rewriting
Recommendations
Refining weakly outermost-needed rewriting and narrowing
PPDP '03: Proceedings of the 5th ACM SIGPLAN international conference on Principles and practice of declaritive programmingOutermost-needed rewriting/narrowing is a sound and complete optimal demand-driven strategy for the class of inductively sequential constructor systems. Its parallel extension (known as weakly) deals with non-inductively sequential constructor systems. ...
Methods for Proving Termination of Rewriting-based Programming Languages by Transformation
Despite the remarkable development of the theory of termination of rewriting, its application to high-level (rewriting-based) programming languages is far from being optimal. This is due to the need for features such as conditional equations and rules, ...
Proving termination of context-sensitive rewriting by transformation
Context-sensitive rewriting (CSR) is a restriction of rewriting that forbids reductions on selected arguments of functions. With CSR, we can achieve a terminating behavior with non-terminating term rewriting systems, by pruning (all) infinite rewrite ...
Comments
Information & Contributors
Information
Published In
October 2023
1825 pages
ISBN:9798400701320
DOI:10.1145/3586183
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
UIST '23: The 36th Annual ACM Symposium on User Interface Software and Technology
October 29 - November 1, 2023
CA, San Francisco, USA
Acceptance Rates
Overall Acceptance Rate 561 of 2,567 submissions, 22%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 372Total Downloads
- Downloads (Last 12 months)252
- Downloads (Last 6 weeks)41
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in