Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1109/AST.2017.5acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

DATm: diderot's automated testing model

Published: 20 May 2017 Publication History

Abstract

Diderot is a parallel domain-specific language for the analysis and visualization of multidimensional scientific images, such as those produced by CT and MRI scanners [6], [14], [5]. Diderot is designed to support algorithms that are based on differential tensor calculus and produces a higher-order mathematical model which allows direct manipulation of tensor fields. One of the main challenges of the Diderot implementation is bridging this semantic gap by effectively translating high-level mathematical notation of tensor calculus into efficient low-level code in the target language.
A key question for a high-level language, such as Diderot, is how do we know that the implementation is correct. We have previously presented and defended a core set of rewriting rules, but the full translation from source to executable requires much more work. In this paper, we present DATm, Diderot's automated testing model to check the correctness of the core operations in the programming language. DATm can automatically create test programs, and predict what the outcome should be. We measure the accuracy of the computations written in the Diderot language, based on how accurately the output of the program represents the mathematical equivalent of the computations.
This paper describes a model for testing a high-level language based on correctness. It introduces the pipeline for DATm, a tool that can automatically create and test tens of thousands of Diderot test programs and that has found numerous bugs. We make a case for the necessity of extensive testing by describing bugs that are deep in the compiler, and only could be found with a unique application of operations. Lastly, we demonstrate that the model can be used to create other types of tests by visual verification.

References

[1]
Ivo Babuska and J. Tinsley Oden. Verification and validation in computational engineering and science: Basic concepts. Computer Methods in Applied Mechanics and Engineering, 193(36--38):4057--4066, 9 2004.
[2]
Jean-Philippe Bernardy, Patrik Jansson, and Koen Claessen. Testing polymorphic properties. In Proceedings of the 19th European Conference on Programming Languages and Systems, ESOP'10, pages 125--144, Berlin, Heidelberg, 2010. Springer-Verlag.
[3]
Junjie Chen, Wenxiang Hu, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, and Bing Xie. An empirical comparison of compiler testing techniques. In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14--22, 2016, pages 180--190, 2016.
[4]
Charisee Chiw. Ein notation in the diderot compiler. http://diderot-language.cs.uchicago.edu/papers/chiw-masters.pdf.
[5]
Charisee Chiw, Gordon L Kindlman, and John Reppy. EIN: An intermediate representation for compiling tensor calculus. In Proceedings of the 19th Workshop on Compilers for Parallel Computing (CPC 2019), July 2016.
[6]
Charisee Chiw, Gordon Kindlmann, John Reppy, Lamont Samuels, and Nick Seltzer. Diderot: A parallel DSL for image analysis and visualization. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 111--120, June 2012. (alphabetical author order).
[7]
Jan Christiansen and Sebastian Fischer. Easycheck: Test data for free. In Proceedings of the 9th International Conference on Functional and Logic Programming, FLOPS'08, pages 322--336, Berlin, Heidelberg, 2008. Springer-Verlag.
[8]
Koen Claessen and John Hughes. Quickcheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, ICFP '00, pages 268--279, New York, NY, USA, 2000. ACM.
[9]
Diderot. website at https://github.com/Diderot-Language/examples.
[10]
Alastair F. Donaldson and Andrei Lascu. Metamorphic testing for (graphics) compilers. In Proceedings of the 1st International Workshop on Metamorphic Testing, MET@ICSE 2016, Austin, Texas, USA, May 16, 2016, pages 44--47, 2016.
[11]
Jonas Duregård, Patrik Jansson, and Meng Wang. Feat: Functional enumeration of algebraic types. In Proceedings of the 2012 Haskell Symposium, Haskell '12, pages 61--72, New York, NY, USA, 2012. ACM.
[12]
Tiago Etiene, Carlos Scheidegger, L Gustavo Nonato, Robert M Kirby, and Cláudio T Silva. Verifiable visualization for isosurface extraction. Visualization and Computer Graphics, IEEE Transactions on, 15(6):1227--1234, 2009.
[13]
R. Feldt, S. Poulding, D. Clark, and S. Yoo. Test Set Diameter: Quantifying the Diversity of Sets of Test Cases. ArXiv e-prints, June 2015.
[14]
Gordon Kindlmann, Charisee Chiw, Nicholas Seltzer, Lamont Samuels, and John Reppy. Diderot: a domain-specific language for portable parallel scientific visualization and image analysis. IEEE Transactions on Visualization and Computer Graphics (Proceedings VIS 2015), 22(1):867--876, January 2016.
[15]
Vu Le, Mehrdad Afshari, and Zhendong Su. Compiler validation via equivalence modulo inputs. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '14, Edinburgh, United Kingdom - June 09 -- 11, 2014, pages 216--226, 2014.
[16]
Christian Lindig. Random testing of C calling conventions. In Proceedings of the Sixth International Symposium on Automated Analysis-driven Debugging, AADEBUG'05, pages 3--12, New York, NY, USA, 2005. ACM.
[17]
William M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100--107, 1998.
[18]
Michal H. Palka, Koen Claessen, Alejandro Russo, and John Hughes. Testing an optimising compiler by generating random lambda terms. In Proceedings of the 6th International Workshop on Automation of Software Test, AST '11, pages 91--97, New York, NY, USA, 2011. ACM.
[19]
Daniel Ratiu and Markus Voelter. Automated testing of DSL implementations: Experiences from building mbeddr. In Proceedings of the 11th International Workshop on Automation of Software Test, AST '16, pages 15--21, New York, NY, USA, 2016. ACM.
[20]
Sympy is a python library. http://www.sympy.org/en/index.html.
[21]
Qiuming Tao, Wei Wu, Chen Zhao, and Wuwei Shen. An automatic testing approach for compiler based on metamorphic testing technique. In 17th Asia Pacific Software Engineering Conference, APSEC 2010, Sydney, Australia, November 30 -- December 3, 2010, pages 270--279, 2010.
[22]
Teem Library. Teem website at http://teem.sf.net.
[23]
Cludio T Silva Tiago Etiene, Robert M Kirby. An Introduction to Verification of Visualization Techniques, volume 7. Morgan and Claypool Publishers, Clifton Park, New York, 3rd edition, December 2015.
[24]
Hui Wu, Jeff Gray, and Marjan Mernik. Unit Testing for Domain- Specific Languages, pages 125--147. Springer Berlin Heidelberg, Berlin, Heidelberg, 2009.
[25]
Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. Finding and understanding bugs in C compilers. SIGPLAN Not., 46(6):283--294, June 2011.

Cited By

View all
  • (2020)Surfacing Visualization MiragesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376420(1-16)Online publication date: 21-Apr-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AST '17: Proceedings of the 12th International Workshop on Automation of Software Testing
May 2017
80 pages
ISBN:9781538615485

Sponsors

Publisher

IEEE Press

Publication History

Published: 20 May 2017

Check for updates

Qualifiers

  • Research-article

Conference

ICSE '17
Sponsor:

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Surfacing Visualization MiragesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376420(1-16)Online publication date: 21-Apr-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media