short-paper

Metamorphic testing for adobe analytics data collection javascript library

Authors:

Zhenyu Wang,

Dave Towey,

Zhi Quan Zhou,

Tsong Yueh ChenAuthors Info & Claims

MET '18: Proceedings of the 3rd International Workshop on Metamorphic Testing

Pages 34 - 37

https://doi.org/10.1145/3193977.3193979

Published: 27 May 2018 Publication History

Get Access

Abstract

In recent years, metamorphic testing has been successfully and systematically adopted within Adobe Systems to improve the cost effectiveness of its software process. In this industry experience report, we present a case where metamorphic testing has been applied to the Data Collection JavaScript Library of Adobe Analytics. This type of software is difficult to test using traditional approaches. The application of metamorphic testing alleviated the oracle problem, and detected real-life bugs in the system under test as well as compatibility problems between the system and its environment, namely, the Internet Explorer browser. Our results further justify the adoption of metamorphic testing as a simple yet effective approach in industrial settings.

References

[1]

Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2015. The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering 41, 5 (2015), 507--525.

Digital Library

Google Scholar

[2]

T. Y. Chen, S. C. Cheung, and S. M. Yiu. 1998. Metamorphic testing: A new approach for generating next test cases. Technical Report HKUST-CS98-01. Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong.

Google Scholar

[3]

Tsong Yueh Chen, Fei-Ching Kuo, Huai Liu, Pak-Lok Poon, Dave Towey, T. H. Tse, and Zhi Quan Zhou. 2018. Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys 51, 1 (2018), 4:1--4:27.

Digital Library

Google Scholar

[4]

T. Y. Chen, T. H. Tse, and Z. Q. Zhou. 2003. Fault-based testing without the need of oracles. Information and Software Technology 45, 1 (2003), 1--9.

Digital Library

Google Scholar

[5]

Darryl C. Jarman, Zhi Quan Zhou, and Tsong Yueh Chen. 2017. Metamorphic Testing for Adobe Data Analytics Software. In Proceedings of the IEEE/ACM 2nd International Workshop on Metamorphic Testing (ICSE MET '17), in conjunction with the 39th International Conference on Software Engineering (ICSE '17). 21--27.

Digital Library

Google Scholar

[6]

William M. McKeeman. 1998. Differential testing for software. Digital Technical Journal 10, 1 (1998), 100--107. http://www.hpl.hp.com/hpjournal/dtj/vol10num1/toc.htm

Google Scholar

[7]

Sergio Segura, Gordon Fraser, Ana B. Sanchez, and Antonio Ruiz-Cortés. 2016. A survey on metamorphic testing. IEEE Transactions on Software Engineering 42, 9 (2016), 805--824.

Crossref

Google Scholar

Cited By

View all

Ayerdi JTerragni VArrieta ATonella PSagardui GArratibel MSpinellis DGousios GChechik MDi Penta M(2021)Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case studyProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3473920(1264-1274)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3473920
Zhou ZTse TWitheridge M(2021)Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact FactorsIEEE Transactions on Software Engineering10.1109/TSE.2019.291506547:6(1164-1183)Online publication date: 1-Jun-2021
https://doi.org/10.1109/TSE.2019.2915065
Ayerdi JSegura SArrieta AArratibel GArratibel M(2020)QoS-aware Metamorphic Testing: An Elevation Case Study2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE5003.2020.00019(104-114)Online publication date: Oct-2020
https://doi.org/10.1109/ISSRE5003.2020.00019
Show More Cited By

Index Terms

Metamorphic testing for adobe analytics data collection javascript library
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation

Recommendations

Metamorphic Testing: A Review of Challenges and Opportunities

Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and ...
Metamorphic testing for Adobe data analytics software
MET '17: Proceedings of the 2nd International Workshop on Metamorphic Testing

It is challenging to test data analytics software because a test oracle might not be available. This study reports our experience of applying metamorphic testing to Adobe's data analytics software that is used for anomaly detection in a set of time ...
Testing web enabled simulation at scale using metamorphic testing
ICSE-SEIP '21: Proceedings of the 43rd International Conference on Software Engineering: Software Engineering in Practice

We report on Facebook's deployment of MIA (Metamorphic Interaction Automaton). MIA is used to test Facebook's Web Enabled Simulation, built on a web infrastructure of hundreds of millions of lines of code. MIA tackles the twin problems of test flakiness ...

Comments

Information & Contributors

Information

Published In

MET '18: Proceedings of the 3rd International Workshop on Metamorphic Testing

May 2018

66 pages

ISBN:9781450357296

DOI:10.1145/3193977

Conference Chairs:
Xiaoyuan Xie
Wuhan University, China
,
Laura L. Pullum
Oak Ridge National Laboratory
,
Pak Lok Poon
RMIT University, Australia

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Australian Research Council

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27, 2018

Gothenburg, Sweden

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
112
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ayerdi JTerragni VArrieta ATonella PSagardui GArratibel MSpinellis DGousios GChechik MDi Penta M(2021)Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case studyProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3473920(1264-1274)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3473920
Zhou ZTse TWitheridge M(2021)Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact FactorsIEEE Transactions on Software Engineering10.1109/TSE.2019.291506547:6(1164-1183)Online publication date: 1-Jun-2021
https://doi.org/10.1109/TSE.2019.2915065
Ayerdi JSegura SArrieta AArratibel GArratibel M(2020)QoS-aware Metamorphic Testing: An Elevation Case Study2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE5003.2020.00019(104-114)Online publication date: Oct-2020
https://doi.org/10.1109/ISSRE5003.2020.00019
Wu CSun LZhou ZXie XPoon PPullum L(2019)The impact of a dotProceedings of the 4th International Workshop on Metamorphic Testing10.1109/MET.2019.00011(17-23)Online publication date: 26-May-2019
https://dl.acm.org/doi/10.1109/MET.2019.00011
Sun LZhou Z(2018)Metamorphic Testing for Machine Translations: MT4MT2018 25th Australasian Software Engineering Conference (ASWEC)10.1109/ASWEC.2018.00021(96-100)Online publication date: Nov-2018
https://doi.org/10.1109/ASWEC.2018.00021

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Metamorphic Testing: A Review of Challenges and Opportunities

Metamorphic testing for Adobe data analytics software

Testing web enabled simulation at scale using metamorphic testing

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Metamorphic Testing: A Review of Challenges and Opportunities

Metamorphic testing for Adobe data analytics software

Testing web enabled simulation at scale using metamorphic testing

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations