Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3487552.3487827acmconferencesArticle/Chapter ViewAbstractPublication PagesimcConference Proceedingsconference-collections
research-article

Understanding the performance of webassembly applications

Published: 02 November 2021 Publication History

Abstract

WebAssembly is the newest language to arrive on the web. It features a compact binary format, making it fast to be loaded and decoded. While WebAssembly is generally expected to be faster than JavaScript, there have been mixed results in proving which code is faster. Little research has been done to comprehend WebAssembly's performance benefit. In this paper, we conduct a systematic study to understand the performance of WebAssembly applications and compare it with JavaScript. Our measurements were performed on three sets of subject programs with diverse settings. Among others, our findings include: (1) WebAssembly compilers are commonly built atop LLVM, where their optimizations are not tailored for WebAssembly. We show that these optimizations often become ineffective for WebAssembly, leading to counter-intuitive results. (2) JIT optimization has a significant impact on JavaScript performance. However, no substantial performance increase was observed for WebAssembly with JIT. (3) The performance of WebAssembly and JavaScript varies substantially depending on the execution environment. (4) WebAssembly uses significantly more memory than its JavaScript counterparts. We hope that our findings can help WebAssembly tooling developers identify optimization opportunities. We also report the challenges encountered when compiling C benchmarks to WebAssembly and discuss our solutions.

References

[1]
2019. IEEE Standard for Floating-Point Arithmetic.
[2]
2020. Project Website. https://benchmarkingwasm.github.io/BenchmarkingWebAssembly/
[3]
Aldeida Aleti, Catia Trubiani, André van Hoorn, and Pooyan Jamshidi. 2018. An efficient method for uncertainty propagation in robust software performance estimation. Journal of Systems and Software 138 (2018), 222--235.
[4]
Android. 2020. Android Debug Bridge (adb). https://developer.android.com/studio/command-line/adb
[5]
asm.js. 2020. asm.js - an extraordinarily optimizable, low-level subset of JavaScript. http://asmjs.org/
[6]
Aart JC Bik, David L Kreitzer, and Xinmin Tian. 2008. A case study on compiler optimizations for the Intel® Core TM 2 Duo Processor. International Journal of Parallel Programming 36, 6 (2008), 571--591.
[7]
Stack Overflow Contributor Blindman67. 2018. Why is webAssembly function almost 300 time slower than same JS function. https://stackoverflow.com/questions/48173979/why-is-webassembly-function-almost-300-time-slower-than-same-js-function
[8]
Caligatio. 2021. Caligatio/jsSHA. https://github.com/Caligatio/jsSHA
[9]
Winston Chen. 2018. Performance Testing Web Assembly vs JavaScript. https://medium.com/samsung-internet-dev/performance-testing-web-assembly-vs-javascript-e07506fd5875
[10]
Clang. 2020. LLVM's Analysis and Transform Passes. https://llvm.org/docs/Passes.html#argpromotion-promote-by-reference-arguments-to-scalars
[11]
Lin Clark. 2018. Calls between JavaScript and WebAssembly are finally fast. https://hacks.mozilla.org/2018/10/calls-between-javascript-and-webassembly-are-finally-fast-%F0%9F%8E%89/
[12]
Stack Overflow Contributor ColinE. 2017. Why is my WebAssembly function slower than the JavaScript equivalent? https://stackoverflow.com/questions/46331830/why-is-my-webassembly-function-slower-than-the-javascript-equivalent/46500236#46500236
[13]
Jason Cong, Bin Liu, Raghu Prabhakar, and Peng Zhang. 2012. A study on the impact of compiler optimizations on high-level synthesis. In International Workshop on Languages and Compilers for Parallel Computing. Springer, 143--157.
[14]
Emscripten Contributors. 2015. File System Overview --- Emscripten 1.39.17 documentation. https://emscripten.org/docs/porting/files/file_systems_overview.html#file-system-overview
[15]
Emscripten Contributors. 2020. Emscripten 1.39.4 documentation. https://emscripten.org/
[16]
WebAssembly Contributors. 2020. Webassembly Use Cases. https://webassembly.org/docs/use-cases/
[17]
Netscape Communications Corporation and Inc. Sun Microsystems. 1995. Netscape and Sun Announce JavaScript, the Open, Cross-Platform Object Scripting Language for Enterprise Networks and the Internet. https://web.archive.org/web/20070916144913/http://wp.netscape.com/newsref/pr/newsrelease67.html
[18]
Luis Cruz and Rui Abreu. 2017. Performance-based guidelines for energy efficient mobile applications. In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft). IEEE, 46--57.
[19]
Mariana Cunha and Nuno Laranjeiro. 2018. Assessing Containerized REST Services Performance in the Presence of Operator Faults. In 2018 14th European Dependable Computing Conference (EDCC). IEEE, 95--100.
[20]
Damianociarla. 2021. Damianociarla/node-ffmpeg. https://github.com/damianociarla/node-ffmpeg
[21]
Damianociarla. 2021. Damianociarla/node-ffmpeg/lib/ffmpeg.js. https://github.com/damianociarla/node-ffmpeg/blob/master/lib/ffmpeg.js
[22]
DcodeIO. 2021. DcodeIO/Long.js. https://github.com/dcodeIO/Long.js/
[23]
DcodeIO. 2021. Long.js Avoiding Overflow. https://github.com/dcodeIO/long.js/blob/master/src/long.js#L56-L59
[24]
DcodeIO. 2021. Long.js JavaScript Source Code. https://github.com/dcodeIO/long.js/blob/master/src/long.js
[25]
DcodeIO. 2021. Long.js WebAssembly Source Code. https://github.com/dcodeIO/long.js/blob/master/src/wasm.wat
[26]
Giovanni Denaro, Andrea Polini, and Wolfgang Emmerich. 2004. Early performance testing of distributed software applications. In Proceedings of the 4th international workshop on Software and performance. 94--103.
[27]
Mozilla developers. 2021. Bugzilla - Bug 37449 - llvm performs less inlining in -O3 than in -O2. https://bugs.llvm.org/show_bug.cgi?id=37449
[28]
Craig Disselkoen, John Renner, Conrad Watt, Tal Garfinkel, Amit Levy, and Deian Stefan. 2019. Position Paper: Progressive Memory Safety for WebAssembly. In Proceedings of the 8th International Workshop on Hardware and Architectural Support for Security and Privacy (Phoenix, AZ, USA) (HASP '19). Association for Computing Machinery, New York, NY, USA, Article 4, 8 pages.
[29]
MDN Web Docs. 2020. Compiling an Existing C Module to WebAssembly. https://developer.mozilla.org/en-US/docs/WebAssembly/existing_C_to_wasm
[30]
Haas et al. 2017. Bringing the web up to speed with WebAssembly. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. 185--200.
[31]
Martín Abadi et al. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/
[32]
FFmpeg. 2021. FFmpeg. https://www.ffmpeg.org/
[33]
ffmpegwasm. 2021. ffmpegwasm/ffmpeg.wasm. https://github.com/ffmpegwasm/ffmpeg.wasm
[34]
ffmpegwasm. 2021. ffmpegwasm/ffmpeg.wasm/dist/ffmpeg.min.js. https://unpkg.com/@ffmpeg/[email protected]/dist/ffmpeg.min.js
[35]
Inc. Figma. 2021. The collaborative interface design tool. https://www.figma.com/
[36]
Free Software Foundation (FSF). 2020. GCC, the GNU Compiler Collection. https://gcc.gnu.org/
[37]
Google. 2020. Google Chrome - Download the Fast, Secure Browser from Google. https://www.google.com/chrome/
[38]
Google. 2020. V8 JavaScript Engine. https://v8.dev/
[39]
WebAssembly Group. 2020. WebAssembly/design. https://github.com/WebAssembly/design/blob/master/FutureFeatures.md
[40]
WebAssembly Community Group. 2020. Use Cases - WebAssembly. https://webassembly.org/docs/use-cases/
[41]
Jakob Gruber. 2021. JIT-less V8. https://v8.dev/blog/jitless
[42]
Antonio Guerriero, Raffaela Mirandola, Roberto Pietrantuono, and Stefano Russo. 2019. A Hybrid Framework for Web Services Reliability and Performance Assessment. In 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 185--192.
[43]
Andreas Haas, Andreas Rossberg, Derek L. Schuff, Ben L. Titzer, Michael Holman, Dan Gohman, Luke Wagner, Alon Zakai, and JF Bastien. 2017. Bringing the Web up to Speed with WebAssembly. SIGPLAN Not. 52, 6 (June 2017), 185--200.
[44]
Yuko Hara, Hiroyuki Tomiyama, Shinya Honda, and Hiroaki Takada. 2009. Proposal and quantitative analysis of the CHStone benchmark program suite for practical C-based high-level synthesis. Journal of Information Processing 17 (2009), 242--254.
[45]
David Herrera, Hangfen Chen, Erick Lavoie, and Laurie Hendren. 2018. WebAssembly and JavaScript Challenge: Numerical program performance using modern browser technologies and devices. University of McGill, Montreal: QC, Technical report SABLE-TR-2018-2 (2018).
[46]
Aaron Hilbig, Daniel Lehmann, and Michael Pradel. 2021. An Empirical Study of Real-World WebAssembly Binaries: Security, Languages, Use Cases. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW '21). Association for Computing Machinery, New York, NY, USA, 2696--2708.
[47]
Raymond Hill. 2019. gorhill/ublock. https://github.com/gorhill/uBlock
[48]
Abhinav Jangda, Bobby Powers, Emery D Berger, and Arjun Guha. 2019. Not so fast: analyzing the performance of webassembly vs. native code. In 2019 {USENIX} Annual Technical Conference ({USENIX}{ATC } 19). 107--120.
[49]
Evan Johnson, David Thien, Yousef Alhessi, Shravan Narayan, Fraser Brown, Sorin Lerner, Tyler McMullen, Stefan Savage, and Deian Stefan. 2021. Trust, but verify: SFI safety for native-compiled Wasm. In NDSS. Internet Society.
[50]
Josdejong. 2021. Josdejong/mathjs. https://github.com/josdejong/mathjs
[51]
Yuriy Kashnikov, Jean Christophe Beyler, and William Jalby. 2012. Compiler optimizations: Machine learning versus o3. In International Workshop on Languages and Compilers for Parallel Computing. Springer, 32--45.
[52]
Heejin Kim, Byoungju Choi, and W Eric Wong. 2009. Performance testing of mobile applications at the unit test level. In 2009 Third IEEE International Conference on Secure Software Integration and Reliability Improvement. IEEE, 171--180.
[53]
Daniel Lehmann, Johannes Kinder, and Michael Pradel. 2020. Everything Old is New Again: Binary Security of WebAssembly. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 217--234. https://www.usenix.org/conference/usenixsecurity20/presentation/lehmann
[54]
Daniel Lehmann and Michael Pradel. 2018. Wasabi: A Framework for Dynamically Analyzing WebAssembly. CoRR abs/1808.10652 (2018). arXiv:1808.10652 http://arxiv.org/abs/1808.10652
[55]
Kai Lei, Yining Ma, and Zhi Tan. 2014. Performance comparison and evaluation of web development technologies in php, python, and node. js. In 2014 IEEE 17th international conference on computational science and engineering. IEEE, 661--668.
[56]
Zhiming Liu, Nafees Qamar, and Jie Qian. 2013. A quantitative analysis of the performance and scalability of de-identification tools for medical data. In International Symposium on Foundations of Health Informatics Engineering and Systems. Springer, 274--289.
[57]
Goran Martinovic, Josip Balen, and Bojan Cukic. 2012. Performance Evaluation of Recent Windows Operating Systems. J. UCS 18, 2 (2012), 218--263.
[58]
MDN. 2020. WebAssembly.Memory(). https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Memory/Memory
[59]
MDN. 2021. Number.MAX-SAFE-INTEGER - JavaScript: MDN. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER
[60]
Tianhui Meng, Katinka Wolter, and Qiushi Wang. 2015. Security and performance tradeoff analysis of mobile offloading systems under timing attacks. In European Workshop on Performance Engineering. Springer, 32--46.
[61]
Microsoft. 2020. Download New Microsoft Edge Browser: Microsoft. https://www.microsoft.com/en-us/edge
[62]
Mnater. 2021. Mnater/Hyphenator. https://github.com/mnater/Hyphenator
[63]
Mnater. 2021. Mnater/Hyphenator/Hyphenopoly-Loader.js. https://github.com/mnater/Hyphenator/blob/master/Hyphenator_Loader.js
[64]
Mnater. 2021. Mnater/Hyphenopoly. https://github.com/mnater/Hyphenopoly
[65]
Mnater. 2021. Mnater/Hyphenopoly/Hyphenopoly-Loader.js. https://github.com/mnater/Hyphenopoly/blob/master/Hyphenopoly_Loader.js
[66]
Mozilla. 2020. Firefox: Internet for people, not profit. https://www.mozilla.org/en-US/
[67]
Mozilla. 2020. WebAssembly Memory. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_objects/WebAssembly/Memory
[68]
Mozilla. 2021. Geckoview. https://mozilla.github.io/geckoview/
[69]
Mozilla. 2021. SpiderMonkey JavaScript/WebAssembly Engine. https://spidermonkey.dev/docs/
[70]
Marius Musch, Christian Wressnegger, Martin Johns, and Konrad Rieck. 2019. New Kid on the Web: A Study on the Prevalence of WebAssembly in the Wild. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 23--42.
[71]
Shravan Narayan, Craig Disselkoen, Daniel Moghimi, Sunjay Cauligi, Evan Johnson, Zhao Gang, Anjo Vahldiek-Oberwagner, Ravi Sahita, Hovav Shacham, Dean Tullsen, and Deian Stefan. 2021. Swivel: Hardening WebAssembly against Spectre. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 1433--1450. https://www.usenix.org/conference/usenixsecurity21/presentation/narayan
[72]
Shravan Narayan, Tal Garfinkel, Sorin Lerner, Hovav Shacham, and Deian Stefan. 2019. Gobi: WebAssembly as a practical path to library sandboxing. arXiv preprint arXiv:1912.02285 (2019).
[73]
Wasm pack contributors. 2019. Wasm Speed Are No Faster Than JS. https://github.com/rustwasm/wasm-pack/issues/558
[74]
Senthil Padmanabhan and Pranav Jha. 2020. WebAssembly at eBay: A Real-World Use Case. https://tech.ebayinc.com/engineering/webassembly-at-ebay-a-real-world-use-case/
[75]
Louis-Noël Pouchet, U Bondugula, and T Yuki. 2016. PolyBench/C 4.2. Polyhedral Benchmark Suite.
[76]
Raghu Ramakrishnan and Arvinder Kaur. 2020. An empirical comparison of predictive models for web page performance. Information and Software Technology (2020), 106307.
[77]
Alan Romano, Xinyue Liu, Yonghwi Kwon, and Weihang Wang. 2021. An Empirical Study of Bugs in WebAssembly Compilers. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE).
[78]
Alan Romano and Weihang Wang. 2020. WASim: Understanding WebAssembly Applications through Classification. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1321--1325.
[79]
Alan Romano and Weihang Wang. 2020. WasmView: Visual Testing for WebAssembly Applications. In Proceedings of the 42nd International Conference on Software Engineering Companion (Seoul, South Korea) (ICSE'20 Companion). Association for Computing Machinery, New York, NY, USA, 4 pages.
[80]
Alan Romano, Yunhui Zheng, and Weihang Wang. 2020. MinerRay: Semantics-Aware Analysis for Ever-Evolving Cryptojacking Detection. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1129--1140.
[81]
Prabhjot Sandhu, David Herrera, and Laurie Hendren. 2018. Sparse matrices on the web: Characterizing the performance and optimal format selection of sparse matrix-vector multiplication in JavaScript and WebAssembly. In Proceedings of the 15th International Conference on Managed Languages & Runtimes. 1--13.
[82]
Marija Selakovic and Michael Pradel. 2016. Performance issues and optimizations in JavaScript: an empirical study. In Proceedings of the 38th International Conference on Software Engineering. 61--72.
[83]
Yuliang Shi, Xudong Zhao, Shanqing Guo, Shijun Liu, and Lizhen Cui. 2016. SRConfig: An Empirical Method of Interdependent Soft Configurations for Improving Performance in n-Tier Application. In 2016 IEEE International Conference on Services Computing (SCC). IEEE, 601--608.
[84]
Daniel Smilkov, Nikhil Thorat, and Ann Yuan. 2020. Introducing the WebAssembly backend for TensorFlow.js. https://blog.tensorflow.org/2020/03/introducing-webassembly-backend-for-tensorflow-js.html
[85]
The Clang Team. 2020. clang - the Clang C, C++, and Objective-C compiler --- Clang 11 documentation. https://clang.llvm.org/docs/CommandGuide/clang.html#cmdoption-o0
[86]
Leaning Technologies. 2020. Cheerp | C/C++ to WebAssembly compiler. https://leaningtech.com/pages/cheerp.html
[87]
Aaron Turner. 2018. WebAssembly Is Fast: A Real-World Benchmark of WebAssembly vs. ES6. https://medium.com/@torch2424/webassembly-is-fast-a-real-world-benchmark-of-webassembly-vs-es6-d85a23f8e193
[88]
Vladimir. 2018. WebAssembly vs. the world. Should you use WebAssembly? https://blog.sqreen.com/webassembly-performance/
[89]
W3C. 2021. Web Cryptography API. https://w3c.github.io/webcrypto/
[90]
Evan Wallace. 2016. Evanw/thinscript: A low-level programming language inspired by TypeScript. https://github.com/evanw/thinscript
[91]
Conrad Watt, John Renner, Natalie Popescu, Sunjay Cauligi, and Deian Stefan. 2019. CT-Wasm: Type-Driven Secure Cryptography for the Web Ecosystem. Proc. ACM Program. Lang. 3, POPL, Article 77 (Jan. 2019), 29 pages.
[92]
Junjun Zheng, Hiroyuki Okamura, and Tadashi Dohi. 2016. Performance Evaluation of VM-based Intrusion Tolerant Systems with Poisson Arrivals. In 2016 Fourth International Symposium on Computing and Networking (CANDAR). IEEE, 181--187.

Cited By

View all
  • (2024)Ring: A Lightweight and Versatile Cross-Platform Dynamic Programming Language Developed Using Visual ProgrammingElectronics10.3390/electronics1323462713:23(4627)Online publication date: 23-Nov-2024
  • (2024)Measuring the performance of techniques for dynamic 2D animation in web browsersJournal of Applied Mathematics, Statistics and Informatics10.2478/jamsi-2024-000920:2(77-110)Online publication date: 22-Dec-2024
  • (2024)Towards Seamless 5G Open-RAN Integration with WebAssemblyProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696864(121-131)Online publication date: 18-Nov-2024
  • Show More Cited By

Index Terms

  1. Understanding the performance of webassembly applications

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      IMC '21: Proceedings of the 21st ACM Internet Measurement Conference
      November 2021
      768 pages
      ISBN:9781450391290
      DOI:10.1145/3487552
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      • USENIX Assoc: USENIX Assoc

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 02 November 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. browser performance
      2. justin-time compilation
      3. web page performance
      4. webassembly

      Qualifiers

      • Research-article

      Conference

      IMC '21
      IMC '21: ACM Internet Measurement Conference
      November 2 - 4, 2021
      Virtual Event

      Acceptance Rates

      Overall Acceptance Rate 277 of 1,083 submissions, 26%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)476
      • Downloads (Last 6 weeks)34
      Reflects downloads up to 13 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Ring: A Lightweight and Versatile Cross-Platform Dynamic Programming Language Developed Using Visual ProgrammingElectronics10.3390/electronics1323462713:23(4627)Online publication date: 23-Nov-2024
      • (2024)Measuring the performance of techniques for dynamic 2D animation in web browsersJournal of Applied Mathematics, Statistics and Informatics10.2478/jamsi-2024-000920:2(77-110)Online publication date: 22-Dec-2024
      • (2024)Towards Seamless 5G Open-RAN Integration with WebAssemblyProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696864(121-131)Online publication date: 18-Nov-2024
      • (2024)eWAPA: An eBPF-based WASI Performance Analysis Framework for Web Assembly Runtimes2024 IEEE International Conference on Software Services Engineering (SSE)10.1109/SSE62657.2024.00054(323-333)Online publication date: 7-Jul-2024
      • (2024)A Study on the Performance of WebAssembly Versus JavaScript in an SVG Editor Environment2024 International Symposium on Parallel Computing and Distributed Systems (PCDS)10.1109/PCDS61776.2024.10743854(1-5)Online publication date: 21-Sep-2024
      • (2024)An Investigative Study of WebAssembly Performance in Cloud-to-Edge2024 International Symposium on Parallel Computing and Distributed Systems (PCDS)10.1109/PCDS61776.2024.10743586(1-5)Online publication date: 21-Sep-2024
      • (2024)Application of WebAssembly Technology in High-Performance Web Applications2024 11th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN)10.1109/IcETRAN62308.2024.10645198(1-6)Online publication date: 3-Jun-2024
      • (2024)Characterizing Dynamic Memory Behavior in WebAssembly Workloads2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)10.1109/ISPASS61541.2024.00047(328-330)Online publication date: 5-May-2024
      • (2024)dTEE: A Declarative Approach to Secure IoT Applications Using TrustZone2024 23rd ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)10.1109/IPSN61024.2024.00021(200-212)Online publication date: 13-May-2024
      • (2024)A Next Generation Web Browser Execution Environment2024 IEEE International Conference on Data and Software Engineering (ICoDSE)10.1109/ICoDSE63307.2024.10829873(1-6)Online publication date: 30-Oct-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media