research-article

Dynamically composing languages in a modular way: supporting C extensions for dynamic languages

Authors:

Matthias Grimmer,

Thomas Würthinger,

Hanspeter MössenböckAuthors Info & Claims

MODULARITY 2015: Proceedings of the 14th International Conference on Modularity

Pages 1 - 13

https://doi.org/10.1145/2724525.2728790

Published: 16 March 2015 Publication History

Abstract

Many dynamic languages such as Ruby, Python and Perl offer some kind of functionality for writing parts of applications in a lower-level language such as C. These C extension modules are usually written against the API of an interpreter, which provides access to the higher-level language's internal data structures. Alternative implementations of the high-level languages often do not support such C extensions because implementing the same API as in the original implementations is complicated and limits performance. In this paper we describe a novel approach for modular composition of languages that allows dynamic languages to support C extensions through interpretation. We propose a flexible and reusable cross-language mechanism that allows composing multiple language interpreters, which run on the same VM and share the same form of intermediate representation - in this case abstract syntax trees. This mechanism allows us to efficiently exchange runtime data across different interpreters and also enables the dynamic compiler of the host VM to inline and optimize programs across multiple language boundaries. We evaluate our approach by composing a Ruby interpreter with a C interpreter. We run existing Ruby C extensions and show how our system executes combined Ruby and C modules on average over 3x faster than the conventional implementation of Ruby with native C extensions, and on average over 20x faster than an existing alternate Ruby implementation on the JVM (JRuby) calling compiled C extensions through a bridge interface. We demonstrate that cross-language inlining, which is not possible with native code, is performance-critical by showing how speedup is reduced by around 50% when it is disabled.

References

[1]

How Not To Lie With Statistics: The Correct Way To Summarize Benchmark Results. Communications of the ACM, 29, March 1986.

Digital Library

[2]

IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2008, pages 1–70, Aug 2008.

[3]

Ruby Summer of Code Wrap-Up. http://blog.bithug.org/2010/ 11/rsoc, 2011.

[4]

Standard ECMA-335. Common Language Infrastructure (CLI). http://www.ecma-international.org/publications/standards/ Ecma-335.htm, 2012.

[5]

CNI (Compiled Native Interface). http://gcc.gnu.org/onlinedocs/ gcj/About-CNI.html, 2013.

[6]

HotSpot JVM. Java version history (J2SE 1.3). http://en. wikipedia.org/wiki/Java_version_history, 2013.

[7]

IBM. Java 2 Platform. Standard Edition. http://www.ibm.com/ developerworks/java/jdk/, 2013.

[8]

Java Native Access (JNA). https://github.com/twall/jna#readme, 2013.

[9]

jruby-cext: CRuby extension support for JRuby. https://github. com/jruby/jruby-cext, 2013.

[10]

Why shouldn’t I use PyPy over CPython if PyPy is 6.3 times faster? http://stackoverflow.com/questions/18946662/ why-shouldnt-i-use-pypy-over-cpython-if-pypy-is-6-3 -times-faster, 2013.

[11]

Extensions to the C Language. https://gcc.gnu.org/onlinedocs/ gcc/C-Extensions.html, 2014.

[12]

Clang/LLVM. http://clang.llvm.org/, 2014.

[13]

Common Object Request Brooker Architecture (CORBA) Specification. http://www.omg.org/spec/CORBA/3.3/, 2014.

[14]

GCC (GNU C Compiler). http://gcc.gnu.org/, 2014.

[15]

XPCOM Specification. https://developer.mozilla.org/en-US/ docs/Mozilla/XPCOM, 2014.

[16]

M. Arnold, S. Fink, D. Grove, M. Hind, and P. Sweeney. A survey of adaptive optimization in virtual machines. Proceedings of the IEEE, 93(2):449–466, 2005. ISSN 0018-9219.

[17]

E. Barrett, C. F. Bolz, and L. Tratt. Unipycation: A case study in cross-language tracing. In Proceedings of the 7th ACM Workshop on Virtual Machines and Intermediate Languages, VMIL ’13, pages 31– 40, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2601-8. URL http://doi.acm.org/10.1145/2542142.2542146.

Digital Library

[18]

E. Barrett, C. F. Bolz, and L. Tratt. Approaches to interpreter composition. To appear, 2014.

[19]

N. Goto, P. Prins, M. Nakao, R. Bonnal, J. Aerts, and T. Katayama. BioRuby: bioinformatics software for the Ruby programming language. Bioinformatics, 26(20):2617–2619, 2010.

Digital Library

[20]

. URL http://bioinformatics.oxfordjournals.org/content/26/20/2617.

[21]

abstract.

[22]

M. Grimmer. High-performance language interoperability in multilanguage runtimes. In Proceedings of the 2014 Companion Publication for Conference on Systems, Programming, Applications: Software for Humanity, SPLASH ’14, New York, NY, USA, 2014. ACM.

Digital Library

[23]

M. Grimmer, M. Rigger, L. Stadler, R. Schatz, and H. Mössenböck. An Efficient Native Function Interface for Java. In Proceedings of the 2013 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools, PPPJ ’13, pages 35–44, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2111-2. URL http://doi.acm.org/10.1145/ 2500828.2500832.

Digital Library

[24]

M. Grimmer, M. Rigger, R. Schatz, L. Stadler, and H. Mössenböck. TruffleC: Dynamic Execution of C on a Java Virtual Machine. In Proceedings of the 2014 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools, PPPJ ’14, New York, NY, USA, 2014. ACM. URL http://dx.doi.org/10.1145/2647508.2647528.

[25]

M. Grimmer, T. Würthinger, A. Wöß, and H. Mössenböck. An Efficient Approach for Accessing C Data Structures from JavaScript. In Proceedings of 9th International Workshop on Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems PLE - Workshop on Programming Language Evolution, 2014, ICOOOLPS ’14, New York, NY, USA, 2014. ACM. URL http://dx.doi.org/10.1145/2633301.2633302.

Digital Library

[26]

U. Hölzle, C. Chambers, and D. Ungar. Optimizing dynamically-typed object-oriented languages with polymorphic inline caches. In P. America, editor, ECOOP’91 European Conference on Object-Oriented Programming, volume 512 of Lecture Notes in Computer Science, pages 21–38. Springer Berlin Heidelberg, 1991. ISBN 978-3-540-54262-9. URL http://dx.doi.org/10.1007/BFb0057013.

Digital Library

[27]

T. Kalibera and R. Jones. Rigorous benchmarking in reasonable time. In Proceedings of the 2013 ACM SIGPLAN International Symposium on Memory Management (ISMM), 2013.

Digital Library

[28]

R. LeFevre. PSDNative, 2013. URL https://github.com/ layervault/psd_native.

[29]

S. Liang. Java Native Interface: Programmer’s Guide and Reference. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1st edition, 1999. ISBN 0201325772.

Digital Library

[30]

E. Meijer and J. Gough. Technical overview of the common language runtime. language, 29:7, 2001.

[31]

C. Nutter, T. Enebo, O. Bini, N. Sieger, et al. JRuby. http://jruby. org/, 2014. URL http://jruby.org/.

[32]

Oracle. OpenJDK: Graal project. http://openjdk.java.net/ projects/graal/, 2013.

[33]

J. R. Rose. Bytecodes meet combinators: Invokedynamic on the jvm. In VMIL ’09: Proceedings of the Third Workshop on Virtual Machines and Intermediate Languages, pages 1–11, New York, NY, USA, 2009.

Digital Library

[34]

ACM. ISBN 978-1-60558-874-2. URL http://portal.acm.org/ citation.cfm?id=1711506.1711508.

[35]

K. S. Ryan LeFevre et al. PSD.rb from Layer Vault, 2013. URL https://cosmos.layervault.com/psdrb.html.

[36]

C. Seaton, M. L. Van De Vanter, and M. Haupt. Debugging at full speed. In Proceedings of the Workshop on Dynamic Languages and Applications, pages 1–13. ACM, 2014.

Digital Library

[37]

M. Slee, A. Agarwal, and M. Kwiatkowski. Thrift: Scalable crosslanguage services implementation. Facebook White Paper, 5, 2007.

[38]

L. Stadler, G. Duboscq, H. Mössenböck, and T. Würthinger. Compilation queuing and graph caching for dynamic compilers. In Proceedings of the Sixth ACM Workshop on Virtual Machines and Intermediate Languages, VMIL ’12, pages 49–58, New York, NY, USA, 2012.

Digital Library

[39]

ACM. ISBN 978-1-4503-1633-0. URL http://doi.acm.org/10. 1145/2414740.2414750.

[40]

L. Stadler, G. Duboscq, H. Mössenböck, and T. Würthinger. Compilation queuing and graph caching for dynamic compilers. In Proceedings of the Sixth ACM Workshop on Virtual Machines and Intermediate Languages, VMIL ’12, pages 49–58, New York, NY, USA, 2012.

Digital Library

[41]

ACM. ISBN 978-1-4503-1633-0. URL http://doi.acm.org/10. 1145/2414740.2414750.

[42]

L. Stadler, T. Würthinger, and H. Mössenböck. Partial escape analysis and scalar replacement for java. In Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO ’14, pages 165:165–165:174, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2670-4. URL http://doi.acm.org/ 10.1145/2544137.2544157.

Digital Library

[43]

W. van Bergen et al. Chunky PNG, 2013. URL https://github.com/ wvanbergen/chunky_png.

[44]

W. van Bergen et al. OilyPNG, 2013. URL https://github.com/ wvanbergen/oily_png.

[45]

M. Wegiel and C. Krintz. Cross-language, type-safe, and transparent object sharing for co-located managed runtimes. Technical Report 2010-11, UC Santa Barbara, 2010.

[46]

A. Wöß, C. Wirth, D. Bonetta, C. Seaton, C. Humer, and H. Mössenböck. An object storage model for the Truffle language implementation framework. In Proceedings of the 2014 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools, PPPJ ’14. ACM, 2014. URL http://dx.doi.org/10.1145/2647508.2647517.

[47]

T. Würthinger, A. Wöß, L. Stadler, G. Duboscq, D. Simon, and C. Wimmer. Self-optimizing AST interpreters. In Proceedings of the 8th Symposium on Dynamic Languages, DLS ’12, pages 73–82, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1564-7. URL http://doi.acm.org/10.1145/2384577.2384587.

Digital Library

[48]

T. Würthinger, C. Wimmer, A. Wöß, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon, and M. Wolczko. One VM to rule them all. In Proceedings of the 2013 ACM international symposium on New ideas, new paradigms, and reflections on programming & software, pages 187––204. ACM, 2013.

Digital Library

Cited By

Nanjekye JBremner DMicic AJul ERacordon D(2023)Towards Reliable Memory Management for Python Native ExtensionsProceedings of the 18th ACM International Workshop on Implementation, Compilation, Optimization of OO Languages, Programs and Systems10.1145/3605158.3605849(15-26)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3605158.3605849
Kreindl JBonetta DStadler LLeopoldseder DMössenböck HMarr S(2020)Multi-language dynamic taint analysis in a polyglot virtual machineProceedings of the 17th International Conference on Managed Programming Languages and Runtimes10.1145/3426182.3426184(15-29)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3426182.3426184
Bertholon GKell SBonetta DLiu Y(2019)Towards seamless interfacing between dynamic languages and native codeProceedings of the 11th ACM SIGPLAN International Workshop on Virtual Machines and Intermediate Languages10.1145/3358504.3361230(38-47)Online publication date: 22-Oct-2019
https://dl.acm.org/doi/10.1145/3358504.3361230
Show More Cited By

Index Terms

Dynamically composing languages in a modular way: supporting C extensions for dynamic languages
1. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

Cross-Language Interoperability in a Multi-Language Runtime

In large-scale software applications, programmers often combine different programming languages because this allows them to use the most suitable language for a given problem, to gradually migrate existing projects from one language to another, or to ...
High-performance cross-language interoperability in a multi-language runtime
DLS 2015: Proceedings of the 11th Symposium on Dynamic Languages

Programmers combine different programming languages because it allows them to use the most suitable language for a given problem, to gradually migrate existing projects from one language to another, or to reuse existing source code. However, existing ...
High-performance language interoperability in multi-language runtimes
SPLASH '14: Proceedings of the companion publication of the 2014 ACM SIGPLAN conference on Systems, Programming, and Applications: Software for Humanity

Programs often consist of parts that are written in different languages because sub-problems lend themselves to being implemented in a particular language. However, multi-language programs often suffer from poor performance, complex cross-language ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MODULARITY 2015: Proceedings of the 14th International Conference on Modularity

March 2015

145 pages

ISBN:9781450332491

DOI:10.1145/2724525

General Chairs:
Robert B. France
Colorado State University, USA
,
Sudipto Ghosh
Colorado State University, USA
,
Program Chair:
Gary T. Leavens
University of Central Florida, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Modularity '15

Modularity '15: 14th International Conference on Modularity

March 16 - 19, 2015

CO, Fort Collins, USA

Acceptance Rates

Overall Acceptance Rate 41 of 139 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
293
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nanjekye JBremner DMicic AJul ERacordon D(2023)Towards Reliable Memory Management for Python Native ExtensionsProceedings of the 18th ACM International Workshop on Implementation, Compilation, Optimization of OO Languages, Programs and Systems10.1145/3605158.3605849(15-26)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3605158.3605849
Kreindl JBonetta DStadler LLeopoldseder DMössenböck HMarr S(2020)Multi-language dynamic taint analysis in a polyglot virtual machineProceedings of the 17th International Conference on Managed Programming Languages and Runtimes10.1145/3426182.3426184(15-29)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3426182.3426184
Bertholon GKell SBonetta DLiu Y(2019)Towards seamless interfacing between dynamic languages and native codeProceedings of the 11th ACM SIGPLAN International Workshop on Virtual Machines and Intermediate Languages10.1145/3358504.3361230(38-47)Online publication date: 22-Oct-2019
https://dl.acm.org/doi/10.1145/3358504.3361230
Gaikwad SNisbet ALuján MHosking AFinocchi I(2019)Hosting OpenMP programs on Java virtual machinesProceedings of the 16th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3357390.3361031(63-71)Online publication date: 21-Oct-2019
https://dl.acm.org/doi/10.1145/3357390.3361031
Kreindl JBonetta DMössenböck HHosking AFinocchi I(2019)Towards efficient, multi-language dynamic taint analysisProceedings of the 16th ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3357390.3361028(85-94)Online publication date: 21-Oct-2019
https://dl.acm.org/doi/10.1145/3357390.3361028
Grimmer MSchatz RSeaton CWürthinger TLuján MMössenböck H(2018)Cross-Language Interoperability in a Multi-Language RuntimeACM Transactions on Programming Languages and Systems10.1145/320189840:2(1-43)Online publication date: 28-May-2018
https://dl.acm.org/doi/10.1145/3201898
Rigger MSchatz RKreindl JHäubl CMössenböck HMarr SSartor J(2018)Sulong, and thanks for all the fishCompanion Proceedings of the 2nd International Conference on the Art, Science, and Engineering of Programming10.1145/3191697.3191726(58-60)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3191697.3191726
Psaroudakis IKaestle SGrimmer MGoodman DLozi JHarris TOliveira RFelber PHu Y(2018)Analytics with smart arraysProceedings of the Thirteenth EuroSys Conference10.1145/3190508.3190514(1-15)Online publication date: 23-Apr-2018
https://dl.acm.org/doi/10.1145/3190508.3190514
Cazzola WChitchyan RRashid AShaqiri A(2018)-DSUComputer Languages, Systems and Structures10.1016/j.cl.2017.07.00351:C(71-89)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1016/j.cl.2017.07.003
Würthinger TWimmer CHumer CWöß AStadler LSeaton CDuboscq GSimon DGrimmer M(2017)Practical partial evaluation for high-performance dynamic language runtimesACM SIGPLAN Notices10.1145/3140587.306238152:6(662-676)Online publication date: 14-Jun-2017
https://dl.acm.org/doi/10.1145/3140587.3062381
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents