A modest proposal: an approach to making the internal R system extensible

Abstract

The R computing environment has become an important part of the statistical community and fostered the development of over a thousand add-on packages, many representing state-of-the-art research in statistical methodology. Although it is relatively easy to develop functionality on top of the system, it is very difficult for developers to directly extend the core system itself—the language, the interpreter and the internal data structures. Yet the ability to easily introduce new core, first-class data structures into the system that are customized and efficient is becoming essential in this era of large, complex data sets and innovative algorithms and data structures. While the community that might use such a facility to introduce new data types may be small, it is potentially very talented and important, and may lead to significant innovations that allow us to continue to leverage R for the next 5 years or more in rich new ways. I describe some of the difficulties that people encounter in extending the system and suggest that an object-oriented architecture for the internal implementation of R (or any system) would make such low-level internals extensible by package developers and not just the core development team. This would promote potentially rich experimentation that would allow us and others to approach new styles of computation in R, while simultaneously maintaining the existing important community which provides so much value-added to the R environment. Specifically, transforming the R implementation from a representation-specific architecture to a C++ abstract/virtual interface-based architecture may be the least disruptive approach to the continued evolution of R, and would bring many advantages and some technical challenges. Such an approach involves many technical details and potential degradations in performance. Due to the length of the this paper, I do not explore these issues in great detail but introduce the basic concepts. I do, however, refer to some technical aspects that are best understood with some knowledge of the implementation of R at the level of using the .Call () interface in R.