Efficient and General On-Stack Replacement

for Aggressive Program Specialization

Sunil Soman Chandra Krintz

Computer Science Department
University of California, Santa Barbara, CA 93106

E-mail: {sunils,ckrintz}@cs.ucsb.edu

Abstract ditions, and then to undo the optimization when condi-

Efficient invalidation and dynamic replacement of tions change. This enables the compiler to identify op-
executing code – on-stack replacement (OSR), is neces- timizations that are likely to be successful in the near-
sary to facilitate effective, aggressive, specialization of term, and then to apply others as it learns more about
object-oriented programs that are dynamically loaded, future program and resource behavior. Examples of
incrementally compiled, and garbage collected. Extant such optimizations include virtual call inlining, excep-
OSR mechanisms restrict the performance potential of tion handler removal, and memory management system
program specialization since their implementations are optimizations.
special-purpose and restrict compiler optimization. Key to the success of such specializations is the
In this paper, we present a novel, general-purpose presence of an efficient, general-purpose, mechanism
OSR mechanism that is more amenable to optimiza- for undoing optimization on-the-fly when associated as-
tion than prior approaches. In particular, we decou- sumptions are rendered invalid. Future method invo-
ple the OSR implementation from the optimization pro- cations can be handled by recompilation. Replacing
cess and update the program state information incre- the currently executing method and re-initiating exe-
mentally during optimization. Our OSR implementa- cution in the new version is termed on-stack replace-
tion efficiently enables the use of code specializations ment (OSR). OSR has previously been successfully em-
that are invalidated by any event – including those ex- ployed for debugging optimized code [14], for deferred
ternal to program code execution. We improve code compilation [7, 23, 10, 19], and method optimization-
quality over the extant, state-of-the-art, resulting in per- level promotion [15, 10, 19].
formance gains of 1-31%, and 9% on average. Each of these prior approaches, however, is specific
Keywords to the particular optimization being performed. More-
Program specialization, on-stack replacement, code in- over, extant approaches inhibit compiler optimizations
validation, virtual machines, Java. by employing special instructions for state collection,
in-line with generated program code. Such imple-
1 Introduction mentations, in addition to increasing code size, increase
Advanced implementations of modern object- variable live ranges and restrict code motion.
oriented languages, such as Java [12], depend on run- We present a general-purpose OSR implementation,
time environments that employ “just-in-time” compi- which can be used for any OSR-based specialization,
lation for performance. These language environments and is more amenable to compiler optimization. We de-
dynamically compile program code that is loaded incre- couple OSR from the program code, and consequently,
mentally, and possibly from a remote target. Although from compiler optimizations. This significantly im-
optimization has the potential to significantly improve proves code quality over an extant, state-of-the-art ap-
program performance, on-the-fly and incremental com- proach, and enables OSR to occur at any point at which
pilation makes optimization implementation challeng- control can transfer out of an executing method. We
ing. In particular, the compiler must predict which op- can, therefore, support existing OSR-based specializa-
timizations will be most successful for the remainder of tions as well as other, more aggressive, optimizations
program execution. that are triggered by events external to the executing
One successful approach to enable more aggressive code, including, class loading, changes in program and
optimizations for such systems is to specialize the pro- JVM behavior, exception conditions, resource availabil-
gram for a particular behavior or set of execution con- ity, and user events, such as dynamic software updates.
 In the sections that follow, we provide background a call instruction – execution of an OSRPoint inserted
on OSR (Section 2) and detail our extensions and im- in a method causes OSR to be performed uncondition-
plementation in the Jikes Research Virtual Machine ally for that method. The OSRPoint records necessary
(JikesRVM) [1] (Section 3). We then describe three execution state which consists of values for bytecode-
OSR-based specializations (Section 4). The first two level, local and stack variables, and the current program
are existing specializations for virtual method dispatch counter. The execution state is a mapping that provides
and for dynamic GC switching. Our OSR implemen- information about runtime values at the bytecode-level
tation enables performance improvement by 6% for the so that the method can be recompiled and execution
former technique, and by 9% for the latter. We also restarted with another version. The JikesRVM OSR im-
present a novel specialization for generational garbage plementation is similar to that used in other systems for
collection in which we avoid adding write-barriers to similar purposes [14, 23, 13, 19], although the authors
generated code until they are required (i.e. until there of each refer to the instruction using different names,
are objects in the old generation(s)). Our results indi- e.g., interrupt points [14], OPC RECOMPILE instruc-
cate that we improve startup time for programs by 6% tions [23], and uncommon traps [19].
on average; for short running programs we reduce ex- Prior OSR implementations restrict compiler opti-
ecution time by 8-14%. Following our empirical eval- mization. Since the compiler considers all method
uation and analysis (Section 5), we present our conclu- variables (locals and stack) live at an OSRPoint, it
sions and plans for future work (Section 6). artificially extends the live ranges of variables and
limits the applicability of optimizations such as dead
2 Background and Related Work code elimination, load/store elimination, alias analy-
On-stack replacement (OSR) [7, 15] is a four step sis, and copy/constant propagation. In addition, the
process. The runtime extracts the execution state (cur- compiler cannot move variable definitions around OS-
rent variable values and program counter) from a partic- RPoints [14, 10].
ular method. The compilation system then recompiles
the method. Next, the compiler generates a stack activa- 3 Extending On-Stack Replacement
tion frame and updates it with the values extracted from Existing implementations for OSR do not signifi-
the previous version. Finally, the system replaces the cantly degrade code quality when there are a small num-
old activation frame with the new and restarts execution ber of OSRPoints [10, 14]. However, our goal is to
in the new version. enable more aggressive, existing and novel, specializa-
Given OSR functionality, a dynamic compilation tions including those triggered by events external to the
system can implement aggressive specializations based executing code, e.g., class loading, exception handling,
on conditions that may change as execution progresses. garbage collection optimization, and dynamic software
If these conditions change as a result of an external or updating. For such specializations, we may be required
a runtime event, the compilation system can recompile to perform OSR at all locations in a method at which ex-
(and possibly re-optimize) the code, and replace the cur- ecution can be suspended. Since there are a large num-
rently executing version [7]. OSR has been employed ber of such locations, many along the critical path of the
by extant virtual execution environments for dynami- program, we require an alternative implementation that
cally de-optimizing code for debugging [14], for de- enables optimization at, and across, these locations.
ferred compilation of method regions to avoid compi- To this end, we extended the current OSR imple-
lation overhead and improve data flow [7, 23, 10, 19], mentation in the JikesRVM with a mechanism that de-
and to optimize methods that execute unoptimized for a couples the OSR implementation from the optimization
long time without returning [15, 10, 19]. process. In particular, we maintain execution state in-
These OSR implementations were specially de- formation without inserting extra instructions at every
signed for a target purpose. For example, OSR for point at which control may transfer out of a method.
deferred compilation is implemented as an uncondi- This includes implicit yield points (method prologues,
tional branch to the OSR invocation routine. OSR for method epilogues, and loop back-edges), call sites, ex-
method promotion and de-optimization is implemented ception throws, and explicit yield points. The data
as part of the thread yield process. The JikesRVM from structure we employ for state collection is called a
IBM Research is a virtual machine that currently em- VARMAP (short for variable map).
ploys a state-of-the-art, special-purpose, OSR for de- A VARMAP is a per-method list of thread-switch
ferred compilation and method promotion [10]. It is this points and bytecode variables that are live at each point.
system that we extend in this work. We maintain this list independent from compiled code
The JikesRVM optimizing compiler inserts an in- so it does not affect liveness of code variables. We up-
struction, called an OSRPoint, into the application code date the VARMAP incrementally as the compiler per-
at the point at which OSR should be unconditionally forms its optimizations. Our VARMAP is somewhat
performed. This instruction is implemented similarly to similar in form to the data structure described in [9]
 Before optimization After optimization present in the VARMAP are removed during optimiza-
.. .. tion, the wrapper replaces its entry with the rvalue in
15: int_move l15i(int)=l8i(int) 15: int_move l8i(int)=l8i(int)
18: int_shl l17i(int)=l15i(int), 2 18: int_shl l17i(int)=l8i(int), 2 the instruction, or records all of the right-hand vari-
20:
25:
call static “callme() V”
int_add l19i(int)= l8i(int),
20:
25:
call static “callme() V”
int_add l19i(int) = l8i(int),
ables along with operators used to derive the lvalue in
l15i(int) l8i(int) case of multiple r-values (we currently only handle sim-
.. ..
ple unary or binary expressions). Similarly, we replace
variables eliminated by escape analysis with their rval-
Intermediate Code (HIR)
ues in the VARMAP.
25@main (..LLL,..),.., l18i(int),
l15i(int), l17i(int), ..
25@main (..LLL,..),.., l18i(int),
l8i(int), l17i(int), ..
The compiler automatically updates the VARMAP
during live variable analysis. We record variables that
transferVarForOsr(l15i, l8i)
are no longer live at an OSR point, and the relative
VARMAP entry
position of each in the map. Every variable that live
Figure 1. Shows how the VARMAP is analysis discovers as dead, is set to a void type in the
maintained and updated. VARMAP. We cannot simply drop the variable from the
entry since we must track the relative positions of vari-
for tracking pointer updates in the presence of com- ables in order to enable their restoration in the correct
piler optimizations, to support garbage collection in location during OSR. During register allocation, we up-
Modula-3. Our implementation is different in that we date the VARMAP with physical register and spill loca-
track all stack, local, and temporary variables across a tions. This enables us to restore these locations during
wide range of compiler optimizations automatically and OSR as well.
transparently, and do so online, during dynamic opti- Figure 1 shows how we maintain and update
mization of Java programs. VARMAP entries. We show the VARMAP before and
To update VARMAP entries during optimization, we after copy propagation. We show a snippet of Java byte-
defined the following system methods: code (left), and high-level, JikesRVM intermediate code
• transferVarForOsr(var1, var2): Record that var2 representation (HIR). We also show the VARMAP entry
will be used in place of var1, henceforth in the (bottom) for the callme() call site which contains the
code (e.g. as a result of copy propagation) bytecode index (25) of the instruction that follows the
call site, as well as three local (l) variables with inte-
• removeVarForOsr (var): Record that var is no ger (i) types (a: l8i, b: l15i, c: l17i).
longer live/valid in the code. The index identifies the place in the code at which ex-
ecution resumes following the call (in this example), if
• replaceVarWithExpression(var, vars[], oper- OSR occurs during callme(). The VARMAP tracks
ators[]): Record that variable var has been method variables (l8i, l15i, and l17i) at this
replaced by an expression that is derivable from point; this entry is used by the system to update the
the set of variables vars and operators. frame of the new version of meth() during OSR.
Our OSR-enabled compilation system handles all In the HIR before optimization, the instruction
JikesRVM optimizations that impact liveness except at bytecode index 15 copies variable l8i to l15i
tail call and tail recursion elimination (which elim- then uses both in the subsequent code. The op-
inate stack frames entirely). This set of optimiza- timization replaces all l15i uses with l8i dur-
tions includes copy and constant propagation, com- ing copy propagation. During this optimization,
mon subexpression elimination (CSE), branch opti- the replacement invokes a wrapper which automati-
mizations, dead-code elimination (DCE), and local es- cally updates the VARMAP, i.e., the wrapper calls
cape analysis optimization. transferVarForOsr(l15i,l8i) prior to re-
When a variable is updated, the compiler calls a placement. DCE will remove bytecode instruction 15
wrapper function that automatically invokes the ap- in a later pass. DCE of the instruction at index 15 will
propriate VARMAP functions. This enables users to cause the VARMAP to record that l15i is no longer
easily extend the compiler with new optimizations, live.
without having to manually handle VARMAP up- When the compilation of a method completes, we
dates. For example, when copy/constant propagation encode the VARMAP using a compact encoding for
or CSE replaces a use of a variable (rvalue) with an- OSRPoints from the existing implementation [10]. The
other variable or constant, the wrapper performs the encoded map contains an entry for each OSR point,
replacement in the VARMAP entry by invoking the which consists of the register map – a bitmap that indi-
transferVarForOsr function. cates which physical registers contain references (which
We handle DCE and local escape analysis using a the garbage collector may update). In addition, the map
similar wrapper function for updates to variable def- contains the current program counter (bytecode index),
initions. When definitions of dead variables that are and a list of pairs (local variable, location) (each pair
encoded as two integers), for every inlined method (in the OSR helper from the specialized method. More
case of an inlined call sequence). The encoded map precisely, apart from re-routing the return address of the
is maintained in the system throughout the lifetime of specialized method’s callee, we set the return address
the the program. All other data structures required for of the OSR helper to point to the current instruction
OSR-aware compilation (including the VARMAP) are pointer in the specialized code, since this value will be
reclaimed during GC. used during OSR to calculate the location to resume ex-
We also handle certain cases specially to enable cor- ecution at the corresponding location in the new version
rect state collection and to enable efficient code gener- of the method. We also update OSR helper’s stack
ation. These cases include call site return values, ex- pointer appropriately.
ception throws and yield points, and machine-specific
instruction selection and optimization. 4 Aggressive Program Specialization
If the callee of a method to be replaced returns a We compare our OSR implementation to the current
value, the OSR process extracts that value from the state-of-the-art as part of our empirical evaluation. In
execution state so that it is available in the new ver- addition, we employ our system for novel specializa-
sion of the method. Return values are typically stored tion for generational garbage collection (GC) and to im-
in specific registers as dictated by the compiler’s call- prove two existing specialization systems, one for dy-
ing convention and the target architecture, and must be namic dispatch of virtual method calls and the other for
extracted from these registers. We handle exception dynamic switching between GCs.
throws and yield points as we do call sites. Moreover,
we do not perform OSR for methods for which execu- Write-Barrier Specialization for Generational GC
tion has reached their epilogue. A generational garbage collector (GC) [18, 24] segre-
gates objects into young and old parts (generations),
based on their lifetimes. This separation enables young
3.1 Triggering On-Stack Replacement
objects to be garbage collected independently and more
Our system requires an appropriate trigger to initiate frequently than the old, without collecting the entire
OSR. The current OSR implementation in JikesRVM heap. Once objects age in the young generation, they
uses a compiler-driven, eager approach. The compiler are promoted to the old. A generational GC must record
inserts a call to a system method, which is guarded by references from mature objects to nursery objects on ev-
a condition that checks the validity of the specializing ery field assignment. The compiler inserts extra instruc-
assumption. Execution of this method causes OSR for tions called write-barriers [25, 16, 3] into application
the callee. code for this purpose, for every pointer mutation (put-
The alternative, an external, lazy trigger, is more ap- fields and array stores in Java). These extra instructions
propriate for enabling OSR due to events in the envi- generally degrade program performance. Researchers
ronment, out of line with application execution, such as have recently found that with intelligent write-barrier
is done in Self-91 for debugging optimized code [14]. code and modern architectures, write-barrier cost for
The runtime triggers OSR when it deems invalid an as- many programs is small [3, 4], compared to the bene-
sumption previously used by the compiler for special- fits of generational GC.
ization. The runtime invokes a helper method (called However, for applications that do not allocate
OSR helper) that performs OSR. The method either enough memory to require a GC, or that trigger GC later
patches the code of the executing method(s) with code in their lifetime, write-barrier execution, regardless of
that invokes the rest of the OSR process, or modifies how efficiently the write-barriers are implemented, is
the return address of each callee of the method to be re- pure overhead. Such applications have become more
placed. This approach does not require the insertion of plentiful as the cost of memory for modern processors
conditional calls into the application code or all inval- has plummeted and memory has become abundant.
idated methods to be replaced at once. As a result, it To address this issue, we introduce a novel special-
enables a fast call stack traversal. ization technique that avoids inserting write-barriers.
The OSR helper reroutes the return address of the That is, we specialize frequently executing (hot) meth-
specialized method’s callee to itself so that OSR it can ods without write-barriers. The compiler inserts write-
perform OSR on the method incrementally – when the barriers for all cold methods. If a GC occurs and objects
callee returns. When this occurs, the OSR helper are promoted to the mature space, we invalidate and re-
sets up a new stack frame with execution state extracted place hot methods that require write-barriers. We do not
from the specialized method’s stack. To preserve values specialize hot methods once GC has occurred and ob-
the contained in registers during the execution of spe- jects have been promoted to the mature space, or when
cialized method, the OSR helper saves all registers the maximum heap size is below 500MB, or if heap res-
(volatiles and non-volatiles) into its stack frame. The idency exceeds 60% (identified empirically).
OSR helper expects to find the specialized method Consequently, programs that perform no GC experi-
by traversing the stack, and hence, we “fake” a call to ence minimal write-barrier overhead (due to execution
of cold, unoptimized methods). Also, programs that do OSR specialized methods to ensure correctness. The
not require GC at startup, experience improved startup system only performs OSR for optimized methods that
and reap the benefits of generational collection, with may execute specialized code after the current program
only minor OSR overhead imposed at the first GC. point. Support for OSR however significantly inhibits
Specialization of Virtual Method Dispatch dynamic GC selection from achieving its full potential.
Virtual method dispatch is a technique for aggressively We modified the switching system to use our VARMAP
binding virtual call sites to their implementations. The extension, in place of the extant (default JikesRVM) ap-
cost of executing virtual methods is higher than static proach.
method invocations since the underlying object type 5 Empirical Evaluation
must be extracted at runtime, to identify the method to
dispatch. [17] discusses many such techniques. To evaluate the efficacy of our OSR implementation,
we measured the performance of our system for the ag-
JikesRVM employs pre-existence [8], code patch-
gressive specialization techniques described previously.
ing [17], and deferred compilation [10] to reduce the
We first detail the experimental methodology and then
overhead of dynamic dispatch. The optimizing com-
present results.
piler speculatively inlines a virtual method using a
heuristic to predict the likely underlying type of an ob- 5.1 Experimental Methodology
ject [5, 11, 15]. The compiler omits a check to protect
We used a dedicated 2.4GHz x86-based single-
inlined code if it can determine pre-existence – if given
processor Xeon machine (with hyperthreading) running
the currently loaded class files, the object type does not
RedHat Linux 9. We focus on the x86 system since it is
change once method execution commences. If a class
currently the most popular processor for desk-side sys-
loaded later violates these assumptions, only future in-
tems and Internet computing. We extended JikesRVM
vocations of the method are recompiled, obviating the
version 2.2.0 with jlibraries R-2002-11-21-19-57-19.
need for OSR [8].
We ran all of our experiments using the Fast Adap-
If the compiler cannot establish a pre-existence guar-
tive JikesRVM configuration with a Generational Mark-
antee, then it must insert a “guard” to protect the spec-
Sweep (GMS) garbage collector. This GC uses a copy-
ulatively inlined code. JikesRVM uses code patching
ing young generation and a mark-sweep collected ma-
for this purpose. The compiler generates code for the
ture generation. We generated and employed off-line
inlined sequence, and instructions that will be executed
profiles to guide hot-method optimization to reduce
if inline specialization is invalidated. If class loading
non-determinism resulting from adaptive selection of
invalidates the assumption used for inlining, the VM
methods to be optimized (our profiles are available upon
overwrites the first instruction of the inlined sequence
request) as is done in similar studies [20, 21]. Our
that occurs after the current point in the execution, with
benchmarks are a subset of the the SpecJVM98 [22]
a branch to the alternative path. The compiler inserts
and the JOlden [6] benchmark suites. We executed each
an OSRPoint at the alternative path to avoid compila-
benchmark multiple times and report the average of all
tion of this unlikely case. Program execution of an OS-
but the first run. We report the compilation overhead
RPoint causes the current method to be replaced with
introduced by our system separately.
a new version that implements only the dynamic dis-
patch code sequence. As an alternative to code patch- 5.2 Results
ing, we extended this JikesRVM implementation to use We first present results from experiments that com-
our VARMAP-based OSR for speculatively inlined call pare our VARMAP-based implementation to a state-of-
sites for which pre-existence cannot be guaranteed. the-art extant OSR implementation. In order to create a
Specialization of Automatic GC-Switching System valid reference to compare our implementation against,
In prior work, we developed an extension to JikesRVM we extended the JikesRVM OSR system to remove OS-
that enables application-specific dynamic GC selec- RPoints (described in Section 2) at the end of compila-
tion [21]. This system employs multiple garbage col- tion. By doing so, we are able to insert OSRPoints at
lectors within a single JikesRVM execution image and every point in a method at which execution can be sus-
can switch between them at runtime in an effort to im- pended, thus potentially allowing OSR at these points,
prove performance. This prior work indicates the poten- without actually triggering OSR unconditionally.
tial for significant performance improvements (11-14% Figure 2 shows the results from this comparison. The
for standard benchmarks and up to 44% for short run- y-axis is percent reduction in execution time enabled
ning applications). by VARMAP OSR over OSRPoints. The Average bar
To enable these improvements, the compilation sys- shows the average across all benchmarks, and Average
tem must specialize the code for the underlying GC, i.e., Spec98 shows the average for only the SpecJVM bench-
inline allocation sites and include or omit generational marks. Both bars are averages across heap sizes (min-
write-barriers, as appropriate. When the underlying GC imum to 12 times the minimum), to include the im-
changes, the system must invalidate and if necessary, pact of any space overhead introduced by VARMAPs.
 )
 Clean With WBSpec
15 6 
Benchmark ET ET % Impr. WBs Elim. OSRs OSR Time
,  (s) (s) (ms)
1- 2 
compress 6.96 6.72 3.45 1209 0 0
+0  jess 2.97 2.96 0.37 6902267 13 3.75
) db 17.03 15.83 7.05 26852028 0 0
34 


,  javac 6.71 6.56 2.24 2598303 1 0.38
1
, 
mtrt 6.23 5.92 4.98 1851952 0 0
1- 2  jack 4.11 4.08 0.73 4685883 7 3.94
+
/0
MST 0.87 0.75 13.79 4683081 0 0
.) Perimeter 0.25 0.23 8 3170646 0 0
-,  Voronoi 1.44 1.21 15.97 16047220 0 0
          
   
%$  &'
+)     !
"      Avg. 5.17 4.92 6.29 7421399 2.33 0.90
*
 # 

  %$  Avg. Spec98 7.34 7.01 3.14 7148607 3.50 1.35
()

# 
Figure 4. Performance of write-barrier
Figure 2. Performance of our VARMAP
specialization (WBSpec) for a heap size
OSR Implementation vs the JikesRVM
of 500MB using the popular Generational
Reference System, across heap sizes.
Mark Sweep GC.

Compilation Time (ms) Space Added (KB) our system. Columns 2 and 3 show the compilation
Benchmark Clean VARMAP Compile Time Runtime time for the reference system and our VARMAP, re-
compress 68 79 14.52 3.16 spectively. Column 4 shows the percentage degradation
db 91 117 24.57 5.26 in compilation time imposed by VARMAP. Columns 5
jack 445 543 139.67 30.00
and 6 show the space overhead introduced by VARMAP
javac 1962 2540 629.94 136.98
jess 504 656 136.80 29.20 during compilation time (collectable) and runtime (per-
mtrt 595 746 154.38 33.50 manent), respectively. On average, our system increases
MST 50 66 17.03 3.73 compilation time by just over 100ms, and adds space
Perimeter 86 66 15.82 3.47 overhead of 133KB that is collected, and 29KB that is
Voronoi 96 129 62.06 13.49
not collectable.
Avg. 433 549 132.75 28.75
Avg. Spec98 611 780 183.31 39.68
Specialization for Generational Garbage Collection
We next present results for a novel OSR-based special-
Figure 3. Overhead of our OSR-VARMAP
ization for write-barrier removal for generational GCs.
Implementation in the JikesRVM refer-
Prior to the initial GC when there are no objects in the
ence system. Cols 2 & 3 indicate compila-
mature space, write-barriers are not required, and thus,
tion times and Cols 5-6 show space over-
impose pure overhead. Our goal with this specialization
head.
is to reduce the overhead of write-barriers for programs
that do not require GC, and to improve the startup per-
VARMAP shows significant improvement in applica- formance of those programs that do.
tion performance – by 9% on average across all bench- For this specialization, we employ the popular Gen-
marks, and by over 10% across the SpecJVM bench- erational Mark Sweep (GMS) collector. The com-
marks. jess and mtrt show the most benefit, 31% and piler checks the maximum heap size to ensure that it
20% respectively. For these benchmarks, the original is large enough to warrant specialization (>=500MB),
implementation severely inhibits optimization, partic- and that heap residency (pages used/pages allocated) is
ularly due to increased register pressure by artificially low (<=60%). We identified both values empirically. If
extending live ranges of variables, past their last actual no GCs have occur, the compiler elides write-barriers.
use. This results in a large number of variable spills to This specialization must be invalidated when objects
memory. With our implementation, we do not need to are first promoted to the mature space. Our system only
maintain conservative liveness estimates, since we track performs OSR for methods that require write barriers
liveness information accurately. and when there are field assignments that will execute
Other benchmarks show benefits of 5% or less. For after the point at which execution has been suspended.
these benchmarks, improved code quality does not im- We present the performance of write-barrier special-
pact overall performance significantly. Since these pro- ization (referred to as WBSpec) in Figure 4. We used
grams are short running, they are not heavily optimized. a heap size of 500MB for these experiments. Column
In addition, OSR VARMAP does impose some GC 2 and 3 show the execution time performance in sec-
overhead, since we maintain information for each pos- onds without and with WBSpec, respectively. Column
sible OSR point – which for short running codes is not 4 shows the percent improvement enabled by WBSpec.
fully amortized, especially for small heap sizes. Column 5 shows the number of write barriers elimi-
Figure 3 details the space compilation overhead of nated. The final two columns show the OSR overhead
 I 8 7
QU V X^W
@ ~
L †Š ‹
? ] X
QM R  ] W
KP > †‚ ‡
I \X
ST = €… \[W
L ~
Q < ˆ‰ ZX
L 
† Z[W
QM R ; 
K Y X
OP : †‚ ‡ Y W
€
NI 9 „… X
ML ƒ~ W
8
KI ‚
7 _m l
_
jeff o c oqp[r s o ec ` u lzy e _ {d|
H IJ €~ bdc e f fhg ikj l jln a e v `c`w x ec r b e
ABCC DFE G E } ~ _`a t e cu a n
e
c lzy
xn e
Figure 5. Performance improvement from
using OSR VARMAP for speculative de- Figure 6. Performance improvement from
virtualization (dynamic dispatch) of vir- using OSR VARMAP for OSR within an au-
tual methods. We omit benchmarks which tomatic GC switching system.
showed no significant change.
We also used our OSR implementation to im-
imposed – column 6 is the number of OSRs that occur prove the performance of a dynamic, garbage-collection
and column 7 is the total time for all OSRs. For many of switching system within JikesRVM that we developed
the benchmarks, no OSR is required since a minor GC in prior work. This system enables performance gains
is not triggered for these. For those that require OSR, (including all overheads) of 11-14% on average. How-
the overhead is very small. ever, this system incurs an overhead of 10% on aver-
On average, WBSpec improves performance by 6% age, even when no dynamic switching is triggered. This
across benchmarks. For the SPECJVM benchmarks overhead inhibits the full performance potential of the
(first 6 in the table), WBSpec improves performance system, the primary source of which is the need for OSR
by 3% on average. For benchmarks that require GC, support. Our OSR implementation reduces the over-
this benefit is during program startup. The JOlden head of the system by almost half.
benchmarks require no GC when we use a heap size of Figure 6 shows the benefits on total execution time
500MB. We believe that such applications are ideal can- due to the use of our OSR implementation in the GC
didates for specialization presented. The average im- switching system. The data is the average improvement
provement in execution time for these benchmarks is across a range of heap sizes for each benchmark (from
13%. the minimum to 12x the minimum). VARMAP im-
proves performance most significantly for jess (26%),
OSR VARMAP for Existing Specializations mtrt (19%), and MST (22%). This is due to the num-
For guard-free dynamic dispatch, we replace code ber of OSR points in the hot methods for these bench-
patching and OSRPoints for deferred compilation with marks, and due to the fact that MST is very short run-
our VARMAP implementation to guard speculatively ning. Across benchmarks, VARMAP shows a 9% im-
inlined virtual method calls. The compiler inlines calls provement. This result is similar to the performance
that meet size constraints and for which a single ob- improvement enabled by VARMAP versus the extant
ject target can be predicted [2]. To preserve correct- state-of-the-art that we presented previously.
ness, we employ OSR to replace code that does not
have checks inserted, when class loading invalidates as- 6 Conclusions and Future Work
sumptions made by the compiler. Upon recompilation, We present a new implementation of on-stack re-
the compiler generates a new version of the method that placement (OSR) that decouples dynamic state collec-
implements dynamic dispatch at the previously inlined tion from compilation and optimization. Unlike ex-
call site. We only perform OSR for methods for which isting approaches, our implementation does not inhibit
the compiler cannot establish pre-existence guarantees compiler optimization, and enables the compiler to pro-
(see Section 4). duce higher quality code. Our empirical measurements
Figure 5 shows the impact of using VARMAP over within JikesRVM show a performance improvement by
using the original OSRPoint and code patching. We 9% on average (from 1% to 31%), over a commonly
only presents the results for the benchmarks (2) that used implementation. We implement a novel, OSR-
implemented call sites for which the compiler was based specialization for write-barrier removal in gener-
unable to make pre-existence guarantees. For jess, ational GC that improves performance by 6% on aver-
a single inlined method with an eliminated guard age. Moreover, we empirically confirm that our system
( 202 jess.jess.ValueVector.size()) constitutes 8% of the is effective for extant specializations: dynamic dispatch
total number of method invocations. In the case of mtrt, of virtual methods, and for automatic GC switching.
3 such methods constitute over 50% of the total invoca- As part of future work, we plan to employ our OSR
tions. system for other aggressive specializations. These in-
clude removing infrequently executing instructions, e.g. [10] S. Fink and F. Qian. Design, Implementation and Eval-
exception handling code. In addition, OSR can be used uation of Adaptive Recompilation with On-Stack Re-
to trigger dynamic software updates in highly available placement. In International Symposium on Code Gen-
server systems. In such environments, control never eration and Optimization (CGO), Mar. 2003.
leaves a particular method (typically, a program loops [11] G. Aigner and U. Hölzle. Eliminating Virtual Function
Calls in C++ Programs. In European Conference on
forever listening for service requests, and issues re-
Object-Oriented Programming (ECOOP), July 1996.
quested work to slave processes). Using OSR, we can, [12] J. Gosling, B. Joy, and G. Steele. The Java Language
hence, upgrade code without affecting service availabil- Specification. Addison-Wesley, 1996.
ity. Finally, we plan to investigate the use of OSR in ag- [13] U. Hölzle. Optimizing Dynamically Dispatched Calls
gressive incremental alias, and escape analysis to per- with Run-Time Type Feedback. In Conference on Pro-
form speculative pointer-based optimizations, such as, gramming Language Design and Implementation, June
stack allocation of objects, memory layout optimiza- 1994.
tions, and synchronization removal. [14] U. Hölzle, C. Chambers, and D. Ungar. Debugging Op-
timized Code with Dynamic Deoptimization. In ACM
Acknowledgements Conference on Programming language design and im-
plementation, June 1992.
We thank Kathryn McKinley and Steve Fink for their [15] U. Hölzle and D. Ungar. A Third Generation Self Im-
invaluable input and suggestions on this work. This plementation: Reconciling Responsiveness With Per-
work was funded in part by NSF grant Nos. CAREER- formance. In Conference on Object-Oriented Program-
CNS-0546737, ST-HEC-0444412, and EHS-0209195 ming, Systems, Languages, and Applications (OOP-
and grants from Intel, Microsoft, and UCMicro. SLA), Oct. 1994.
[16] A. Hosking, J. E. Moss, and D. Stefanović. A Com-
References parative Performance Evaluation of Write Barrier Im-
plementations. In Conference on Object-Oriented
[1] B. Alpern, C. R. Attanasio, J. J. Barton, M. G. Burke, Programming, Systems, Languages, and Applications
P.Cheng, J.-D. Choi, A. Cocchi, S. J. Fink, D. Grove, (OOPSLA), Oct. 1992.
M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. [17] K. Ishizaki, M. Kawahito, T. Yasue, H. Komatsu, and
Mergen, T. Ngo, J. R. Russell, V. Sarkar, M. J. Serrano, T. Nakatani. A Study of Devirtualization Techniques for
J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srini- a Java Just-In-Time Compiler. In Conference on Object-
vasan, and J. Whaley. The Jalapeño Virtual Machine. Oriented Programming, Systems, Languages, and Ap-
IBM Systems Journal, 39(1):211–221, 2000. plications (OOPSLA), Oct. 2000.
[2] M. Arnold, S. Fink, D. Grove, M. Hind, and P. Sweeney. [18] H. Lieberman and C. Hewitt. A Real-Time Garbage
Adaptive Optimization in the Jalapeño JVM. In Pro- Collector based on the Lifetimes of Objects. Communi-
ceedings of the ACM SIGPLAN Conference on Object- cations of the ACM, 26(6):419–429, 1983.
Oriented Programming Systems, Languages, and Appli- [19] M. Paleczny, C. Vick, and C. Click. The Java
cations (OOPSLA), Oct. 2000. HotSpot(TM) Server Compiler. In USENIX Java Vir-
[3] S. Blackburn and K. McKinley. In or Out? Putting tual Machine Research and Technology Symposium
Write Barriers in Their Place. In International Sympo- (JVM’01), Apr. 2001.
sium on Memory Management (ISMM), June 2002. [20] N. Sachindran, J. Eliot, and B. Moss. Mark-copy: Fast
[4] S. M. Blackburn and A. L. Hosking. Barriers: Friend or Copying GC with less Space Overhead. In Conference
Foe? In International Symposium on Memory Manage- on Object-Oriented Programming, Systems, Languages,
ment (ISMM), Oct. 2004. and Applications (OOPSLA), Oct. 2003.
[5] M. Burke, J. Choi, S. Fink, D. Grove, M. Hind, [21] S. Soman, C. Krintz, and D. F. Bacon. Dynamic
V. Sarkar, M. Serrano, V. Shreedhar, H. Srinivasan, and Selection of Application-Specific Garbage Collectors.
J. Whaley. The Jalapeño Dynamic Optimizing Compiler In International Symposium on Memory Management
for Java. In ACM JavaGrande Conference, June 1999. (ISMM), Oct. 2004.
[6] B. Cahoon and K. McKinley. Data Flow Analysis for [22] SpecJVM’98 Benchmarks.
Software Prefetching Linked Data Structures in Java http://www.spec.org/osg/jvm98.
Controller. In International Conference on Parallel Ar- [23] T. Suganuma, T. Yasue, and T. Nakatani. A Region-
chitectures and Compilation Techniques (PACT), Sept. Based Compilation Technique for a Java Just-In-Time
2001. Compiler. In Conference on Programming Language
[7] C. Chambers and D. Ungar. Making Pure Object-
Design and Implementation, June 2003.
Oriented Languages Practical. In Conference on Object-
[24] D. Ungar. Generation scavenging: A non-disruptive
Oriented Programming, Systems, Languages, and Ap-
high performance storage recalamation algorithm. In
plications (OOPSLA), Oct. 1991.
Software Engineering Symposium on Practical Soft-
[8] D. Detlefs and O. Agesen. Inlining of Virtual Methods.
ware Development Environments, Apr 1992.
In European Conference on Object-Oriented Program-
[25] P. Wilson and T. Moher. A Card-Marking Scheme for
ming (ECOOP), June 1999.
[9] A. Diwan, E. Moss, and R. Hudson. Compiler Support Controlling Intergenerational References in Generation-
for Garbage Collection in a Statically Typed Language. Based Garbage Collection on Stock Hardware. SIG-
In ACM Conference on Programming language design PLAN Notices, 24(5):87–92, 1989.
and implementation, June 1992.