Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Retrieve

Download as pdf or txt
Download as pdf or txt
You are on page 1of 16

DOI: 10.1111/cgf.

14771

EUROGRAPHICS 2023 / K. Myszkowski and M. Nießner COMPUTER GRAPHICS forum


(Guest Editors) Volume 42 (2023), Number 2

Non-linear Rough 2D Animation using Transient Embeddings

Melvin Even, Pierre Bénard , Pascal Barla

Inria, Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, France

(a)

(b) (c)

Figure 1: Our rough animation system supports the inbetweening of complex special effects with multiple topological events, here a flame
splitting into smaller components. In (a), we show the input key drawings traced over a reference from Gilland’s book [Gil09], while in
(b) we show their decomposition into transient embeddings. Even though we use a unique color per embedding throughout the animation
for visualization, each embedding only exists between a pair of keyframes in this example. This allows the handling of topological changes,
which occur at the third and fifth keyframes. In (c), we display in magenta a subset of the inbetween frames generated by our animation
system in real-time. Please see the supplemental results video for the full sequence.

Abstract
Traditional 2D animation requires time and dedication since tens of thousands of frames need to be drawn by hand for a typical
production. Many computer-assisted methods have been proposed to automatize the generation of inbetween frames from a set
of clean line drawings, but they are all limited by a rigid workflow and a lack of artistic controls, which is in the most part due
to the one-to-one stroke matching and interpolation problems they attempt to solve. In this work, we take a novel view on those
problems by focusing on an earlier phase of the animation process that uses rough drawings (i.e., sketches). Our key idea is to
recast the matching and interpolation problems so that they apply to transient embeddings, which are groups of strokes that only
exist for a few keyframes. A transient embedding carries strokes between keyframes both forward and backward in time through
a sequence of transformed lattices. Forward and backward strokes are then cross-faded using their thickness to yield rough
inbetweens. With our approach, complex topological changes may be introduced while preserving visual motion continuity.
As demonstrated on state-of-the-art 2D animation exercises, our system provides unprecedented artistic control through the
non-linear exploration of movements and dynamics in real-time.
CCS Concepts
• Computing methodologies → Graphics systems and interfaces; Animation;

1. Introduction their dynamics (e.g., their speed and acceleration) through the tim-
ing & spacing of drawings [JT95, Wil01]. Even though the rough
Traditional 2D animation requires a lot of planning, not only at the
animation itself is not directly visible in the final movie, its impact
level of storyboards, but also for the animation itself where rough
on motion design is vividly retained, since it serves as a guide for
drawings (i.e., sketches) are used to test out the motion of different
“inbetweeners” – the artists who draw all the intermediate frames.
characters or special effects (e.g., water, smoke). These tests aim
at defining the trajectories of the characters or objects, as well as

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
412 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

In this paper, we introduce a system for the design and explo- dings are presented in Section 5, featuring non-linear control over
ration of rough 2D animations; a problem which, to the best of timing & spacing and direct artistic control over trajectories be-
our knowledge, has never been addressed in previous work. Our tween and across keyframes. Implementation details including a
main contribution is in the assembly and adaptation of a set of ex- novel real-time ARAP registration technique for vector strokes are
isting techniques for the previsualization of 2D animations. The exposed in Section 6. We demonstrate that our system allows users
main goal is to provide real-time feedback at intermediate frames to produce complex 2D animations in Section 7, by reproducing
between rough key drawings, to both significantly speed up the an- typical animation exercises [Wil01], such as walk cycles, special ef-
imation process and allow artists to experiment with different cre- fects, articulated motion, and some principles of animation [JT95],
ative alternatives. This is useful not only for experienced animators, such as anticipation and follow-through, or squash and stretch.
who may try variations in early tests for discussions with art direc-
tors and quickly converge to final rough animations to pass down
2. Related work
to inbetweeners; but also for animation students, who may benefit
from the ability to observe interactively the look-and-feel of differ- The design of computer-aided 2D animation systems dates back to
ent animation choices. As described in supplemental material, we the inception of Computer Graphics in the late ’60s and early ’70s
relied on an observational study of a professional animator at work [MIT67, Bae69, BW71]. As already observed by Catmull [Cat78],
followed by interviews to guide the design of our system. By work- automatic inbetweening is a central problem tackled by most of
ing at the rough animation stage, we leverage the fact that draw- such systems, and yet – more than forty years later – current com-
ings are sketchy and the global perception of movement is more mercial solutions [Ado, Too, CAC, Com] often remain too limited
important than the appearance of the strokes themselves that will or time consuming for most use cases in production. In addition,
eventually be redrawn at the cleaning stage. However, it brings two by focusing on inbetweening of final clean line drawings, we be-
fundamental challenges. First, artists may create drawings through lieve that those systems and most previous work in academia have
very different workflows such as “shift-and-trace” (drawings are missed the real potential benefit of computer assistance, that is, us-
traced over deformations of previous ones) or “pose-to-pose” (all ing the words of Durand [Dur91], to “boost user creativity by al-
key drawings are created in advance then interpolated). Second, the lowing them to concentrate on the most interesting part of their
drawings themselves are most often composed of different numbers work”: the design and exploration of motion.
of strokes and routinely differ in their number of parts.
The key idea of our approach is to shift focus from the animation
As described by Fekete et al. [FBC∗ 95], automated inbetween- of individual strokes to the animation of groups of strokes that are
ing systems can be divided in two main families: those based on only defined between a pair of keyframes, which we call transient
templates or embeddings (e.g., [BW75]), and those relying on ex- embeddings. Nevertheless, it requires to revisit the two main stages
plicit correspondences between strokes (e.g., [MIT67]). The for- of explicit correspondence techniques: matching and interpolation.
mer family is mostly well-suited for “cut-out” animations since the
movement of the embedded objects or characters is restricted by Matching. Most early methods require the user to manually iden-
the motion of their template (e.g., skeleton, control polygon, cage) tify correspondences between the strokes of consecutive keyframes
whose topology is usually fixed throughout the animation. Explicit and do not handle occlusions or topological changes [MIT67,
correspondence systems are more flexible as the stroke-to-stroke Bae69, Ree81, Dur91]. More recent techniques support such fea-
mapping is transient, changing between each pair of keyframes. tures using manually populated 2.5D [DFSEVR01, RID10] or
However they are restricted to “tight inbetweening” of clean line space-time [DRvdP15] data structures. Despite their appeal, these
drawings due to the challenge (or chore) of matching complex net- approaches require ad hoc, rather constraining drawing represen-
works of strokes. In this work, we propose transient embeddings to tations which are not suitable for rough drawings. To partially au-
keep the best of both approaches, hence allowing template-based tomatize the stroke correspondence process, a large body of work
animation of rough drawings with topological changes. In practice, represents the drawings as a graph of strokes and try to match
this requires adapting two common problems to deal with transient those graphs at subsequent keyframes [Kor02, WNS∗ 10, LCY∗ 11,
embeddings: the matching problem where two drawings must be YBS∗ 12, CMV17, Yan18, YSC∗ 18, MFXM21]. They differ by the
registered, here with drawings having different numbers of strokes; graph matching algorithm they employ, and the way users inter-
and the interpolation problem, where the movement from one key act with the system to guide or correct correspondences, espe-
drawing to the next must be generated while providing flexible cially when strokes appear or disappear. To resolve occlusions in a
and interactive artistic control over timing, spacing and trajecto- user-controllable fashion, Jiang et al. [JSL22] introduce “boundary
ries. A key feature of our approach is to enable changes of topol- strokes”, i.e., strokes with an occluding side that acts as occluding
ogy at keyframes (i.e., key drawings may have different numbers of surfaces. However, none of these methods can handle rough draw-
embeddings), while ensuring visual continuity through constrained ings with an highly dissimilar number of strokes per keyframe, and
trajectories that persist over multiple keyframes. despite recent advances in rough sketch cleanup [YVG20], no algo-
rithm is currently able to produce a sequence of clean line drawings
Our main contribution is a novel animation system that relies
that can be automatically inbetweened.
on transient embeddings to provide full non-linear artistic control
at the rough animation stage, as described in Section 3. Methods Alternatively, some methods aim at estimating region (rather
for matching embeddings at keyframes are introduced in Section 4: than stroke) correspondences between consecutive frames based on
they work with shift-and-trace and pose-to-pose workflows, or any their appearance (color, shape, distance) [Xie95, MSG96, SBv05,
combination of them. Methods for interpolating between embed- dJB06,BBA09,ZHF12,LMY∗ 13] and motion features [ZLWH16],

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 413

but they are limited to polygonal shapes or cel animations (i.e., Whited et al. [WNS∗ 10] present an interpolation scheme that pro-
mostly flat color regions with clean line boundaries). Taking in- duces arc trajectories for a full graph of strokes. It first computes
spiration from As-Rigid-As-Possible (ARAP) shape deformation motion paths for stroke endpoints along logarithmic spirals, and
techniques [IMH05, WXXC08], Sýkora et al. [SDC09] present an then deform the intermediate stroke vertices using intrinsic interpo-
image registration algorithm that decouples the matching resolution lation [SGWM93] followed by curve fitting and a tangent-aligning
from the image complexity by embedding it into a square lattice. warp to ensure continuity between adjacent strokes. The trajectory
Noris et al. [NSC∗ 11] use this method to estimate a global warp of any stroke vertex can be edited, albeit without considering its
between two drawings of an existing rough animation, abstract- dynamics. This scheme was later used by Noris et al. [NSC∗ 11] for
ing the input strokes by their rasterized distance fields. Then, each generating smooth stroke trajectories between pairs of strokes.
stroke of the first drawing, deformed by the ARAP transformation,
An alternative solution to minimize shape distortion is to rely
is matched with the most similar stroke in the second one. We also
on ARAP interpolation [ACOL00, XZWB05] of 2D embeddings
embed strokes into square lattices, but extend the registration algo-
of the drawings. The interpolated trajectories can be controlled
rithm to directly take as input vectorial strokes. Furthermore, we
through point and vector constraints [BBA08, KHS∗ 12] or even a
make the assumption that stroke-to-stroke correspondences are not
full skeleton [YHY19]. However, the boundary polygon of those
required to depict motion in rough animations, which we demon-
embeddings must be compatible across keyframes and put into cor-
strate in our results.
respondence, and a compatible triangulation of their interior must
Closest to our work, Xing et al. [XWSY15] present an interac- also be built. Baxter et al. [BBA09] describe the most relevant tech-
tive system that combines a global shape similarity metric with an niques to solve this challenging problem along with their own so-
embedding-free ARAP deformation model [SSP07] to match an lution. Zhu et al. [ZPBK17] extend these approaches to handle ex-
existing drawing with a new set of hand-drawn guidelines. We dis- treme shape deformations and topological changes, but it requires
cuss the benefits of our explicit embeddings in Section 8.1 and pro- significant manual intervention and involves an expensive numeri-
vide visual comparison in the supplemental results video. cal optimization that prevents its use in an interactive system.
Following the current trend in computer science, learning-based Yang [Yan18] combines the strokes deformation technique of
techniques [Yag17,NHA19,LZLS21,SZY∗ 21] have also been pro- Whited et al. [WNS∗ 10] with a simpler embedding, called “con-
posed to estimate per-pixel correspondence between two raster text mesh”, that better preserves the global layout of the stroke
clean line drawings. In the work of Casey et al. [CPL21], line- network. He presents an automatic construction algorithm of these
enclosed segments are first extracted from the two drawings, and compatible meshes based on the matched strokes, and an edge-
then correspondences between segments are estimated using a com- based rigid interpolation technique inspired by the method of
bination of convolutional and transformer neural networks. Extend- Igarashi et al. [IMH05] which is robust to degenerated configura-
ing such approaches to rough drawings, whose style may consider- tions (e.g., collapsing edges) and may be constrained to follow a
ably vary from one artist to another, seems extremely difficult for given trajectory. It is however unclear how such “context meshes”
such data-driven approaches. could be built for rough drawings. Instead, we use even simpler lat-
tice embeddings, which are compatible between two keyframes by
Interpolation. Once the key drawings have been put into corre- construction, but do not need to extend further in time.
spondence, inbetween frames can be generated by interpolation.
Dvorožnák et al. [DLKS18] use similar embeddings to build de-
As already noted by Burtnyk et al. [BW75], linear interpolation
formable puppets, but since those are connected at fixed junctions
and thus linear trajectories sampled at uniform rates do not produce
driven by a skeleton, their results suffer from the “cut-out” look-
natural motion in the great majority of cases.
and-feel. The animation system of Bai et al. [BKLP16] integrates
To offer maximum artistic control, the animation system of handle-based shape deformations with example-based simulations
Reeves [Ree81] allows the user to specify the trajectory and dy- to interpolate drawings embedded into triangular meshes. This ap-
namics of a set of “moving points” spanning multiple keyframes. proach manages to reproduce many of the principles of anima-
This effectively decomposes the full 2D+t space of the animation tion [JT95], supports manual topological changes and local control
into a network of Coons patches, into which interpolation can be of the dynamics, but user interaction is restricted to handles manip-
performed independently but with continuity at boundaries. How- ulation, hence once again following the “cut-out” metaphor rather
ever heuristics are required to complete the patch network, and user than hand-drawn animation.
manipulation of moving points in space and time may be laborious.
For image-based approaches, interpolation turns into an image
Kort [Kor02] models trajectories of stroke vertices by quadratic morphing (i.e., deformation and blending) problem. Many solu-
splines. The user can correct these paths when needed and spec- tions have been proposed for photographs (e.g., [FZP∗ 20,PSN20]),
ify their spacing. Since this simple interpolation scheme does not cartoon animations [LZLS21, SZY∗ 21, CZ22] and, closest to our
take the shape of the strokes into account, it may lead to local or inputs, concept sketches [ADN∗ 17]. Yet, rough drawings have a
global distortions. Sederberg et al. [SGWM93] introduce an in- very specific style which requires preserving the distribution, spa-
trinsic interpolation technique which minimizes shape distortion. tial continuity and color or gray-level intensity of the strokes. Pre-
Similar approaches [FTA05, SZGP05] attempt to preserve local vious approaches are unlikely to satisfy all three criteria. In this
differential quantities (Laplacian coordinates or edge deformation work, we do not attempt to solve this problem, and use simple
gradients). But those three methods only apply to a single poly- cross-fading of stroke thicknesses that turns out to be sufficient for
line. Motivated by classical 2D animation books [JT95, Wil01], motion previsualization in practice.

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
414 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

3. Animation system (a)

Most computer-aided 2D animation systems (e.g., [TVP, Too, Ado,


CAC]) provide a timeline which displays the timing (i.e., frame
number) and the layers in a simplified and systematic manner. Lay-
ers are organized in a stack defining the compositing order of the
drawings on the canvas. Each layer is animated independently and (b)
populated by keyframes that may hold a drawing and be “exposed”
(i.e., repeated) over multiple frames.
The design of our animation system is guided by extended dis-
cussions with a professional 2D animator and 2D animation soft-
ware developers. As detailed in supplemental material, we arrived Start Breakdown End
at the following conclusions and thus guidelines: keyframe keyframe keyframe

G1: Contours in rough drawings are depicted with multiple strokes


which implies working with groups of strokes rather than indi- (c) *
vidual elements; the artist must be able to redefine the number of
groups according to the complexity of the motion.
Invertible
G2: Topological changes, such as parts of the drawing appear- transformations
ing, disappearing, splitting or merging, are explicitly drawn at
keyframes, hence do not need to be automatically in-betweened. Figure 2: A transient embedding is a sequence of transformed lat-
G3: Artists alternately use pose-to-pose and shift-and-trace work- tices carrying strokes between keyframes. (a) A first set of strokes
flows that must be both supported; they are used to provide indi- (in red), stored in the start keyframe, is propagated forward via a
cations to inbetweeners, but their creative flow should be inter- transformation, while another set of strokes (in green), stored in
rupted as least as possible. the end keyframe, is propagated backward via the inverse transfor-
G4: Control over timing, spacing and interpolation trajectories mation. Forward and backward strokes are cross-faded in between.
is crucial to design motion, yet extremely complex and time- (b) An additional inbetween lattice transformation may be added
consuming since all intermediate frames must be redrawn. through a breakdown keyframe, which holds a single set of strokes
(in blue) that are propagated both backward and forward in time.
Based on these guidelines, we extend the structure of a classic (c) Symbolic representation of a transient embedding: each type of
animation system to work on groups of strokes, which we call em- keyframe is represented by a different symbol, which are connected
beddings, as detailed in the remainder of this section. Embeddings by segments that represent lattice transformations. The regular ex-
act as units of motion (e.g., the forearm of a character), which may pression notation [· · · ]∗ indicates that breakdowns are optional.
be refined throughout the animation process when needed (G1).
Each embedding is transformed from its start keyframe to the next,
carrying along its strokes so that they come into alignment with a
different subpart of the next key drawing. Embeddings may thus
be said to be transient in the sense that they do not last past the
next keyframe, where new embeddings take over the animation
avoiding popping artifacts when transitioning from one embedding
process, possibly with a different topology (G2). This novel ani-
to the next. But they may also differ significantly from the strokes
mation structure is well adapted to existing artistic workflows (G3)
found in the next keyframe, hence allowing topological changes
and grants new non-linear controls (G4), as described in Sections 4
to occur, as demonstrated in Section 3.3. However, in our system,
and 5 respectively.
topological changes never occur between keyframes.

3.1. Transient embeddings A same embedding may be used over multiple keyframes, as
shown in Figure 2(b). To make this possible, we introduce break-
In its simplest form, as shown in Figure 2(a), an embedding is de- down keyframes, inspired by the traditional animation technique of
fined by a pair of lattices with the same topology at two keyframes. the same name [Wil01]. In our system, they amount to storing a
The lattice at the start keyframe holds strokes (a subset from the new transformed lattice in the embedding, along with an additional
corresponding key drawing) that are propagated forward in time. A set of strokes that is propagated both backward and forward in time.
second set of strokes is stored in a transformed lattice at the end
keyframe, which is lined up in time with the next keyframe. How- Figure 2(c) abstracts the structure of a transient embedding with
ever, the end keyframe itself is never displayed. The transformation a simple sequence of symbols: a square for the start keyframe, cir-
between the two lattices must be invertible so that strokes from the cles for (optional) breakdown keyframes, and a crossed circle for
end keyframe can be propagated backward in time. The two sets of the end keyframe. In effect, the end keyframe is a special case of
forward and backward strokes are cross-faded using stroke thick- breakdown that is only propagated backward in time, and is not it-
ness instead of opacity. Such a representation opens up a number self displayed. Segments between symbols represent both forward
of possibilities. For instance, backward strokes may be obtained and backward embedding transformations, which may be evaluated
by copying a subset of the strokes in the next key drawing, hence at any time step to yield cross-faded strokes.

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 415

(a) Split
(a)

(b) *
Remove
(c)
(b)
Figure 3: In our animation system, the timeline is segmented
into intervals whose boundaries are shown with tall ticks, while Figure 4: Non-linear editing of the animation structure is made
short ticks delimit frames. An animation is created by (a) adding possible by special updates of the transient embeddings. Splitting
transient embeddings that span one or more intervals, with their an interval in two is done by (a) inserting a breakdown keyframe in
keyframes lined up at the beginning of each interval. Timing is read- all involved embeddings, which may optionally be converted into
ily modified by (b) automatically updating embeddings when inter- an end keyframe followed by a new, automatically generated em-
val boundaries are moved. Edits are local since (c) the removal of bedding. Removing an interval yields (b) three different types of
an embedding does not affect other embeddings. results depending on the type of keyframe: extension of the embed-
ding, removal of a breakdown, or removal of the embedding.

3.2. Animation structure


With this formal definition of a single transient embedding at hand,
we must now formalize the global animation structure used in our
system. This is required for a non-linear animation system as we Eb
need to ensure that a valid animation structure is maintained irre- Eg
spective of the order in which operations are performed on it. Er

The first structural constraint is that the timeline must be seg-


mented into contiguous intervals, which act as containers for em-
beddings. Intervals may hold zero, one or more embeddings. The Figure 5: We illustrate our animation system on a simple example
keyframes of all embeddings in the same layer must be lined up in made of three intervals. A pair of green Eg and red Er embeddings
time with the first frame of an existing interval, as shown in Fig- meet a blue embedding Eb at the second keyframe, where the topol-
ure 3(a). Interval boundaries may then be modified, carrying with ogy of the drawing changes. This is achieved by using backward
them all the embeddings they store, as shown in Figure 3(b). This strokes for Eg and Er (dashed lines) that are different from forward
amounts to modifying the timing of an animation, in which case strokes for Eb . The latter spans the two remaining intervals via a
embeddings must be evaluated at different time steps. An additional breakdown keyframe, which enables additional deformation. The
advantage of this structure is that removing an embedding does not end keyframe of Eb is empty and strokes are faded out. The gray
affect the rest of the animation as illustrated in Figure 3(c), hence triangles indicate the frames at which the above embeddings are
making the animation process globally non-destructive. evaluated, both at and in between keyframes.

Intervals that do not contain any embedding may be safely


deleted or split into two intervals. New intervals may similarly
be added before or after existing ones. A more complex situation 3.3. A simple example
arises when the same operations are applied to an interval that con-
We conclude this section with a simple example animation,
tains one or more embeddings. When splitting an interval in two,
shown in Figure 5. It demonstrates several structural properties
each of its stored embeddings must also be split at the same frame
of our animation system: a topological change with different for-
to preserve the aforementioned keyframe alignment constraint. As
ward/backward strokes, a breakdown keyframe, and a fade out. End
shown in Figure 4(a), we do so by inserting a breakdown keyframe
keyframes are shown as dashed lines since they are not displayed,
for each involved embedding. These breakdown keyframes may be
but their strokes are propagated backward to be used for interpola-
converted into end keyframes if needed, which results in the auto-
tion. We also rely on this example for illustration purposes in Sec-
matic creation of a new embedding as shown in the second row of
tions 4 and 5, and in the supplemental demo video.
the figure. When removing a key interval, all the involved embed-
dings must be updated according to three different rules, depending Such a simple example leaves open a number of questions that
on the type of keyframe involved. This is illustrated in Figure 4(b): we address in the next two sections. They precisely correspond to
an end keyframe is not removed but merely extended to the start the matching and interpolation problems recast on transient embed-
frame of the next interval; removing a breakdown keyframe re- dings: How are embeddings deformed at keyframes so that back-
quires reconnecting the start or previous breakdown keyframe to ward and forward strokes are put in alignment, yielding visually
the next breakdown or end keyframe; when a start keyframe is re- continuous animations (Section 4)? How are embeddings interpo-
moved, there is no other choice than deleting the entire embedding lated between keyframes, while providing control over motion dy-
since there is no key drawing to rely on anymore. namics, trajectories and smoothness (Section 5)?

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
416 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

T Id

Id Shift T Trace Shift Trace Shift


E0 Id
T
E0 E0
E1
E0
Eb start Eb breakdown Eb end
Figure 6: With the shift-and-trace workflow, an initial embedding keyframe keyframe keyframe

E0 is first manually modified through a target transformation T ⋆


– the shift step. The embedded strokes (symbolized by the light blue Figure 7: The lattice of Eb is first transformed using ARAP de-
color) are then copied either to a breakdown keyframe, or to a new formation from the start to the breakdown keyframe. New, slightly
embedding E1 – the trace step. different strokes are then traced in the breakdown. Finally, ARAP
deformation is once again used to yield the lattice transformation
in the end keyframe, which is tagged as fading out.
4. The matching problem
Recall that an embedding in our system corresponds to a unit of Id Id
motion. By default, all strokes in a key drawing are part of a single Id Register T
embedding, but this may be refined at any time by the user to assign E1 E1
different subsets of strokes to different embeddings. This choice E0 E0
must be made based on which strokes of the current keyframe are
expected to move together to the next keyframe, i.e., based on mo- Figure 8: In the pose-to-pose workflow, the initial embeddings E0
tion complexity rather than the complexity of the drawing itself. and E1 are provided as input, each storing its own set of embedded
As shown in Figure 2, each embedding stores its strokes in a strokes (symbolized by light green and light blue colors). A single
square lattice, which is built axis-aligned at the start keyframe. registration step is used to automatically find a target transforma-
Recall that the lattice does not change topology throughout a tion T ⋆ that aligns the embedded strokes of E0 with a subset of
transient embedding, but may undergo transformations at break- strokes in E1 , and to copy the latter to the end keyframe of E0 .
down and end keyframes. The main goal of the matching prob-
lem is then to establish lattice transformations that put embed-
ded strokes in alignment with strokes at the next keyframe. At the end of the shift-and-trace process, we end up with a new
Such a target transformation T ⋆ consists t1 interval in a configuration similar to the one we started with; the
in a discrete vector field {⃗ti⋆ } defined t2 process may thus be repeated to produce a longer animation se-
t3
on lattice corners (see inset figure). We t4 quence. The embedded strokes of any keyframe may be redrawn
present two basic matching approaches, t5
t6 at any time during that process, allowing artists to add visual com-
inspired by traditional animation work- t7 plexity. Redrawing strokes in a breakdown or end keyframe is con-
flows: shift-and-trace (Section 4.1), and t8
strained to lie within the deformed lattice, since the new strokes
pose-to-pose (Section 4.2). must be propagated forward and/or backward using the same lat-
tice topology. Redrawing strokes in the start keyframe of a new
4.1. Shift-and-trace embedding does not impose any constraint since the newly created
lattice can be trivially extended to cover the new strokes, which are
The main idea of the shift-and-trace workflow is to deform a lattice
only propagated forward.
to the next keyframe (the “shift” step), and then to redraw a similar
set of strokes over the deformed, embedded ones (the “trace” step). The shift-and-trace workflow is illustrated in Figure 7 on the blue
Any combination of tools may be used to yield the transformation embedding Eb of Figure 5, which spans two intervals.
T ⋆ . We have implemented classic linear transformations (transla-
tion, rotation and scaling), as well as an ARAP deformation tool
4.2. Pose-to-pose
similar to the “Warp tool” in TVPaint Animation [TVP].
The pose-to-pose workflow instead starts from an existing sequence
The shift-and-trace workflow is illustrated in Figure 6. An em-
of drawings (i.e., “poses”), each stored in its respective keyframe,
bedding E0 is subject by default to an identity transformation, and
and registers subsets of strokes from one keyframe to the next. Con-
acts on a set of strokes, symbolized by the light blue color. The ini-
sequently, the target transformation T ⋆ is not manually specified by
tial configuration is thus a static drawing exposed over the whole
the artist but automatically inferred from a pair of drawings.
interval, similarly to traditional animation systems. During the shift
step, a target transformation T ⋆ is applied to E0 , but no stroke is The pose-to-pose workflow is detailed in Figure 8. A sequence
stored yet in its end keyframe. The trace step creates a new interval of two embeddings E0 and E1 is shown, each with an identity
and populates it in one of the following two ways: the embedded transformation by default, but acting on different sets of strokes
strokes may either be duplicated to a new breakdown keyframe, symbolized by different colors. As before, the initial configura-
thus extending E0 in time; or a new embedding E1 may be created, tion is identical to the one found in traditional animation systems:
with embedded strokes separately copied to the end keyframe of E0 a sequence of static drawings exposed over their whole interval.
and the start keyframe of E1 . Next, a vector registration algorithm – adapting the method of

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 417

Register Redraw

Eg

Figure 10: Applying the default ARAP interpolation to the simple


Eb
Eg end Eb start example of Figure 5 produces a reasonable initial result, but lacks
keyframe keyframe
control over spacing and trajectories.
Figure 9: A subset of strokes is first selected in Eb to be matched by
Eg . Then registration automatically finds a lattice transformation
for Eg that aligns strokes in both embeddings, and matching strokes by T (t) = V (t) −V0 , with V (t) obtained by ARAP interpolation be-
in the start keyframe of Eb are copied to the end keyframe of Eg . We tween V0 and V1 (see Appendix A for details). Similarly, the inverse
then partially redraw these strokes. lattice transformation is T (t)−1 = V (1 − t) −V1 .
With this formulation, we obtain results such as in Figure 10,
where embedded strokes are transformed forward and backward
Sỳkora et al. [SDC09] to vector drawings to yield real-time perfor- according to T and T −1 respectively. A first limitation of this de-
mance (see Section 6) – is used to find the target transformation T ⋆ fault interpolation behavior is that the spacing of interpolated draw-
that best aligns the strokes embedded in E0 with any user-selected ings is not readily controlled. We introduce control over spacing in
subset of strokes in E1 , which are immediately copied inside the Section 5.1 through a formalization of the “spacing chart”, which
end keyframe of E0 . The transformed forward strokes of E0 may is ubiquitous in traditional 2D animation. A second limitation is
alternatively be copied to its end keyframe. that trajectories are only defined per embedding, which may lead
to velocity discontinuities across keyframes, or to dissociation of
Once again, this process ends up with an interval in a configura- embeddings between keyframes. We address these issues in Sec-
tion similar to the one it started with, such that registrations may be tion 5.2 by sharing trajectories across embeddings and keyframes.
applied over and over again to match pairs of drawings over several
keyframes. Shift-and-trace and pose-to-pose workflows may even
be interleaved since they both end up in the same configuration. 5.1. Spacing charts
Moreover, the transformation T ⋆ found through registration may Providing control over spacing through interpolation is cru-
optionally be adjusted manually or semi-automatically – thanks to cial to convey motion dynamics. In existing animation software
the real-time performance of our vector registration algorithm – at (e.g., [CAC]), interpolated drawings may be manually positioned
any time during that process, while the strokes in any keyframe on a per-frame basis, which is inspired by charts in traditional an-
might be redrawn as before. imation [Wil01]. The process may be tedious, even though presets
The pose-to-pose workflow is illustrated in Figure 9 on the em- (e.g., “ease in / ease out”) can help. Furthermore, it does not grant
beddings Eg and Eb of Figure 5 (a similar process is used to match non-linear control over the animation since it must be redone af-
Er with Eb ), with an additional redrawing step. ter each retiming. Alternatively, animation curves may be directly
edited (e.g., [Ado,Too]), but this looses the intuitive appeal of spac-
ing charts. We keep the best of both worlds by formalizing the spac-
5. The interpolation problem ing chart in a way that preserves editing through ticks, while allow-
ing automatic updates after retiming.
In our system, interpolation is available at all times. As shown in the
demo video, after little interaction (sometimes a single click) our A spacing chart is defined as a monotonically increasing 1D
system provides instant interpolation results. The process is trivial remapping function S : [f0 , f1 ] → [0, 1], where f0 and f1 are natu-
for newly created embeddings since their lattice is static (e.g., Id ral numbers locating two contiguous keyframes on the timeline. As
arrows in Figures 6 and 8). As soon as a target transformation T ⋆ illustrated in Figure 11(a-b), we let users manually adjust each tick
is introduced, we must find an invertible family of transformations value S(f) for any discrete interpolated frame f ∈ {f0 , ..., f1 } ⊂ N,
T (t) that satisfies T (0) = {⃗0} and T (1) = T ⋆ . We choose ARAP while enforcing S(f − 1) < S(f) < S(f + 1). In contrast, the spacing
interpolation [ACOL00] for T (t) since it provides a natural default function S is defined over the real range [f0 , f1 ]. In practice, we use
behavior in general, while enabling artistic control through addi- a monotone cubic piecewise function [FC80] for S. It interpolates
tional constraints. This should not be mistaken with the ARAP de- the prescribed S(f) at discrete frame values, with tangents automat-
formation or registration tools of Section 4. Indeed, matching and ically computed to ensure smoothness and monotonicity.
interpolation are decoupled in our approach.
The main advantage of this smooth continuous representation
In practice, we use the symmetric formulation of Bax- is non-linear editing. When an interval is extended for retiming,
ter et al. [BBA08] which is fast to evaluate, easy to implement, and we linearly stretch S over the new [f′0 , f′1 ] range to get S′ , which
offers control through linear constraints. We triangulate the axis- is then evaluated at discrete frame positions f ∈ {f′0 , ..., f′1 } ⊂ N to
aligned lattice and store the initial and transformed coordinates of yield the new chart ticks S′ (f), as demonstrated in Figure 11(b-c).
its corners in two matrices V0 and V1 . By construction, we have Other timeline operations, such as those depicted in Figure 4, in-
T ⋆ = V1 − V0 , and the time-varying transformation is then defined duce straightforward operations on spacing functions, since these

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
418 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

(a) 1 2 3 4 5 (b) 1 2 3 4 5 (c) 1 2 34 5 6 7 8 9

5 1 5 1 9 1
Id S S'
4
Spacing Retiming 8
4 7
3
6
2 3 5
4
2 3
2
1 0 1 0 1 0
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 6 7 8 9

Figure 11: We illustrate our spacing function on the green embedding Eg . By default, we use (a) the identity spacing function, resulting in a
linear time parameter t. It may be modified through (b) user-provided ticks in the vertical spacing chart at left, which results in a new spacing
function S (in red). After retiming (c), S is stretched to S′ and new ticks on the spacing chart (in red) are automatically recomputed.

are defined on frame ranges. Splitting an interval merely results in (a)


splitting a spacing function in two functions. Removal of an end
keyframe results in an extension of the embedding which is equiv-
alent to a retiming, whereas removal of a breakdown keyframe re-
sults in the merging of the two neighboring spacing functions.
The spacing function outputs a normalized time value that is (b)
used to compute V (S(f)), the interpolated lattice at frame f. By con-
struction, V (S(f0 )) = V0 and V (S(f1 )) = V1 . Note that even a linear
spacing function may not lead to a transformation that behaves ex-
actly linearly between V0 and V1 , even though they are close in
practice. Indeed, lattice corners will not move at constant speed
Figure 12: Constrained trajectories help control inbetweening.
owing to the global optimization involved in ARAP interpolation.
In (a), we use one constraint for Eg and one constraint for Er . Tra-
However, constrained trajectories may be added if a precise local
jectories for the same points in Eb are also visualized with dashed
control over motion speed is required, as explained next.
lines. Since each embedding is interpolated separately, we can ob-
serve that trajectories are not G1 -continuous (pointed out by ar-
5.2. Constrained interpolation rows). In (b), we activate the same constraint points in Eb and
align the tangent of their trajectory starting points with the ones
Besides spacing, the default ARAP interpolation may be controlled
in Eg and Er , which restores continuity.
by linear constraints, CV = D(t) (see Appendix A). The C ma-
trix identifies which point of the embedding should be subject to a
constraint through time. Any point inside the lattice may be used
as a constraint (even away from drawn strokes) as it can be ex- through D(t) = D(S(f)), where f is the current frame and S the spac-
pressed with barycentric coordinates in a cell. The D(t) matrix pro- ing function of the embedding. The geometry of the fitted trajectory
vides trajectories for the identified constraints, hence locally driv- may then be edited to drive lattice motion explicitly, as illustrated
ing lattice motion. In the formulation of Baxter et al. [BBA08], in Figure 12(a) on the red and green embeddings of Figure 5. Since
these constraints are handled by Lagrange multipliers. ARAP fac- the curve endpoints are fixed, only the tangents of the trajectory
torization only needs to be recomputed when constraints are added may be modified.
or removed (through C); whereas trajectory editing (through D(t))
Constrained trajectories may be trivially chained across break-
works in real time.
down keyframes since the lattice topology is maintained through-
In practice, we choose to represent a constrained trajectory with out a same embedding. More interestingly, constrained trajectories
a cubic Bézier curve, which permits the reproduction of arc mo- may be chained across different embeddings. Consider for instance
tions as prescribed in traditional animation guidelines (e.g., [JT95, a pair of embeddings E0 and E1 , and a constrained trajectory D0 (t)
Wil01]). We have favored this representation over logarithmic spi- that drives the lattice of E0 from its start to its end keyframe. In
rals [WNS∗ 10] for two reasons: it better captures the motion gener- the likely case where D0 (1) ∈ E1 (assuming E0 and E1 have been
ated by ARAP interpolation, and it provides intuitive direct controls properly matched), a new constrained trajectory D1 (t) may be im-
through tangents. A Bézier curve is initially fit to the discrete tra- posed on E1 , with D1 (0) = D0 (1) by construction. Note that the
jectory (position and parameterization) of the selected constrained trajectories remain independent, hence removing E0 will not affect
point using Schneider’s algorithm [Sch90]. In our system, we al- the chained trajectories in E1 . Any chain of trajectory constraints
ways re-parameterize the Bézier curve uniformly along arc-length constructed this way can be made G1 -continuous at keyframes by
after any edit. This gives a more accurate local temporal control aligning the tangents of adjacent constrained trajectories. This is a

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 419
(a) (b) transitivity of ARAP interpolation. When removing a breakdown
keyframe, the associated transformation and trajectories must be re-
moved as well, which potentially leads to a simpler motion. In prac-
tice, we keep all active constraints, and fit new trajectories from the
default ARAP interpolation on the new interval to preserve artistic
inputs as much as possible.

6. Implementation details
Figure 13: Hierarchical constraints maintain spatial relationships Strokes & lattices. In our system, strokes are represented by poly-
between adjacent embeddings throughout interpolation. In (a), the lines with stylus pressure recorded at vertices when available. As
arm, forearm and hand embeddings interpenetrate at intermediate mentioned in previous sections, embeddings may apply to subsets
frames, as shown by the dashed trajectories. In (b), by placing two of strokes, which consist of stroke segments. Moreover, since em-
hierarchical constraints – one (in red) on the arm driving the fore- beddings are transient, different subsets of strokes may be manipu-
arm, and the other (in blue) on the forearm driving the elbow – the lated at different keyframes, as demonstrated in the last interval of
full articulated chain remains properly connected at constrained Figure 17. Each new lattice is enforced to be connected and auto-
points, which follow smooth arc trajectories. matically initialized in an axis-aligned manner. We make only one
exception: when converting a breakdown keyframe into a new em-
bedding (see Figure 4(a)), we keep the previously transformed lat-
notable feature of our approach, since it provides persistent con- tice to retain its target transformation T ⋆ . This is a small limitation
trol over trajectories while relying on transient embeddings. We of the current implementation that could be solved by transferring
demonstrate that feature in Figure 12(b), where the trajectories of T ⋆ to a new axis-aligned lattice. We provide an additional tool to
Eg and Er are chained with G1 continuity to trajectories of Eb . This add contiguous empty cells inside an embedding, which is particu-
produces a visually continuous motion even in the presence of a larly useful to fill in lattices (enforcing their rigidity through inter-
topological change, as is best seen in the supplemental demo video polation), or to provide support for constraints away from strokes.
where we use more inbetween frames. The size of lattice cells may also be adjusted to capture fine details.

Trajectories might not only be used to “knit” embeddings across Vector registration. The registration problem consists in finding
time but also in space. This is a useful feature when two embed- a transformation T ⋆ that aligns the strokes in an embedding (the
dings that are visually connected at contiguous keyframes get dis- source) with a given subset of stroke segments in the next keyframe
connected or interpenetrate during interpolation, as shown in Fig- (the destination). Our solution relies on the image registration tech-
ure 13(a). For a shared constraint to be added, lattices in each em- nique of Sýkora et al. [SDC09], which works in two phases that are
beddings must overlap. One of the embeddings is identified as the iterated until convergence: a “push” phase that moves lattice cor-
leader El and the other one as the follower E f . Only the constrained ners towards locations where the source and destination are similar;
trajectory of El is edited, while E f follows that same trajectory. and a “regularize” phase that reintroduces local rigidity.
This raises an issue when the spacing functions Sl and S f of the
respective embeddings are different, as El and E f are then driven In our approach, we adapt the push phase to work on vec-
by the same trajectory but at different speeds. The issue is trivially tor strokes, while the regularize phase is left unchanged, as il-
solved by using the spacing function of the leader Sl on the follower lustrated in Figure 14. In our push phase, we compute the opti-
E f , only at the constrained embedding point; S f is retained other- mal rigid transformations that minimizes the sum of squared dis-
wise. Such hierarchical constraints bear a resemblance to skeletons tances between the stroke points in the source and their nearest-
used in cut-out animation systems, as they open up to the control of neighbors in the destination, using the closed-form solution of
articulated structures. In particular, they may be persistent as with Schaefer et al. [SMW06]. This process results in a set of discon-
any other constrained trajectory, as shown in Figure 13(b). How- nected transformed cells (step 1 in Figure 14). The lattice connec-
ever, they may also be deactivated at any keyframe, which is useful tivity is then restored by averaging lattice corner positions (step 2).
for imposing temporary constraints, such as an object temporarily The regularize phase is similar, except we compute optimal rigid
attached before being thrown away (see the supplemental results transformations that aligns the source cells with the lattice obtained
video for an example). at the end of the push phase (step 3), before restoring connectivity
(step 4). The regularize phase is repeated Nr times, with greater
Constrained trajectories are easily updated after each of the time-
values of Nr increasing rigidity of the lattice (we use Nr = 10 by
line operations of Figure 4. When an embedding is split at a frame
default). Repetitions are crucial to prevent the lattice from collaps-
fsplit , the lattice positions Vsplit = V (S(fsplit )) are used for the break-
ing due to the naive initial nearest-neighbor correspondences.
down keyframe and a new ARAP interpolation is recomputed on
each new interval, i.e., from V0 to Vsplit and from Vsplit to V1 . Bézier An alternative solution would have been to use the block match-
trajectories are split as well, and the tangents at the split points re- ing approach of Sýkora et al. [SDC09] applied to rasterized strokes,
computed, for instance using De Casteljau’s algorithm. In all our or their distance transform, similarly to Noris et al. [NSC∗ 11]. We
tests, we have observed that motion before and after splitting are have found in practice that a vectorial solution is much more ef-
visually identical, even though we could not find a proof of the ficient since it provides direct initial correspondences. As a result,

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
420 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

Push Matching 10.22 ms



Push phase 8.41ms

Regularize phase 1.81ms
1 2 Interpolation 6.67 ms

Factorization 5.55ms

Solve 0.90ms
Stroke rendering 0.22ms
Total 16.89 ms
Regularize
Table 1: Performance of our single-threaded implementation in a
worst-case scenario, recorded on a i7-4790K 4GHz CPU and a
3 4 Nvidia GeForce GTX 980Ti GPU. We report timings for the second
keyframe of the fish animation (Figure 19), holding 284 strokes in a
lattice of 753 cells. We use Nr = 10 for regularization, while solve
and stroke rendering timings correspond to 10 rendered frames.

Nr repetitions

Figure 14: Our vector registration algorithm matches a source strokes, using a function c(t) : [0, 1] → [0, 1]:
drawing (bottom strokes) with a destination drawing (top strokes) (
in two phases. During the push phase, lattice cells are (1) indepen- (1 − (2(t − 12 )2 )2 if t ∈ [0, 12 ],
dently rigidly transformed to match the closest destination stroke c(t) =
1 otherwise.
points (red arrows), after which (2) the lattice connectivity is re-
stored through averaging. During the regularize phase (repeated For backward strokes we simply use c(1 − t). Hence at any time
Nr times), the shape of the source lattice is partly restored by (3) t, there is always one set of strokes (forward or backward) that
finding independent rigid cell transformations that match the cur- is displayed at full thickness. Note that cross-fading is also sub-
rent target lattice before (4) restoring connectivity. jected to spacing since t = S(f) for a frame f. By default, we de-
activate cross-fading for embeddings whose end keyframe does not
store any stroke, so as to keep forward propagated strokes displayed
over the full interval. Cross-fading is reactivated whenever the end
there is no need to iterate over the sequence of push and regular- keyframe is populated with strokes, or the embedding is tagged as
ize phases as in the raster version: both phases are only applied fading in or out. Yet, we use linear cross-fading in the latter case to
once, and our vectorial registration achieves real-time performance. make strokes appear or disappear at the same rate as interpolation.
Thanks to its efficiency, vectorial registration may be used interac-
tively in a semi-automatic fashion. For instance, our system allows Performance. Our system is implemented in C++, using the Qt li-
the user to deform the lattice using the tools mentioned in Sec- brary for the GUI and an OpenGL Geometry Shader for the final
tion 4.1 and to start the registration from this deformed configura- stroke rendering. We use the sparse LU solver of Eigen [GJ∗ 10] to
tion, hence allowing plastic deformations of the lattice. Registra- efficiently factorize and solve the linear system involved in ARAP
tion might even be run continuously during the deformation so that interpolation (Appendix A). Table 1 reports the performance of
the source strokes glide over the destination. Please see the supple- our implementation recorded for the second keyframe of Figure 19
mental demo video for a live illustration. that uses the largest lattice from all our examples, and relies on
the more demanding pose-to-pose workflow with semi-automatic
Spacing & trajectories. The analytic spacing function S may be (and thus interactive) registration. Interpolation times are also re-
controlled by adjusting the position of each individual tick. For in- ported since we display the interpolated results during matching
tervals holding many frames, this may be tedious; hence we provide via an “onion skin” visualization. We obtain real-time performance
several interface tools to control multiple ticks at once. As shown in such a worst-case scenario, as well as in all our experiments.
in the supplemental demo video, we provide typical “ease-in/ease-
out” controls, as well as options to place ticks on “halves”, as rou-
tinely done by 2D artists when creating spacing charts [Wil01]. To 7. Results
facilitate constrained trajectory editing, besides tangents manipu-
In this section and the supplemental video we present complex re-
lation, we have implemented a sketching technique that linearly
sults obtained using our animation system. Note that the video does
transforms a curve drawn by the user such that its endpoints match
not merely show final inbetweened results, but first and foremost
the constrained positions at keyframes, and then fit a cubic Bézier
the ability of our system to enable a fast and flexible creative pro-
curve with uniform arc-length parameterization to the result. This
cess to get to those results.
is also demonstrated in the supplemental demo video.
Figure 1 shows the particularly complex example of a special ef-
Cross-fading. Each interpolated frame is the result of cross-fading fect animation traced from Gilland’s book [Gil09], featuring mul-
forward and backward transformed strokes. To limit ghosting ar- tiple topological changes made possible by the use of several tran-
tifacts, we apply cross-fading to control the thickness of forward sient embeddings. Figure 13 demonstrates a first Articulated arm

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 421
(a) (b)
(a) (b) (c)

Figure 16: Starting from the same two extreme key drawings (in
gray) traced over an animation exercise by Williams [Wil01] and
using the same decomposition into four embeddings throughout
(see inset image), we produce (a) a “normal” walk cycle by adding
Figure 15: Starting from the same first extreme key drawing (in three breakdowns (in black) at, and around, the passing posi-
gray), three ball drop animations are produced with a different se- tion, and (b) a very different animation with a single breakdown
quence of breakdowns (in black) using a shift-and-trace workflow. keyframe and a constrained trajectory (red curve) to make the char-
In (a), the ball is slightly deformed in the direction of motion and acter bow down inbetween steps.
acceleration is conveyed through an ease-in/ease-out spacing, with
a high rebound conveying a light object. In (b), the ball is acceler-
ated, with no deformation on contact and a low rebound, all effects
conveying a heavier object. In (c), we apply a large “squash-and- 8. Discussion
stretch” deformation to the ball before and after it hits the ground, Our animation system relies on the concept of transient embed-
giving it an elastic and cartoony look-and-feel. dings. In Section 8.1, we justify this choice, comparing it to an al-
ternative stroke-level animation system. We then discuss practical
limitations and future work in Sections 8.2 and 8.3.
animation exercise taken from Williams’ guide [Wil01], showcas-
ing arc motions controlled through hierarchical trajectories. In Fig-
ure 15 through 18, we reproduce three other classic animation ex- 8.1. Comparison with stroke-level inbetweening
ercises. Details are provided in figure captions; we summarize the
Xing et al. [XWSY15] present an interactive 2D animation system
main demonstrated features below:
that aims at assisting in both the drawing of new keyframes and
Ball drop (Figure 15): different impressions of weight are con- the matching of drawings across keyframes. In their approach, the
veyed through variations in breakdowns and spacing; user provides guiding strokes that are used by the system to predict
Walk cycle (Figure 16): breakdown keyframes are used to produce a new drawing based on past spatial and temporal repetitions. The
drastically different animations, starting from the same pair of user may then either directly reuse the suggested drawing or instead
extreme key drawings; rely on guide strokes, either case yielding strokes matched between
Flour sack jump (Figure 17): plausible motion dynamics are ob- the current and next keyframes. In effect, this amounts to a different
tained by adding anticipation and follow-through via a combina- kind of workflow that works at the stroke level and couples drawing
tion of breakdowns, deformations and spacing; with matching, alternating between shift and trace steps.
Head turn (Figure 18): 3D-like rotation is conveyed through sev-
Since strokes are only matched from one keyframe to the next,
eral embeddings, some being tagged as fading in or out.
the representation may be considered transient, like ours. However,
Note that even a simple ball drop animation (Figure 15) would unlike our method, the embedded deformation model [SSP07] is
require many trials-and-errors for novice 2D animators to produce carried by the strokes themselves, at a coarse sampling rate. To
a first result. Exploring alternate results afterwards would require a generate inbetween frames, the parameters of this model – an affine
significant amount of work (tens of minutes, perhaps hours) since transformation per sample – are interpolated in time, and the full-
all inbetween frames must be redrawn. In contrast, our system al- resolution strokes are reconstructed by spatial diffusion.
lows the exploration of different alternatives instantly while retain-
We reproduce one of their animations with our system in Fig-
ing a plausible rough hand-drawn animation style.
ure 19(a), achieving a very similar result using a pose-to-pose
The Head turn example (Figure 18) reveals the main limitation workflow. The comparison is not intended to show the superior-
of our approach (see Section 8.3): even with a special treatment ity of one workflow over another. Indeed, we believe that their
of the occluded embeddings (such as the ears), our system is not auto-completion algorithm – which is the core contribution of their
yet designed to let parts of embeddings appear or disappear be- work – could be adapted to compute lattice transformations in our
hind other embeddings, as it would require to interpolate topologi- system as well, providing an additional matching solution to artists.
cal changes. The method of Zhu et al. [ZPBK17] is able to produce Instead, we want to stress the implications of choosing to work at
such inbetween frames, but the user must specify correspondences the stroke level. First, coupling matching with drawing of strokes
for cuts, openings and boundaries on every key drawing, and com- has the undesired property that when strokes are erased, matching
patible embeddings must then be computed throughout the anima- is lost in the process. This is obviously not the case with our sys-
tion with a prohibitively expensive optimization. tem, since matching is done on embedding lattices. Second, it is

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
422 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

of trajectories; but other solutions could be devised. Soft constraints


could be introduced [BBA08] to mimic exterior forces (such as
wind); or constraints may be allowed to slide inside an embedding
to precisely control motion at articulations. This latter improvement
should be more computationally-demanding as it would a priori re-
quire a new ARAP factorization at each interpolated frame.

8.3. Future work


We have chosen to rely on ARAP interpolation to implement trans-
formations between embeddings, as it produces natural results by
Figure 17: For this flour sack jump animation, we start from three default, and may be further controlled efficiently through con-
initial keyframes (in black and red/blue), which are registered in strained trajectories. However, it does not appear to be the most
a pose-to-pose approach using a single embedding for the whole suitable solution in some cases. For instance, ARAP interpolation
drawing. We then add a breakdown (in dark gray) to produce an is not well adapted to animations that are expected to follow an un-
anticipation effect, then two additional breakdowns (in light gray) derlying path, such as roots growing in a soil as shown in the sup-
to refine animation. For follow-through, the last key drawing is di- plemental results video. More generally, 3D-like motions (e.g., ro-
vided into two embeddings: the red one is held fixed on screen while tations out of the canvas plane) remain hard to reproduce with our
the blue one is slightly deformed through a shift step. approach. A solution might be again to rely on an underlying guide
during interpolation, this time a 3D proxy surface.
3D-like animations raise another, more difficult problem: the
unclear how the diffusion-based interpolation could be adapted to handling of occlusions. As shown in Figure 18, occluded embed-
provide control over trajectories and motion smoothness, whereas dings simply fade in or out during interpolation in our current sys-
such control is direct in our system as demonstrated in Figure 19(b) tem. Even though cross-fading is enough in most cases, it is in-
and the supplemental results video. adequate in the case of occlusions: parts of strokes should be re-
vealed or hidden during interpolation, which requires the handling
8.2. Practical limitations of depth order relationships between embeddings and the interpo-
lation of topological changes at interactive rates with minimal user
Our implementation is intended as a proof-of-concept prototype intervention. Extending our system to handle occlusions while re-
demonstrating the potential of transient embeddings. Yet our cur- taining non-linear editing abilities is a challenging and exciting di-
rent animation system could be improved in several ways. rection of future work.
On the matching side, we could improve the registration tool
with an automatic global non-rigid registration prior to our more
Acknowledgments
local ARAP registration; or we could give artists the option to pin
some correspondences to guide that global alignment process. Such We are grateful to Antoine Antin, Clément Berthaud, Eric Scholl
pinned points should be related to interpolation constraints (any and Fabrice Debarge from Praxinos for their participation to the
constraint should be a pin), but they should not be equivalent since observational study and for very helpful discussions. This work is
artists may not need to control the trajectory of all pinned points. supported by the ANR MoStyle project (ANR-20-CE33-0002).
As mentioned in Section 8.1, we would also like to adapt stroke
auto-completion [XWSY15] to guide the matching of embeddings.
Working with lattices may allow us to alleviate the influence of References
drawing order on prediction, which may prove especially problem-
[ACOL00] A LEXA M., C OHEN -O R D., L EVIN D.: As-rigid-as-possible
atic for rough drawings. shape interpolation. In Proceedings of the 27th Annual Confer-
ence on Computer Graphics and Interactive Techniques (2000), ACM,
On the interpolation side, we have identified three directions of
p. 157–164. doi:10.1145/344779.344859. 3, 7, 15
improvements. First, we would like to explicitly model the rigid
component of a transform T through a pivot transform, such that [ADN∗ 17] A RORA R., DAROLIA I., NAMBOODIRI V. P., S INGH K.,
B OUSSEAU A.: Sketchsoup: Exploratory ideation using design sketches.
the non-rigid component may be handled by ARAP interpolation Computer Graphics Forum (2017). doi:10.1111/cgf.13081. 3
relative to that pivot. This would simplify the control of a whole
[Ado] A DOBE I NC .: Adobe animate. URL: https://www.adobe.
embedding motion through the trajectory of its pivot (e.g., a bounc- com/products/animate.html. 2, 4, 7
ing ball moving along an arc), which could exhibit interesting de-
fault behaviors (e.g., logarithmic spirals [WNS∗ 10]). Second, our [Bae69] BAECKER R. M.: Picture-driven animation. In Proceedings of
the May 14-16, 1969, Spring Joint Computer Conference (1969), ACM,
trajectory constraints only allow to enforce G1 continuity across pp. 273–288. doi:10.1145/1476793.1476838. 2
keyframes due to potential discontinuities of the spacing functions
[BBA08] BAXTER W., BARLA P., A NJYO K.-I.: Rigid shape interpola-
S. We would like to investigate ways to ensure full C1 continu- tion using normal equations. In Proceedings of the International Sym-
ity by giving the option to adjust S on both sides of abutting con- posium on Non-photorealistic Animation and Rendering (2008), ACM,
straints. Finally, we have focused on hard constraints for the control pp. 59–64. doi:10.1145/1377980.1377993. 3, 7, 8, 12, 15

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 423

Figure 18: This head turn animation is generated from only three key drawings whose embeddings are color-coded – the last keyframe is
made of a single embedding (in black). To make the ears appear or disappear, we must draw and/or deform them occluded (blue and magenta
dashed lines) and tag them for fading in or out.

Digital light table – inbetweening for 2d animations using guidelines.


Computers & Graphics 65 (2017), 31–44. doi:https://doi.org/
10.1016/j.cag.2017.04.001. 2
[Com] C OMMUNITY B. O.: Blender - a 3d modelling and rendering
package. URL: http://www.blender.org. 2
[CPL21] C ASEY E., P ÉREZ V., L I Z.: The animation transformer: Visual
correspondence via segment matching. In 2021 IEEE/CVF International
Conference on Computer Vision (ICCV) (2021), pp. 11303–11312. doi:
10.1109/ICCV48922.2021.01113. 3
[CZ22] C HEN S., Z WICKER M.: Improving the perceptual quality of 2d
animation interpolation. In Proceedings of the European Conference on
(a) (b) Computer Vision (2022). 3
[DFSEVR01] D I F IORE F., S CHAEKEN P., E LENS K., VAN R EETH F.:
Automatic in-betweening in computer assisted animation by exploiting
Figure 19: We traced three keyframes (in black) from the fish ex- 2.5d modelling techniques. In Proceedings Computer Animation 2001.
ample in the supplemental video of [XWSY15], and matched them Fourteenth Conference on Computer Animation (2001), pp. 192–200.
using a pose-to-pose workflow. In (a), we show the interpolated doi:10.1109/CA.2001.982393. 2
drawings produced by default by our method: as highlighted by [dJB06] DE J UAN C. N., B ODENHEIMER B.: Re-using traditional ani-
the dashed line, the trajectories are discontinuous at the middle mation: Methods for semi-automatic segmentation and inbetweening. In
keyframe. In (b), we constrained the trajectory depicted in solid Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on
Computer Animation (2006), Eurographics Association, pp. 223–232. 2
red to follow a smooth arc. Notice how this constraint is propa-
[DLKS18] DVOROŽNÁK M., L I W., K IM V. G., S ÝKORA D.: Toon-
gated to neighbor trajectories (e.g., the whiskers, see the dashed
synth: Example-based synthesis of hand-colored cartoon animations.
red curves). ACM Trans. Graph. 37, 4 (2018). doi:10.1145/3197517.
3201326. 3
[DRvdP15] DALSTEIN B., RONFARD R., VAN DE PANNE M.: Vector
[BBA09] BAXTER W., BARLA P., A NJYO K.- I .: Compatible embedding graphics animation with time-varying topology. ACM Trans. Graph. 34,
for 2d shape animation. IEEE Transactions on Visualization and Com- 4 (2015). doi:10.1145/2766913. 2
puter Graphics 15, 5 (2009), 867–879. doi:10.1109/TVCG.2009. [Dur91] D URAND C. X.: The “toon” project: Requirements for a com-
38. 2, 3 puterized 2d animation system. Computers & Graphics 15, 2 (1991),
[BKLP16] BAI Y., K AUFMAN D. M., L IU C. K., P OPOVI Ć J.: Artist- 285–293. doi:https://doi.org/10.1016/0097-8493(91)
directed dynamics for 2d animation. ACM Trans. Graph. 35, 4 (2016). 90081-R. 2
doi:10.1145/2897824.2925884. 3 [FBC∗ 95] F EKETE J.-D., B IZOUARN É., C OURNARIE É., G ALAS T.,
TAILLEFER F.: Tictactoon: A paperless system for professional 2d
[BW71] B URTNYK N., W EIN M.: Computer-generated key-frame ani-
animation. In Proceedings of the 22nd annual conference on Com-
mation. Journal of the SMPTE 80, 3 (1971), 149–153. 2
puter graphics and interactive techniques (1995), pp. 79–90. doi:
[BW75] B URTNYK N., W EIN M.: Computer animation of free form 10.1145/218380.218417. 2
images. In Proceedings of the 2nd Annual Conference on Computer
[FC80] F RITSCH F. N., C ARLSON R. E.: Monotone piecewise cubic
Graphics and Interactive Techniques (1975), ACM, p. 78–80. doi:
interpolation. SIAM Journal on Numerical Analysis 17, 2 (1980), 238–
10.1145/563732.563743. 2, 3
246. doi:10.1137/0717021. 7
[CAC] CACAN I P TE LTD .: Cacani. URL: https://cacani.sg/.
[FTA05] F U H., TAI C.-L., AU O. K.-C.: Morphing with laplacian coor-
2, 4, 7
dinates and spatial-temporal texture. In Proceedings of Pacific Graphics
[Cat78] C ATMULL E.: The problems of computer-assisted animation. In (2005), pp. 100–102. 3
Proceedings of the 5th annual conference on Computer graphics and [FZP∗ 20] F ISH N., Z HANG R., P ERRY L., C OHEN -O R D., S HECHT-
interactive techniques (1978), vol. 12, ACM, pp. 348–353. doi:10. MAN E., BARNES C.: Image morphing with perceptual constraints
1145/800248.807414. 2 and stn alignment. Computer Graphics Forum 39, 6 (2020), 303–313.
[CMV17] C ARVALHO L., M ARROQUIM R., V ITAL B RAZIL E.: Dilight: doi:10.1111/cgf.14027. 3

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
424 M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings

[Gil09] G ILLAND J.: Elemental Magic, Volume I: The Art of Special [SBv05] S ÝKORA D., B URIÁNEK J., Ž ÁRA J.: Colorization of black-
Effects Animation. Focal Press, 2009. 1, 10 and-white cartoons. Image and Vision Computing 23, 9 (2005), 767–782.
[GJ∗ 10] G UENNEBAUD G., JACOB B., ET AL .: Eigen v3. doi:10.1016/j.imavis.2005.05.010. 2
http://eigen.tuxfamily.org, 2010. 10 [Sch90] S CHNEIDER P. J.: An Algorithm for Automatically Fitting Digi-
[IMH05] I GARASHI T., M OSCOVICH T., H UGHES J. F.: As-rigid- tized Curves. Academic Press Professional, Inc., 1990, p. 612–626. 8
as-possible shape manipulation. ACM Trans. Graph. 24, 3 (2005), [SDC09] S ÝKORA D., D INGLIANA J., C OLLINS S.: As-rigid-as-
1134–1141. doi:10.1145/1073204.1073323. 3 possible image registration for hand-drawn cartoon animations. In Pro-
[JSL22] J IANG J., S EAH H. S., L IEW H. Z.: Stroke-based drawing and ceedings of the 7th International Symposium on Non-Photorealistic An-
inbetweening with boundary strokes. Computer Graphics Forum 41, 1 imation and Rendering (2009), ACM, p. 25–33. doi:10.1145/
(2022), 257–269. doi:10.1111/cgf.14433. 2 1572614.1572619. 3, 7, 9
[JT95] J OHNSTON O., T HOMAS F.: The illusion of life : Disney anima- [SGWM93] S EDERBERG T. W., G AO P., WANG G., M U H.: 2-d shape
tion. Disney Press, 1995. 1, 2, 3, 8 blending: An intrinsic solution to the vertex path problem. In Proceed-
ings of the 20th Annual Conference on Computer Graphics and Inter-
[KHS∗ 12] K AJI S., H IROSE S., S AKATA S., M IZOGUCHI Y., A NJYO
active Techniques (1993), ACM, p. 15–18. doi:10.1145/166117.
K.: Mathematical Analysis on Affine Maps for 2D Shape Interpolation.
166118. 3
In Eurographics/ ACM SIGGRAPH Symposium on Computer Animation
(2012). doi:10.2312/SCA/SCA12/071-076. 3, 15 [SMW06] S CHAEFER S., M C P HAIL T., WARREN J.: Image deformation
[Kor02] KORT A.: Computer aided inbetweening. In Proceedings of using moving least squares. ACM Trans. Graph. 25 (2006), 533–540.
the 2nd international symposium on Non-photorealistic animation and doi:10.1145/1179352.1141920. 9
rendering (2002), ACM, pp. 125–132. doi:10.1145/508530. [SSP07] S UMNER R. W., S CHMID J., PAULY M.: Embedded defor-
508552. 2, 3 mation for shape manipulation. ACM Trans. Graph. 26, 3 (2007).
[LCY∗ 11] L IU D., C HEN Q., Y U J., G U H., TAO D., S EAH H. S.: doi:10.1145/1276377.1276478. 3, 11
Stroke correspondence construction using manifold learning. Com- [SZGP05] S UMNER R. W., Z WICKER M., G OTSMAN C., P OPOVI Ć
puter Graphics Forum 30, 8 (2011), 2194–2207. doi:10.1111/j. J.: Mesh-based inverse kinematics. ACM Trans. Graph. 24, 3 (2005),
1467-8659.2011.01969.x. 2 488–495. doi:10.1145/1073204.1073218. 3
[LMY∗ 13] L IU X., M AO X., YANG X., Z HANG L., W ONG T.-T.: [SZY∗ 21] S IYAO L., Z HAO S., Y U W., S UN W., M ETAXAS D., L OY
Stereoscopizing cel animations. ACM Trans. Graph. 32, 6 (2013). C. C., L IU Z.: Deep animation video interpolation in the wild. In
doi:10.1145/2508363.2508396. 2 2021 IEEE/CVF Conference on Computer Vision and Pattern Recog-
[LZLS21] L I X., Z HANG B., L IAO J., S ANDER P.: Deep sketch- nition (2021), pp. 6583–6591. doi:10.1109/CVPR46437.2021.
guided cartoon video inbetweening. IEEE Transactions on Visualiza- 00652. 3
tion and Computer Graphics (2021). doi:10.1109/TVCG.2021.
[Too] T OON B OOM A NIMATION I NC .: Toon boom harmony. URL:
3049419. 3
https://cacani.sg/. 2, 4, 7
[MFXM21] M IYAUCHI R., F UKUSATO T., X IE H., M IYATA K.: Stroke
correspondence by labeling closed areas. In 2021 Nicograph Interna- [TVP] TVPAINT D EVELOPPEMENT: Tvpaint. URL: https://www.
tional (2021), IEEE Computer Society, pp. 34–41. doi:10.1109/ tvpaint.com. 4, 6
NICOINT52941.2021.00014. 2 [Wil01] W ILLIAMS R.: The animator’s survival kit. Faber and Faber,
[MIT67] M IURA T., I WATA J., T SUDA J.: An application of hybrid curve 2001. 1, 2, 3, 4, 7, 8, 10, 11
generation: cartoon animation by electronic computers. In AFIPS ’67 [WNS∗ 10] W HITED B., N ORIS G., S IMMONS M., S UMNER R. W.,
(Spring) (1967). 2 G ROSS M., ROSSIGNAC J.: Betweenit: An interactive tool for tight in-
[MSG96] M ADEIRA J. S., S TORK A., G ROSS M. H.: An approach to betweening. Computer Graphics Forum 29, 2 (2010), 605–614. doi:
computer-supported cartooning. The Visual Computer 12, 1 (1996), 1– 10.1111/j.1467-8659.2009.01630.x. 2, 3, 8, 12
17. doi:10.1007/BF01782215. 2 [WXXC08] WANG Y., X U K., X IONG Y., C HENG Z.-Q.: 2d shape de-
[NHA19] NARITA R., H IRAKAWA K., A IZAWA K.: Optical flow based formation based on rigid square matching. Computer Animation and
line drawing frame interpolation using distance transform to support in- Virtual Worlds 19, 3-4 (2008), 411–420. doi:10.1002/cav.251. 3
betweenings. In 2019 IEEE International Conference on Image Process- [Xie95] X IE M.: Feature matching and affine transformation for 2d cell
ing (2019), pp. 4200–4204. 3 animation. The Visual Computer 11, 8 (1995), 419–428. doi:10.
[NSC∗ 11] N ORIS G., S ÝKORA D., C OROS S., W HITED B., S IM - 1007/BF02464332. 2
MONS M., H ORNUNG A., G ROSS M., S UMNER R. W.: Tempo-
ral noise control for sketchy animation. In Proceedings of the ACM [XWSY15] X ING J., W EI L.-Y., S HIRATORI T., YATANI K.: Auto-
SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation complete hand-drawn animations. ACM Trans. Graph. 34, 6 (2015).
doi:10.1145/2816795.2818079. 3, 11, 12, 13
and Rendering (2011), ACM, p. 93–98. doi:10.1145/2024676.
2024691. 3, 9 [XZWB05] X U D., Z HANG H., WANG Q., BAO H.: Poisson shape inter-
[PSN20] PARK S., S EO K., N OH J.: Neural crossbreed: Neural based polation. In Proceedings of the 2005 ACM symposium on Solid and phys-
image metamorphosis. ACM Trans. Graph. 39, 6 (2020). doi:10. ical modeling (2005), ACM, pp. 267–274. doi:10.1145/1060244.
1145/3414685.3417797. 3 1060274. 3, 15
[Qui10] Q UILEZ I.: Inverse bilinear interpolation, 2010. URL: https: [Yag17] YAGI Y.: A filter based approach for inbetweening. CoRR
//iquilezles.org/articles/ibilinear/. 15 abs/1706.03497 (2017). 3
[Ree81] R EEVES W. T.: Inbetweening for computer animation utiliz- [Yan18] YANG W.: Context-aware computer aided inbetweening. IEEE
ing moving point constraints. In Proceedings of the 8th Annual Confer- Transactions on Visualization and Computer Graphics 24, 2 (2018),
ence on Computer Graphics and Interactive Techniques (1981), ACM, 1049–1062. doi:10.1109/TVCG.2017.2657511. 2, 3
p. 263–269. doi:10.1145/800224.806814. 2, 3 [YBS∗ 12] Y U J., B IAN W., S ONG M., C HENG J., TAO D.: Graph based
[RID10] R IVERS A., I GARASHI T., D URAND F.: 2.5d cartoon mod- transductive learning for cartoon correspondence construction. Neu-
els. ACM Trans. Graph. 29, 4 (2010). doi:10.1145/1778765. rocomput. 79 (2012), 105–114. doi:10.1016/j.neucom.2011.
1778796. 2 10.003. 2

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
M. Even, P. Bénard & P. Barla / Non-linear Rough 2D Animation using Transient Embeddings 425

[YHY19] YANG W.-W., H UA J., YAO K.-Y.: Cr-morph: Control- Appendix A: Controllable ARAP interpolation
lable rigid morphing for 2d animation. Journal of Computer Sci-
ence and Technology 34, 5 (2019), 1109–1122. doi:10.1007/ The original ARAP formulations of Alexa et al. [ACOL00] and
s11390-019-1963-3. 3 Xu et al. [XZWB05] do not offer any control over motion trajecto-
[YSC∗ 18] YANG W., S EAH H.-S., C HEN Q., L IEW H.-Z., S ỲKORA ries. Baxter et al. [BBA08] introduce such controls through linear
D.: Ftp-sc: Fuzzy topology preserving stroke correspondence. Computer constraints thanks to their reformulation of the problem in terms
Graphics Forum 37, 8 (2018), 125–135. doi:10.1111/cgf.13518. of normal equations. Katji et al. [KHS∗ 12] present an even more
2
generic mathematical framework, but since we do not need such
[YVG20] YAN C., VANDERHAEGHE D., G INGOLD Y.: A benchmark a generalization, we choose the method of Baxter et al. [BBA08]
for rough sketch cleanup. ACM Trans. Graph. 39, 6 (2020). doi:10.
1145/3414685.3417784. 2
whose implementation is simpler and very efficient.
[ZHF12] Z HANG L., H UANG H., F U H.: Excol: An extract-and- More precisely, to compute the interpolated positions of the lat-
complete layering approach to cartoon animation reusing. IEEE Transac- tice corners at time t ∈ [0, 1], we use Equation 5 in their paper:
tions on Visualization and Computer Graphics 18, 7 (2012), 1156–1169.
 ⊤ −1  ⊤
doi:10.1109/TVCG.2011.111. 2

P WP C P WA(t)
V (t) = ,
[ZLWH16] Z HU H., L IU X., W ONG T.-T., H ENG P.-A.: Globally opti- C 0 D(t)
mal toon tracking. ACM Trans. Graph. 35, 4 (2016). doi:10.1145/
2897824.2925872. 2 where the sparse matrix P encodes the triangulated lattice connec-
[ZPBK17] Z HU Y., P OPOVI Ć J., B RIDSON R., K AUFMAN D. M.: Planar tivity, the matrix A(t) stores the target affine triangle deformations,
interpolation with extreme deformation, topology change and dynam- the diagonal matrix W allows to specify a weight per triangle (we
ics. ACM Trans. Graph. 36, 6 (2017). doi:10.1145/3130800. use its area), the matrix C expresses hard linear constraints defined
3130820. 3, 11 on lattice vertices, and the matrix D(t) stores the constrained driven
positions. When no constraint is provided by the user, we com-
pel the mean position of the lattice to follow a linear and uniform
trajectory in order to have a unique solution. This implies setting
C = [1/N . . . 1/N], with N the number of lattice triangles, and set
D(t) to the linearly interpolated position of the lattice barycenter.
Otherwise, each constrained point p maps to a row C p in C with
four non-zero values, one for each barycentric coordinate relative
to its cell corners. Denoting {i, j, k, l} the indices of these corners,
enumerated in clockwise order starting from top-left, yields:
i j k l
 
Cp = . . . (1 − u)(1 − v) . . . u(1 − v) . . . uv . . . (1 − u)v . . . ,
with (u, v) the coordinates of p in the quad cell obtained by in-
verse bilinear interpolation [Qui10]. The corresponding row in D(t)
stores the position along the trajectory curve evaluated at t.
Note that matrix inversion, which is the most computationally
expensive part of the method, needs to be performed in only two
cases: when a lattice is created or its topology changed, since P is
modified; or when a linear constraint is added to C. The matrix A(t)
is updated whenever matching is modified, whereas D(t) is updated
whenever a constrained trajectory is edited.
With this formulation, the inverse lattice transformation
T (t)−1 = V (1 − t) −V1 might result in a non-symmetric behavior,
which is obviously problematic for strokes cross-fading. We thus
use the symmetric formulation of Baxter et al. [BBA08] which is
slightly more complex but equally fast to compute. We refer the
interested reader to their paper for details.

© 2023 Eurographics - The European Association


for Computer Graphics and John Wiley & Sons Ltd.
Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content
may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for
individual use.

You might also like