Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and ... more Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep understanding, improvement, and parallel execution. This paper develops a simple, generalized AD algorithm calculated from a simple, natural specification. The general algorithm is then specialized by varying the representation of derivatives. In particular, applying well-known constructions to a naive representation yields two RAD algorithms that are far simpler than previously known. In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler p...
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discre... more Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person modeling an animation with the help of a computer. Using a continuous model of time for animation allows for natural specification, avoiding some artificial details, but is difficult to implement with generality, robustness and efficiency. This paper presents and motivates continuous modeled animation, and sketches out a naive functional implementation for it. An examination of some of the practical problems with this implementation leads to several alternate representations, all of which have difficulties in themselves, some quite subtle. We hope that the insights and techniques discussed in this paper lead to still better representations, so that animation may be specified in natural terms without significant loss of performance.
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animate... more This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is that when a hand grasps the image, the image stuck to the hand at the grasped location. When only one hand is grasping, the image pans (translates). When two hands are grasping, the image pans, zooms and rotates as required in order to keep the two grasped locations under the hands. While the C++ program that inspired this exercise is quite long and complex, the Fran version is short, modular and we feel captures the essence of the behavior being modeled.
Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and ... more Automatic differentiation (AD) in reverse mode (RAD) is a central component of deep learning and other uses of large-scale optimization. Commonly used RAD algorithms such as backpropagation, however, are complex and stateful, hindering deep understanding, improvement, and parallel execution. This paper develops a simple, generalized AD algorithm calculated from a simple, natural specification. The general algorithm is then specialized by varying the representation of derivatives. In particular, applying well-known constructions to a naive representation yields two RAD algorithms that are far simpler than previously known. In contrast to commonly used RAD implementations, the algorithms defined here involve no graphs, tapes, variables, partial derivatives, or mutation. They are inherently parallel-friendly, correct by construction, and usable directly from an existing programming language with no need for new data types or programming style, thanks to use of an AD-agnostic compiler p...
Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discre... more Animation is a temporally continuous phenomenon, but is typically programmed in terms of a discrete sequence of changes. The use of discreteness serves to accommodate the machine that is presenting an animation, rather than the person modeling an animation with the help of a computer. Using a continuous model of time for animation allows for natural specification, avoiding some artificial details, but is difficult to implement with generality, robustness and efficiency. This paper presents and motivates continuous modeled animation, and sketches out a naive functional implementation for it. An examination of some of the practical problems with this implementation leads to several alternate representations, all of which have difficulties in themselves, some quite subtle. We hope that the insights and techniques discussed in this paper lead to still better representations, so that animation may be specified in natural terms without significant loss of performance.
This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animate... more This paper describes a program for allowing a user to navigate smoothly in a 2D (possibly animated) image, performing pan, zoom and rotation. The program is written in Haskell, using the Fran interactive animation library. The idea is that when a hand grasps the image, the image stuck to the hand at the grasped location. When only one hand is grasping, the image pans (translates). When two hands are grasping, the image pans, zooms and rotates as required in order to keep the two grasped locations under the hands. While the C++ program that inspired this exercise is quite long and complex, the Fran version is short, modular and we feel captures the essence of the behavior being modeled.
Uploads
Papers by Conal Elliott