Visualizing light transport phenomena with a primal-dual coding video camera

We present a primal-dual coding video camera, an optical device that captures video in which the flow of light through a scene has been manipulated. This camera has the ability to visualize indirect transport effects (caustics, inter-reflections, volumetric scattering, etc.), measure direct-only light paths (surface reflections), capture transport that occurs within specific regions of 3D space, and perform structured light imaging in the presence of complex indirect effects. The operating principle behind this technology is to simultaneously code the light that goes into a scene with the light that comes out. Specifically, over the course of a single camera exposure period, we project a sequence of light patterns onto the scene in lockstep with a second sequence of mask patterns that modulates the light incident on the camera sensor. This optical procedure creates RAW, unprocessed photos where a scene's light transport function appears to have changed.