“Computational Plenoptic Imaging” by Ihrke, Lanman, Heidrich and Akeley
Conference:
Type:
Entry Number: 09
Title:
- Computational Plenoptic Imaging
Presenter(s)/Author(s):
Abstract:
Prerequisites
This introductory-level course has no prerequisites.
Abstract
Spawned by the introduction of the Lytro light field camera to the consumer market and recent accomplishments in the speed at which light can be captured [Raskar et al. 2011] , a new generation of computational cameras is emerging. By exploiting the co-design of camera optics and computational processing, these cameras capture unprecedented details of the plenoptic function — a ray-based model for light that includes the color spectrum as well as spatial, temporal, and directional variation. Although digital light sensors have greatly evolved in the last years, the visual information captured by conventional cameras has remained almost unchanged since the invention of the daguerreotype. All standard CCD and CMOS sensors integrate over the dimensions of the plenoptic function as they convert photons into electrons; in the process all visual information is irreversibly lost, except for a two-dimensional, spatially-varying subset — the common photograph. In this course, we review the plenoptic function and discuss approaches that aim at optically encoding high-dimensional visual information that is then recovered computationally in post-processing.
This course is intended to review the state of the art in joint optical light modulation and computational reconstruction of visual information transcending that captured by traditional photography. In addition to the plenoptic dimensions, we also consider high dynamic range image acquisition as common sensors have a limited dynamic range. In contrast to prior courses on general computational photography [Raskar and Tumblin 2006,2007] and a recent survey on the topic [Wetzstein et al. 2011], this course gives a broad, well-structured, and intuitive overview of all aspects of plenoptic image acquisition and focuses on two recent developments: light field acquisition and ultra-fast cameras. We unveil the secrets behind capturing light at a trillion frames per second and the Lytro camera. Our course serves as a resource for interested parties by providing a categorization of recent research and help in the identification of unexplored areas in the field.
We will discuss all aspects of plenoptic image acquisition in detail. Specifically, we begin by giving an overview of the plenoptic dimensions and show how much of this visual information is irreversibly lost in conventional image acquisition. We proceed by discussing the state of the art in joint optical modulation and computation reconstruction for the acquisition of high dynamic range imagery as well as spectral information. Two parts, focusing on light field acquisition and ultra-fast optics respectively, will unveil the secrets behind imaging techniques that have recently been featured in the news. We outline other aspects of light that are of interest for various applications and wrap the course up with a short summary, while leaving enough time for questions and a short discussion.
