When there is relative motion along the optical axis between a real-aperture camera and a 3D scene, the sequence of images captured will not only be space-variantly defocused but will also exhibit pixel motion due to motion parallax. Existing single viewpoint techniques such as shape-from-focus (SFF)/depth-from-defocus (DFD) and axial stereo operate in mutually exclusive domains. SFF and DFD assume no pixel motion and use the focus and defocus information, respectively, to recover structure. Axial stereo, on the other hand, assumes a pinhole camera and uses the disparity cue to infer depth. We show that in real-aperture axial stereo, both blur and pixel motion are tightly coupled to the underlying shape of the object. We propose an algorithm which fuses the twin cues of defocus and parallax for recovering 3D structure. The effectiveness of the proposed method is validated with many examples. © 2009 Springer Berlin Heidelberg.

A. N. Rajagopalan

Department of Electrical Engineering

3D SCENES

3D structure

Defocus

Depth from defocus

MOTION PARALLAX

OPTICAL AXIS

PIN-HOLE CAMERAS

RELATIVE MOTION

SEQUENCE OF IMAGES

SHAPE FROM FOCUS

TIGHTLY-COUPLED

Astrophysics

Cameras

Geometrical optics

Pattern Recognition

Pixels

three dimensional

IIT Madras is a public technical and research university located in Chennai, Tamil Nadu. Founded in 1959, it is recognised as an Institute of National Importance.

IIT Madras has been ranked as the top engineering institute in India for four years in a row by the National Institutional Ranking Framework of the MHRD

It currently offers undergraduate, postgraduate and research degrees across 16 disciplines in Engineering, Sciences, Humanities and Management. About 596 faculty belonging to science and engineering departments and centres of the Institute are engaged in teaching, research and industrial consultancy.

IIT Madras

Real Aperture Axial Stereo: Solving for Correspondences in Blur

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Traditional depth estimation methods typically exploit the effect of either the variations in internal parameters such as aperture and focus (as in depth from defocus), or variations in extrinsic parameters such as position and orientation of the camera (as in stereo).When operating off-the-shelf (OTS) cameras in a general setting, these parameters influence the depth of field (DOF) and field of view (FOV). While DOF mandates one to deal with defocus blur, a larger FOV necessitates camera motion during image acquisition. As a result, for unfettered operation of an OTS camera, it becomes inevitable to account for pixel motion as well as optical defocus blur in the captured images. We propose a depth estimation framework using calibrated images captured under general camera motion and lens parameter variations. Our formulation seeks to generalize the constrained areas of stereo and shape from defocus (SFD)/focus (SFF) by handling, in tandem, various effects such as focus variation, zoom, parallax and stereo occlusions, all under one roof. One of the associated challenges in such an unrestrained scenario is the problem of removing user-defined foreground occluders in the reference depth map and image (termed inpainting of depth and image). Inpainting is achieved by exploiting the cue from motion parallax to discover (in other images) the correspondence/color information missing in the reference image. Moreover, considering the fact that the observations could be differently blurred, it is important to ensure that the degree of defocus in the missing regions (in the reference image) is coherent with the local neighbours (defocus inpainting). © 2011 Springer Science+Business Media, LLC.

International Journal of Computer Vision

Towards unrestrained depth inference with coherent occlusion filling

We propose a new method that extends the capability of shape-from-focus (SFF) to estimate the depth profile of 3-D objects in the presence of structure-dependent pixel motion. Existing SFF techniques work under the constraint that there is no parallax in the captured stack of frames. However, in off-the-shelf cameras, there can be appreciable pixel motion among the observations when there is relative motion between the object and the camera. In such a scenario, the depth estimates will be erroneous if the parallax effect is not factored in. Our degradation model accounts for pixel migration effects in the observations due to parallax resulting in a generalization of the SFF technique. We show that pixel motion and defocus blur therein are tightly coupled to the underlying shape of the 3-D object. Simultaneous reconstruction of the underlying 3-D structure and the all-in-focus image is carried out within an optimization framework using local image operations. The proposed method when tested on many examples, both synthetic and real, is very effective and delivers state-of-the-art performance. © 2011 IEEE.

IEEE Transactions on Image Processing

Dealing with parallax in shape-from-focus

Real aperture axial stereo: Solving for correspondences in blur

Journal	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN	03029743
Open Access	No