Shape from Specular Flow

An image of a purely specular (mirror-like) object is just a distortion of its surrounding illumination environment. and there is interest in understanding when and how a specular shape can be recovered from these distortions. Unlike most of the previous studies, we assume that only little or nothing about the illumination environment is unknown. However, when motion is incorporated the problem became more traceable.

We explore the specular flow which is invariant to the environment content and hence can be exploited for the reconstruction task when illumination environment is unknown. We have developed the foundations of the shape-from-specular-flow problem and showed that the specular flow is directly related to surface shape through a non-linear partial differential equation (ICCV2007, CVPR2008, PAMI2010). Recently, we have shown that a suitable re-parameterization leads to a linear formulation of the shape from specular flow equation (ICCV2009).

Complex Motion Estimation

In (CVPR2011) we explore a polar representation of optical flow in which each element of the brightness motion field is represented by its magnitude and orientation instead of its Cartesian projections. This seemingly small change in representation provides more direct access to the intrinsic structure of a flow field, and when used with existing variational inference procedures it provides a framework in which regularizers can be intuitively tailored for very different classes of motion.
Our evaluations reveal that a flow estimation algorithm that is based on a polar representation can perform as well or better than the state-of-the-art when applied to traditional optical flow problems concerning camera or rigid scene motion. At the same time, it facilitates both qualitative and quantitative improvements for untraditional cases such as fluid flows and specular flows, whose structure is very different. Our source code is made available here .

Specular Flow Benchmark



Recently, the specular flow -- the vector field that is induced on the image plane as a result of a relative motion between the camera, object, or environment - is exploited to facilitate diverse tasks in computer vision such as shape inference, 3D pose estimation and detection of rigid objects. The specular flow is by definition an instance of optical flow and yet the estimation of specular flow from image sequences is challenging task.

We have established a benchmark dataset with real image sequences with their corresponding specular flow ground truth (BMVC2010). Establishing this database involves the creation of specular objects with ground truth shape using a state-of-the-art, high precision 3D printer and the acquisition of specular image sequences using a custom made device. This benchmark fills several missing aspects in the Middlebury optical flow dataset and crucial for the shape from specular flow research. Additionally, it supplies a unique case-study for estimation of complex motion field in which the motion structure is complicated enough to raise new challenges to existing algorithms. At the same time, there is no violation of the brightness constancy assumption between frames in this benchmark; it allows focusing in the motion structure separately then the diffusion which is common in many cases.