3D Motion and Shape Reconstruction from 2D Images
People
 A. Chiuso, P. Favaro, H. Jin, S. Soatto.
References
  • Reducing Structure From Motion: Part 1 and Part 2 (IEEE Trans. PAMI, 1998)
  • Motion Estimation using Subspace Constraints (IJCV, 1997)
  • Motion Estimation via Dynamic Vision (IEEE Trans. TAC, 1996)
Synopsis
 The problem of ``Structure From Motion'' (SFM) deals with extracting three-dimensional information about the environment from the motion of its projection onto a two-dimensional surface. The most outstanding example of machinery to deal with this problem is the combination of the human eye and brain: from the projection of moving objects onto the retina, we are able to gather a three-dimensional representation that is sufficient for us to reach for them, manipulate them, walk around them etc. In engineering, Control Systems using Vision as a sensor  on structured environments (for instance freeways or interior of buildings) have recently achieved astonishing performance and robustness (see for instance the work of Dickmanns and his coworkers). However, engineering systems are far from achieving the flexibility of primates, in that changes of the environment (for instance from a freeway to a dirt road) require a complete restructuring and reconfiguration of the system.

At the highest level of generality, SFM is an extraordinarily complicated problem.  Most of the research on SFM during the past 20 years has concentrated on a  representation of the environment  as a set of points in 3-D space that move rigidly relative to the imaging surface (the retina or the CCD sensor of a video-camera). The goal of SFM is then to estimate the 3-D shape and motion of such a collection of feature points given either the velocity of their perspective projection onto the imaging sensor (optical flow), or the correspondence between projections taken from different vantage points (feature correspondence).
Such a restriction, however, is a deceptive simplification of the original problem of SFM, for the representation of the environment using feature points does not account for visually complex phenomena such as the motion of the foliage of a tree or a that of a silk gown.  The most recent research in SFM aims at exploring novel representation of the environment. Such representation depend crucially upon the use the estimates of shape are intended for. In this sense, sensing and control are very tightly connected. It is a primary agenda of our laboratory to explore the issue of shape and motion estimation in a variety of contexts that range from autonomous robotics (navigation, manipulation, docking, tracking), to industrial automation, CAD, Computer Graphics, Human-Machine interfaces, medical robotics etc.