PhD Subject 2012 - Learning Rigidities
Estimation of local surface rigidity parameters of tracked 3D shapes
Advisors
Jean-Sebastien Franco, Morpheo team
Edmond Boyer, Morpheo Team
LJK & Inria
E-mail : jean-sebastien.franco@inria.fr
Tel.: 04 56 52 71 30
E-mail : edmond.boyer@inrialpes.fr
Tel.: 04 76 61 53 54
Context
This project is part of the long term research effort in the Morpheo team to identify intrinsic properties of objects in motion, as observed with multi-camera video streams. We are particularly interested in learning, by simple observation of the sequence, rigidity properties of the observed surface. The work is encouraged by recent breakthroughs in recent surface tracking methods, including in those presented in the Morpheo team, which allow robust tracking of humans and objects under large and deformable motion. Such methods provide a powerful building block for analysis of intrinsic properties of the underlying surface.
While the subject of marker-less temporal surface tracking from severals is gaining more and more attention, few methods exist to extract such surface deformation properties at a local level. The problem is challenging, because of the high dimensionality required to parameterize local deformation, noise, and partial observability issues. Intrinsic surface properties are hidden to the observer and must be extracted from redundant information among the different views and time instances. Observed surface motion is often corrupt or biased due to observation conditions, lack of local texture, visual ambiguity and occlusion, imposing the use of robust, probabilistic methods. Our goal in this thesis is to explore new models of such surface deformation and conceive the associated new robust estimation algorithms, to retrieve their parameters given a set of multi-view videos of the moving object. We typically expect breakthrough results building on geometric analysis, probabilistic reasoning on geometric structures, spectral analysis of meshes, computational geometry and deformable tracking.
Pre-requisites include a background and strong interest in computer vision and probabilistic modeling, skills in computer graphics or applied geometry are a plus.
Background
Different strategies exist to identify the rigid structure of a geometric model. A number of methods segment static models based on geometry alone [1], but this assumes a correlation between geometric features and the underlying motion partition, and as such does not segment according to the kinematic definition of rigidity.
A number of computer vision techniques successfully segment moving point clouds into rigid parts based on point motions alone [2], yet do not attempt to leverage or infer proximity information on the observed surface.
A family of recent methods have brought solutions to the rigid segmentation problem based on spectral clustering [3], in the context of the more general shape matching problem.
Recent surface tracking techniques over-segment the template in a set of rigid patches as initialization, then use this patch-rigid structure to constrain surface tracking [4]. An attempt to learn this segmentation during tracking has also been presented [5]. Our interest and goal in this internship is to exhibit a model similar to [4], but where the characteristics of local rigid links between patches are learned during tracking. One could for example link patches with an inter-patch spring model, whose spring rigidity parameters are to be inferred online by the estimation and tracking method.
Development will be carried out in C++ and/or Matlab.
Keywords: rigidity, multi-view shape modeling, rigid segmentation, mass-spring model.
References
[1] A. Shamir. A survey on mesh segmentation techniques. Computer Graphics Forum, 27(6):1539–1556, 2008
[2] J. Yan and M. Pollefeys. A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, de- generate and nondegenerate. In ECCV, pages 94–106, 2006.
[3] A. Sharma, E. von Lavante, and R. Horaud. Learning shape segmentation using constrained spectral clustering and prob- abilistic label transfer. In ECCV, volume 6315, pages 743– 756, 2010.
[4] C. Cagniart, E. Boyer, and S. Ilic. Probabilistic deformable surface tracking from multiple videos. In ECCV, pages 326– 339, 2010.
[5] Franco, Jean-Sebastien; Boyer, Edmond; , Learning temporally consistent rigidities, Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on , vol., no., pp.1241-1248, 20-25 June 2011