Master2: Motion tracking of a plant during its growth

We are looking for a highly motivated Master student to work on the subject of tracking the motion of a plant during the growth process.

Fig. 1. 3D mesh model of an Averrhoa Carambola leaf with and without texture.

Fig. 2. Left: approximation of the leaf spine for 70 successive time steps. Right: spine for the first and last time steps.

Context

The Carambole project aims at constructing a virtual space-time 3D model of a growing plant and using this model to give accurate measurements about the plant motion. Of particular interest is the Averrhoa Carambola, which possesses a nutation motion of unknown origin. Targeted measurements include this nutation motion, as well as the growth along the spine of a leaf.

Reconstruction is done using a multi-camera set-up, which enables to create independent mesh models at each time step [2], see Fig. 1 above. In order to recover the motion of the plant, leaves should then be tracked over time. State-of-the-art shape tracking approaches such as [3] were developed for human shapes and rely on the hypothesis that the motion is quasi-rigid. As a consequence, they cannot be used here. Even more general approaches such as [4] do not account for the appearance of new organs such as leaflets. The goal of this Master internship is thus to develop a new tracking method which would enable to process shapes with appearing or disappearing geometries, such as growing plants.

A first approach has been implemented last summer by two interns. It starts by computing a skeleton [1] inside the leaf spine at each time step (see Fig. 2 above). Matching between successive meshes is then done by detecting and matching SIFT features. This approach will be the starting point of the internship, but needs to be completed, evaluated and possibly extended to other plants such as lentils and fruit trees if results are promising.

Objectives

The master student will perform the following tasks:

  • Study the relevant bibliography about non-rigid shape tracking;
  • Discuss and propose a solution relevant to this problem with advisors;
  • Exhibit a preliminary implementation of the proposed solution;
  • Validate this solution on acquired datasets of Averrhoa Carambola and other plants. The student will have access to the Kinovis platform to perform her/his own 3D acquisitions;
  • Write a Master’s thesis with details of the proposed method, bibliography and experiments.

Keywords
Shape tracking, motion, mesh processing, skeleton.

Student profile

  • Master student – preferably in Computer Science or Applied Mathematics.
  • Creative and highly motivated.
  • Solid programming skills; the project involves programming in Python and using Matlab.
  • Solid mathematics knowledge, especially in linear algebra and discrete geometry.
  • Prior courses or knowledge in the areas of computer vision, mesh processing, computational geometry is a plus.
  • Fluent English or French spoken.

Duration: 5-7 months

Start date: February 2017.

Location: Inria Grenoble Rhône-Alpes, France.

How to apply

Please send applications through this page:
http://morpheo.inrialpes.fr/job-applications/

  • a complete CV
  • graduation marks, rankings
  • the name and email address of references, if relevant
  • previous thesis reports, if available

Advisors

Frannck Hétroy-Wheeler, Morpheo team
Julien Pansiot, Morpheo Team
LJK & Inria
E-mail : franck.hetroy@grenoble-inp.fr
E-mail : julien.pansiot@inria.fr

References

  1. Harmonic Skeleton for Realistic Character Animation , Grégoire Aujay, Franck Hétroy, Francis Lazarus, Christine Depraz. ACM-SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) 2007.
  2. QuickCSG: Arbitrary and Faster Boolean Combinations of N Solids, Matthijs Douze, Jean-Sébastien Franco, Bruno Raffin. Inria Research Report RR-8687, 2015.
  3. Volumetric 3D Tracking by Detection, Chun-Hao Huang, Benjamin Allain, Jean-Sébastien Franco, Nassir Navab, Slobodan Ilic, Edmond Boyer. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016.
  4. Computing temporal alignments of human motion sequences in wide clothing using geodesic patches, Aurela Shehu, Jinlong Yang, Jean-Sébastien Franco, Franck Hétroy-Wheeler, Stefanie Wuhrer. International Conference on 3D Vision (3DV) 2016.