Master2: Deep Learning for 3D Human Motion

We are looking for a highly motivated Master student to work on the subject of acquiring and identifying shapes in motion.

Context

Recent works in the scientific community show the increasing success of deep learning techniques in the field of computer vision, in which they were applied for a wide variety of tasks, such as feature extraction, segmentation, motion detection, object and action recognition. Recently the research community has taken interest in its applicability to a range of specific 3D problems, such as whether a Deep Convolutional Neural Network can capture feature space mappings between two 3D feature spaces [5]. For human body modeling, deep networks have recently been used to model body shape based on silhouettes [3] and to model the motion of sparse motion capture data [4].

The Morpheo team deals with the capture and motion analysis of human subjects from multi-camera studios [1,2], and operates its own 68 camera acquisition platform and cluster, http://kinovis.inrialpes.fr, shown in the figure above. Many promising applications exist for these techniques for 3D performance capture, 3DTV, cultural heritage preservation, and interactive & entertainment applications. In this internship, we wish to study whether deep learning techniques can be applied to enhance such capture, and more precisely or more robustly extract 3D information from captured images. Deep learning techniques offer the prospect of automatically modeling useful mappings that allow to directly extract elements of the 3D shape and motion of an observed human, as well as limiting this extraction to plausible human shapes, as automatically inferred from a relevant human 3D model database.

During the internship, the student will familiarize with deep Learning CNN techniques, and relevant elements of 3D vision, in order to provide a model and application for an initial subset of human capture situations.

Objectives

The master student will perform the following tasks:

  • Study the relevant bibliography, identify relevant datasets available from other research groups
  • Discuss and propose a solution relevant to this problem with advisors
  • Exhibit a preliminary implementation of the proposed solution
  • Validate this solution on a reasonable size available or acquired dataset. The student will have access to the Kinovis platform to perform her/his own 3D acquisitions.
  • Write a Master’s thesis with details of the proposed method, with bibliography and experiments.

Student profile

  • Master student – preferably in Computer Science or Applied Mathematics.
  • Creative and highly motivated
  • Solid programming skills, Python, C++ and/or Matlab
  • Solid mathematics knowledge in linear algebra, geometry, and statistics.
  • Fluent English or French spoken.
  • Prior courses or knowledge in the areas of computer vision, computational geometry, mesh processing, computer graphics, signal processing, machine learning or bayesian inference is a plus

Duration: 5-7 months

Start date: February 2018.

Location: Inria Grenoble Rhône-Alpes, France.

How to apply

Please send applications through this page:
http://morpheo.inrialpes.fr/job-applications/

  • a complete CV
  • graduation marks, rankings
  • the name and email address of references, if relevant
  • previous thesis reports, if available

Advisors

Jean-Sébastien Franco, Morpheo team
Stefanie Wuhrer, Morpheo Team
LJK & Inria
E-mail : jean-sebastien.franco@inria.fr
E-mail : stefanie.wuhrer@inria.fr

References

[1] Estimation of Human Body Shape in Motion with Wide Clothing, Jinlong Yang, Jean-Sébastien Franco, Franck Hétroy-Wheeler, Stefanie Wuhrer, ECCV, 2016

[2] Volumetric 3D Tracking by Detection, Chun-Hao Huang, Benjamin Allain, Jean-Sébastien Franco, Nassir Navab, Slobodan Ilic, Edmond Boyer. CVPR, 2016

[3] HS-Nets : Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks, Endri Dibra, Himangshu Jain, Cengiz Oztireli, Remo Ziegler, Markus Gross, 3DV, 2016

[4] Learning Human Motion Models for Long-term Predictions, Partha Ghosh, Jie Song, Emre Aksan, Otmar Hilliges, 3DV, 2017

[5] Learning shape correspondence with anisotropic convolutional neural networks, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M. Bronstein, NIPS, 2016