Master2: Deep Learning for 3D Human Motion

We are looking for a highly motivated Master student to work on the subject of acquiring and identifying shapes in motion.

Context

Recent works in the scientific community show the increasing success of deep learning techniques in the field of computer vision, in which they were applied for a wide variety of tasks, such as feature extraction, segmentation, motion detection, object and action recognition. Recently the research community has taken interest in its applicability to a range of specific 3D problems, such as whether a Deep Convolutional Neural Network can capture feature space mappings between two 3D feature spaces [5], or a mapping between any available visual information of a subjects and its 3D shape [4], sometimes for specific families of objects, e.g. chairs [3].

The Morpheo team deals with the capture and motion analysis of human subjects from multi-camera studios [1,2], and operates its own 68 camera acquisition platform and cluster, http://kinovis.inrialpes.fr, shown in the figure above. Many promising applications exist for these techniques for 3D performance capture, 3DTV, cultural heritage preservation, and interactive & entertainment applications. In this internship, we wish to study whether deep learning techniques can be applied to enhance such capture, and more precisely or more robustly extract 3D information from captured images. Deep learning techniques offer the prospect of automatically modeling useful mappings that allow to directly extract elements of the 3D shape of an observed human, as well as limiting this extraction to plausible human shapes, as automatically inferred from a relevant human 3D model database.

During the internship, the student will familiarize with deep Learning CNN techniques, and relevant elements of 3D vision, in order to provide a model and application for an initial subset of human capture situations.

Objectives

The master student will perform the following tasks:

  • Study the relevant bibliography, identify relevant datasets available from other research groups
  • Discuss and propose a solution relevant to this problem with advisors
  • Exhibit a preliminary implementation of the proposed solution
  • Validate this solution on a reasonable size available or acquired dataset. The student will have access to the Kinovis platform to perform her/his own 3D acquisitions.
  • Write a Master’s thesis with details of the proposed method, with bibliography and experiments.

Student profile

  • Master student – preferably in Computer Science or Applied Mathematics.
  • Creative and highly motivated
  • Solid programming skills, Python, C++ and/or Matlab
  • Solid mathematics knowledge in linear algebra, geometry, and statistics.
  • Fluent English or French spoken.
  • Prior courses or knowledge in the areas of computer vision, computational geometry, mesh processing, computer graphics, signal processing, machine learning or bayesian inference is a plus

Duration: 5-7 months

Start date: February 2017.

Location: Inria Grenoble Rhône-Alpes, France.

How to apply

Please send applications through this page:
http://morpheo.inrialpes.fr/job-applications/

  • a complete CV
  • graduation marks, rankings
  • the name and email address of references, if relevant
  • previous thesis reports, if available

Advisors

Jean-Sébastien Franco, Morpheo team
Stefanie Wuhrer, Morpheo Team
LJK & Inria
E-mail : jean-sebastien.franco@inria.fr
E-mail : stefanie.wuhrer@inria.fr

References

[1] Estimation of Human Body Shape in Motion with Wide Clothing, Jinlong Yang, Jean-Sébastien Franco, Franck Hétroy-Wheeler, Stefanie Wuhrer, European Conference on Computer Vision 2016, Oct 2016, Amsterdam, Netherlands

[2] Volumetric 3D Tracking by Detection, Chun-Hao Huang, Benjamin Allain, Jean-Sébastien Franco, Nassir Navab, Slobodan Ilic, Edmond Boyer. CVPR 2016 – IEEE Conference on Computer Vision and Pattern Recognition, Jun 2016, Las Vegas, United States

[3] Learning to Generate Chairs with Convolutional Neural Networks, Alexey Dosovitskiy, J. Springenberg, Thomas Brox, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2015

[4] Learning camera viewpoint using CNN to improve 3D body pose estimation, Mona Fathollahi Ghezelghieh, Rangachar Kasturi, Sudeep Sarkar, accepted at 3DV 2016

[5] Learning shape correspondence with anisotropic convolutional neural networks, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M. Bronstein, accepted at NIPS, 2016