EPVH

What is EPVH?

EPVH is a visual hull reconstruction tool, to build 3D models of objects from their silhouettes as obtained by a set of calibrated cameras. A binary executable is provided for Linux, Mac and Windows systems. The tool is accessible from the command line (reconstruct executable) or as a simple C++ API to be included in your programs (linux-only). It builds a polyhedron mesh representation of a scene, the silhouette images and calibrated view information of which you provide. It is based on research code developed during my PhD, and is based on the method presented in the following research papers: Efficient Polyhedral Modeling from Silhouette. EPVH is provided as-is, with no warranty, and is free for non-commercial use. It is subject to a free-software License. EPVH has minor system requirements. Please let me know if this program is useful to you. If so, please cite my work/link to my web page.

What's new in version 1.1?

• Algorithm is made O(n^2 p log p) instead of O(n^2 p^2), with n the number of views, and p the maximum number of input contour points in a view. The optimization brings significant speed increases to all datasets. The speedup for datasets with input contour polygons exhibiting high vertex counts (~10'000) is very large (x30). Details can be found in the journal article.
• Many more command line controls, documentation improved.
• Binaries made available for Mac and Windows platforms!

Program usage

The input for the executable "reconstruct" is a calibration file with N 3x4 calibration matrices, and N greyscale image files containing the silhouette mask, in P5 raw PGM format (0 = background, non-zero = foreground). Silhouette contour polygons can also be used as input on the command line (see command line help for details). An example dataset with 12 views is provided in the Example directory of the distribution.
The output is either an OFF model file with triangles (geomview readable, very simple to input), or a PMESH description (a home-breed OFF extension described here). The program decides which format to generate according to the extension of the provided output file (.off -> OFF, .pmesh -> PMESH). Further instructions can be obtained by typing "reconstruct" with no arguments on the command line.

File formats

• The input calibration file is a simple ASCII file containing the 3x4 projection matrices as blank separated floats, in sequence for all of the N views.
   
• Input silhouette contour polygons can be used instead of PGM images, using a single ASCII file with the following information:
  
  IMGa NVa X1 Y1 X2 Y2 ... X_NVa Y_NVa    <-- contour polygon a
  ...
  IMGz NVz X1 Y1 X2 Y2 ... X_NVz Y_NVz    <-- contour polygon z
  
  where IMGx is the view number the contour x belongs to, numbered in 0..N-1; NVx is the number of vertices in contour polygon x, followed by 2*NVx floats corresponding to the 2D euclidean coordinates of each contour vertex in the image plane. Several outside (ccw) and inside (cw) contours can be used for the same IMG to describe the silhouette as a general polygon, with possibly several unconnected components and holes. Contours must appear ordered in the file, IMGa <= ... <= IMGz. Such a file can be generated from a set of PGM images with the reconstruct tool using the -oc option (see command line help).



• Output mesh files:
  
  There are two output file formats to the 'reconstruct' program: OFF (.off extension) triangle mesh and PMESH (.pmesh extension). For all purposes both of these formats share the exact same OFF layout, and differ only in the interpretation of contour geometry:
  
  OFF
  V C 0                               <-- nb of verts, contours, edges(unused)
  X Y Z                               <-- coords for vertex 0
  ...
  X Y Z                               <-- coords for vertex V-1
  S IDX0 IDX1 ... IDX_S-1 # IMG ID    <-- contour 0 descr., 2 extra fields
  ...
  S IDX0 IDX1 ... IDX_S-1 # IMG ID    <-- contour C-1 descr., 2 extra fields
  
  'OFF' appears on the first line for geomview compatibility. On the second line, the number of vertices appear, then the number of contours, then 0. Then on the next lines the vertex array is described. Finally, a contour is described per file line: first the number of vertices appear, then the corresponding number of indexes in the vertex array. Two additional EPVH-specific fields are added for convenience behind the comment character '#': IMG is the view number of the silhouette which generated the current polygonal face (numbered in 0..N-1), and ID is an identifier describing which polygonal face this contour belongs to.
  
  OFF/Geomview interpretation: Each contour describes a single convex polygon of size S. When this mode is used in EPVH, faces of the visual hull polyhedron are tesselated with triangles. Coplanar triangles, which originate from the same viewing cone face, share the same IMG and ID identifiers. For a complete reference of the OFF file format see the geomview documentation.
  
  PMESH interpretation: Each contour describes one contour among several possible in a general polygonal face of the polyhedron. A face here is meant in the most general sense: a face is a general polygon, with possibly several outside contours (meaning there can be several non-connected coplanar components to the same face) AND several inside contours (meaning each component can have one or several planar holes). Distinction between outside and inside contours is implicit in orientation (ccw for outside contours, cw for inside contours). In practice, two contours which belong to the same face will have the same IMG number and the same ID number.
  It is not guaranteed that this file is a geomview file, thus the distinction. This is because of the possible presence of inside contours (holes), and non-convex contours in the polygon face representation. Both will be misrepresented by geomview. This is why the other mode (OFF triangle mesh) is provided, because the only way to correctly represent the polyhedron in a simple graphics framework is to tesselate such non-trivial faces. Nevertheless, you can always display the PMESH using geomview, while staying aware of these limitations. In all cases you will have a very good idea of what the visual hull looks like, even if some of the geometry is misdrawn.

   

EPVH Mini-API (linux only)

A small API and default command-line front-end source is provided as a means to link your program against EPVH in linux. The source for the front-end (see file src/reconstruct.cpp in the archive), as well as the .pro and qmake-generated makefiles are provided as examples on how to use the linux shared library libepvh in your code. After checking the requirements for EPVH, you can compile the project by typing in a shell "qmake project.pro" while in the main directory of the archive, then "make".
You can use src/reconstruct.cpp as a basis to build your own program. Data structures are straightforward and explained in the provided headers (particularly include/epvh.h). Here is a short introduction to these classes, in include/epvh.h:

PMESH

• the EPVH class is the main reconstructor class. You must instantiate this class once in your program, build the necessary input structures, and instantiate output structures. You must then call the reconstruct method with input and output structures passed as argument. After this call, the output structure is filled up and can be freely accessed to get the mesh data.
• the EPVH::Input class is simply an STL vector of per-camera data structures, each structure holding the projection matrix and image corresponding to a camera. An example of how to load projection matrices and images into this structure is provided in the main source file for the reconstruct program, src/reconstruct.cpp, in the load function.
• there are two possible output classes, EPVH::Mesh and EPVH::PMesh. Each of these output types comes with its own overloaded version of the EPVH::reconstruct method, to reconstruct the visual hull, and fill one ouptut structure or the other appropriately.

EPVH::Mesh is a triangular mesh description of the visual hull polyhedron. Two STL vectors are used to store vertex coordinates, and triangles (described by 3 indices in the vertex array). This internal structure is straightforwardly used by the reconstruct program to generate the OFF triangle mesh. As such, an example of usage of the EPVH::Mesh class is given in the dumpOff function of the main source file src/reconstruct.cpp.

EPVH::PMesh is a polygonal mesh description of the visual hull polyhedron. Three STL vectors are used to store vertex coordinates, contour headers, and contour index tables. This internal structure is straightforwardly used by the reconstruct program to generate the description file. As such, an example of usage of the EPVH::PMesh class is given in the dumpPMesh function of the main source file src/reconstruct.cpp.

FAQ

here

• What do I need to run EPVH?
  
  Most of the time nothing: it should work out of the box. Under all platforms, the standard OpenGL and GLU libraries should be available.
  
  Windows The binary only depends on the standard OpenGL DLLs and GLU DLLs. Both default Windows and vendor provided drivers should work, because EPVH only uses the GLU tesselation, usually implemented in software.
  
  Mac The Mac binary should function directly on MacOSX Tiger 10.4, and is a universal binary which runs under both PowerPC and Intel Macs.
  
  Linux Manual intervention might be required to install GL and GLU drivers, if not available by default with your distribution. Gcc 3.4 or above should be available on your platform, gcc 4 or above is very strongly recommended (to compile AND to run epvh). This can be checked by typing "gcc -v" in a shell. Systems with versions of gcc below 3.4 simply can't be expected to work with the provided binary, because of a change in the ABI specification from gcc 3.4 on (previous versions link to libstdc++.so.5). The binary .so provided thus links dynamically to libstdc++.so.6, which should be available with recent systems, shipping the newer gcc's. If you run into problems, you can use the command "ldd reconstruct" in a shell when in the EPVH directory, to make sure libstdc++.so.6 is found by the system. Also, ensure that the path to libepvh.so.* (in the lib directory of the distribution package) is listed in your LD_LIBRARY_PATH environment variable, or else when running reconstruct you will get an error message, "libepvh.so not found".
  
   
• What do I need to compile and link with EPVH under linux?
  
  First ensure you can run EPVH, with the reconstruct binary already provided. Under linux, this executable can be rebuilt using qmake, which is available by default on most systems shipping with KDE. Qmake can be bypassed by removing references to the pro file build, in the provided makefile. Again g++ 3.4 or above is required, g++4 or above recommended.



• Any future open source release? This is unlikely to happen due to commercial applications being developed on the basis of EPVH. Commercial licenses to EPVH will possibly be available in the near future.



• I run the program with my input but absolutely nothing is reconstructed. Why?

When no geometry is reconstructed, the problem is very likely to come from the calibration matrices.
Check your coordinate systems: the chosen convention is the same as in the vision textbooks (see for example)

Possible errors include:
- non direct basis chosen for the world or camera coordinates system
- erroneously shuffled order of view matrices and their respective images
- wrong camera basis chosen: x and y base vectors must be oriented in raster order (x pointing right, y pointing down, z toward the scene)
For example if the camera z points outwards (with respect to the scene) it won't work.



• Why do the projection matrices in the provided example have an identical last column?
  
  The fourth column of the projection matrix represents where the origin of the world coordinate system projects in image space. In the provided example all views are placed on a sphere and point directly to the origin of the world coordinate system. Consequently in all views, the fourth column in this dataset exceptionally matches the principal point of the 300x300 image, in homogeneous coordinates which turn out identical (270 270 1.8), or (150 150). The fact that they are exactly identical digit-to-digit here is also because the views were synthetically generated, with identical intrinsic parameters. This would be numerically impossible to achieve with real-life calibrations, which are a result of numerical optimization.
   

History

• Version 1.1: released January 22th, 2007
   
• Version 1.0 (first non-beta release): released June 28th, 2006
  Many bugs corrected. Tested against a dataset of 1760 reconstructions with only one error. See  journal paper for details about the algorithm.
   
• Version 1.0.1-beta: released January 4th, 2005
File format enhancements. Introduction of PMESH, and additional OFF fields.


• Version 1.0.0-beta: released July 27th, 2004
  Added library support in this version: the main program source is released and the library implementation provided in an .so file. Only the triangle mesh structure and OFF triangle output format is supported. Directory tree reorganized.
   
• Versions 0.1 and 0.11 beta: released February, 2004
  Initial stand-alone executable versions dumping a tessellated OFF triangle mesh of the visual hull, with no shared library.

If you encounter problems, feel free to contact me.