Abstract

In this work we present an investigation of recent advances for implicit representations of 3D shapes for the task of 3D reconstruction from single images, and present a model that naturally extends to prediction of full color RGB meshes. In this context, we present SOTA results IoU and L2-Chamfer distance reconstruction metrics for the ShapeNet-cars dataset, and show qualitative results for its full color mesh outputs. The presented architecture relies local multi-scale 2D and 3D feature extraction, and on incorporating recent advances for training implicit functions, especially the 1-Cycle policy [1] and Fourier embedding of coordinates [2]. Furthermore, the contribution of different feature modalities (2D images, 3D colored density voxels) are quantitatively and qualitatively evaluated. Using multiple feature modalities shows distinctive advantages over related works, i.e. higher visual clarity of reconstructed meshes and lower memory requirement during training.

Input image	Resulting 3D colored Mesh

Download a summary of the project

View on Github

[1] Leslie N. Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learningrates, 2018.

[2] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains, 2020.