Deep Single-View 3D Object Reconstruction with Visual Hull Embedding
Hanqing Wang1 Jiaolong Yang2 Wei Liang1 Xin Tong2
1Beijing Institute of Technology 2Microsoft Research Asia
Abstract
3D object reconstruction is a fundamental task of many robotics and AI problems. With the aid of deep convolutional neural networks (CNNs), 3D object reconstruction has wit- nessed a significant progress in recent years. However, possi- bly due to the prohibitively high dimension of the 3D object space, the results from deep CNNs are often prone to miss- ing some shape details. In this paper, we present an approach which aims to preserve more shape details and improve the reconstruction quality. The key idea of our method is to lever- age object mask and pose estimation from CNNs to assist the 3D shape learning by constructing a probabilistic single- view visual hull inside of the network. Our method works by first predicting a coarse shape as well as the object pose and silhouette using CNNs, followed by a novel 3D refinement CNN which refines the coarse shapes using the constructed probabilistic visual hulls. Experiment on both synthetic data and real images show that embedding a single-view visual hull for shape refinement can significantly improve the re- construction quality by recovering more shapes details and improving shape consistency with the input image.
Publication
Deep Single-View 3D Object Reconstruction with Visual Hull Embedding
Hanqing Wang,
Jiaolong Yang,
Wei Liang,
Tong Xin
Association for the Advancement of Artificial Intelligence 2019 (AAAI 2019 Oral)
Paper
, Video
, Slide, Code
BibTex
@inproceedings{wang2019deep,
title=
{Deep Single-View 3D Object Reconstruction with Visual Hull Embedding},
author = {Wang, Hanqing and Yang, Jiaolong and Liang, Wei and Tong, Xin},
booktitle = {Proceedings of the AAAI},
year = {2019}
}
- 媒体计算与智能系统实验室
- Media Computing and Intelligent Systems Lab
Beijing Institute of Technology Copyright Address: 5 South Zhongguancun
Street, Haidian District, Beijing Postcode: 100081