Header logo is

Patches, Planes and Probabilities: A Non-local Prior for Volumetric {3D} Reconstruction


Conference Paper



In this paper, we propose a non-local structured prior for volumetric multi-view 3D reconstruction. Towards this goal, we present a novel Markov random field model based on ray potentials in which assumptions about large 3D surface patches such as planarity or Manhattan world constraints can be efficiently encoded as probabilistic priors. We further derive an inference algorithm that reasons jointly about voxels, pixels and image segments, and estimates marginal distributions of appearance, occupancy, depth, normals and planarity. Key to tractable inference is a novel hybrid representation that spans both voxel and pixel space and that integrates non-local information from 2D image segmentations in a principled way. We compare our non-local prior to commonly employed local smoothness assumptions and a variety of state-of-the-art volumetric reconstruction baselines on challenging outdoor scenes with textureless and reflective surfaces. Our experiments indicate that regularizing over larger distances has the potential to resolve ambiguities where local regularizers fail.

Author(s): Ali Osman Ulusoy and Michael J. Black and Andreas Geiger
Book Title: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2016
Month: June

Department(s): Autonomous Vision, Perceiving Systems
Research Project(s): Deep, Probabilistic and Semantic 3D Reconstruction
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2016

Links: YouTube
Attachments: pdf


  title = {Patches, Planes and Probabilities: A Non-local Prior for Volumetric {3D} Reconstruction},
  author = {Ulusoy, Ali Osman and Black, Michael J. and Geiger, Andreas},
  booktitle = { IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  month = jun,
  year = {2016},
  month_numeric = {6}