Abstract

This paper studies implicit surface reconstruction leveraging differentiable ray casting. Previous works such as IDR and NeuS overlook the spatial context in 3D space when predicting and rendering the surface, thereby may fail to capture sharp local topologies such as small holes and structures. To mitigate the limitation, we propose a flexible neural implicit representation leveraging hierarchical voxel grids, namely Neural Deformable Anchor (NeuDA), for high-fidelity surface reconstruction. NeuDA maintains the hierarchical anchor grids where each vertex stores a 3d position (or anchor) instead of the direct embedding (or feature). We optimize the anchor grids such that different local geometry structures can be adaptively encoded. Besides, we dig into the frequency encoding strategies and introduce a simple hierarchical positional encoding method for the hierarchical anchor structure to flexibly exploit the properties of high-frequency and lowfrequency geometry and appearance. Experiments on both the DTU and BlendedMVS datasets demonstrate that NeuDA can produce promising mesh surfaces.

overview

Motivation

Previous approaches (such as NeuS, IDR, and UNISURF) overlook the spatial context in 3D space when predicting and rendering the surface, thereby may fail to capture sharp local topologies such as small holes and structure. To mitigate the limitation, we propose a Neural Deformable Anchor(NeuDA), for high-fidelity surface reconstruction. The main differences between the hierarchical deformable anchors representation and some baseline variants can be found as follows:


(1) Methods such as NeuS, volSDF, and UNISURF sample points along a single ray; (2) Standard voxel grid approaches store a learnable embedding feature at each vertex. Spatial context could be simply handled via the feature aggregation operation; (3) The hierarchical voxel grid can further explore different receptive fields; (4) Neural deformable anchor maintains a 3D position instead of a feature vector at each vertex. We optimize the anchor points such that different geometry structures can be adaptively represented.


Neural Deformable Anchor

We propose Neural Deformable Anchor (NeuDA), a new neural implicit representation for high-fidelity surface reconstruction leveraging multi-level voxel grids. Specifically, we store the 3D position, namely the anchor point, instead of the regular embedding (or feature) at each vertex. The input feature for a query point is obtained by directly interpolating the frequency embedding of its eight adjacent anchors. The anchor points are optimized through backpropagation, thus would show flexibility in modeling different fine-grained geometric structures.

Results on DTU dataset


Results on BMVS dataset


Citation

Acknowledgements

The website template was borrowed from Michaël Gharbi.