RNb-NeuS2: Multi-View Surface Reconstruction
Using Normal and Reflectance Cues

Abstract

Achieving high-fidelity 3D surface reconstruction while preserving fine details remains challenging, especially in the presence of materials with complex reflectance properties and without a dense-view setup. In this paper, we introduce a versatile framework that integrates multi-view normal and optionally reflectance maps into radiance-based surface reconstruction. Our approach employs a pixel-wise joint re-parameterization of reflectance and surface normals, representing them as a vector of radiances under simulated, varying illumination. This formulation enables seamless integration into standard surface reconstruction pipelines, such as traditional multi-view stereo (MVS) frameworks or modern neural volume rendering ones. Combined with the latter, our approach achieves state-of-the-art performance on multi-view photometric stereo (MVPS) benchmark datasets, including DiLiGenT-MV, LUCES-MV and Skoltech3D. In particular, our method excels in reconstructing fine-grained details and handling challenging visibility conditions. The present paper is an extended version of the earlier conference paper by Brument et al. [1], featuring an accelerated and more robust algorithm as well as a broader empirical evaluation.

Reconstruction of the Cypriot Woman's Head with Calathos¹ from the Musée Saint-Raymond (Toulouse, France) with normals estimated via [2]. (Acquisition credit: Antoine Laurent)

Reconstructing High-Frequency Surface Details from Sparse Views

3D viewer of the reconstructed Cypriot Warrior Head² from the Musée Saint-Raymond (Toulouse, France). The left view represents the reconstruction from RNb-NeuS2 (ours) using normals from [3] with 13 viewpoints. The right view corresponds to the reconstruction from Colmap [4] using 121 viewpoints. (Acquisition credit: Antoine Laurent)

Reconstructing High-Frequency Surface Details Under Complex Reflectance

3D reconstruction of the golden_snail from Skoltech3D [5]. The left view represents the reconstruction from RNb-NeuS2 (ours) using normal and albedo maps from SDM-UniPS [6] with 20 viewpoints. The right view corresponds to the reconstruction from NeuS2 [7] using 20 viewpoints.

Reconstructing Transparent Materials

3D reconstruction of the orange_mini_vacuum from Skoltech3D [5]. The left view represents the reconstruction from RNb-NeuS2 (ours) using normal and albedo maps from SDM-UniPS [6] with 20 viewpoints. The right view corresponds to the reconstruction from NeuS2 [7] using 20 viewpoints.

Accurate Reconstruction of Concave Structures

3D reconstruction of the wooden_trex from Skoltech3D [5]. The left view represents the reconstruction from RNb-NeuS2 (ours) using normals from UniMS-PS [8] with 20 viewpoints. The right view corresponds to the reconstruction from NeuS2 [7] using 20 viewpoints.

Archeological Objects

¹Cypriot Woman's Head with Calathos
This first archaeological object is a small ceramic piece from the collections of the Musée Saint-Raymond, Toulouse, France. Originating from Cyprus and dating from the 4th century BC, it depicts a woman's head adorned with a calathos, broken at the neck. (Dimensions: 5.7cm x 4.2cm x 1.4cm). The 3D acquisitions of the Cypriot woman's head were conducted within the reserves of the Musée Saint-Raymond. An initial acquisition was performed using an Artec Spider hand-held scanner, which offers a 3D point accuracy down to 50 µm and a 3D mesh resolution down to 100 µm. Photogrammetry involved capturing 48 photographs of the front face with a Nikon Z7 II camera and a 50 mm lens from an average distance of 22 cm. For multi-view photometric stereo, 5 sequences of 35 shots each were taken from 5 different viewpoints using an RTI dome, with the same camera and lens.

²Cypriot Warrior Head
The second object is a terracotta Cypriot warrior head, also from the Musée Saint-Raymond in Toulouse, France. This piece dates back to the 6th century BC and originates from Cyprus. (Dimensions: 45cm x 18.2cm x 19.5cm). For photogrammetry, 121 photographs were taken using a Nikon Z7 II camera equipped with a 35mm lens and a Godox AD100pro flash. For multi-view photometric stereo, 13 viewpoints were captured, with 12 lighting conditions per viewpoint, totaling 156 images.

Citation

Acknowledgements

Robin Bruneau's postdoctoral fellowship was funded by the Department of Quantitative Biomedicine at the University of Zurich. Baptiste Brument's doctoral student fellowship was funded by the CNRS through the project Open-DOPAMIn. Lilian Calvet's postdoctoral fellowship was supported by the University of Zurich, the University Hospital of Balgrist, and the OR-X - a Swiss national research infrastructure for translational surgery. The research leading to these results received funding from the French National Research Agency (ANR) through the LabCom project ALICIA-Vision, and by the Department of Computer Science at the University of Copenhagen (DIKU) through the Copenhagen Data+ project PHYLORAMA.
We would also like to thank our partners for their support. We thank the Musée Saint-Raymond in Toulouse for providing access to the archaeological objects. We also thank Antoine Laurent for his work on the 3D acquisitions and Benjamin Coupry for the data processing and generation of normal maps. We thank Antoine Laurent for his work on the 3D acquisitions and Benjamin Coupry for the data processing and generation of normal maps. We also thanks Tim Flueckiger for the creation of the video.

References

[1] RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction , Brument et al. CVPR 2024
[2] Assessing the Quality of 3D Reconstruction in the Absence of Ground Truth: Application to a Multimodal Archaeological Dataset , Coupry et al., WACV 2025
[3] Self-calibrated Near-light Photometric Stereo using a Geometric Proxy , Coupry et al. SSVM 2025
[4] Pixelwise View Selection for Unstructured Multi-View Stereo , Schoenberger et al. ECCV 2016
[5] Multi-sensor large-scale dataset for multi-view 3D reconstruction , Voynov et al. CVPR 2023
[6] Scalable, Detailed and Mask-free Universal Photometric Stereo , Ikehata et al. CVPR 2023
[7] NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction, Wang et al. ICCV 2023
[8] Uni MS-PS: A multi-scale encoder-decoder transformer for universal photometric stereo , Hardy et al. CVIU 2024