Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering

Jonathan Dyssel Stets1,†, Alessandro Dal Corso1,†, Jannik Boll Nielsen1, Rasmus Ahrenkiel Lyngby1, Sebastian Hoppe Nesgaard Jensen1, Jakob Wilm1, Mads Brix Doest1, Carsten Gundlach2, Eythor Runar Eiriksson1, Knut Conradsen1, Anders Bjorholm Dahl1, Jakob Andreas Bærentzen1, Jeppe Revall Frisvad1,*, and Henrik Aanæs1

1Technical University of Denmark, Department of Applied Mathematics and Computer Science, Richard Petersens Plads, 2800 Kongens Lyngby, Denmark
2Technical University of Denmark, Department of Physics, Fysikvej, 2800 Kongens Lyngby, Denmark
Joint primary authors
*Corresponding author

Transparent objects require acquisition modalities that are very different from the ones used for objects with more diffuse reflectance properties. Digitizing a scene where objects must be acquired with different modalities, requires scene reassembly after reconstruction of the object surfaces. This reassembly of a scene that was picked apart for scanning seems unexplored. We contribute with a multimodal digitization pipeline for scenes that require this step of reassembly. Our pipeline includes measurement of bidirectional reflectance distribution functions and high dynamic range imaging of the lighting environment. This enables pixelwise comparison of photographs of the real scene with renderings of the digital version of the scene. Such quantitative evaluation is useful for verifying acquired material appearance and reconstructed surface geometry, which is an important aspect of digital content creation. It is also useful for identifying and improving issues in the different steps of the pipeline. In this work, we use it to improve reconstruction, apply analysis by synthesis to estimate optical properties, and to develop our method for scene reassembly.

Supplementary Visualizations


Stets, J. D., Dal Corso, A., Nielsen, J. B., Lyngby, R. A., Jensen, S. H. N., Wilm, J., Doest, M. B., Gundlach, C., Eiriksson, E. R., Conradsen, K., Dahl, A. B., Bærentzen, J. A., Frisvad, J. R., and Aanæs, H. Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering. Applied Optics 56(27):7679-7690, September 2017. [abstract] [video] [data] [lowres pdf] [news release] [front cover image] [editor's pick]