NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis

The main purpose of this paper is based on NeRF Heavy light . The original NeRF It requires the illumination direction of the pictures in all perspectives , It's all consistent , Therefore, the light in the synthesized results is naturally the same , Can't change , This paper hopes to allow arbitrary manipulation of lighting conditions by modeling light .

NeRF Essentially, the scene is modeled as an entity that emits light , and NeRV Is to treat the scene as an entity that can reflect light . Because the essence of object color is the result of illumination , therefore NeRV Model density and reflected illumination separately （NeRF It is a joint modeling of density and color ）, Density and NeRF It makes no difference , Reflected light is based on BRDF Reflection equation of , That is, the corresponding MLP Predict the roughness and 3D Reflectivity . in addition , It is worth mentioning that , The normal of the object surface can be inferred from the density information .

in addition , Because the reflection of light between objects is almost endless , Therefore, if you really render objects according to the actual situation, the amount of calculation will definitely explode , So the author puts forward a series of optimization schemes , Make the model more efficient for lighting simulation .

The first is to propose a visibility MLP, It is used to predict the ambient light visibility and the end depth of the light at the input point .

And through the reality of random sampling （ simulation ） Light do loss To ensure the authenticity of the results .

Later, it involves more knowledge about lighting , A little incomprehensible , You may need to make up lessons first and then reread .