Application of 3D Data Processing Using Neural Networks

Abstract

Neural Fields model the "function values corresponding to each point in space" using neural networks and describe shapes as functions that represent surfaces. One of the surface models using Neural Fields, DeepSDF, not only represents smooth surfaces but also models complex shapes with a simple network, revolutionizing 3D representations. However, there were limitations. The volumetric model NeRF, which was introduced later, made it possible to generate images from arbitrary viewpoints using only multi-view images. Its optical phenomena are modeled with density and radiance, allowing for changes in radiance based on direction and enabling the handling of scenes involving scattering, such as smoke. The network structure takes coordinates and viewpoint angles as input and outputs density and color. However, as queries increase in space, it becomes computationally intensive, and through various techniques such as spatial decomposition and splitting NeRF, fast learning and inference were achieved. Finally, an applied research example that the author worked on collaboratively will be introduced.