当前位置:网站首页>3D content generation based on nerf
3D content generation based on nerf
2022-07-07 12:45:00 【Nismilesucc】
source : Deep Blue College 《 be based on NeRF 3D content generation 》 Zhang Kai
3D content
Generate from pictures 3D content 【 Labor takes time 、 energy 】 => Automatically generate by computer-aided means
picture : It's very easy to get => Anti rendering Generate 3D content
Computer graphics : How to generate high-quality rendered images
Computer vision : Given picture => Anti rendering generates 3D content needed in computer graphics 【 Can change the light , Insert objects and so on 】
The three elements of anti rendering
1. Shape representation
Triangle Mesh、Point Cloud、Occupancy field、Signed distance filed
Different shape representations may determine the difficulty of solving the problem 【 Different optimization methods 】
2. appearance
The left side represents the material and light separately 【 Ideal situation , You can change the light , Edit material , But it's very difficult to understand ( It involves the rendering process in graphics )】, On the right, pack the material and light together 【 Can't edit well , Put the object in a new environment and observe its appearance ,】
Anti rendering is relative to 3D reconstruction , The key is differentiable,2D->3D->2D
3D reconstruction is a special case of inverse rendering , 3D reconstruction was not focus Rendering quality .
3. Rendering process 【 Optimize 】
Ray tracing rendering: Track the propagation of light , Pass through each in the image pixel The process of weighted summation of light direction color distribution => Get the color of pixels
NeRF
Right picture : The quality of depth map is very high
Soft Shape: Like fog , A little bit appears in every part of the space , Unlike solid objects, they only occupy a small part of space .
Success factors :
1. Shape representation 【 Soft shape representation ( Foggy )】
2. appearance 【 Materials and light 】
3. Rendering process 【 Functions are all differentiable 】
The geometric details on the right may not be good enough 【NeRF shortcoming 】
The previous work with neural network is not good enough => Hard representation is selected (eg. Triangle Mesh)
Reasons for the success of selecting soft shapes :
shortcoming : Every point in the light predict Color =>Expensive
=》 Prove the validity of soft shape , Better rendering can still be achieved without neural networks
There is no neural network (evaluation Very slowly ), For every pixel of light, go querry
At the beginning, neural network is introduced to represent the scene :
Another question : Neural networks have special bias, Tends to fit smooth shape=> Introduced a map Y Y Y
Five scenarios :
- 360 No background , Only the prospect
- Only move the camera within a small range
- Panoramic shooting mode
- Take photos casually with your mobile phone in your room 【 Cameras are more irregular 】
- 360in and outforward Scene , Want to rebuild both the foreground and the background
Local or The overall There is one trade off (a) Choose the point on the foreground (b) Divide the points into the foreground and the background
Yes resolution problem
NeRF++
Draw a ball to deal with the foreground and background
such wrapping The property of can well solve the problem of resolution : Space is squeezed
NeRF It has good composition The nature of
NeRF At present, it cannot run on real-time online devices , It can't support editing well .
NeRF The sampling frequency corresponds to pixel size
NeRF Core assumptions : Objects are static
The sawtooth problem occurs when downsampling , It is related to image sampling frequency .
Nyquist frequency problem
边栏推荐
- SQL lab 21~25 summary (subsequent continuous update) (including secondary injection explanation)
- Tutorial on the principle and application of database system (011) -- relational database
- Vxlan 静态集中网关
- Realize all, race, allsettled and any of the simple version of promise by yourself
- 2022A特种设备相关管理(锅炉压力容器压力管道)模拟考试题库模拟考试平台操作
- The road to success in R & D efficiency of 1000 person Internet companies
- Financial data acquisition (III) when a crawler encounters a web page that needs to scroll with the mouse wheel to refresh the data (nanny level tutorial)
- SQL injection -- Audit of PHP source code (take SQL lab 1~15 as an example) (super detailed)
- 【二叉树】删点成林
- [Q&A]AttributeError: module ‘signal‘ has no attribute ‘SIGALRM‘
猜你喜欢
【统计学习方法】学习笔记——第五章:决策树
Tutorial on principles and applications of database system (009) -- conceptual model and data model
Epp+dis learning road (2) -- blink! twinkle!
Epp+dis learning path (1) -- Hello world!
图形对象的创建与赋值
Pule frog small 5D movie equipment | 5D movie dynamic movie experience hall | VR scenic area cinema equipment
Tutorial on principles and applications of database system (010) -- exercises of conceptual model and data model
金融数据获取(三)当爬虫遇上要鼠标滚轮滚动才会刷新数据的网页(保姆级教程)
The hoisting of the upper cylinder of the steel containment of the world's first reactor "linglong-1" reactor building was successful
[pytorch practice] write poetry with RNN
随机推荐
Several methods of checking JS to judge empty objects
【PyTorch实战】用PyTorch实现基于神经网络的图像风格迁移
用mysql查询某字段是否有索引
Epp+dis learning path (1) -- Hello world!
【PyTorch实战】用RNN写诗
Object. Simple implementation of assign()
BGP third experiment report
[statistical learning methods] learning notes - Chapter 5: Decision Tree
2022-07-07日报:GAN发明者Ian Goodfellow正式加入DeepMind
[statistical learning methods] learning notes - Chapter 4: naive Bayesian method
Static routing assignment of network reachable and telent connections
Utiliser la pile pour convertir le binaire en décimal
sql-lab (54-65)
Inverted index of ES underlying principle
SQL head injection -- injection principle and essence
[Q&A]AttributeError: module ‘signal‘ has no attribute ‘SIGALRM‘
leetcode刷题:二叉树27(删除二叉搜索树中的节点)
[pytorch practice] use pytorch to realize image style migration based on neural network
Decrypt gd32 MCU product family, how to choose the development board?
gcc 编译报错