Turn 2D photos into 3D models to see NVIDIA's new AI "magic"!

Arrangement | Zheng Liyuan

Produce | CSDN

Computer vision and pattern recognition conference held this week （CVPR） On , NVIDIA has launched a product called 3D MoMa Of AI “ magic ”： Will be static 2D Convert photos to easily editable 3D Model .

According to INVISTA ,3D MoMa Available to architects 、 The designer 、 Artists and game developers convert images 3D Model quick import graphics engine , You can also modify the scale during use 、 Change the material or try different lighting effects .

from 2D Extract... From the image 3D Model

In reality , If you want to enable artists or engineers to make full use of 3D Model , Put it into the game engine 、3D Most widely used tools, such as modelers or movie renderers, are best , But only if ： This kind of 3D The model is a triangular mesh with textured material （ Mainly used to define 3D Basic framework for graphics and modeling shapes ）.

Generally speaking , Many game studios and creators habitually use sophisticated photogrammetric techniques to create such 3D Model , Not only do you have to work hard and manually , It will take a lot of time —— but 3D MoMa The emergence of may change this situation .

according to the understanding of ,3D MoMa Behind the , Is vice president of NVIDIA graphics research David Luebke be called “ The Holy Grail of unifying computer vision and computer graphics ” Reverse rendering technology .

Reverse rendering , A series of static 2D A photograph reconstructed into an object or scene 3D Model technology , and 3D MoMa By defining each reverse rendering problem as GPU Accelerated differentiable components , Use modern AI Machines and NVIDIA GPU To quickly generate 3D object , Enable the creator to import it without restriction in the existing tools 、 Edit and extend .

Yingweida official blog said ,3D MoMa Can be in a single Tensor Core GPU On ,1 The triangular mesh model will be generated within hours , The output will be directly related to the 3D Graphics engine and modeling tools are compatible .

More Than This , because 3D MoMa What is generated is a triangular mesh model , The mesh is like a triangle 3D Shape paper model , So the creator can reconstruct it ： Change its mesh model 、 Materials and lighting effects .

fictitious 3D Musical instrument show jazz performance

For more intuitive display 3D MoMa The function of , NVIDIA research team first collected five jazz band instruments from different perspectives （ trumpet 、 Trombone 、 saxophone 、 Drums and clarinets ） Close to 100 Zhang image .

then , The research team used 3D MoMa Will these 2D The image is converted to a triangular mesh model of each instrument , Take it out of the original scene and import it Omniverse 3D Simulation platform for editing .

As mentioned above , The creator can make comments on 3D MoMa Reconstruction of the output results of , Therefore, NVIDIA research team has also tried .

• The material can be changed , Convert the original plastic material into gold 、 Marble 、 Wood or cork ;

• The newly edited model can be put into the virtual scene , These instruments are put into the classic Cornell box for graphics rendering quality test （Cornell Box） in , It is found that the reflection of virtual instruments on light is the same as that in the real world ;

• These new models generated by reverse rendering can be used as components of complex animation scenes , As shown in the following video .

Although at present 3D MoMa Still under development , But NVIDIA is looking forward to it ： It is believed that it can allow game developers to quickly modify this and other designers 3D object , Then add them to any virtual scene .

Reference link ：

https://blogs.nvidia.com/blog/2022/06/21/inverse-rendering-3d-research-cvpr/

https://thenextweb.com/news/nvidias-new-ai-magic-turns-2d-photos-into-3d-graphics

Looking back

Matplotlib Two methods of drawing torus ！

13 individual python Necessary knowledge , Recommended collection ！

Artifact , Easy visualization Python Calling process ！

Get it done Python Several common data structures ！

 Share 
 Point collection 
 A little bit of praise 
 Click to see