CG kit explore high performance rendering on mobile terminal

Content sources ： Huawei Developers Conference 2021 HMS Core 6 Graphics Technology Forum , keynote speech 《CG Kit Explore mobile high-performance rendering 》

Speakers ： Huawei Haisi Qilin GPU Team Engineer

Hello everyone , I'm a member of Huawei Hisilicon Qilin team GPU R & D Engineer , The theme for you today is “CG Kit Explore mobile high-performance rendering ”.

Huawei graphics computing service （Computer Graphics Kit, abbreviation “CG Kit”） Provides the most cutting-edge rendering framework in the field of computer graphics 、 plug-in unit SDK And GPU Extension interface , Assist developers to explore the ultimate performance of hardware , Greatly reduce the difficulty of development , Help developers create products with better experience .

that ,CG Kit What solutions can be provided ？

One 、Vulkan Rendering framework practice

Vulkan Is the latest graphics standard API, Compare with OpenGL Have some advantages , also OpenGL This standard has stopped evolving , Future graphics is basically based on Vulkan Move forward . for instance , Ray tracing at the mobile end is in Vulkan Description in , And in OpenGL None of them . But the problem is ,Vulkan Very flexible , Compared with OpenGL There is a certain threshold for use , So there is a need to be based on Vulkan High performance rendering framework , So as to provide practice architecture and Scheme , Give full play to the hardware performance .

Two 、 High performance rendering plug-ins

CG Kit Provides multithreaded rendering 、 Taking pictures is super 、 Volume clouds 、 Occlusion culling and other plug-ins with a series of rendering capabilities , So as to facilitate the smooth progress of tripartite cooperation , For example, Netease last year 《 Heaven's Oracle 》 The mobile game integrates the algorithm of taking photos and super points .

3、 ... and 、 Developer tool chain

CG Kit It also provides a complete developer tool chain , We can help developers quickly locate and solve rendering function and performance problems . You can imagine , If future tools can automatically identify some functions in the game scene / Performance issues and feedback to developers , This will greatly improve our development efficiency .

Four 、 Exploration of cutting-edge graphics technology

all the time CG Kit The team is exploring cutting-edge graphics technology , such as , How do we put AI The technology is applied to graphics rendering , Now we are in AI Over score 、 Automatically 3D Face modeling ,AI Research achievements in collaborative animation generation .

Vulkan Rendering framework practice

Vulkan Rendering framework practice , For developers Vulkan Rendering solution optimization 、 Rendering enhancement plug-ins and rendering technical documentation , such as , Some... Will be introduced in the document API Use policy 、 Resource cache usage optimization 、Vulkan Compatibility problems, etc . Mentioned earlier Vulkan It's relatively new API, its Command Buffer The mechanism can naturally support multi-threaded rendering . because Vulkan More flexible ,CG Kit It's about Vulkan Of API Simplified packaging , More convenient to use .

secondly , CG Kit Physical based rendering is also supported ,PBR It can truly reflect the attribute of object surface illumination , adopt PBR The objects we render will be more realistic .

meanwhile ,CG Kit Provides a resource manager , A large number of pipeline resources can be reused as much as possible , So as to reduce the cost , Improve rendering performance .

Last ,CG Kit Also support the provision of HDR10 Display capability , coordination HDR Screen display optimization , The final plot will have a better dynamic range .

Static super component

Let's introduce the plug-in of game photography super points , Games are social , Sharing is a very important part of social networking . In the course of the game, we often encounter some wonderful moments worth sharing , Such as completing a milestone task , Got a limited edition of beautiful skin , Or have completed a virtual character customization …… In these moments , Gamers are eager to share these wonderful moments with their friends , Then you need to use the function of taking pictures , But how to make the picture better ？

For photographic effects ,CG Kit There are two solutions .

The first solution is to have independent NPU High end mobile phones , For example, Kirin 980、990 Kirin 9000 The phone corresponding to the chip , These have independent high-end features NPU Your phone can directly call AI Super split interface , In this way, the picture details can be significantly increased , The picture style is more clear and natural , As can be seen from the comparison chart below , After processing, the facial details of the characters in the picture have been significantly improved .

And for those who are not independent NPU Cell phone with chip , We also provide a general solution , called FilterSR, This is based on general GPU An algorithm of , After applying this algorithm , No, NPU Your phone can also greatly improve the picture effect , And compared with the traditional method, its sawtooth feeling and picture definition are also greatly improved . It is worth mentioning that ,《 Heaven's Oracle 》 It integrates our camera super division algorithm .

Volume cloud components

CG Kit It also provides high-performance volume cloud plug-ins for mobile terminals , Volume cloud technology can dynamically render the global illumination scene in a highly realistic cloud sea in real time , And support players to shuttle freely in the cloud , The picture can truly restore the physical characteristics of clouds . such as , The sense of particles coming from players in the process of shuttling , And real lighting effects , Including details of the edge of the cloud , Will have a very good presentation . As can be seen from the figure below , The color of this cloud changes gradually with time , Because we did to the light 24 Hours of adaptation .

In terms of the shape of the volume cloud , We support both real and cartoon styles , It also supports arbitrary shapes Mash Convert to volume cloud , This can meet the different customization needs of developers . From a performance point of view , In kylin 980 On this platform , We can do 4 Render in milliseconds 1 frame , The plug-in displays , It also provides PC Plug in for mobile terminal and mobile terminal , Generally speaking, developers edit and debug on the mobile terminal , And on the mobile end with SO The way to integrate , Its inclusion size is only 400K about .

Volume fog assembly

Next, let's introduce the body fog assembly , In some specific game style games , The effect of highly realistic light and fog will bring you a very good experience , For example, there was a very popular game last year , It's called 《 Cyberpunk 2077》, It has a lot of fog effect .

host 3A In the big work , Volume fog is generally realized by volume rendering , Unlike surface rendering, you only need to render the object surface , Volume rendering renders the interior of an object , The rendering workload may be dozens or hundreds of times that of surface rendering , Volume rendering requires very high hardware performance , This is also the reason why it has not landed on the mobile terminal .

and CG Kit The volume fog component has done a lot of optimization to the traditional volume rendering , Let me briefly introduce two optimizations .

The first optimization is for multi light scene . When there are multiple light sources , The workload of rendering is increased dozens of times , To solve this problem, we introduce the light source elimination technology , Let the lighting calculation occur only in the local range affected by the light source , In this way, the workload of rendering can be greatly reduced .

The second optimization is , In order to improve the efficiency of rendering , We use the method of reducing the sampling rate , However, after reducing the sampling rate, there will be a problem of insufficient picture details , To solve this problem , We introduce time domain filtering technology , Fuse the results of the current frame and the previous frame , This will also get a very good result . The data structure is also relatively adapted for these optimizations .

After a series of previous optimizations , We can perform real-time high-performance volume fog rendering on the mobile terminal , From a performance point of view , In kylin 980 On the platform , It can be done 4 Render a frame in milliseconds ; In the form of integration , It can also provide PC And mobile plug-ins , Support PC End to end editing and debugging , Mobile terminal SO Way integration , The package of the plug-in at the mobile end is less than 500KB.

、

AI Pinch your face

Finally, I'd like to introduce AI Pinch your face , In the game scene, if you can achieve a virtual image of thousands of people and thousands of faces , And its virtual image can have some facial features of game players , It will enhance the user's sense of substitution .AI Pinch your face and input in the form of positive self photographing , Calling our AI After the interface , A set of kneading parameters that highly restore the user's facial features will be generated , Automatic generation based on kneading parameters 3D Model , At the same time, we pursue geometric similarity , Also real textures 、 Model texture fusion , So that its texture can present some facial features of the player , More reductive . We provide a very simple interface for integration , Developers can easily adapt , It can also greatly improve its development effect .

Because we are pure end-to-side operation , In medium and high-end models AI The face pinching time is less than 4s, Very stable . At the same time, our face pinching range not only supports the face , But also support facial features 、 hairstyle 、 Glasses, etc ; We also incorporate highly restored real textures , Better reduction , And provide beauty adjustment function , You can control the beauty of the final output graph . In terms of development efficiency , Developers only need two steps to call our functions , First step PC Model free adaptation at the end , The second step is to call our interface at the mobile terminal with the user's self photographing as the input , The whole process is very simple and convenient .

I hope you can have in-depth cooperation , thank you ！

More highlights , Please see the official Huawei Developer Forum →https://developer.huawei.com/consumer/cn/forum/home?ha_source=sanfang