2022 year 2 month 4 Japan , The Winter Olympics officially opened . During the opening ceremony and the Winter Olympics , Artificial intelligence 、5G、AR、 Naked eye 3D And cloud and other hard technologies have shown their magic . among , Virtual digital human technology, as the entrance and connecting link of the meta universe scene, is particularly eye-catching . Both the sign language anchor and the weather anchor of this winter Olympics are virtual digital people .
- Virtual sign language anchor ： CCTV virtual sign language anchor broadcasts news on the Winter Olympics 、 Live broadcast of the event and live interview , Provide real-time sign language translation services for people with hearing impairment . The virtual anchor is a super realistic virtual digital human , expression 、 mouth 、 Hair 、 Clothes are highly close to real people .
- Virtual weather anchor ： With Feng Shu as the training object AI Virtual human Feng Xiaoshu , During the Beijing Winter Olympics, the weather conditions for the Winter Olympics will continue to be broadcast , Provide health guidelines for the public .
This article will lead you from the Winter Olympics AI The anchor walks into the virtual digital human , Understand its connotation in detail 、 Technical realization 、 Development status and promising application fields .
1. What is a virtual digital human ： Highly anthropomorphic 、 Natural interaction
Pseudo digital people can be traced back to 20 century 80 years .1982 year , Japanese animation 《 Robotech: The Macross Saga 》 After the broadcast , The producer packaged the heroine Lin Mingmei as a singer who sang the animated episode , And made a music album , The world's first virtual singer was born . At this time, the virtual digital human is mainly hand drawn .21 At the beginning of the century , Computer graphics （CG, Computer Graphics) And motion capture technology gradually mature , It has been widely used in the production of virtual digital human , Such as “ Hatsune Miku ”, Its early image is to use CG Technology synthesis . near 5 Over the years , Thanks to the rapid development and application of artificial intelligence technology , Making virtual digital human is more convenient 、 Precision and intelligence . stay AI Under the support of Technology , The image of virtual digital human can reach realism , And have the intelligent interaction ability of emotional expression and communication .
At present, the virtual digital people discussed in the market are mainly AI Under the blessing of high simulation 、 Interactive virtual digital human , virtualization 、 Digitalization and personification are its core elements .
virtualization ： Exist in the non physical world , At present, it is mainly based on pictures 、 video 、 Live in real time 、 Real time animation exists in APP、 Applet 、 Hardware and software integrated display device ; future ,VR Equipment and holographic projection will also become an important way of existence .
Digitization ： Rely on multiple digital technologies , Include CG（Computer Graphics, Use computer for visual design and production ）、 speech recognition 、 Image recognition 、 Motion capture and other related technologies .
Personification ： Virtual human in appearance 、 Behavior as well as thought and interaction , Are highly similar to people .
- appearance ： Having a specific appearance 、 Character characteristics such as gender and character . The appearance of virtual digital human will be affected by the category of virtual digital human （ Such as directly borrowing human images 、 High fidelity modeling 、 Stylized ）、 Production details （ To sweat 、 The skin 、 Modeling of hair and other details ）、 Rendering level 、 Design aesthetics and other influences .
- Behavior ： Have the ability to use language 、 The ability to express facial expressions and body movements . The behavior of virtual digital human will be driven （ Human driven 、 Intelligent drive, etc )、 Training data 、 The accuracy of the driving model .
- Interaction ： Have the ability to identify the external environment 、 And the ability to communicate and interact with people . The interaction ability of virtual digital human will be affected by speech recognition ability 、 Natural language understanding and processing level 、 Knowledge map 、 The influence of preset knowledge base .
2. Technical realization ： Multimodal AI And computing power as the core support
Virtual digital human is mainly supported by three core technical links ： modeling （ That is, the generation of virtual digital human image ）, drive （ That is, identify the user's intention , And determine the subsequent voice and action of the digital human according to the current intention of the user , Drive virtual digital people to start the next round of interaction with users ） And rendering （ Let the skin texture of virtual digital human become real , The image is more personified ）. These technical links are inseparable from the support of artificial intelligence technology and computing resources .
2.1 modeling ： Computer vision technology blessings
At present, modeling mainly depends on CG(Computer Graphics, Computer graphics ） And static scanning technology . But it's expensive , Universal Pictures uses CG Technology restores the dead Paul in 《 Fast and furious 7》 Performance in , The related rendering costs have increased by about 5000 Thousands of dollars . Besides ,CG Virtual human synthesized by technology also faces rough image 、 The challenge of insufficient simulation .
In recent years , With the development of artificial intelligence technology , Dynamic 3D reconstruction technology using deep learning gradually shows its edge . Dynamic 3D reconstruction technology mainly adopts camera array to collect geometric forms 、 texture 、 texture of material 、 Dynamic data such as 3D motion information , At the same time, combine the direction and angle data of all rays in the light field , Provide rich information for modeling 、 Fine information , Improve the effect of modeling .
meanwhile ,AI Technology also greatly reduces the threshold of modeling . Anthropomorphic expressions and actions , Such as a simple frown , It also affects bones 、 muscle 、 A series of skin changes . Using the traditional manual method to adjust the workload is huge , And the use of AI Technology can greatly reduce the workload .
2.2 drive ： Multimodal AI Technology empowerment
According to the difference of interactive driving methods , Virtual digital human can be divided into human driven and AI There are two types of driving . Human driven virtual digital human is expressed by human action and cooperated with motion capture technology , Drive the virtual human to interact with the audience in real time .AI Driven virtual digital human can automatically read, analyze and recognize external input information through intelligent system , According to the analysis results, the subsequent output text of the virtual digital human is decided , Then drive the character model to generate the corresponding voice and action to make the digital human interact with the user .
chart 1 Classification of virtual digital human - From a technical point of view The core technology support of human driven virtual digital human is motion capture technology . The mainstream motion capture technology is optical capture and inertial capture , But the equipment is more expensive . This year , With the in-depth study of technology and expression 、 Advances in algorithms such as motion recognition , Motion capture based on computer vision is gradually emerging , With low price 、 Simple 、 Advantages of ease of use , Ordinary cameras combined with high-quality recognition algorithm can also achieve good driving .
AI Driven virtual digital human relies on multimodal AI technology , Comprehensive use of vision 、 Hearing, etc “ sensory organs ”, Give Way AI The image is more personified . The specific technology mainly includes ：
- speech recognition （ASR）： auditory perception , Realization “ I can hear you ”
- natural language processing （NLP）： Accurately understand the needs of users , namely “ Understand ”
- speech synthesis （TTS）： Response and interaction , And the language expression conforms to the vocal habits of real people
- Voice driven facial animation （ADFA）： The lip shape of virtual digital human is driven by voice
Compared with single mode algorithm development , The development and training of multimodal models are more complex , Involving multimodal representation learning 、 Mode conversion 、 alignment 、 Complex technical details and large-scale training needs such as multimodal fusion and collaborative learning . therefore , High usability development and production tools and high-performance computing engines , For improving the productivity of virtual digital human development , Reducing training costs is also crucial .
2.3 Rendering ： Large scale computing support
Rendering technology is used to improve the fidelity of virtual human , It can be divided into real-time rendering and offline rendering , It requires high fineness 、 Scenes with low timeliness requirements can be rendered offline , Games and real-time interactive scenes need real-time rendering . Due to the limitation of computing power resources and transmission speed , Real time rendering still has great room for improvement in image quality and effect performance .
The rendering of virtual digital human needs to consume a lot of computing resources , It is a huge cost expenditure for enterprises , It greatly limits the exploration and application of virtual digital human . At present, the management of computing power in most enterprises is still relatively extensive , The efficient utilization and refined management of computing resources are expected to effectively reduce the development and production cost of virtual digital human , Promote its application .
Sum up , From the perspective of the technical requirements of the whole life cycle of virtual digital human production , Multimodal AI Technology and sufficient computing support are the core , And easy to use 、 High performance AI The development of production tools and efficient computing resource management are the basic guarantee to promote its application . White sea technology new generation AI Develop production platform -IDP, Provide easy-to-use with interactive programming and engineering functions IDE、 High performance distributed computing engine and refined resource scheduling management （ Including resource isolation 、 Task level breakpoint continuation 、 Automatic sleep, etc ）, It can effectively accelerate the development of virtual digital human , Reduce the consumption of computing resources .
3. Current situation and trend of Technology Application ： The game field may have great prospects
At present, virtual digital human has been in financial customer service 、 Media anchor 、 Game characters and other fields have been applied . In the decentralized world Web 3.0 And under the trend of meta universe , We believe that the commercialization of digital avatars operated by individuals or communities has broad prospects , The game track or its first explosive field .
chart 2 Application classification of virtual digital human
In the near future ：RPG The demand for high-quality Games
As regulation tightens and the number of game players tends to be saturated , China's game industry has entered the stage of stock game . In a highly competitive environment , Providing users with a richer and immersive experience has become a key success factor . A typical recent case is ARPG game 《 Protogod 》, With quality and reputation , Attracted a lot of traffic . The rich and exquisite virtual avatars are 《 Protogod 》 One of the most important components . As of version 2.2,《 Protogod 》 The number of playable characters has reached 41 individual . The production team is expected to be 《 Protogod 》 Add about 17 A character , This pair of open world ARPG It's not a small number .
In terms of game categories , Role games highly related to virtual digital people （ Including role-playing games and multiplayer games ） In China, the contribution of game revenue ranks first , Proportion of revenue in mobile games ~35%. The refinement of this kind of game , Implement the application of virtual digital human technology . Virtual digital human technology can effectively simplify and speed up the process of game animation , Let more virtual characters in the game have rich body movements and fine facial expressions , Bring players a more immersive game experience .
chart 3 Chinese game market trend
Medium and long term ： The demand for digital identity in metauniverse Games
Game as a simulation and extension of reality , The form is very similar to the metauniverse , It may become the first application scenario of the metauniverse . Satya, CEO of Microsoft · NADELLA said after Microsoft acquired Activision Blizzard ：“ On all platforms today , Games are the most dynamic 、 The most exciting entertainment category , In the future, it will play a key role in the development of the meta universe platform .”
From a technical point of view , The core of metauniverse game is decentralization , Open social interaction and self creation , Users can use multi platform interworking 、 Security 、 Autonomous digital identity , Interact with your players . The open meta universe game world will increase the demand for virtual digital people . for example , At present, it is recognized that it is closest to the metauniverse game Roblox in , Everyone can customize their digital identity to socialize with others , Including viewing nearby players 、 Online parties and meetings 、 Virtual concert, etc .
At present, the virtual digital human market is still in the early stage of cultivation , But the attention of the market has continued to rise , More and more enterprises will invest in the virtual digital human industry chain . Game enterprises have unique advantages in the demand of virtual digital people and the mask of landing scene , With the strong support of technology , It is expected to reshape the game concept based on digital identity .
4. IDP Help game enterprises accelerate AI Application and innovation
The design of the game 、 Make 、 The whole life cycle of user operation and other sub technologies of artificial intelligence intersect , It is the application of artificial intelligence technology , Realize social intelligence (Social Intelligence)、 Emotional interaction (Affective Interaction) And general intelligence (General Intelligence) The ideal scene of waiting for the goal .
In addition to reducing the threshold of virtual digital human development and Application ,IDP In the game enterprise intelligent user analysis 、 Game intelligent production design and optimization also have cost reduction and efficiency increase 、 The role of accelerating iteration . Widely embrace in game enterprises AI In the process of Technology ,IDP Together with the enterprise , high efficiency 、 Achieve... At a low cost AI The innovative application of , Promote the intelligent and disruptive innovation of game enterprises .
chart 4 IDP Accelerate game enterprises AI innovation — User insight and operational analysis
chart 5 IDP Accelerate game enterprises AI innovation — Game production design
【 Reference material 】
- Deloitte Consulting , Metauniverse series white papers — The future has to ,2021.12
- The boom securities , Virtual human “ soul ” What is it? ？2021.12
- China Artificial Intelligence Industry Development Alliance , Virtual digital human development white paper ,2020
- Shengang securities , Microsoft acquires Activision Blizzard Yuanuniverse ecological layout is the next city ,2022.01
understand IDP: https://baihai.co/
Focus on IDP: official account Baihai IDP
