The "advertising maniacs" in this group of programmers turned Tiktok advertisements into ar games

In most people's impression , Advertisements always occupy the screen, but nothing strange . But now , Advertisements can also have rich playing methods and interesting interactions , Full of innovation .

 

such as , When you are brushing Tiktok , Have you ever played these AR Interactive advertising ？

 

Aim at the dishes floating in the air , They all become targets for shooting ：

 

 

Make a heart gesture , You can summon a series of bubbles , You can also get KFC coupons ：

 

 

Catch the red envelope falling from the sky ：

 

 

You can participate in the lottery ：

 

 

Participation within a limited time , You can also get preferential benefits ：

 

 

AR The form of advertising has turned advertising into an interactive experience full of games and fun , Not only served customers , It also creates entertainment space for users .

 

therefore , These Tiktok AR Dynamic creative advertising , How on earth did it come out ？

 

The R & D team of ByteDance , And what little colored eggs are buried inside ？

 

 

 

How to make H5 Learn to do AR？

 

This kind of content is rich 、 Various forms of interaction are mostly used H5 To develop , adopt app End embedding webview To present .

 

however H5 and AR There are many adaptation problems between . Such as some App in ,AR Capabilities are integrated in App in ,H5 Unable to call ; Some also need to verify the gyroscope .

 

therefore , Need another technical capability , To get through H5 and App Barriers between the two .

 

The R & D team chose Lynx.

 

Lynx, Byte hop client cross engine framework , With JavaScript For developing languages , It can be used by front-end R & D personnel Web Technology builds quickly Native View , And it can also realize cross end development .

 

In the Spring Festival Gala this year ,Lynx Also had a very good performance , Greatly reduce the cost of client release .

 

The R & D team found Lynx Developing AR Three advantages of interaction ：

 

First of all ,Lynx Framework support canvas Dynamic rendering , fit AR Development ;

 

second ,Lynx It is very consistent with front-end development , Match the front-end technology stack ;

 

Last , use Lynx To develop , All content updates do not rely on cloud publishing , In other words, when there are new creative advertising effects, there is no need to do it specially App Version update .

 

The classmates of this team JackHong say ：“ You can understand it as Lynx It's a piece of white paper , There are all kinds of brushes and pigments in the rendering Library , It can be used for drawing .”

 

But sure Lynx As technology selection , It means huge and complex preparatory work ——Lynx Are not compatible H5, Need modification H5 The underlying rendering library to fit Lynx Of API.

 

This is a very complicated project ,JackHong It takes threeorfour months to change with his partners , screening bug、 Location problem , After gradually debugging , Render all kinds of in the Library 「 Brush paint 」 It can finally come in handy .

 

Earliest , They first realized the use of H5 stay Lynx Ability to draw graphics on a frame ：

 

 

With the ability to draw graphics , You can add more interactive elements on the basis of graphics , Make one by yourself 2048 Little games ：

 

 

Even realize the natural light and shadow 、 Smooth 3D Rendering ：

 

 

After these abilities are completed , Access Lynx Open end API, obtain AR Recognized rendering capabilities .

 

Final , take AR Ability to integrate into dynamic creativity SDK in , The preliminary preparation is finally completed . Now? ,H5 The ability to express 、 Interaction ability , Terminal AR Capabilities are integrated .

 

JackHong Look at the smooth operation in the mobile phone Demo, A sense of accomplishment , I feel that my efforts over the past few months have not been in vain . He recalled the scene with some shyness and pride ：“ When it's done , Of course, all kinds of shows , I showed it to the students in our team , to Lynx The classmates of the team showed it once , Show it to the design team .”

 

Several students of the docking team were very impressed by the results ,JackHong Also full of sense of achievement .

 

Now? ,H5 With AR The ability of , You can start to realize all kinds of AR Creative . It's like getting a treasure knife , You can start practicing martial arts .

 

 

 

A picture creates a dynamic effect ,50 Kind of + Gestures are recognizable

 

JackHong The first one for colleagues Demo It's a gesture AR, This function was also successfully used in the marketing activities of KFC Christmas season .

 

 

To achieve this recognition 「 finger heart 」 Gesture and then bubble effect , The front end needs to call the mobile camera , Draw the video stream onto the screen ; At the same time, send the video circulation to App Gesture recognition , Return results , After confirming the success of bixin gesture , Play bubble effect , A series of bubbles flew out of the palm of the user .

 

It is worth noting that , Here is a series of 「 Bubble 」 There is only one . In essence, they are all like this png picture , The size is only 4k, Very memory efficient .

 

 

With the help of such a picture , The R & D team controls its frequency with code 、 path of particle 、 Changes in size and transparency , A series of smooth and natural bubbles can appear on the user's screen .

 

Now? , This gesture AR The function has been recognized 50 A variety of gestures , Whether it's one handed gesture or two handed gesture , Even clap your hands 、 Dynamic hand movements such as boxing can be achieved .

 

 

 

Particles flying in all directions

 

several AR In dynamic creative advertising , The most difficult thing is the shooting effect .

 

 

After hitting , All kinds of food that used to float in the air will be broken , Into countless pieces , Fly in all directions .

 

If you look carefully , We'll find out 「 debris 」 Are round particles of different sizes , And their colors are taken from floating food . If the user shoots strawberry cream cake , The colors of round particles are pink and white ; If the shot is chocolate pie , Particles are brownish brown in different shades .

 

and , After each shot , The trajectory of particles flying in all directions is random , The direction and speed of each particle flying out are inconsistent .

 

Realizing such a function is very complicated , R & D students made a 「 Particle engine 」, stay 3D Environment , According to the resolution and color of the picture , Turn the picture into particles , Define the moving direction of particles 、 Size and speed , And the order of divergence . There are many parameter adjustments .

 

however , Why do we have to use such a complex scheme ？

 

JackHong say ：“ Although it can be made into the special effects of every shot , But the visual effect is not good , For example, the target is a red thing , Then after hitting it, some blue particles fly out , It seems to be more contrary , Users will get tired of reading too much . also , Make a separate picture for each frame of dynamic effect , One 25 Frame animation , One second 25 Zhang png picture , It takes a long time to load , And it takes up more mobile memory , influence App Operational stability .”

 

With this scheme, the memory consumption is greatly reduced , But the performance consumption is improved , So I call GPU Speed up . Because if you use CPU Rendering , You need to traverse the motion state and color change of all particles ,GPU Then, multi-pixel parallel computing can be realized , Each thread can calculate the change of a particle separately .

 

 

 

Can write code 「 Madman 」

 

these AR Dynamic creative advertising effect , All from ByteDance Creative Center .

 

Compared with other R & D teams , They are more like an advertising creative studio , From the front end 、 Back end development , Underlying algorithm , They also produce all kinds of interactions and special effects , Service Tiktok 、TikTok、 Today's headlines and other platforms .

 

One of the characteristics of this team is internationalization , The office is located in Beijing 、 Shanghai 、 Hangzhou 、 Seattle 、 There are five cities in mountain view city , Except Chinese classmates , And from Europe 、 The United States 、 South Korea 、 Members from different places such as Africa , This collision of multiple backgrounds often makes this creative team burst out with new inspiration .

 

Team leader Mu Yi said ：“ Different background 、 Different cultures 、 After students with different ways of thinking join , Always bring some surprises , Their historical background 、 educational background 、 Cross cultural cognition , Are very good supplements to our team .”

 

In addition to various forms of rich interactive advertising , Their other job is to develop various creative tools , For example, shooting videos 、 Tools for making pictures , There are even machines that write automatically 、 A series of solutions for automatic video generation by machines .

 

stay JackHong It seems , This creative advertising research and development team is really very different , Everyone has a kind of 「 Madman 」 The temperament of ：“ in my opinion , How a technology can be played is not just something designers have to do , Research and development should also help designers think . Design students' understanding of technology will be somewhat estranged , I often ask if we have any new technology , In fact, there are so many new technologies to be creative , What is more needed is how to skillfully use technology , At this time, we need to research and develop to provide some wonderful ideas .”

 

This creative space is inseparable from the loose management style of the team .

 

As a team Leader, Muyi said ：“ I personally spend little time on Management , Our style is excellent 、 frank 、 free . Excellence does not necessarily mean how strong the experience background , Although we do have many MIT、 Students graduated from Stanford and other top universities , But more important is the ability to think and solve problems . I believe some excellent people come together , It must be because there is a common goal , This allows everyone to do a lot of things by themselves , Instead of relying on rules and regulations .”

 

Now? ,JackHong We are also working towards a new goal . He hopes to put these AR Advertising creative play into tools , Thus, only design materials can be replaced , Similar playing methods can be reproduced without repeated research and development .