Audio and video technology development weekly | 232

Once a week , Overview of audio and video technology in the field of dry goods .

Press release ：[email protected]

The innovative practice of short video content understanding and generation technology in meituan

Meituan focuses on the rich local life service e-commerce scene , Accumulated massive video data . How to use relevant data through computer vision technology , Provide better services for users and businesses , It is an important research and development topic . This time LiveVideoStackCon 2021 Audio and Video Technology Conference Beijing station , We invited Ma Bin, a senior algorithm expert of meituan, to share the understanding and generation technology of short video content , The landing practice of meituan business scenario .

FFmpeg Tools ： It is used in audio and video development , fast @ Your brother, look at the audio and video tools

Almost all programmers engaged in audio and video development should know or have used FFmpeg.FFmpeg It's open source software , use LGPL or GPL license （ Note the open source protocol here , It has 『 Infectious 』, Will require its users to open source ）. We can use FFmpeg To record audio and video in various formats 、 transformation 、 Streaming capabilities .

Collection of basic concepts of audio and video ：148 A question takes you to get on the car quickly. Audio and video - audio and video foundation

This article is to build an index for the basic concepts of audio and video , Easy to remember and consult . If you are a developer interested in the direction of audio and video , It is strongly recommended to like 、 Collection 、 Share .

General purpose for efficient cross format low latency delivery CMAF Containers

In this paper, we introduce a common media application format （CMAF） As a way to standardize the container format , Combined with low latency HLS（LL-HLS） And low latency DASH（LL-DASH） Specific content encoding constraints and addressing mode constraints in , Provide cross format solutions , Maximize edge cache efficiency , Minimize source storage costs and client request rates .

Audio Orchestrator： Use multiple devices to orchestrate immersive interactive audio

This paper mainly introduces BBC R&D The development of audio multi device editing tool Audio Orchestrator .Audio Orchestrator You can use the Internet , Arrange audio and image output for multiple devices , To enhance the performance of media content .

Perceptual video coding based on saliency

This speech is about the thesis 《AI based saliency-aware video coding》 Explanation . In this paper , This paper introduces the pre-processing of coding using the method of saliency map prediction based on deep learning , The saliency graph information is used to improve the process of rate control to achieve better objective quality and rate saving .

Incomplete collection ： Bytes were open source in the past year 10 Items - byte Technology Annual goods

ByteDance technical team in the past year , Be prudent in the open source field , Many open source projects that have been verified by internal businesses have been opened to the community in succession , It also feeds back the new features of many mainstream open source projects to the community .

ffmpeg Source code analysis - Command line 3-4

This article is based on FFMpeg 4.2.5 The source code of version shall prevail , This series mainly analyzes ffmpeg_parse_options() Internal logic of transcoding . This series begins with a simple transcoding command ,ffmpeg -i a.mp4 b.flv 、 a.mp4.

https://juejin.cn/post/7052334889235906567

https://juejin.cn/post/7052334393947324429

Network damage tools are fighting in disorder

From the birth of the Internet , The limitation of network throughput 、 Loss of data packets 、 The delay and delay jitter of data transmission are closely accompanied by the development of the Internet . Today's Internet is even more crowded 、 A huge system of busy and complex , Different network services and applications share the same network infrastructure in a competitive way .

What is? HLS（HTTP Live Streaming）?

2009 year ,Apple Launched HLS（HTTP Live Streaming）—— be based on HTTP Adaptive rate streaming media transport protocol .HLS This paper describes a set of tools and programs to provide audio and video services through the Internet .

Behind the Chinese team's gold medal 「AI Sign language interpreter 」： First time on duty , Sign language can be understood beyond 90%

In many events of CCTV video , tencent 3D Sign language counts Homo sapiens 「 Listening 」 As 「AI Sign language interpreter 」, Sign language interpretation service is provided , Let the special people in the silent world also 「 listen 」 The grand occasion of hosting ice and snow events in China , It further improves the viewing experience of hearing-impaired people .

Mimir： adopt AI Provide video services to everyone

This is an example for Vimeo General purpose of player ABR Solution , The algorithm can adapt to different global network conditions and network fluctuations throughout the day .

AI | The mathematical basis behind optimization

Optimization in deep learning is an extremely complex task , This is a basic guide , It aims to understand the optimizer from a mathematical point of view .

Detailed explanation AI Accelerator （ One ）：2012 Year of AlexNet What did you do right ？

AI、 machine learning 、 The concept of deep learning can be traced back to decades ago , However , They have really become popular in the past ten years , Why is that ？AlexNet The basic structure of CNN There is no essential difference in architecture , Why can it be a blockbuster ？ In this series of articles , Former apple 、 philips 、Mellanox（ Now it belongs to NVIDIA ） The engineer 、 Dr. Princeton University Adi Fuchs Try from AI From the perspective of accelerator, we can find the answers to these questions .

Detailed explanation AI Accelerator （ Two ）： Why is it AI The golden age of accelerators ？

In the last article , Former Apple Engineer 、 Dr. Princeton University Adi Fuchs Explained AI The motivation for the birth of accelerators . In this article , We will follow the author's ideas to review the whole development process of processors , have a look AI Why can accelerators become the focus of the industry .

Uncover the Spring Festival hot money of Tiktok AR Behind the prop “ Secret ”

near 2022 Spring Festival in , Tiktok officially launched the warm China year . In order to increase the online spring festival atmosphere of the year of the tiger , A series of AR The camera effects are integrated into the Tiktok camera , The landscape of daily life will show various wonderful effects under the lens of Tiktok .

Most complete overview | Image segmentation algorithm

Image segmentation is a classical problem in computer vision research , It has become a hot spot in the field of image understanding , Image segmentation is the first step of image analysis , It is the basis of computer vision , It's an important part of image understanding , It is also one of the most difficult problems in image processing .

The spring of graphics

Through the author's way of learning , And work experience , From Zhejiang University CAD&CG laboratory , Go to Microsoft Research Asia , Then to Autodesk （Autodesk）, as well as VR Start-up company , Experienced an important stage of the development of graphics . Let readers look back on the experience of the author's 20th anniversary , See the vitality of graphics in the future .

in the light of YUV420 Research on depth image compression in color space

In this paper , The author puts forward two methods to adjust to RGB Image design depth image compression framework to compress YUV420 Images ; Based on lightweight framework , The adjustment is further studied YUV The influence of channel training distortion weight on coding performance .

Color image to gray conversion Summary and comparison of common methods

Today, let's talk about the most basic knowledge of image processing , Color image and gray image conversion , The commonly known formula of color image to gray scale is as follows .

【 Forum 】 Talk about how driverless vehicles perceive and understand the world

Driverless vehicles need to face the daytime during operation 、 Night 、 dusk 、 The wind 、 Heavy rains 、 Haze and other natural environment information , And pedestrians on the road 、 vehicle 、 Traffic lights and other object information , How on earth does the vehicle understand these complex information ？

Annual summary report of autonomous driving industry | Attached Download

《2021-2022 Annual summary report of China's automatic driving industry 》 Through to 2021 Sorting out the self driving events in , From the value chain of automobile industry 、 Technological development path 、 Multi scene applications 、 Business model 、 Supply of core parts 、 The six dimensions of data security are reviewed and analyzed annually .

Learn about autonomous vehicle “ location ”

This section will introduce autopilot positioning technology, including ：GNSS（ GNSS ）,RTK（ Real time motion location ） Inertial navigation .

---

Read recommendations

When talking about the metauniverse , What are we talking about ？

“ Meta universe ” The word has different meanings for people ： For some people , It is an immersive virtual reality experience in a lasting landscape ; For others , It is a specific technology stack ; For some people , It is a prospect of the future society .

MPEG The first 137 Progress of the meeting

MPEG The first 137 The first meeting was held in 2022-01-17 to 2022-01-21 Online . In the 137 Time MPEG The meeting ,MPEG Demand group （WG 03） A proposal solicitation order was issued （CfP）, Solicit technologies to realize encoder and packer synchronization and related distributed media asset storage .

Mux Acquisition web End streaming live broadcast tool StreamClub

Stream Club The platform is about to join Mux, This is a platform that allows customers to easily build real-time video broadcasting and create a studio like experience .Mux and Stream Club There is a common goal —— Let everyone use streaming media live .Mux Provide infrastructure for developers ,Stream Club Provide creators with on-site content production experience .

https://mux.com/blog/stream-club-is-joining-mux/

LiveVideoStack Year end Technical Inventory summary

stay 2021 end of the year ,LiveVideoStack Planned a year-end Technical Inventory , We have sent an invitation to the front-line technical workers in the audio and video field , I hope they can output some technical content in the direction of audio and video .

FPGA Exchange format : Realize interoperable FPGA Tools

Google and Antmicro Cooperative FPGA Exchange format items , Provide a unified framework , To reduce the entry threshold for developers to quickly migrate from one tool to another .Antmicro And others CHIPS Alliance members are cooperating to develop Interchange Format definition and related tools , Aims to become FPGA Development standards required by the industry .

https://opensource.googleblog.com/2022/02/FPGA%20Interchange%20format%20to%20enable%20interoperable%20FPGA%20tooling.html

MPEG-LA Release VVC Patent pool

Last month, ,MPEG-LA Released VVC Patent pool . before ,Access Advance stay 2021 year 7 First released in October VVC Patent pool . Two big VVC The patent pool forms a confrontation , But none of them attracted the most important enterprises to join .

Datadog Announcement of acquisition CoScreen

CoScreen Now it is Datadog Part of ,Datadog It is a monitoring and security platform for Cloud Applications , It is also one of the most successful technology companies .Datadog Let the technical team have an in-depth understanding of their systems and Applications , Help them find 、 Analyze and solve problems .

https://www.coscreen.co/blog/coscreen-joins-datadog/

---

Activity recommendation

LiveVideoStackCon Registration for special jury activities

In order to be more perfect LiveVideoStackCon Speech content of audio and Video Technology Conference , Such as ： accuracy 、 Logicality 、 Readability 、 The dryness of the content 、 And the benefits to participants , Ensure the accuracy of content output & rigorous ; secondly , Self improvement of the speaker （ The output content is more professional 、 Reduce errors 、 Increase your confidence in speaking 、 Multi party communication to promote exchanges ）; For those who participate in the review , When reviewing the content, you can learn about the latest developments in the industry , Communicate with the members of the jury , Sharing experience .

Therefore, we are particularly established LiveVideoStackCon Jury mechanism , Click to read the original for more details , Sign up to participate ：

http://livevideostack.mikecrm.com/mVxofCi