Talk about multimodality of fire

Multimodal machine learning , English full name MultiModal Machine Learning (MMML), The aim is to achieve the ability to process and understand multi-source modal information by means of machine learning .

Each of these Source or form of information , You can call it a mode . for example , People have a sense of touch , auditory , Vision , The sense of smell ; The message has voice 、 video 、 Words and other media ; A variety of sensors , Such as radar 、 infrared 、 Accelerometer, etc. . Each of the above can be called a mode .

Modes can also be very broadly defined , For example, we can think of two different languages as two modes , Even the data sets collected in two different cases , You can think of it as two modes .

The present , Multimodal technology has a wide range of application scenarios , Such as Taobao Search 、AI subtitle 、AI Virtual digital human 、 Humanoid interaction 、 Intelligent assistant 、 Product recommendation and information flow advertising 、 Image vector retrieval of video frame and face frame 、 Voice interaction, etc .

We are honored to invite He has many patents and many years of working experience as an algorithm engineer in large factories Peng teacher , utilize 2 About an hour or so , Systematically sort out multimodal technology for you .

Live sharing

01

PART

Day1 live broadcast

• 01  Development and future of multimodality

• 02 Extensive reading of papers ：CLIP— Hongmeng clock masterpiece in multimodal field

• 03 Learning path recommendation

  
  

Day2 live broadcast

• Intensive reading —CLIP： Hongmeng clock masterpiece in multimodal field

• 01 Research background

• 02 Introduction

• 03 Model

• 04  experiment

• 05 Conclusion

02

PART

Lecturer

Live time

03

PART

• 7 month 28 Japan （ Thursday ） On the evening of 20:00-21:00

• 7 month 29 Japan （ Friday ） On the evening of 20:00-21:00
  

Students interested in multimodal Technology , Scan the QR code below , Reservation live broadcast .

Sweep code payment 0.1 Yuan means the appointment is successful

Live broadcast when the party staff contact you ~

04

PART

Multimodal learning path

01  Fundamentals of multimodal theory

Study multimodal pre training related papers ——CLIP、ALIGN、VILT

02  Self supervised algorithm

Learn some self-monitoring schemes that may be used in multimodal pre training ——MAE、DINO、MOCO

03  Introduction to multimodal downstream tasks

Mainly understand VQA The tasks and nlvr Mission

04  Multimodal applications

Image Captioning Case study 、 Alibaba e-commerce cross modal retrieval case . Understand the task introduction 、baseline build 、 Model optimization 、 Result display .

05  Multimodal project

AI Smart copywriting 、 Mobile photo album management and retrieval based on multimodal pre training model 、AI Lip recognition 、 Automatic driving based on deep multimodal target detection and semantic segmentation

Students interested in multimodal Technology , Scan the QR code below , Reservation live broadcast .

Sweep code payment 0.1 Yuan means the appointment is successful

Live broadcast when the party staff contact you ~