Dialogue with Jia Yangqing, vice president of Alibaba: pursuing a big model is not a bad thing

In 60 years of development ,AI After ups and downs and ups and downs . In the face of early “ Machine learning is in 80% It's time to solve 80% The problem of , But I don't know which 80% What have you solved in your time 80% The problem of ” The challenge of , Up to now AI Standardization 、 Pratt & Whitney has made great progress , stay 《 New programmers 004》 in , We invite Jia Yangqing, vice president of Alibaba, based on himself , from AI frame 、 Computational power of model and algorithm 、 engineering 、 Open source 、 And the characteristics that developers need to have , Share will AI Algorithm and application: mental process and method in the process of large-scale landing .

dialogue  |  Tang Xiaoyin      author  | Du min

Guests interviewed  | Gu Yangqing

Produce | 《 New programmers 》 Editorial Office

The change of AI , It not only leads the ideological trend of the new digital era , It also affects the technical life of thousands of programmers , This also includes the current Vice President of Alibaba 、 Senior researcher of Alibaba cloud computing platform division 、 Hands on the court AI Jiayangqing, the person in charge of the platform .

Jia Yangqing doesn't think he is a person with high IQ , Just because of interest . He didn't think he was a hard-working man , But what should be done will be done . Now he doesn't want to mention his past achievements , The so-called gentleman acts according to his position , I don't want to , Because it's the past , All is prologue , For him , I want to waste my life on interesting things , For example, explore the landing of large models of artificial intelligence 、 How to engineer artificial intelligence .

therefore , When you turn on the Internet , Jia Yangqing's figure is looming . For their personal , The media reports on him mostly stay in 2019 Year of 3 month 19 Japan . Because on this day , In addition to the mainstream deep learning framework Caffe The author of 、TensorFlow One of the co founders 、PyTorch 1.0 Our cooperative leaders 、ONNX（Open Neural Network Exchange） In addition to the founders of , A new vice president of Alibaba Technology . In just two years , Jia Yangqing handed over his successor Caffe、TensorFlow Another innovation of —— use 4S Standard leads the team to forge big data +AI Product system Ali Lingjie , Open source drives AI Engineering landing , help AI Become a sharp tool for industrial digital upgrading .

《 New programmers 》 Launched an in-depth dialogue with Jia Yangqing , What we are deeply familiar with is that although it has gone through thousands of sails , But technical people who are still particularly pure about technology , The original heart has never changed . In a new era of Technology , He wanted to get out of the lab AI The algorithm can scatter all over the sky , Walk into and illuminate all walks of life .

This article is excerpted from 《 New programmers 004》 Paper journal + Electronic journals are listed synchronously

From automation to AI, Jia Yangqing's persistence and opinions

As is well known to the outside world , Jia Yangqing is not a pure computer major .2015 year ,《 The programmer 》（《 New programmers 》 The predecessor ） When I first met Google, I buried myself in AI Jia Yangqing of China , When it comes to AI When deeply learning the story of marriage , This Tsinghua student who specializes in automation at the undergraduate and graduate levels , Once joked that , Automation mainly does two things , One is a boiler , One is to drive the elevator .

therefore , After he graduated from Graduate School , I decided to go to the University of California, Berkeley to study my favorite computer , Got a doctorate .

in the meantime , He is studying a project entitled “ How do humans form the concept of category in the process of personal growth ” When it comes to Psychology , suffer Alex Krizhevsky stay ImageNet Inspiration for success , Developed a relatively complete in-depth learning framework Caffe, The reason for R & D is as humorous as that of men of science and technology ,“ I am writing Caffe The reason is that I don't want to write my thesis ”.

Take this as a guide , Jia Yangqing is studying deeply 、 neural network 、 Large scale training and other technology support go deeper and deeper on the road of artificial intelligence , And have their own different views and Thoughts on this field .

Gu Yangqing

《 New programmers 》： In the past two years , Many practitioners say that there is no major breakthrough in the theoretical research of artificial intelligence , And many AI scientists after entering the industry from scientific research , Choose to return to academia again . Under this phenomenon , Whether artificial intelligence will enter the next cold winter ？

Gu Yangqing ： I think “ The cold winter ” The word may not be used . Today's AI is more like returning to normal temperature after experiencing the heat . At present, whether in the Internet industry , Or traditional industries , We use more intelligent algorithms to solve the previous problems , It has been normalized . After returning to normal temperature , We can really calm down and do something down-to-earth .AI Technology is always a process from theoretical research breakthrough to large-scale spread , At present, its theory and practice have realized the state of walking on two legs .AI The ups and downs are essentially the conclusion drawn from the perspective of ordinary melon eaters in the industry , You may see that many enterprises that once grew rapidly are slowing down , I mistakenly thought it was AI The cold winter has come , In fact, this matter itself has little to do with the artificial intelligence industry itself .

《 New programmers 》： Now many universities and even primary and secondary schools have set up artificial intelligence majors or courses , What do you think of this phenomenon ？

Gu Yangqing ： I think it's a good thing to have access to more cutting-edge technologies from my student years . Think back ten years ago , Carnegie Mellon University has set up Machine Learning Department system . however , This phenomenon may bring some short-term invisible disadvantages , There is the possibility of ripening and causing many people to fish in troubled waters . however , We believe that the market economy itself has a mechanism of self-regulation , Will gradually filter out some useless things , therefore , There is no need to worry about these shortcomings .

Under the trend of large model , Alibaba AI Engineering practice

《 New programmers 》： More computing power 、 The development of large models for larger data sets seems to be a trend . Now many practitioners don't pay special attention to the design details of the model , How do we balance AI The relationship between model refinement and scale ？

Gu Yangqing ： I think this phenomenon is very typical , It has also appeared in the field of computer vision . With 2012 Took part in ImageNet Convolutional neural network, which was a great success in the large-scale visual recognition challenge AlexNet For example , The total number of parameters of the model is 6000 ten thousand . Its rise has made many AI Practitioners come up with a relatively simple idea , That is, the larger and deeper the model or the more model parameters , The better .

But here we are 2014 year , be based on Inception Deep neural network model of module GoogLeNet In the presence of 600 The same or even better effect can be achieved on the basis of 10000 model parameters . therefore , In the field of super large models , Many people pursue the promotion effect , Create a phenomenon that the larger the parameter scale, the better the simulation effect . Over time , When users are aesthetic tired of the scale of the model , It will be found that the details such as the structure of the model and the interpretability of the model become more important .

however , This phenomenon is also a typical development process of technology iteration in the field of scientific research , The technology of instant fire has attracted countless people , When people find that this direction is too one-sided, they will return to their original position . This does not mean that pursuing big models is a bad thing , Because the bigger the model , It will also put forward higher requirements for the model algorithm supported by the bottom layer , It also helps to promote the birth of many innovations in the system .

《 New programmers 》： all the time ,AI Developers see model transformation as a very troublesome thing , Do you think ONNX Whether it is possible to become an industry norm , So as to reduce the work of model transformation ？

Gu Yangqing ： This is a very interesting question , At first, I was designing ONNX when , We hope it can be in various frameworks 、 As a standard between software and hardware service providers , Let developers better implement the model to get through .

After years of practice , We find that the model is constantly changing , Such as BERT、GPT-3 And other models continue to emerge , These models will be closely coupled with the framework that designed the model , For example, many large-scale distributed models are now TensorFlow On the basis of implementation . therefore , In the latest application fields , Model transformation has not yet become a core issue , There is no need to put TensorFlow Transform the model into other requirements using the framework .

Essentially ,ONNX It works on some existing typical standard models , For example, it provides a hardware service provider for computer vision to better cut into this field . On the value chain , In my submission ONNX Not supporting the latest model transformation , But after the model is standardized, let its popularity become more rapid . therefore , I think ONNX Exist for a long time and become a standard in the industry , But it won't cover the whole industry , Because new research will always be in a state of relative fragmentation and no need for standards .

《 New programmers 》：2021 At the Alibaba cloud habitat Conference , Ariel Lingjie officially released , You also talked about AI The essence of engineering includes Yunyuan biochemistry 、 Scale and standard inclusive , This will give AI What changes developers bring ？

Gu Yangqing ： As a way for enterprises and developers “ Open the box ” The platform of , Ali Lingjie integrates Ali big data +AI Ability , More followed by “ Big model (Scale)、 high efficiency (Speed)、 Ease of use (Simplicity)、 Scene (Scenario)” Core 4S standard .  

• Scale： Flexible expansion and contraction of big data 、 Big model 、 Big app . Because of the existence of cloud infrastructure , Whether it's AI Training and model training are becoming easier and easier . Through the cloud native way , With the help of Ali's Max Compute platform , Developers can use almost unlimited flexibility with zero start-up cost , Also let computing power no longer become AI Bottlenecks in development .  

• Speed： Extreme operation 、 Development 、 O & M efficiency .“ How to improve the efficiency of developers' development and iteration through cloud primitives ”, It is Alibaba's original intention to put forward this standard . The current in GPU、CPU、 As new chips continue to develop , Users have not been too concerned about the efficiency of hardware execution , Instead, how to get rid of the past manual installation of software and optimization is what people care about , And based on Speed The management of cloud native container brought by the standard 、 Management of cloud native operating environment 、 Cloud native scheduling and other tools , Enable developers to write code 、 The whole link from the model to the final model landing and promotion becomes more rapid .  

• Simplicity： standard 、 Easy to use like a calling function . With the past AI The full stack requirements put forward by developers are different , Now engaged in AI Development , The division of labor of engineers is very clear .Simplicity Means , Standing in the cloud today 、AI From the perspective of algorithm developers , The industry has been able to provide a series of standardization API, Make the application of the upper layer 、 data 、 Business engineers don't have to worry at all AI Implementation details , Develop directly as if calling a function AI Application or product . 

• Scenario： Born to the scene . No scene AI Ability is useless . As mentioned above Scale、Speed、Simplicity It's all about solving problems in different Scenario The process of landing in . For developers , In a deep understanding of all walks of life 、 After the requirements of different scenarios , In the AI Algorithm iteration 、AI The application is more targeted when landing .

《 New programmers 》： Based on the above , Some capabilities can be encapsulated as API It is convenient for developers to directly call . In this trend , future AI What capabilities should developers have ？

Gu Yangqing ： today AI Developers should jump out of AI frame 、 Shackles of traditional thinking , Don't limit your goal to simple optimization 、 Parameter adjustment mode , We don't need to carve a new convolutional neural network model and further improve the accuracy , Instead, we need to learn how to abstract the required computer vision problem into a deep learning problem in different scenes .

future , Model innovation will be a direction of scientific research , But the bigger opportunity lies in the data modeling of the upper layer 、 This layer of model modeling .

Open source for AI What did it bring ？

《 New programmers 》： today , We have ushered in an era when open source is extremely beneficial . stay GitHub On , The growth rate of developers from China has become the fastest in the world , Open source to AI What are the benefits of the development of ？

Gu Yangqing ：AI The development of the field is inseparable from open source . What we have seen in recent years AI、 Algorithms are open source , Open source also allows you to reproduce code 、 The algorithm becomes very easy .

From the perspective of open source , The mentality of upward and open source in China has developed very rapidly , And the enthusiasm of domestic developers for open source is not lower than that of foreign countries . however , I think the biggest challenge of open source today is the top-down mechanism design or mentality .

Now , Many companies talk about open source , Often still stay in the process of opening up the code for everyone to admire . however , From the perspective of the global open source community , The most important point of open source should be to let more people participate in open source projects . therefore , When the code is open , How to further build an open source ecological community and let more developers participate in development and iteration together , This is also a path that the domestic open source community must take .

《 New programmers 》： In the new environment , What kind of open source strategy does Ali have , How to encourage more people to participate in open source ？ in addition , In the process of open source application , Many enterprises even set up KPI, What do you think ？

Gu Yangqing ： This is a good question . First , Ali has set up an internal open source Committee , To ensure that the enterprise can see the demand for open source of different projects . This means that open source does not simply open up the code , You also need a license to include the code 、 The operation of the community 、 Community interaction has a relatively standardized mode , Furthermore, through the open source Committee, we can better promote developers in the company to participate in open source more smoothly .

As for how to design for open source KPI, I think it's very difficult , Because the construction of community itself is a problem driven by interest and enthusiasm of participants . Instead of thinking about how to set up a reasonable for open source KPI, Instead, think about how to help open source jump out of KPI frame , Let more people explore things more openly . therefore , To some extent, this problem is the cultural construction within a company , Not everything needs KPI " . Open source oriented , We need to set aside some time for engineers to make their own passion driven judgments . If open source once KPI turn , When using standardized metrics to bind open source developers , Perhaps the process of promoting open source is the beginning of transformation .

《 New programmers 》： Open source commercialization , What's your point of view ？

Gu Yangqing ： Open source and commercialization are a matter of different opinions . In the near future , Many practitioners believe that , As long as open source software is made, it can raise a lot of money , But this is not the result of rational thinking .

Turn the clock back five years ago , What many people often say is ： There is absolutely no prospect of commercialization of open source software . As for why there are such remarks , This is mainly because the commercialization was not realized in that year .

Open source itself is something that has nothing to do with commercialization . There are just a lot of open source projects , There is a very strong demand in commercial applications , Some companies provide enterprise level services on the basis of open source , This is actually a natural process . Because the application of open source software has certain requirements for users' Technology , If a company can directly solve users' problems , meanwhile , When users don't want or don't have a team to do basic things , There will be companies that offer flexibility 、 No operation and maintenance , Including services such as enterprise level technical support . Above all , Open source and commercialization can be measured at two levels , The first is how to achieve better service ; The second is to see how much the enterprise can pay for enterprise level capacity , In order to achieve a relatively balanced state .

however , Open source software itself , We still hope it has strong purity , Let the industry have a content space for technology sharing and technology iteration such as open source .

Breaking the end of top programmers is the traditional thinking that there is no code to write ！

《 New programmers 》： From fighting on the front line of research and development , To become a manager 、 Entrepreneur , Many people think that in practice , Good programmers gradually have no time to write code 、 Do research . Do you think so ？

Gu Yangqing ： First, break an idea , Good programmers don't write code in the end , All have to be managed . in fact , At present, the development of the industry needs a variety of talents , Maybe a very good architect , It could also be a manager , Among them, there are many excellent system architects who hold very high positions . Here it is , I also recommend 《 One month myth 》 This book , It refers to different professional division of labor , And why a team needs to have a good role as a manager and architect .

In fact, good programmers should pursue their own heart , And in order to achieve one thing, you may wear different clothes at different times “ hat ”, Sometimes dealing with people , Sometimes dealing with code . To achieve a thing, it will never be able to be promoted by a single point of skill . therefore , All you need to do is find the direction you are good at , At the same time, it can adapt to different occasions to a certain extent 、 demand , As a manager 、 Different roles such as system design .

《 New programmers 》： This is also what a good programmer should have ？

Gu Yangqing ： Yes . The system software used in the current industry , Include IT The size of the infrastructure , It's impossible for a person to do things alone . therefore , Even if you are a good architect , Also need to communicate with people 、 Iteration together 、 Solve the problem together , This is a skill that any of us need to have today .

《 New programmers 》： For the younger generation of developers and those who are engaged in AI Practitioners of , Guidance and suggestions on employment direction ？

Gu Yangqing ： because AI It covers a wide range of fields , For general purpose AI For talent , Relatively successful AI Talents are often people who keep curiosity, walk on the road and can define problems through broader needs . Take a large-scale model as an example , In daily practice , Many developers often train single point models in pre training scenarios , In order to improve the accuracy of the whole model . Compared with this method , Try to get rid of single model optimization or single framework optimization , It's redefining a problem . For example, in the pre training scenario, a model is trained independently through a large number of existing but irrelevant data , This makes it work well in many application scenarios , Thus, it evolves into the improvement of some algorithms or systems .

AI New challenges and opportunities

《 New programmers 》： Whether different in-depth learning frameworks can be unified in the future ？

Gu Yangqing ： I don't think it's possible . If a new framework appears today , What we need to consider is what kind of problems it solves in the whole industry . once ,TensorFlow The emergence of has solved the problem of large-scale system ; When very large-scale systems are inconvenient to use , Developers began to think about how to use a more like Python、 Let's develop and iterate the algorithm in a way that is easier to iterate , therefore PyTorch emerge as the times require . There are many frameworks in the current market , If you just develop a better than TensorFlow、PyTorch There are slightly different frameworks , No value .

I don't think we need to worry too much about whether the framework can be unified , Unification is not a goal , Can solve the problem that we are at the top today AI application 、 Problems in scientific research , Is the more important point .

《 New programmers 》： From a programmer's point of view ,AI Completing code has also become a very common function , Will there be a program in the future that can learn from another program , Realize programming ？

Gu Yangqing ： I think learning from existing programs or writing new programs , The algorithm is possible , But don't be overly optimistic . Some existing products in the current market, such as GitHub Copilot, More is to use their own computing power to search in a large amount of code , Then complement the functional realization of the code . This is related to AI There is still a difference between writing a program truly .

AI The computational power of the algorithm 、 Memory is obviously higher than human beings , So at the level of simple and repetitive work ,AI It can be a good supplement to human ability and energy . This problem is to liberate people from simple and repetitive work , Let's think more intelligently . therefore , In my submission AI There is great potential in this field .

《 New programmers 》： Under the wave of the meta universe ,AI What are the scenarios or applications that can promote landing ？

Gu Yangqing ： in fact , There are too many concepts piled up in today's metauniverse , For example, many people think it is a component of blockchain or other technologies . From what I understand , Metauniverse contains two relatively important concepts ：  

• One is VR/AR, They break the traditional writing 、 video 、 The form of voice and other content , Brought a new way .  

• Second, because of the different ways of content presentation , The tools for content creation will also change dramatically .AI In the process of collision and fusion with the meta universe , because AI It has tool properties , Therefore, many of its algorithms make the meta universe gradually become a reality .

Let's break it down , In content creation , A few years ago VR/AR The biggest bottleneck encountered in the development of is the lack of equipment and content . Now with AI It has brought AliceMind、M6 Equal depth language model , Make content creation more simple and efficient .

At the tool level , In the use of VR When the device turns its head or looks at a position ,VR The device needs to quickly grasp the changes in the position of our eyes , Render at the same time . There will be pupil recognition or pupil tracking AI Algorithm implementation , This algorithm becomes faster 、 After efficiency, it makes VR When rendering, you can set aside more time for rendering , Instead of setting aside more time to identify . This makes us in VR/AR The environment seen in can become more real .

The above two directions ,AI Can play a good role in helping .

《 New programmers 》： Facing the future , What kind of scene do you think will make you feel most fulfilled ？

Gu Yangqing ： In a word, the conclusion is “AI to ground ”, The algorithm coming out of the laboratory can be more widely used in all walks of life .

Looking back

It's too voluminous ！AI High accuracy of math exam 81%

Data analysis you choose Pandas Or choose SQL？

2D Transformation 3D, Look at NVIDIA's AI“ new ” magic ！

How to use Python Realize the security system of the scenic spot ？

 Share 
 Point collection 
 A little bit of praise 
 Click to see