Ma Yi, Shen Xiangyang, Cao Ying's latest AI overview is hot! It took 3 months to build, netizens: required papers

Come out , Professor Ma Yi's AI The review paper finally came out ！

It took more than three months , United neuroscientist Cao Ying 、 Computer Daniel Shen Xiangyang , Work together .

According to my description , This paper is about him “ The work of the past five years and the development of intelligence for more than 70 years are organically combined ”, And it also means ：

I have never spent so much energy and time on an article in my life .

To be specific , Namely “ The origin of intelligence and the basic outline and framework of computing principle are sorted out , It can make everyone realize the possibility of integrating theory with practice .”

When previewing on Social Networks , Many netizens are looking forward to it .

The results just came out today , Some scholars said , Just to design a new generation model , It inspired me .

Let's see what kind of paper this is ？

Two principles ： Simplicity and self consistency

In the past ten years , The progress of artificial intelligence mainly depends on the black box model of training homogeneity , The decision-making process 、 Feature representation and other aspects are difficult to explain to a large extent .

And this end-to-end rough training , It not only leads to the size of the model 、 Training data and computing costs are increasing , And there are many problems in practice .

The learned representation lacks richness ; Lack of stability in training ; Lack of adaptability , Prone to catastrophic forgetting ……

Based on this background , Researchers hypothesized , One of the root causes of these problems in practice , It is the lack of systematic and comprehensive understanding of the functions and organizational principles of intelligent systems .

And whether there is a unified method to explain .

For this reason, this article puts forward two basic principles —— Simplicity and self consistency , Answer two basic questions about learning .

1、 What to learn ： What is the goal of learning from data , How to measure ？

2、 How to learn ： How can we achieve such a goal through efficient and effective computing ？

They believe that these two principles restrict the function and design of any intelligent system , And it can be completely rephrased in a measurable and calculable way .

Take simplicity as an example . The foundation of intelligence is the low dimensional structure in the environment , This makes prediction and generalization possible , This is the principle of simplicity . But how to measure ？ This paper presents a geometric formula to measure simplicity .

Based on these two principles , Get the perception / Intelligent general architecture ： Closed loop transcription between compressor and generator .

It implies that the interaction between the two should be a pursuit game . In this game , They act as the opposite of the joint objective function , Not an automatic encoder .

This is also one of the main advantages of this framework , Self learning through self-monitoring and self-criticism , This is the most natural and effective .

Basically , This framework can be extended to completely unsupervised environments , At this time, we only need to regard each new sample and its increment as a new tired .

Self monitoring type + The game mechanism of self-criticism , Such a closed-loop transcription will be easy to learn .

It is worth mentioning that , The structure of this learning feature , Similar to the structure of category selection regions observed in primate brains .

Cao Ying, one of the authors, said , This framework is similar to many previous ideas , Including predictive coding 、 Comparative learning 、 Generate models 、Transformer……

besides , They also gave some new directions , such as CV And graphics .

Finally integrated together , An autonomous agent , The best strategy for learning a task , Yes, will perceive （ feedback ）、 Study 、 Optimization and action are integrated into a closed loop .

It unifies and explains the evolution of modern deep networks and many AI practices . Although the whole article mainly takes the modeling of visual data as an example .

But researchers believe in these two principles , It will unify the understanding of a wide range of autonomous intelligent systems , And provide a framework for understanding the brain .

Last , They also ended with the words of the father of thermodynamics .

It roughly means that only when you can measure , And use numbers to express what you say , You know it ; If not , It could be the beginning of knowledge , But in your mind , You have hardly advanced to the stage of Science .

One More Thing

What's interesting is that , Just submit arXiv Occasion , Professor Ma Yi also asked for help on the Internet ：

Can't someone write an interface ？ The kind of paper submitted with one click . Instead of spending a lot of time revising the paper .

Netizens have also offered advice ： Use LaTeX compiler , instead of PdfLaTeX.

There are also people who go straight to the top .

Okay , Interested friends can click the link below to learn more ～

Thesis link ：
https://arxiv.org/abs/2207.04630
Reference link ：
[1]https://weibo.com/u/3235040884?profile_ftype=1&is_all=1#1657668935671
[2]https://twitter.com/YiMaTweets/status/1546703643692384256
[3]https://twitter.com/doristsao/status/1546722505091534851