当前位置：网站首页>How strong a mathematical foundation does deep learning need?

How strong a mathematical foundation does deep learning need?

2022-06-28 19:40:00 【woshicver】

link ：https://www.zhihu.com/question/266478287
edit ： Deep learning and computer vision
Statement ： Just for academic sharing , Invasion and deletion

author ：EddyLiu
https://www.zhihu.com/question/266478287/answer/587489884

With the foundation 《 probability / Statistics 》、《 linear algebra 》、《 Differential and integral calculus 》 knowledge , You can start the algorithm and practice of deep learning . But after a period of engineering practice , I gradually feel that I spend most of my time choosing models , Supernumerary parameter , Or the arrangement and combination of network structure . The black box characteristic of deep learning is becoming more and more obvious . Are deep learning engineers really data “ Alchemist ” Do you ？
If , You have this feeling , The following video might as well take time to see ( All need to climb over the wall )：

Li Hongyi 《Machine Learning and having it deep and structured》

Not much to say , Just look at the catalogue .
Course address ：http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLDS18.html
《Theory 1 - Why Deep Structure》
Can shallow network fit any function
Potential of Deep
Is Deep better than Shallow
《Theory 2 - Optimization》
When Gradient is Zero
Deep Linear Network
Does Deep Network have Local Minima
Geometry of Loss Surfaces (Conjecture)
Geometry of Loss Surfaces (Empirical)
《Theory 3 - Generalization 》
Capability of Generalization
Indicator of Generalization

Sanjeev Arora《The mathematics of machine learning and deep learning》

Video address ：https://www.youtube.com/watch?v=r07Sofj_puQ
This is a ICM2018 Keynote speech of , although Sanjeev Arora As a professor of computer science at Princeton , But the content of the speech is simple , It does not involve a large number of mathematical formulas and derivation , Here is an outline ：

Summary

The contents of these two parts echo each other , Let's look at Mr. lihongyi's course first , Then I'm looking at Sanjeev Arora Professor's sharing summary .

author ： Running https://www.zhihu.com/question/266478287/answer/313813956

There is no need to read another math Master 了 , Even if you plan to do machine learning （ Including deep learning ） Theoretical research , It is enough for the students majoring in mathematics . If you plan to do deep learning , It is suggested to read a book about machine learning Master, At the same time, improve your programming ability （ Like brush LeetCode）.

author ：Cv Dafa code sauce
https://www.zhihu.com/question/266478287/answer/2479263874

And Compared with machine learning , Most of the contents of deep learning do not have such high requirements for mathematics .

If it is for the purpose of engineering application and non theoretical academic research , That is, some operations of linear algebra , Various loss functions , Gradient descent method , Back propagation algorithm .

Compared with the support vector machine in machine learning ,EM Algorithm , Probability graph model , Probability inference , Various sampling algorithms , It's much easier .

It should be friendly to use flower books , In the first few chapters, a large amount of space is devoted to the introduction of mathematical knowledge , It basically covers the main mathematical knowledge points of in-depth learning . Include ：

linear algebra 、 Probability theory and information theory 、 Numerical calculation

You should be able to feel , The first chapter of Huashu 1 part “ Fundamentals of Applied Mathematics and machine learning ” And the 2 part “ Deep networks ： Modern practice ” Relatively easy to understand , As long as there is some mathematical foundation , Can read . The problem is that 3 part - Deep learning research ： Linear factor model 、 Self encoder 、 It means learning 、 In depth learning Structured probability model 、 Monte Carlo method 、 Straight face partition function 、 Approximate inference 、 Depth generation model

The mathematical knowledge in these chapters has increased significantly , And many of them are unfamiliar to us . And then there's something that's bothering everyone , such as MCMC sampling ,EM Algorithm ： Approximate inference and variational inference Variation .

Just look at the description , It is difficult to understand the concept of functional and the principle of variational method .

From the above, we can see , If you don't lay a good foundation in Mathematics , It is also unrealistic to learn deeply .

however , After graduation, there is no need to further study mathematics as a graduate student , Education is not the solution , Have the ability to learn to solve problems .

Flower Book （《 Deep learning 》, People's post and Telecommunications Press ） They are the teaching materials with the largest sales volume in the field of in-depth learning in China . It is generally acknowledged that their quality is very high , But an embarrassing situation is ： Most people don't understand these two books after they buy them , I didn't insist on reading ！

There are many mathematical concepts and formulas in flower books , It is difficult for most readers , In particular, a lot of mathematical knowledge goes beyond the undergraduate course “ Differential and integral calculus ”,“ linear algebra ”,“ Probability theory and mathematical statistics ”3 The scope of the course . Seeing strange mathematical symbols and formulas makes us at a loss .

So I'm recommending a resource --《 The mathematics of machine learning 》, Cooperate with it to learn , Basically, you can clean up your watermelon book , The mathematical barrier of Huashu . When you see mathematical symbols and formulas, you will no longer feel strange , How to apply these mathematical knowledge to machine learning and deep learning , There is also a clear understanding .

This book accurately covers machine learning in the smallest space 、 Deep learning 、 Strengthen the core mathematical knowledge required for learning . Chapter structure design is scientific and reasonable , Something you don't need , Not at all , This can effectively reduce the learning cost of readers .

If you want to engage in academic research , It can also lay a good foundation in mathematics .

author ： Machine consciousness
https://www.zhihu.com/question/266478287/answer/1911910013

If you want to do pioneering research , probability theory 、 linear algebra 、 Calculus is not enough , These mathematical tools do not provide a blueprint , We need to understand some less ancient fields of mathematics , For example, modern algebra 、 Differential geometry 、 Functional analysis, etc .

For example , The problem of finding roots of quintic equation is studied , Learning math is not enough , You need to be able to expand Galois groups and domains

A lot of work is about recommendation , If you know differential geometry , We can use the concept of fiber bundles to abstract this problem , The user feature space is the base space , The combination of user features and content features is the whole space , But how to use the known conclusions in differential geometry to optimize the recommended modeling , It's still under exploration , But at least there is more room to think

* END *

If you see this , Show that you like this article , Please forward 、 give the thumbs-up . WeChat search 「uncle_pn」, Welcome to add Xiaobian wechat 「 woshicver」, Update a high-quality blog post in your circle of friends every day .

↓ Scan QR code and add small code ↓