History of deep learning

Conceptually speaking , Deep learning is inspired by the brain .

Just like the design of the airplane is inspired by birds .

Although their basic principles are the same , But the details are quite different .

The history of in-depth learning can be traced back to the current name 「 cybernetics 」（cybernetics） The field of .

It begins with 1940 s McCulloch and Pitts, They proposed that neurons are threshold units with on and off states , Boolean circuits can be constructed by constructing logical reasoning and connecting neurons to each other .

The brain is basically a logical reasoning machine , Because the on and off states of neurons are binary .
Neurons calculate the weighted sum of inputs , And compare the sum with its threshold .
If it is above the threshold, it turns on , If it is lower than the threshold, it turns off , This is a simplified view of how neural networks work .

1947 year , Donald · Herb (Donald Hebb) The idea that neurons in the brain learn by changing the strength of connections between neurons , This is called super learning . If two neurons are activated together , Then the connection between them will increase ; If they are not motivated together , Then the connection will be reduced .

1948 year , Norbert · Wiener （Norbert Wiener） Put forward Cybernetics , That is, by building a system with sensors and actuators , There is a feedback loop and a self regulating system . for example , The feedback mechanism of a car , It all comes from the idea .

1957 year ,Frank Rosenblatt Put forward 「 perceptron 」, This is a learning algorithm , You can modify the weights of very simple neural networks .

Overall speaking , The idea of trying to build intelligent machines by simulating a large number of neurons was born in 1940 years , stay 1950 The s developed rapidly , And in 1960 It died out completely in the late s .

The field is in 1960 The main reason why the year died out is ：
● The researchers used binary neurons . However , The way to make back propagation work is to use successive activation functions . at that time , The researchers did not have the idea of using continuous neurons , They can't train with gradients , Because binary neurons are not differential .
● For continuous neurons , The activation function of the neuron must be multiplied by the weight to contribute to the weighted sum . However , stay 1980 Years ago , Multiplication of two numbers , Especially floating point numbers , Very slow . This is another reason to avoid using continuous neurons .
●
With the emergence of back propagation , Deep learning is in 1985 Take off again in .
1995 year , The field is dying out again , The machine learning community has abandoned the idea of neural networks .
2010 Beginning of the year , People began to use neural networks in speech recognition , Performance improvement , Then it was widely used in the commercial field .
2013 year , Computer vision is turning to neural networks .
2016 year , The same shift has taken place in naturallanguageprocessing , Soon , A similar revolution will take place in robots 、 Control and many other areas .