[reading notes] deep learning Chapter 1: Introduction

Preface

Deep learning （deep learning） As a palace work in the field of deep learning , It covers the field of mathematics, including linear algebra 、 probability theory 、 Information theory and other knowledge , And related content in machine learning , At the same time, it also introduces the deep learning technology used by practitioners in the industry , It is the most comprehensive field of deep learning 、 Authoritative works , I have heard about this book for a long time , But because of its substantial content 、 There are many mathematical formulas, and I have never dared to get involved , Take advantage of this summer's free time , Read this work , Hope to gain something . The content of this blog involves the first chapter of Huashu , It mainly introduces the concept of deep learning and the development history of deep learning .

---

introduction

Artificial intelligence

• advantage ： Describe the problem through a series of formal mathematical rules .
• Inferiority ： Difficult to solve is easy for people , But it's hard to formally describe the task , As said 、 The face in the image .

resolvent ：

• Let computers learn from experience .
• Understand the world according to the hierarchical conceptual system .

The key challenge of AI ： How to convey informal knowledge to computers .

Resolution process ：

• Hard code knowledge , That is, artificial intelligence knowledge base method （ Clumsy method , There is no ability to acquire knowledge by oneself ）.
• The ability to extract patterns from raw data —— machine learning , Performance depends on the representation of the given data （ It's a feature , But what features are extracted is a problem ）.
• Use machine learning to discover the representation itself , Instead of just mapping the representation to the output , The method is called It means learning （ Such as self encoder composed of encoder and decoder ）.

Variation factors ： The goal of designing features or designing algorithms for learning features , Privacy refers to different sources of influence , These factors are usually not directly observable , But it affects the observable quantity （ namely target or label）.

Indicates the disadvantage of learning ：

• Multiple variation factors simultaneously affect each data observed .
• Extract high-level data from raw data 、 Abstract features are difficult .

Deep learning Express complex representations through other simpler representations , It solves the core problem of expression learning .

The following figure shows how the deep learning system combines simpler concepts （ Such as corners and contours ） To represent the concept of objects in the image .

There are two ways to measure the depth of the model ：

• Based on the number of sequential instructions to be executed by the evaluation architecture . The longest path of calculating the corresponding flow chart can be regarded as the depth of the model .
• Consider the depth of the graph that describes how concepts relate to each other as the depth of the model . For example, the face layer and eye layer in the face image .

Summary of deep learning ： One of the ways to artificial intelligence , It is a kind of machine learning , Can make the computer system from Experience and data Technology improved in .

Different AI The relationship between disciplines ：

1. This book is for the readers

Applicable people ： Students and researchers who study machine learning , And engineers who have no relevant background but want to quickly master applicable skills .
The framework of this book is organized ： Basic mathematical tools and machine learning concepts 、 Mature deep learning algorithm 、 Future outlook .

Flow chart of high-level organization structure of books ：

2. The historical trend of deep learning

The reasons for the development of deep learning ：

• The continuous increase of training data
• The computer software and hardware foundation for deep learning has been improved

2.1 Many names and fate changes of Neural Networks

3 Second development wave ：

1. 20 century 40 years ——60 years , The prototype appears in cybernetics （ The first wave of neural network research ） in , Adaptive linear units appear ADALINE, Use Random gradient descent method .
2. 20 century 80 years ——90 years , Expressed as Connectionism . Related concepts that appear include Distributed representation , Back propagation algorithm .
3. 2006 year , Revive in the name of deep learning . The research direction begins to focus on Unsupervised learning and Small sample learning .

The reasons for the first two recessions ：

1. Limitations of linear models , Cannot learn XOR function .
2. The entrepreneurial formula is ambitious but unrealistic , At the same time, the kernel method and graph model have achieved good results .

2.2 The increasing amount of data

The figure above shows that the size of the dataset increases significantly over time .

reason ： Use computers frequently ——> Record data increases ——> Machine interconnection makes it easy to manage data ——> The era of big data is coming

2.3 Increasing model size

Faster CPU、 Universal GPU、 Faster network connectivity and better software infrastructure for Distributed Computing , It is the historical trend of deep learning .

2.4 Increasing precision 、 Complexity and impact on the real world

ImageNet The error rate of large-scale visual recognition challenge competition changes .

Improvement of scale and accuracy of deep learning stochastic networks , It can solve increasingly complex tasks , This trend of increasing complexity pushes logical conclusions —— Neural Turing machine , It can learn to read and write any content in the storage unit and item storage unit .

Another achievement of deep learning is Reinforcement learning . Without the guidance of human operators , Tasks performed through trial and error learning .

Read the summary

The first chapter summarizes the history of deep learning, the development and reasons of deep learning , The three waves of deep learning are introduced in detail , And the future development of deep learning is prospected .
In my case , Because it has the theoretical basis related to deep learning , So it's not difficult to understand , But the biological basis mentioned in the first chapter , Knowledge related to brain neuroscience , It gave me a lot of inspiration , Neural network itself is the simplest and most basic simulation of the stacking of neurons in the human brain , But it does not involve the relevant scientific basis , Because really in-depth study , Between neurons “ Activation function ” More complex , So if one day , Human beings can penetrate the secrets between neurons in the human brain , So real AI It's not far .