Deep learning and neural networks: the six most noteworthy trends

The basic idea of neural network is to simulate computer “ The brain ” Multiple interconnected cells in , So it can learn from the environment , Identify different patterns , And then make decisions similar to human beings . A typical neural network is composed of thousands of interconnected artificial neurons , Neuron is the basic unit of neural network .

The basic idea of neural network is to simulate computer “ The brain ” Multiple interconnected cells in , So it can learn from the environment , Identify different patterns , And then make decisions similar to human beings .

A typical neural network is composed of thousands of interconnected artificial neurons , Neuron is the basic unit of neural network . These neurons are stacked together in order , Millions of connections are formed in the form of layers . The units are divided as follows :

○ input unit ： Used to receive information about the external environment ;

○ Hidden units ： The hidden layer passes the required calculation and output results to the output layer ;

○ output unit ： The output signal shows how the network responds to the recently acquired information .

Most neural networks are “ All of the connection ”, in other words , Each hidden unit and output unit are connected to all units on the other side . The connection between each unit is called “ The weight ”, The weight can be positive or negative , It depends on how much it affects the other unit . The greater the weight , The greater the impact on relevant units .

Feedforward neural network is one of the simplest neural networks , The neurons are layered . Each neuron is connected only to the neurons of the previous layer . Receive the output of the previous layer , And output to the next layer , There is no feedback between the layers . Is currently the most widely used 、 One of the fastest growing artificial neural networks .

The following will discuss several important trends in the development of neural networks and deep learning ：

Capsule network （Capsule Networks）

Capsule network is a new kind of deep neural network , It processes information in a way similar to the human brain .

Capsule network is opposite to convolution neural network , Although convolutional neural network is one of the most widely used neural networks so far , But it fails to consider the key spatial hierarchy between simple objects and complex objects . This leads to misclassification and a higher error rate .

When dealing with simple identification tasks , Capsule network has higher accuracy , Fewer errors , And do not need a large number of training model data .

Deep reinforcement learning （Deep Reinforcement Learning, DRL）

Deep reinforcement learning is a form of neural network , Its way of learning is through observation 、 Actions and rewards , Interact with your surroundings . Deep reinforcement learning has been successfully used in game strategy making , Such as Atari and Go.AlphaGo Defeated the human champion chess player , It is the most famous application of deep reinforcement learning .

Data to enhance （Lean and augmented data learning）

up to now , The biggest challenge for machine learning and deep learning is ： A lot of tagged data is needed to train the system . There are two widely used techniques that can help solve this problem ：

（1） Synthesize new data

（2） The migration study

“ The migration study ”, That is to transfer the experience learned from one task or field to another task or field ,“ A study ” It refers to the application of transfer learning to extreme situations , There is only one relevant example , Even without examples . So they become “ Streamline data ” Learning skills . Similar to , When using simulation or interpolation to synthesize new data , It helps to get more training data , Thus, existing data can be enhanced to improve learning .

By using the above techniques , We can solve more problems , Especially when there are few historical data .

Monitoring model （Supervised Model）

A form of learning when monitoring models , It learns or builds a pattern based on pre tagged training data , And infer new instances according to this pattern . The supervised model uses a supervised learning algorithm , The algorithm consists of a set of inputs and labeled correct outputs .

Compare the input of the tag with the output of the tag . Given the change between the two , Calculate an error value , Then an algorithm is used to learn the mapping between input and output .

Network memory model （Networks With Memory Model）

A typical difference between humans and machines is the ability to work and think critically . We can program the computer , Make it complete specific tasks with high accuracy . But if we want it to work in a different environment , There are still many problems to be solved .

To adapt the machine to the real world environment , The neural network must be able to learn continuous tasks without generating “ Catastrophic forgetfulness （catastrophic forgetting）”, This requires the help of many methods , Such as ：

○ Long term memory network （Long-Term Memory Networks）： It can process and predict time series

○ Elastic weight consolidation algorithm （Elastic Weight Consolidation, EWC）： This method can selectively slow down the learning rate of weights that are important to these tasks

○ Progressive neural networks （Progressive Neural Networks）： Will not produce “ Catastrophic forgetfulness ”, It can extract useful features from learned networks , For new tasks

Blended learning model （Hybrid Learning Models）

Different types of deep neural networks , For example, generative countermeasure network （GANs） And deep reinforcement learning （DRL）, It shows great potential in performance improvement and wide application . however , Deep learning models cannot model uncertain data scenarios like Bayesian probability .

The blended learning model combines the advantages of the two approaches , Typical hybrid learning models include Bayesian generation of countermeasure networks （Bayesian GANs） And Bayesian condition generation countermeasure network （Bayesian Conditional GANs）.

The blended learning model expands the scope of business problems , So that it can solve the deep learning problem with uncertainty , So as to improve the performance of the model , Enhance the interpretability of the model , To achieve wider application .