Deep confidence network (DBN) [the classical DBN network structure is a deep neural network composed of several layers of RBM (restricted Boltzmann machine) and one layer of BP]

DBN (deep belief network, Deep confidence network ), It's using RBM(Restricted Boltzmann Machines, Restricted Boltzmann machine ) A deep neural network .

One 、RBM brief introduction

RBM It belongs to an unsupervised learning method , The purpose of unsupervised learning is to fit the training data as much as possible .

The following figure for RBM Structure , Among them, the lower neurons form a display layer (visible layer), By Xianyuan (visible units) form , Used to enter data ; The upper neurons form the hidden layer (Hidden layer), By Yin yuan (hidden units) form , For feature extraction .

Two 、 Training DBN

Training DBN The process is carried out layer by layer . In every layer , Using data vectors to infer hidden layers , Take this hidden layer as the next layer ( A higher level ) The data vector of .

classical DBN The network structure consists of several layers RBM And the first floor BP A deep neural network , The structure is shown in the following figure .

DBN The process of training the model is mainly divided into two steps :

• The first 1 Step ： Train each layer in order RBM The Internet , Ensure that when the feature vector is mapped to different feature spaces , Retain as much feature information as possible ;

• The first 2 Step ： stay DBN The last layer is set BP The Internet , At the same time, the last RBM The output eigenvector of is taken as BP The input eigenvector of the network , Supervised training of entity relationship classifiers . Then the back propagation network propagates the error information from top to bottom to each layer RBM, Fine tune the whole DBN The Internet .

In the training model , The first 1 This step is called pre training , The first 2 This step is called fine tuning . Supervised learning is not necessarily BP The Internet , It can be replaced with any classifier model as needed .

DBN The essence of the algorithm

In terms of its unsupervised learning , The purpose is to retain the characteristics of the original features as much as possible , At the same time, reduce the dimension of features ;

In terms of supervised learning , The purpose is to make the classification error rate as small as possible .

Whether supervised learning or unsupervised learning ,DBN The essence of algorithms is Feature Learning The process of , That is, how to get better feature expression .

---

---

---

Reference material ：
Deep confidence network （DBN）