Recommendation System - Sorting Layer - Model (1): Embedding + MLP (Multilayer Perceptron) Model [Deep Crossing Model: Classic Embedding + MLP Model Structure]

First, the structure of Embedding+MLP model

MLP (Multilayer Perceptron) is the abbreviation of Multilayer Perceptron.A perceptron is another name for a neuron, so a multilayer perceptron is a multilayer neural network.

Second, get user characteristics, item characteristics

• Item features can be obtained from raw data;
• User features may require secondary analysis and processing to obtain;

Three, Deep Crossing model

The deep learning Deep crossing model proposed by Microsoft in 2016 is this structure-Deep Crossing can be divided into 5 layers from bottom to top, namely Feature layer, Embedding layer, Stacking layer, MLP layer and Scoring layer (as shown below).).Used for ad recommendations.

• The Feature layer is also called the feature input layer (the bottom layer), as the input of the entire model.Feature has two cases. The first is to connect the Embedding layer, and the second is to directly connect the Stacking layer. The main difference is: Feature #1 represents the feature vector generated by one-hot encoding of categorical features, while Feature #2Represents a numerical feature.Due to the sparsity, the one-hot feature cannot be directly input into the subsequent neural network for training, so it needs to be converted into a denser Embedding vector by connecting to the Embedding layer.
• The Embedding layer is set to convert the sparse One-hot vector into a dense Embedding vector. It should be noted that the Embedding layers are not all connected, but each feature corresponds to an Embedding layer. Different Embedding layersnot interfere with each other.
• Stacking layer Chinese name The stacking layer is also called the connection (Concatenate) layer. Its function is to stitch together different Embedding features and numerical features to form a new feature vector containing all the features.
• The MLP layer is the multi-layer neural network layer we mentioned at the beginning. The Deep Crossing in the figure uses the Multiple Residual Units layer, also called the multi-layer residual network.Deep Crossing selects residual neurons for specific problems. In fact, there are many kinds such as Relu, tanh, and sigmoid.The feature of the MLP layer is that it is fully connected, and there are connections between neurons in different layers.
• Scoring layer is also called output layer.The ultimate goal of deep learning to predict is the probability of a classification.If the click-through rate estimation is a binary classification problem, then we can use logistic regression as the output layer neuron, and if it is a multi-classification problem like image classification, we often use a multi-classification model such as softmax in the output layer.

Summary: For category features, first use the Embedding layer to densify the features, then use the Stacking layer to connect other features, input the multi-layer structure of the MLP, and finally use the Scoring layer to predict the results.

---

---

---

References:
How recommender systems use TensorFlow to implement classic deep learning models (Embedding+MLP)
[Wang Zhe-Recommendation System] Model Chapter-(task4)Embedding+MLP Model