Product recommendation system based on deep learning (WEB)

Product recommendation system based on deep learning （ECRS_Web）

Project brief introduction

Technology stack

The technology used in the project is as follows ：
Language ：Python3Java
Web End ：Layui,Flask,Nginx,Gevent,Flask_Cache
model training ： PaddleRec , PaddlePaddle
Deep learning model :DSSM, DeepFM
Vector recall ：milvus
data storage ： Redis
Model reasoning ： PaddleServing
Module communication ：gRPC,protobuf

Quick start

Project deployment depends on

Python3、PaddlePaddle2.2.2、PaddleServing、milvus1.0、redis、nginx、Gevent

PaddlePaddle Install the reference ：PaddlePaddle Propeller installation

Milvus Please refer to ：Milvus install

Milvus Of python Drive and PaddleServing install

python3 -m pip install redis pymilvus==1.0.1 paddle_serving_app==0.3.1

gevent Module installation

pip install gevent

Flask_Cache Module installation

pip install Flask-Cache

Project start

When you install the above dependencies , Guarantee Redis and Milvus In operation

1. towards Redis Store user and product data in

python to_redis.py

2. towards Milvus Store product vector data in

python to_milvus.py

3. Start user services

python um.py

4. Start the content service

python cm.py

5. Start the recall service

python recall.py

6. Start the sorting service

python rank.py

7. start-up Web application

python controller.py

python controller2.py

You can use the system here , Access in browser http://localhost:5000 and http://localhost:5001 You can experience
If load balancing is required, you can run Nginx, In this project Nginx Please refer to Nginx To configure
If you have configured Nginx, Access... In the browser http://loclhost:5158 Use this system

Cross platform function description

A cross platform demo is built into this project Demo stay ECRS_jav_demo Under the folder , Run directly under this folder Client.java file , You can experience Java visit python The recommended service module , If you need to customize access to other services, please refer to Cross platform access

System architecture and recommendation process

System architecture

System workflow

Recommended function flow chart

(1) Sign in .
The default start page of the system , When all users access the system for the first time , Will access this page , At the same time, the page also bears the function of requesting traffic carrying .
(2) Customer service / Goods and services .
Split the experimental data set , One copy of user data and one copy of product data , The data is parsed and saved to a non relational database Redis in , Unique for external incoming users and products id As key To Redis Find the corresponding value And the result is returned to the corresponding module in the form of binary stream .
(3) Recall service .
Recall service has three main tasks , When there are new users in the system , The information of the user cannot be queried in the above user information database . The system can use the original user information training model to fit the new users , The new user will get the feature vector of the user through the user model ; In order to complete the recall task , All product information needs to be converted into product feature vector through product model and imported in advance Milvus in , Empathy , When a new product goes on the shelves , The above steps need to be performed in advance so that new products can be recommended in time ; Finally, the user vector and the product vector are divided into Milvus Do vector approximation search in , Returns only items id List of candidate sets of information .
(4) Sort Services .
Candidate set of products obtained in the recall stage id List to query the details of goods through the goods service , Then it is combined with the user vector as the input of the sorting service , The ranking model scores all the candidates Select the products to be recommended and sort them from high to low , Finally, return to the recommended list with detailed product information .
(5) Interface services .
Each function module in the background passes gRPC The framework is divided into micro service modules , Each service has its own communication port channel, The developer can provide the required information to the specified module channel Send request parameters , The module returns the result after receiving the parameter processing .

Result demonstration example （ Show only some function pages ）

You can also view the demo video of the system Bilibili The online demo

Models and data files

Limited by file size , The project has not uploaded models and data files . When training the model file of this project , We did the training under the two experimental conditions of binary classification and logistic regression , The final results of the model on the test set are as follows ：

Two classification

Model	Total	Correct	ACC	AUC
DSSM(user)	1373344	857725	0.62455	0.7312
DSSM(item)	1373344	857725	0.62455	0.7306
DeepFM	1373344	858895	0.6254	0.73428

Logical regression

Model	Total	Correct	ACC	MAE
DSSM(user)	1373344	797789	0.58090	2.57662
DSSM(item)	1373344	798387	0.58134	2.56567
DeepFM	1373344	765238	0.55720	2.43759

DSSM Model ACC The highest rise 7.5%,DeepFM Model ACC The highest rise 12.24%.

If you need to train your model file , You can refer to The official project or My project Train yourself to acquire models or complete a new recommendation system , Or send a private letter to me to obtain the missing model and corresponding data file of the project

Complete project

Please visit ECRS