Multi objective Optimization Practice Based on esmm model -- shopping mall

19 year 3 month , We began to explore multi-objective optimization .19 In, it was practiced and put into practice in the scene of mushroom Street mall ,2020 In, the live slicing scene was tried again on the home page of mushroom street . Practice has proved , Multi objective optimization can indeed significantly improve GMV And other core business indicators . This essay will elaborate on what we did last year ESMM The practical details of the model applied to the scene of mushroom Street Mall , Including offline model structure optimization and online sorting formula optimization . The recent multi-objective practice in the new scenario will be seen in the next article .

This article is the core content of a recently accepted patent .

Originality is not easy. , If it helps you , Please use your finger to like or collect , Encourage me to continue to create ！

Directory as follows （ We strongly recommend that you finish reading , The later , The more dry goods ）：

1. The relationship between click through rate and conversion rate —— Four quadrant analysis
2. Three strategies for solving multi-objective problems
3. Sharing mechanism of multi task learning
4. ESMM Model review
5. Mushroom Street Mall is based on ESMM Multi objective practice of
6. summary

1、 The relationship between click through rate and conversion rate —— Four quadrant analysis

Designing and deploying a real large-scale personalized search recommendation system will encounter many challenges . for instance , In the mushroom Street Mall category and search scene , There may be conflict between click and transaction target . Product click through rate ctr（ Click on ／ Exposure ） And conversion rate cvr（ Clinch a deal ／ Click on ） The relationship is not exactly proportional . Imagine a scene , The main picture cover of a product displayed to users is not attractive enough , The probability of being clicked by the user is low , But the quality of the product ／ evaluation ／ Historical sales and other indicators performed very well , Once the user clicks and enters the product details page 、 After observing a series of properties of the commodity , This kind of commodity has a high probability of being converted by users . In the sorting system with click through rate as the main goal , It is difficult for such high-quality products with low click through rate to get the top ranking position , Therefore, competitive exposure cannot be obtained . This is a completely underestimated commodity , The inside is often much higher than the outside .

So here comes the question , If you want to improve the ranking of these high conversion Commodities , We can introduce the transformation factor in the off-line model training stage or the online sorting formula fusion stage . How to introduce transformation factors into model training ？ You can think of the following options ：

• Mining statistical class features related to transformation , But this kind of feature belongs to the generalized feature , Naturally, it can not play a personalized recommendation effect for thousands of people , And the impact on the model is limited .
• stay ctr Real time dynamic changes are added to the prediction model id Class transaction sequence ／ Additional purchase sequence ／ Collection sequence, etc , Although this kind of sequence can also play a certain role in the transformation target , However, the relationship between click through rate and conversion rate cannot be completely balanced , The target of the model is still the click through rate .
• adopt gmv reweight Multi objective optimization , Although this model is simple , Low online cost , But it mainly depends on manual experience ,reweight Manual adjustment of formulas requires strong business background and more experience , And in essence, it can not reach the optimal . About reweight Practical details of , You can refer to my previous article , Very detailed ：

The above measures can solve the symptoms but not the root causes .

I analyzed the distribution of commodity attributes of mushroom Street mall , And draw a picture of the click through rate ctr And conversion rate cvr The four quadrant relationship between , As shown in the figure below . As you can see from the diagram , The first quadrant is the best 、ctr and cvr Are higher commodities , The price of this kind of goods is generally low . The second quadrant is ctr low , however cvr A high class commodity , Like the number and sales are high , The price is moderate , This kind of commodity is a high potential commodity . The third quadrant is ctr and cvr Goods below the average , Their prices are high , Low number of likes and sales , We hope to eliminate this batch of goods . The fourth quadrant is ctr high , however cvr A low class commodity , This batch of goods is flashy , They often have beautifully made cover pictures and eye-catching titles , But the sales volume and evaluation are few , Or more negative comments . However, it does not rule out that some of the commodities in the three and four quadrants belong to high potential commodities newly launched on the shelves , After the cold start phase and sufficient data have been accumulated , May be able to get a better deal .

2、 Three strategies for solving multi-objective problems

There are generally three strategies to solve multi-objective problems in the industry , Multi model fractional fusion 、 Sequence learning （Learning To Rank,LTR）、 Multi task learning （Multi-Task Learning,MTL）.

Our implementation strategy in this article , It combines two schemes of multi model fusion and multi task learning .

The so-called multi model , It is to build training samples and features for each target individually , And select appropriate models to train respectively , Each model calculates a score . Its advantage is that different targets can be individually optimized in depth , And on the model inference, That is, online scoring and forecasting , According to the current main business objectives , Flexibly control the weight of each goal . For example, the company's goal is to improve gmv indicators , Then we can appropriately improve the cvr The weight of . However , The disadvantages of this scheme are also obvious , We need to invest multiple human and machine resources to train and store the model , There is no unified and appropriate standard paradigm for online formula adjustment , It can only be determined through continuous practice . If our scenario needs to optimize two objectives , Then we need two models , An optimization ctr, An optimization cvr.

In recent years, academia and industry have paid more attention to multi task learning （Multi-task Learning, MTL） There are a lot of studies , It is an excellent multi-objective sorting solution . Multi-objective learning is a kind of transfer learning , By sharing certain parameters , Learn the parameters of different goals , And then combine . Typical algorithms are Google's MMOE（Multi-gate Mixture-of-Experts） And Ali's ESMM（Entire Space Multi-Task Model）. Multi task learning is a branch of transfer learning , To be more exact , It is a special solution in transfer learning . When the task is highly relevant , For example, the click through rate and conversion rate in some scenarios , At this time, multi task learning has obvious advantages . Multi task learning, learning several tasks at the same time , Do not distinguish between source and target , The learner of multi task learning does not know the target task at first , It receives information about multiple tasks at once , Treat all tasks equally .

3、 Sharing mechanism of multi task learning

Multi task learning （Multitask learning） Is based on shared representation （shared representation）, A machine learning method that puts several related tasks together to learn .

Two key points of multi task learning ：1、 Multiple tasks must have dependencies ;2、 Multiple tasks have underlying representations that can be shared .

The core challenge of multitasking is how to design the sharing mechanism between tasks . It is not too much to say that shared design is the soul of multitasking , Yes, of course , Long period of division , A long time must be divided. , Sharing evolves later , It needs to be separated , Since then, a series of better soft sharing multi-objective structures have been born . It is difficult to introduce shared information into traditional machine learning algorithms ; But in the neural network model , Model sharing is relatively simple and convenient . The multi task sharing mechanism is very flexible , There are many shared models . The following summary about sharing mode is taken from qiuxipeng's 《 Neural networks and deep learning 》, Very comprehensive ：

1） Hard sharing mode of parameters ： The neural network models of different tasks use the underlying shared modules （Shared Layers） To extract some general features , Then set up high-level private modules for each different task （Task-specific layers）.

2） Parameter soft sharing mode ： No explicit shared modules , But each task can steal information from other tasks , To improve your ability . The method of stealing includes directly copying and using the implicit state of other tasks , Or use the attention mechanism to actively select useful information . Like Google's MMOE, It is a typical soft sharing mode .

3） Hierarchical sharing mode ： The feature types extracted from different layers of neural network are different , Especially for image tasks . The bottom layer generally extracts some low-level local features , High level extraction of some high-level abstract semantic features . therefore , If multi task learning , Different tasks can also be classified into advanced levels , A reasonable sharing mode is to let low-level tasks output at the bottom , High level tasks output at high level .

4） share - Private model ： The division of labor in this mode is more clear , Module and task specific will be shared （ private ） The responsibilities of the modules are separated , The sharing module captures some shared features across tasks , Private modules only capture features related to specific tasks . The final representation consists of shared features and private features .

4、ESMM Model review

Let's briefly review ESMM Structure of model .

Take the recommendation system for example , User behavior follows a serialization pattern , That is exposure -> Click on -> conversion . therefore ,CVR The model predicts the conversion probability after clicking , namely

. differ ctr Estimate the problem ,cvr Estimates face two important issues ：1）sample selection bias (SSB) problem： Tradition CVR The model usually takes the click data as the training set , Click and convert to positive sample , Click not converted to negative sample , But we are online inference When predicting , But the whole exposure sample space .SSB The problem will reduce the generalization performance of the model .2）data sparsity (DS) problem：cvr The click training sample size used by the model is much smaller than ctr The exposure training sample size used by the task .

Some strategies can alleviate these two problems , For example, from the exposure set, the samples that are not clicked are sampled for negative case mitigation SSB（ Cause to be right cvr An underestimation of the estimated probability ）, Oversampling of transformed samples DS（ Sensitive to sampling rate ） etc. . These methods cannot solve the problem in essence .

CVR The essence of prediction model , It's not prediction “ The item is clicked , Then transformed ” Probability （CTCVR）, It is “ Suppose the item is clicked , Then it is transformed ” Probability （CVR）. This is that you can't train directly with all the samples CVR Why the model , Because we don't know this information ： Those products that have not been clicked , Suppose they are clicked by the user , Whether they will be transformed . If used directly 0 As their label, Will be largely misleading CVR Model learning .

Proposed by the author ESMM Can be solved simultaneously in essence SSB and DS The problem of .ESMM Two auxiliary task： Estimate the click probability after exposure and the click and conversion probability after exposure . stay ESMM in , All of them are estimated in the full sample space , So I got

It also belongs to the full sample space , This eliminates SSB problem . meanwhile ,CVR And CTR The parameters expressed by the underlying features of the network are shared , obviously CTR Training can use more samples , So as to assist CVR Learning from , send CVR Tasks can implicitly learn from a large number of exposure samples that are not clicked , This parameter sharing mechanism similar to transfer learning can greatly alleviate DS problem .

In this formula ,

The probability of clicking and converting after exposure ,

Is the click probability after exposure ,

Is the condition that the current product is clicked by the user , The probability of being transformed .CTR Corresponding label yes click,CTCVR Corresponding label yes conversion & click. among ,click It means to click ,conversion It means transformation . These two tasks can use global exposure samples . In the thesis, through studying these two tasks , Then according to the formula , Learn implicitly CVR Mission .

Implicit learning pCVR refer to ,pCVR Just a variable in the network structure , We don't take it as our goal , And will not pCVR Add to the objective function .