[multimodal] transferrec: learning transferable recommendation from texture of modality feedback arXiv '22

《TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback》 Arxiv’22

NLP and CV Domain pre training and large models have become very popular , Come forth BERT、GPT-3、ViT etc. , Realized one4all normal form , That is, a general large model serves almost all downstream tasks . But the development of recommendation system in this direction is slow , The mobility range of the model is limited , It is usually only applicable to the business scenario within a company , It is impossible to achieve portability and versatility in a broad sense .

The article first points out ： This is mainly because RS Over reliance on users ID With items ID Information , be based on ID Collaborative filtering paradigm makes RS Break away from complex content modeling , also DL+GCN It makes CF The performance of has experienced a period of improvement , Has dominated the field of recommendation systems . But based on ID Of RS There has been a serious bottleneck in the performance of , Approaching the ceiling , and ID Its unshareable nature leads to almost no migration .

Therefore, it is proposed that ID Back to content-based recommendations , Realization General recommendation system for large-scale mixed mode .

Mixed mode scene

The implementation of general recommendation is based on a common recommendation scenario , That is, the interaction behavior of users' items is determined by ** Mixed mode （MoM: Mixture-of-modality）** Composition of items , The object of user interaction can be text （text） form , Vision （vision）（ Images / Video etc. ） form , Or both modes exist . This paper begins with MoM Of source domain Next pre training model , This allows you to migrate to any domain Downstream tasks of .

The dataset is QQ News recommendation scenario of browser ,7 A record of days .

TransRec

Item Encoder

First item encoder It's pre training BERT and ResNet-18, That is, the yellow and green color block in the above figure .

For the text item i, take word token Sequence $\mathbf{t}=\left[t_1,t_2,\dots ,t_k\right]$ Input BERT, And then pass by self- attention pooling Get the text item The final characterization of ：
$$Z_{i,t}=\mathrm{SelfAtt}(\mathrm{BERT}(\mathbf{t}))$$
For the picture item i, take ResNet Output. feature map Have a time MLP, Get pictures item The final characterization of ：
$${\mathbf{Z}}_{\mathbf{i},\mathbf{v}}=\mathrm{MLP}(\mathrm{ResNet}(\mathbf{v}))\text{. }$$

User Encoder

The user is represented by his item interaction sequence , therefore User Encoder The input of is user interaction item Of embedding, And then use BERT（ Write it down as $BER{T}_u$） Get the representation of user interaction sequence , As a user embedding, And then item embedding Calculate similarity , there BERT Is one-way , Use the last item The output of as a representation of the sequence ：
$$\begin{array}{rl}& {\mathbf{S}}^u={\mathbf{Z}}^u+{\mathbf{P}}^u\\ & {\mathbf{U}}^u=E_u\left({\mathbf{S}}^u\right)={\mathrm{BERT}}_{\mathrm{u}}\left({\mathbf{S}}^u\right)\end{array}$$

Training methods

Two stage pre training

User Encoder Preliminary training

Conduct self-monitoring on user encoder pretraining . say concretely , The generation pre training from left to right is used to predict the next in the user interaction sequence item, That is, pre training one-way BERT Use softmax Cross entropy loss as objective function
$$\begin{array}{rl}& {\tilde{\mathbf{y}}}_t=\mathrm{Softmax}\left(\mathrm{RELU}\left({\mathbf{S}}^{\prime }{}_t{\mathbf{W}}^{\mathbf{U}}+{\mathbf{b}}^{\mathbf{U}}\right)\right)\\ & {\mathcal{L}}_{\text{UEP }}=-\sum _{u\in U}\sum _{t\in [1,\dots ,n]}\left({\mathbf{y}}_{\mathbf{t}}\log \left({\tilde{\mathbf{y}}}_{\mathbf{t}}\right)\right)\end{array}$$
there $W^U,b^U$ Then discarded , Because it uses inner product as similarity matching .$S^{\prime }{}_t$ It is the representation of sequence .

End-to-End Twin tower training

Training at the same time item encoder and user encoder, The purpose is similar to that of previous hybrid experts , In order to make the characteristics of text and pictures encoder Adapt to the current situation as soon as possible domain. utilize Contrastive Predictive Coding (CPC) Method training , The idea is shown in the figure above , Divide a sequence into two segments , According to the prediction of the previous paragraph, all in the next paragraph item, Therefore, it is the same as the task scenario of collaborative filtering ：
$${\mathcal{L}}_{\mathbf{CPC}}=-\sum _{u\in U}\left[\sum _{t=n+1}^{n+l}\log \left(\sigma \left({\mathbf{r}}_{\mathbf{u},\mathbf{t}}\right)\right)+\sum _{g=1}^j\log \left(1-\sigma \left({\mathbf{r}}_{\mathbf{u},\mathbf{g}}\right)\right)\right]$$
$g$ Is a random negative sample .

experiment

be based on ID The method of domain The effect is not as good as the method based on modal content , Pre training is also better than direct training .

The paper also verifies the upper limit of the data to the model , More pre training data for TransRec The greater the performance improvement of , There is an endless stream of data in industry that can be expanded .