Reading papers on false news detection (4): a novel self-learning semi supervised deep learning network to detect fake news on

Paper title ：A novel self-learning semi-supervised deep learning network to detect fake news on social media
date ：2021
# Based on news text 、# Semi supervision 、# Self learning 、# Pseudo label

One 、 The basic content

The model trained with labeled data is unlabeled data Fake labels , The innovation of this work is to use a confidence function The way is to evaluate the false label , Select high-quality pseudo tags Put the sample into the labeled data , This improves the quality of pseudo tags , So as to better carry out semi supervised false news detection .

Two 、 The motivation of the article

（1） In the actual , Annotated datasets are difficult to obtain （ Because fake news spreads through websites ）;
（2） Supervised learning model cannot realize self-study , Because it ignores the correlation between real and false data .

3、 ... and 、 Main work

This paper studies a self-learning semi supervised deep learning network , Added one Confidence network layer , It can automatically return and add the correct results , Help neural networks accumulate positive sample cases , So as to improve the accuracy of neural network .
The network trains both supervised and unsupervised tasks to detect fake news , Specifically ：
（1） Design a semi supervised deep learning network , Use the modified deep learning machine to train both supervised and unsupervised tasks ;
（2） Very accurate unlabeled data can be automatically added to the training set , And gradually expand the training set in the multiple iterative training process to realize self-learning （self-learning）.

Four 、 Model framework

• Data processing stage ： Data cleaning 、 Divide the data into marked data and unmarked data .

• Semi supervised self-learning stage ：
  The model uses an improved deep learning machine $L$ Training at the same time Supervised and Unsupervised Mission . There are supervision tasks Only a small part of the marked data is required in , and Unsupervised task Predict the remaining unlabeled data , And return unmarked data Highly reliable fake tags To enrich the marked data set , So as to achieve the effect of self-learning .

1、 Model training process

$D_l$—— Examples of tags in the training dataset , The size is $|L|$,$D_l^0=(X1,y1),(X2,y2),\dots ,(Xl,yl)$;
$D_u$—— Test unmarked examples in the training set , The size is $|U|$,$D_u=Xl+1,Xl+2,\dots ,Xl+u$.
Workflow ：
（1） initialization ： In the supervised deep learning module , Use $D_l^0$ Train deep learning machines as training sets $L$. then , In the unsupervised deep learning module ,$D_u^{{}^{\prime }}=(X_{l+1},{\hat{y}}_{l+1}),(X_{l+2},{\hat{y}}_{l+2}),\dots ,(X_{l+u},{\hat{y}}_{l+u})$ The pseudo tags are trained by the deep learning machine L And their confidence values σ Generate , If ${\sigma }_0$ It's filtering $D_u^{{}^{\prime }}$ The threshold value of the false label of self-confidence , be $D_u^{{}^{\prime }}$ The set of confidence pseudo tags can be expressed as $D_{pseu}^0=((X_{l+i},{\hat{y}}_{l+i}),(X_{l+i+1},{\hat{y}}_{l+i+1}),\dots ,(X_{l+p+i},{\hat{y}}_{l+p+2}))$, The size is $|P_0|$.
（2） repeat ： New training set $D_l^1=|D_l^0\cup D_{pseu}^0|=(X_1,y_1),(X_1,y_1),\dots ,(X_l,y_l),\dots ,(X_{l+p},y_{l+p})$ For retraining deep learning machines L To generate a new set of confidence tags $D_{pseu}^2$, The size is $|P_1|$ And a new training set $D_l^2=|D_l^1\cup D_{pseu}^1|$. Repeat this step , until $D_{pseu}^t=D_{pseu}^{t+1}$.

2、 Deep learning machine $L$ Basic framework

$L$ Through existing neural networks （ for example RNN、CNN、LSTM and Bi-LSTM） Add confidence layer to build .
（1）Embedding layer
（2）Dropout layer（ Set to 0.5）
（3）Bi-LSTM layer
（4）Softmax layer
（5）Confidence-function layer：
It is believed that the network layer will automatically return and add correction results , So as to help the neural network accumulate positive sample cases .
This layer is used to calculate $D_u$ Confidence value of each element in $\sigma$, And in $D_u^{{}^{\prime }}$ Generate pseudo tags in . For each input $X_i$,${\sigma }_{(}{X}_i)=max(0,p(y=j|X_i))$;
hypothesis ${\sigma }_0$ It's filtering $D_u^{{}^{\prime }}$ The threshold value of the false label of self-confidence , be $D_u^{{}^{\prime }}$ The elements in $X_i$ False label of confidence , If ${\sigma }_{(X_i)}>{\sigma }_0$, be $\hat{y}={\{}_{0,otherwise}^{1,ifj=argmaxp(y=j|X_i)}$.
And then we get $D_u^{{}^{\prime }}$ The entire set of confidence pseudo tags $D_{pseu}^0$, Its size is $|P_0|$.
The final new training set $D_l^1=|D_l^0\cup D_{p}se{u}^0|=(X_1,y_1),(X_1,y_1),\dots ,(X_l,y_l),\dots ,(X_{l+p},y_{l+p})$ For retraining deep learning machines L To generate a new set of confidence pseudo tags $D_{pseu}^2$, The size is $|P_1|$ And a new training set $D_l^2=|D_l^1\cup D_{pseu}^1|$. Repeat this step , until $D_{pseu}^t=D_{pseu}^{t+1}$.

5、 ... and 、 Data sets

FakeNewsNet In the repository PolitiFact and GossipCop Data sets , Each data set contains news content 、 Social background and space-time information .

6、 ... and 、 experimental result