Data reverse attack under federated learning -- gradinversion

This time I'll introduce you to an attack , yes NVIDIA A job of , Recently was CVPR2021 Collected .

“See through Gradients: Image Batch Recovery via GradInversion”

The reason why we introduce this work , Because this attack restores other people's training data through gradient , The effect is also very good .

Previous attacks were mostly member inference attacks （membership inference）, We use differential privacy （DP,Differential Privacy） To protect data . The purpose of member inference attack is to infer whether a data is used for model training , But generally speaking, we assume that the attacker has a lot of data in his hand , Including part of the training data , It also includes some additional data . This assumption is relatively strong , In fact , An attacker may not get part of the training data at all .

There is still a lack of a strong attack , This time, we reverse the training data by gradient , It turned out pretty good , It's worth sharing ！

About federal learning

First of all, we need to introduce federal learning , As shown in the figure below ：

There will be many participants involved in the training process , Each participant has their own data , And train locally , Model parameters will be uploaded after local training , Aggregation of models by a central node pair , Then it is distributed to each participant to synchronize the model .

The advantage of federal learning is , Each node's data is kept local , Ensure data privacy , The access of heterogeneous data is realized （ That is, each participant solves the problem of data access by himself , Even if the data is heterogeneous, it doesn't affect the whole ）.

however , Participants still have to upload the model , Will this lead to data privacy leakage ？

Gradient based data restoration

First , Let's formalize the goal first ：
$$x^{\ast }=\arg \underset{\hat{x}}{\min }{\mathcal{L}}_{grad}(\hat{x};W,\Delta W)+{\mathcal{R}}_{aux}(\hat{x})$$
among $\hat{x}\in {\mathbb{R}}^{K\times C\times H\times W}$ ($K$ yes batch size, $C,H,W$ The number of channels 、 Height 、 Width ), In the formula $W$ It's the weight of the model ,$\Delta W$ It's the weight change of the aggregated model .

among ${\mathcal{L}}_{grad}$ Is the purpose of , Find some possible inputs , So that the weight of training with these inputs , As much as possible consistent with the aggregated weights .

The specific form is
$${\mathcal{L}}_{grad}(\hat{x};W,\Delta W)={\alpha }_G{\Sigma }_l||{\nabla }_{W^{(l)}}\mathcal{L}(\hat{x},\hat{y})-\Delta W^{(l)}|{|}_2$$
among $\Delta W^{(l)}={\nabla }_{W^{(l)}}\mathcal{L}(x^{\ast },y^{\ast })$, Represent the real training data lead to the second $l$ The change of layer weight

There is another term in the previous optimization formula , It's called auxiliary regular term （auxiliary regularization）, The specific form is
$${\mathcal{R}}_{aux}(x)={\mathcal{R}}_{fidelity}(x)+{\mathcal{R}}_{group}(x)$$
It's made up of two , The first drive $x$ Similar to real training samples , The second is consistency , We'll explain later .

Batch label recovery (Batch Label Restoration)

Consider categorizing tasks , Remember that the real data is $x^{\ast }=[x_1,x_2,...,x_K]$ , The corresponding label is $y^{\ast }=[y_1,y_2,...,y_K]$

The corresponding real gradient is
$${\nabla }_W\mathcal{L}(x^{\ast },y^{\ast })=\frac{1}{K}\sum _k{\nabla }_W\mathcal{L}(x_k,y_k)$$
The error function can be understood as cross entropy （Cross-Entropy） error

In the classification task , The last layer of a network is usually a fully connected linear layer , We record it as $W^{(FC)}\in {\mathbb{R}}^{M\times N}$

among $M$ Is the dimension of the input feature , $N$ Is the total number of target categories

For training samples $(x_k,y_k)$ for , Note that the increment of the linear layer is $\Delta W_{m,n,k}^{(FC)}={\nabla }_{w_{m,n}}\mathcal{L}(x_k,y_k)$

Apply the chain rule , You can get ：
$$\Delta W_{m,n,k}^{(FC)}={\nabla }_{z_{n,k}}\mathcal{L}(x_k,y_k)\times \frac{\partial z_{n,k}}{\partial w_{m,n}}$$
among $z_{n,k}$ Represents the input as $x_k$ The final process of softmax Layer of the first $n$ Outputs , The form of the gradient is
$${\nabla }_{z_{n,k}}\mathcal{L}(x_k,y_k)=p_{k,n}-y_{k,n}$$
That is, the probability of the corresponding category minus the tag value

be aware :
$$\frac{\partial z_{n,k}}{\partial w_{m,n}}=o_{m,k}$$
among $o_{m,k}$ It's the second part of the whole link layer $m$ Inputs .

There's a little bit of explanation here , because $W^{(FC)}\in {\mathbb{R}}^{M\times N}$ , The input is a $M$ Dimension vector $v\in {\mathbb{R}}^M$

For a particular category $n$ , The output of $z_n=v\cdot W[:,n]={\sum }_{i=1}^M{v}_i{w}_{i,n}$

that , Immediately $\frac{\partial z_{n,k}}{\partial w_{m,n}}=\frac{\partial {\sum }_{i=1}^M{v}_i{w}_{i,n}}{\partial w_{m,n}}=v_m$

$v_m$ That is to say $o_{m,k}$

Because of the input of the linear layer , It's usually through ReLU perhaps sigmoid Activation , So it's generally non negative ,

that , The increment of the parameters of the previous full connection layer $\Delta W_{m,n,k}^{(FC)}={\nabla }_{z_{n,k}}\mathcal{L}(x_k,y_k)\times \frac{\partial z_{n,k}}{\partial w_{m,n}}$ , Among them ${\nabla }_{z_{n,k}}\mathcal{L}(x_k,y_k)$ This part , If and only if $n=n_k^{\ast }$ ( That is, corresponding to the correct category ) This part is negative .

therefore , For input $x_k$ , We can identify the target category by the sign of the change amount mentioned above , remember
$$S_{n,k}=\sum _m\Delta W_{m,n,k}^{(FC)}=\sum _m{\nabla }_{z_{n,k}}\mathcal{L}(x_k,y_k)\times o_{m,k}$$
once $S_{n,k}<0$ , It means $x_k$ The category of is $n$

however , All of this is based on a single input $x_k$ Of , in the light of $K$ Inputs , We have
$$s_n=\frac{1}{K}\sum _k{S}_{n,k}=\sum _m(\frac{1}{K}\sum _k\Delta W_{m,n,k}^{(FC)})$$
This creates a problem ： The increment after averaging , There's a loss of information , How to infer categories ？

There is a discovery in this work , namely
$$|S_{n_k^{\ast },k}|\gg |S_{n\ne n_k^{\ast },k}|$$
It means , The significance of the label is still relatively high , We can still infer from its absolute value , also , After gradient aggregation of multiple samples , The negative part is still negative , Showing the information of the original tag .

To make the minus sign more robust , The article uses a column by column minimum , Instead of summing according to characteristic dimensions

namely
$$\hat{y}=\arg \text{sort}(\underset{m}{\min }{\nabla }_{W_{m,n}^{(FC)}}\mathcal{L}(x^{\ast },y^{\ast }))[:K]$$
Let's explain the above formula , First notice ${\nabla }_{W_{m,n}^{(FC)}}\mathcal{L}(x^{\ast },y^{\ast })\in {\mathbb{R}}^{M\times N}$ It's a $M\times N$ Matrix

$\min {\nabla }_{W_{m,n}^{(FC)}}\mathcal{L}(x^{\ast },y^{\ast })$ That is to find the smallest line of the matrix , have $N$ dimension , And then sort it from small to large （ The negative ones are all ahead ）

$\arg \text{sort}$ In fact, it returns the subscript after sorting , This corresponds to the category , Before going straight back $K$ Small value , Which corresponds to $K$ Categories of samples

Here's a hypothesis , That is, there is no duplicate category data in a batch , You need to pay attention to ！

Authenticity regularization （Fidelity/Realism Regularization）

Here is a reference DeepInversion In view of the picture natural optimization

Dreaming to distill: Data-free knowledge transfer via DeepInversion.

In this paper, a regularization term is added ${\mathcal{R}}_{fidelity}(\cdot )$, To drive generation $\hat{x}$ Keep it as real as possible , The specific form is ：
$${\mathcal{R}}_{fidelity}(\hat{x})={\alpha }_{tv}{\mathcal{R}}_{TV}(\hat{x})+{\alpha }_{l_2}{\mathcal{R}}_{l_2}(\hat{x})+{\alpha }_{BN}{\mathcal{R}}_{BN}(\hat{x})$$
among ${\mathcal{R}}_{TV}$ and ${\mathcal{R}}_{l_2}$ Penalize the variance of the image and $L2$ norm , Belongs to the standard image prior .

DeepInversion The key part of this is the use of BN To constrain... A priori
$${\mathcal{R}}_{BN}(\hat{x})=\sum _l||{\mu }_l(\hat{x})-B{N}_l(mean)|{|}_2+\sum _l||{\sigma }_l^2(\hat{x})-B{N}_l(variance)|{|}_2$$
among ${\mu }_l(x)$ and ${\sigma }_l^2(x)$ It's No $l$ Layer convolution , Estimation of the mean and variance of a batch of data

This regularization of authenticity can make the image more realistic

Group consistent regularization （Group Consistency Regularization）

In the process of training data recovery , There will be a challenge , That is, the determination of the actual position of the object , As shown in the figure below ：

In the experiment , The author uses different random seeds to restore the image , The result is different degrees of offset , But these samples are semantically consistent .

Based on this observation , A regularization method for group consistency is proposed , That is to say, different random seeds are used to generate , And then fuse these results .

The regularized form is ：
$${\mathcal{R}}_{group}(\hat{x},{\hat{x}}_{g\in G})={\alpha }_{group}||\hat{x}-\mathbb{E}({\hat{x}}_{g\in G})|{|}_2$$

among , We need to work out this expectation $\mathbb{E}({\hat{x}}_{g\in G})$, In fact, that is Average image

As shown in the figure above , First, average by pixels , Get an average image , Then all the images are aligned based on this average image , Then take the average again , Get the final aligned average image .

Final update details

An energy based model is used in this paper , suffer Langevin Inspired by the , The specific form is
$$\begin{gathered} {\Delta }_{{\hat{x}}^{(t)}}\leftarrow {\nabla }_{\hat{x}}({\mathcal{L}}_{grad}({\hat{x}}^{(t-1)},\nabla W)+{\mathcal{R}}_{aux}({\hat{x}}^{(t-1)})) \\ \eta \leftarrow \mathcal{N}(0,I) \\ {\hat{x}}^{(t)}\leftarrow {\hat{x}}^{(t-1)}+\lambda (t){\Delta }_{{\hat{x}}^{(t)}}+\lambda (t){\alpha }_n\eta \end{gathered}$$
among $\eta$ It's sampling noise , For searching ;$\lambda (t)$ It's the learning rate ;${\alpha }_n$ It's the zoom factor .

experimental analysis

First of all, let's look at the correct rate of label recovery

You can see , As the batch size increases , The accuracy will drop , The reason is actually the problem of repeating categories , But compared to iDLG Is much better .

iDLG: Improved deep leakage from gradients.

Then there is the Ablation Experiment of each error term

You can see , Add authenticity and group consistency , It does improve the quality of the picture , The gain of alignment also exists .

As shown in the figure above , Basically, the result of restoration is close to the original image

then , And current SOTA Compare the effect , Here is a part of the result graph

You can see , The effect is better than DeepInversion,Latent Projection It's much better to wait for work ！

after , We have to look at the batch size , That is to say BatchSize, The effect on the reduction effect

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-LlXEZOc0-1620807309508)(…/…/…/…/…/Application Support/typora-user-images/image-20210512160851739.png)]

You can see , As the batch size increases , The reduction effect will be worse , It's also common sense , Because the information loss caused by aggregation will increase .

Conclusion

This work is a milestone , In the context of federal learning , Achieved a powerful attack ！

It's going to be a great inspiration , For the follow-up defense work under the scenario of distributed training such as federated learning .

DP And so on ？ Whether changes in participants will have an impact ？ Does the local training increase the difficulty of recovery ？

There is still a lot of work we need to explore together .