Living to detect the Adaptive Normalized Representation Learning for GeneralizableFace Anti - Spoofing reading notes

论文链接：Adaptive Normalized Representation Learning for Generalizable Face Anti-Spoofing | Proceedings of the 29th ACM International Conference on Multimedia

预备知识

（1）Batch Normalization

定义：Suppose the size of the input features is ,包含 N 个样本,The number of channels per sample is C,高为 H,宽为 W.When calculating the mean and variance of it,将在 N、H、W 上操作,and keep the channel C 的维度.

适用范围：Often used in discriminative models,比如图片分类模型.因为 BN 注重对每个 batch 进行归一化,从而保证数据分布的一致性,而判别模型的结果正是取决于数据整体分布.但是 BN 对 batchsize 的大小比较敏感,由于每次计算均值和方差是在一个 batch 上,所以如果 batchsize 太小,则计算的均值、方差不足以代表整个数据分布.

BN 的公式如下：

（2）Instance Normalization

定义：Originally used for style transfer of images,在生成模型中,The mean and variance of each channel of the feature map will affect the style of the final generated image,So you can put the image in the normalized channel level,Then use the mean and standard deviation of the corresponding channel of the target style image“去归一化”,to get the style of the target image.IN within a single sample,不依赖于 batch 的大小.

适用范围：Often used in generative models,比如图片风格迁移.因为图片生成的结果主要依赖于某个图像实例,所以对整个 batch Normalization is not suitable for image stylization,在风格迁移中使用 IN 不仅可以加速模型收敛,并且可以保持每个图像实例之间的独立.

IN 的公式如下：

引言

Due to the emergence of various face representation attacks,The security of face recognition systems has become a key issue of public concern.为了解决这个问题,Researchers have proposed many FAS 方法,These methods originally utilized LBP (Local Binary Patterns,局部二值模式) 和 HOG (Histogram of Oriented Gradient,方向梯度直方图) and other handcrafted features to distinguish real and fake faces. 随后,Some ways to get started CNN powerful representation capability to detect face attacks.Although these methods in the same scenario has achieved remarkable performance,But when testing in unknown scenarios,Their performance drops significantly.The reason for the performance degradation is that these methods are only suitable for training data with biased features,while ignoring the domain gap between the source domain and the unknown target domain,This leads to poor model generalization ability.

为了解决这个问题,Some methods introduce domain generalization into FAS 任务中.具体来说,They map features from multiple source domains into a common feature space for generalized representation,This is a good way to transfer to the unknown target domain.However, these methods only focus on the characteristics of the final alignment,The process of feature extraction is ignored.如下图所示,In this paper, the author used on the same source domain BN 以及 IN 分别训练模型,and test on unknown target domain.author in target domain A 上进行测试 (如下图左所示),其中具有 BN The model is superior to have IN 的模型.然而,在目标域 B 上测试时,IN 优于 BN.This is because when the unknown target domain has little domain offset compared to the source domain,BN 保持高性能.由于 BN Vulnerable to domain information,So performance degrades when encountering large domain offsets.IN then the specific style information for each sample is eliminated by using statistics,Therefore, the performance is better in the scene of domain shift.

Since both normalization methods have their own limitations,Therefore, researchers have proposed some normalized combination methods to deal with more cases.但是,由于 FAS The sample in the task is in the scene、There are differences in lighting, etc.,Combining directly with shared parameters,可能会导致性能下降.为了解决上述问题,作者提出了一个新的框架,Adaptive Normalization Representation Learning(Adaptive Normalized Representation Learning,ANRL),Obtain feature representations in unknown domains by adaptively selecting features from different normalizations.具体来说,Use the adaptive feature normalization module (Adaptive Feature Normalization Module,AFNM) to get the difference between the samples.此外,为了帮助 AFNM learning sample factors,The authors also propose a double calibration constraint (Dual Calibrated Constraints,DCC),Contains inter-domain compatible loss and inter-class separable loss.The former loss is able to align feature distributions across different domains,The latter loss is then used to enlarge the distance between the real sample and the prosthesis.主要创新点如下：

（1）propose to adaptively select different normalized features to obtain generalizable discriminative representations.

（2）double calibration constraint,Including inter-domain compatible loss and inter-class separable loss,以指导 AFNM 实现更好的泛化.

方法论

整体框架

由于 BN Extracted features are inherently vulnerable to domain transfer,而 IN The extracted features may lose some discriminative information,Therefore, their generalization ability in unknown domains is limited..如下图所示,作者提出了 ANRL To obtain a generalization of the characteristics,It not only preserves discriminative information,And also eliminates domain changes.具体来说,Due to the small domain bias of some samples,while some samples have large domain bias,因此设计了 AFNM,adaptively BN 和 IN The features of are combined with the most appropriate sample factors.此外,作者提出了 DCC,It includes inter-domain compatible loss and inter-class separable loss.具体来说,The former is a two-way design,Not only can the inter-domain distance be reduced,And to spread samples of the same domain,Further promotes domain mixing,The latter uses a larger interval to separate true and false samples,Thus constructing a compact and discriminative normalized representation space.

AFNM

尽管 BN The discriminative information on the source domain is extracted for anti-spoofing,But when transferring to an unknown target domain with large variation,It may be poor performance.为了解决这个问题,IN proven to be an effective solution,Because of its ability to eliminate domain differences. 作者因此设计了 AFNM,以将 BN 和 IN The extracted features are combined with an adaptive balance factor customized for each sample.

The feature map of the input image is ,作者利用 BN 和 IN get the normalized representation separately,分别用  和  表示.Since the balance factor should fit each sample,Hence mining information from their corresponding features to generate balance factors (对输入特征图 X 进行 Conv+Pooling 操作).具体来说,First use global average pooling to get channel-wise statistics ,公式如下：

然后,Use a fully connected layer to obtain a compact feature representation,公式如下：

式中, 表示 ReLU 激活函数, 表示全连接层的权重.由于  和  focus on different information,The authors exploit cross-channel attention to adaptively select useful information, respectively,公式如下：

式中, 表示 sigmoid 激活函数,,  分别表示 BN 和 IN 层的权重.

随后,Introduce a normalized balance factor ,公式如下：

最终的特征图 Y 通过对来自 BN 和 IN The weighted summation of different feature maps of,公式如下：

式中,,.

DCC

为了指导 AFNM Estimates from better balance factor to fusion BN 和 IN 的特征,The author proposed the double constraints of calibration (DCC). 与 FAS The triplet loss commonly used in tasks differs only in building triples based on the class,DCC More comprehensive constraints on features from both domain and class perspectives.

Inter-domain compatibility loss

由于 BN Easily affected by domain differences,Shuffle the inter-domain distribution to close the gap between multi-source domains.为此,Designed for bidirectional inter-domain compatibility (Inter-Domain Compatible,IDC) 损失,to bring samples from different domains closer together,and spread out samples from the same domain. 具体来说,assume a local mini-batch 中有 𝐾 源域,将域 𝐾 The local centroid in is introduced as：

式中, 表示 𝐾 the centroid of the domain, 表示样本数量, 表示域 𝐾 中第 𝑖 个样本的提取特征.

此外,For a more accurate estimate of the centroid,Compute the global centroid over different batches using a momentum update mechanism,公式如下：

式中, 表示动量因子.

然后,for a specific domain 𝑘,Compute the in-domain distance between the sample and the associated global centroid,公式如下：

接着,计算域 𝑘 The sample and the distance between the global center of mass of the other domain,公式如下：

IDC 损失如下：

two-way design IDC The loss not only reduces the inter-domain distance,and also disperse samples within the domain,Further facilitates mixing of different domains.

Between the separation and loss

由于 IN may eliminate some pairs FAS useful information for the task,So it is necessary by widening the distance between the real sample and prosthesis sample to keep identifying information.因此,The author designed the class to be separable（Inter-Class Separable,ICS）loss to cluster samples of the same class together,并将不同类的样本分开.

Due to the variety of attacks,The distribution of prosthetic samples may be scattered,Forcing them together has negative consequences. However, the distribution of real samples is relatively stable,Suitable for increased compactness.So the inter-class distance is calculated from real samples,公式如下：

ICS 损失如下：

通过 ICS 损失,The model is not only forced to mine more discriminative features to distinguish between real and fake samples,But also narrowed the difference in real samples,Both are helpful for learning FAS discriminative representation of.

实验

在泛化 FAS The comparison results with other methods on the four test tasks are as follows：

结论

基于 DG 的 FAS methods perform better than traditional methods,This proves that the traditional method is only suitable for extracting the discriminative information of the source domain.The proposed method outperforms these based on four test settings. DG 的方法,这证明了 ANRL Can adaptively from the BN 和 IN select features in to obtain a domain-independent discriminative representation.