【CVPR 2021】 Lifelong Person Re-Identification via Adaptive Knowledge Accumulation

Methods an overview

1, This paper puts forward a lifelong learning person re-ID Method , This method can continuously learn across multiple domains .
2, This paper puts forward a method for lifelong learning AKA frame , The framework contains a learnable knowledge map for updating previous knowledge , meanwhile , The framework transfers knowledge to improve generalization in the invisible domain .
3, The article is LReID Provides a baseline and evaluation strategy .

Contents summary

Title of thesis	abbreviation	meeting / Periodical	Year of publication	baseline	backbone	Data sets
Lifelong Person Re-Identification via Adaptive Knowledge Accumulation	-	CVPR	2021	-	ResNet-50	Market、SYSU、Duke、MSMT17、CUHK03

Online links ：https://openaccess.thecvf.com/content/CVPR2021/html/Pu_Lifelong_Person_Re-Identification_via_Adaptive_Knowledge_Accumulation_CVPR_2021_paper.html
Source link ： https://github.com/TPCD/LifelongReID

Work Overview

1,lifelong person re-identification (LReID), which enables to learn continuously across multiple domains and even generalise on new and unseen domains.
2,Following the cognitive pro- cesses in the human brain, we design an Adaptive Knowl- edge Accumulation (AKA) framework that is endowed with two crucial abilities: knowledge representation and knowl- edge operation.

Summary of results

1,Our method alleviates catastrophic for- getting on seen domains and demonstrates the ability to generalize to unseen domains.
2,we also provide a new and large-scale benchmark for LReID. Ex- tensive experiments demonstrate our method outperforms other competitors by a margin of 5.8% mAP in general- ising evaluation.

Methods,

Methods the framework

Figure 1: Pipeline of the proposed lifelong person re- identification task. The person identities among the in- volved domains are completely disjoint.

Concrete realization

1, This article adopts a method of learning by constantly adding new domains Lifelong learning methods , And suppose the number of fields in is T.
2, The data set corresponding to each domain contains two parts: training set and test set . And they are marked .
3,baseline The basic loss of is Cross entropy loss , As formula 1 Shown . Considering the knowledge preservation of the front and back domains , The article also designs a knowledge distillation loss , As formula 2 Shown . The total loss function is weighted by both , As formula 3 Shown .

4, The process of knowledge accumulation , Contains Knowledge expression and knowledge operation . Pictured 2 Shown . In which knowledge is expressed It consists of two figures , One is entity based similarity graph （ISG）, The other is cumulative knowledge graph （AKG）.
5, ISG By a mini-batch The full connection graph of the sample characteristics of , The weight of the edge is calculated as the formula 4 Shown . AKG The composition and ISG be similar , However, the data source and how to update the article are not clear , Its edge weight is calculated as the formula 5 Shown .

6, Knowledge operation is divided into two steps ： Knowledge transfer and knowledge accumulation . In knowledge transfer , Connect the above two figures , The weight of the street side is calculated as the formula 6 Shown . The resulting new graph is called a union graph , Its calculation formula is formula 7.

7, Convolution network by graph （GCN） Spread relevant knowledge （ The formula 8）

8, The accumulation of knowledge declines The original features are averaged with the knowledge obtained in the previous step , Get expressed F, Then introduce a plasticity goal （plasticity objective）, As formula 9 Shown .

9, To avoid forgetting problems , This paper also proposes a stability loss , As formula 10 Shown . The formula 9 and 10 Used to optimize AKG Parameters of .

10, The total loss function , The formula 11.

11, Propose a new baseline ,LReID-Seen and LReID-Unseen.

experimental result

Overall evaluation

1, As one can imagine , It is a very large training process . It belongs to supervised pedestrian re identification .
2, It is easy to think of two graphs in the process of knowledge accumulation , Full connection is a little violent , No need to consider the computational complexity and time cost
3, Some of the strategies in this article are inspired by the human learning process , This and hpla similar .
4, We need to study the expression of others' formulas .

Small sample learning and intelligent frontier （ below ↓ official account ） The background to reply “LLAKA", The paper electronic resources can be obtained .

Citation format

@inproceedings{DBLP:conf/cvpr/Pu0LBL21,
author = {Nan Pu and
Wei Chen and
Yu Liu and
Erwin M. Bakker and
Michael S. Lew},
title = {Lifelong Person Re-Identification via Adaptive Knowledge Accumulation},
booktitle = { {CVPR}},
pages = {7901–7910},
publisher = {Computer Vision Foundation / {IEEE}},
year = {2021}
}

reference

[1] Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018. 4
[2] Francisco M Castro, Manuel J Mar´ın-Jim´enez, Nicol´as Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incre- mental learning. In ECCV, pages 233–248, 2018. 2
[3] Wei Chen, Yu Liu, Weiping Wang, Tinne Tuytelaars, Er- win M Bakker, and Michael Lew. On the exploration of incremental learning for fine-grained image retrieval. In BMVC, 2020. 2
[4] Rosemary A Cowell, Morgan D Barense, and Patricnow S Sadil. A roadmap for understanding memory: Decompos- ing cognitive processes into operations and representations. Eneuro, 6(4), 2019. 1, 3, 5
[5] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212, 2017. 4
[6] Douglas Gray and Hai Tao. Viewpoint invariant pedestrian recognition with an ensemble of localized features. In ECCV, pages 262–275, 2008. 6
[7] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In de- fense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017. 6
[8] Martin Hirzer, Csaba Beleznai, Peter M Roth, and Horst Bischof. Person re-identification by descriptive and discrim- inative classification. In scandinavian conference on image analysis, pages 91–102. Springer, 2011. 6
[9] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439, 2020. 5
[10] Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, and Li Zhang. Style normalization and restitution for generalizable person re-identification. In CVPR, pages 3143–3152, 2020. 1
[11] Thomas N Kipf and Max Welling. Semi-supervised classi- fication with graph convolutional networks. In ICLR, 2017. 4
[12] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sci- ences, 114(13):3521–3526, 2017. 5
[13] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Uni- versity of Toronto, 2009. 1, 2
[14] Yann LeCun, L´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 2
[15] Qingming Leng, Mang Ye, and Qi Tian. A survey of open- world person re-identification. IEEE Trans. Circuit Syst. Video Technol., 30(4):1092–1108, 2019. 1
[16] Wei Li and Xiaogang Wang. Locally aligned feature trans- forms across views. In CVPR, pages 3594–3601, 2013. 6
[17] Wei Li, Rui Zhao, and Xiaogang Wang. Human reidentifica- tion with transferred metric learning. In ACCV, 2012. 6
[18] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. Deep- reid: Deep filter pairing neural network for person re- identification. In CVPR, pages 152–159, 2014. 6
[19] Wei-Hong Li, Zhuowei Zhong, and Wei-Shi Zheng. One- pass person re-identification by sketch online discriminant analysis. Pattern Recognition, 93:237–250, 2019. 2
[20] Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. Graph matching networks for learning the similarity of graph structured objects. In ICML, pages 3835– 3845. PMLR, 2019. 4
[21] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell., 40(12):2935–2947, 2017. 2, 3, 6
[22] Yutian Lin, Xuanyi Dong, Liang Zheng, Yan Yan, and Yi Yang. A bottom-up clustering approach to unsupervised per- son re-identification. In AAAI, volume 33, pages 8738–8745, 2019. 2
[23] Jialun Liu, Yifan Sun, Chuchu Han, Zhaopeng Dou, and Wenhui Li. Deep representation learning on long-tailed data: A learnable embedding augmentation perspective. In CVPR, pages 2970–2979, 2020. 2
[24] Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. arXiv preprint arXiv:1705.03550, 2017. 2
[25] Chen Change Loy, Tao Xiang, and Shaogang Gong. Time- delayed correlation analysis for multi-camera activity under- standing. Int. J. Comput. Vis., 90(1):106–129, 2010. 6
[26] Chuanchen Luo, Yuntao Chen, Naiyan Wang, and Zhaoxi- ang Zhang. Spectral feature transformation for person re- identification. In ICCV, pages 4976–4985, 2019. 3
[27] Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, vol- ume 24, pages 109–165. Elsevier, 1989. 1
[28] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and StefanWermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. 2
[29] Anastasia Pentina and Christoph Lampert. A pac-bayesian bound for lifelong learning. In ICML, pages 991–999, 2014. 2
[30] Angelo Porrello, Luca Bergamini, and Simone Calderara. Robust re-identification by multiple views knowledge distil- lation. In ECCV, pages 93–110, 2020. 1, 2
[31] Nan Pu, Wei Chen, Yu Liu, Erwin M Bakker, and Michael S Lew. Dual gaussian-based variational subspace disentangle- ment for visible-infrared person re-identification. In ACM MM, pages 2149–2158, 2020. 1, 2
[32] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classi- fier and representation learning. In CVPR, pages 2001–2010, 2017. 1, 2
[33] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large
scale visual recognition challenge. Int. J. Comput. Vis., 115(3):211–252, 2015. 1, 2
[34] Konstantin Shmelkov, Cordelia Schmid, and Karteek Ala- hari. Incremental learning of object detectors without catas- trophic forgetting. In ICCV, pages 3400–3409, 2017. 2
[35] Jifei Song, Yongxin Yang, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. Generalizable person re- identification by domain-invariant mapping network. In CVPR, pages 719–728, 2019. 1, 2
[36] Frederick Tung and Greg Mori. Similarity-preserving knowl- edge distillation. In ICCV, pages 1365–1374, 2019. 6
[37] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Re- port CNS-TR-2011-001, California Institute of Technology, 2011. 2
[38] DongkaiWang and Shiliang Zhang. Unsupervised person re- identification via multi-label classification. In CVPR, pages 10981–10990, 2020. 1, 2
[39] Wei-Chun Wang, Nadia M Brashier, Erik A Wing, Eliza- beth J Marsh, and Roberto Cabeza. Knowledge supports memory retrieval through familiarity, not recollection. Neu- ropsychologia, 113:14–21, 2018. 1, 5
[40] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re- identification. In CVPR, pages 79–88, 2018. 1, 2, 6
[41] Zheng Wei-Shi, Gong Shaogang, and Xiang Tao. Associat- ing groups of people. In BMVC, pages 23–1, 2009. 6
[42] ChenshenWu, Luis Herranz, Xialei Liu, Joost van deWeijer, Bogdan Raducanu, et al. Memory replay gans: Learning to generate new categories without forgetting. In NeurIPS, pages 5962–5972, 2018. 2
[43] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xi- aogang Wang. End-to-end deep learning for person search. arXiv preprint arXiv:1604.01850, 2(2), 2016. 6
[44] Mang Ye, Jianbing Shen, Xu Zhang, Pong C Yuen, and Shih- Fu Chang. Augmentation invariant and instance spreading feature for softmax embedding. IEEE Trans. Pattern Anal. Mach. Intell., 2020. 4
[45] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. arXiv preprint arXiv:1708.01547, 2017. 2
[46] Hong-Xing Yu and Wei-Shi Zheng. Weakly supervised dis- criminative feature learning with state information for person identification. In CVPR, pages 5527–5537, 2020. 1
[47] Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen. Relation-aware global attention for person re- identification. In CVPR, pages 3186–3195, 2020. 1, 2
[48] Bo Zhao, Shixiang Tang, Dapeng Chen, Hakan Bilen, and Rui Zhao. Continual representation learning for biometric identification. arXiv preprint arXiv:2006.04455, 2020. 2, 5, 6, 7
[49] Fang Zhao, Shengcai Liao, Guo-Sen Xie, Jian Zhao, Kai- hao Zhang, and Ling Shao. Unsupervised domain adap- tation with noise resistible mutual-training for person re- identification. In ECCV, pages 526–544, 2020. 1, 2
[50] Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang. Spindle
net:net: Person re-identification with human body region guided feature decomposition and fusion. In CVPR, pages 1077– 1085, 2017. 6
[51] Feng Zheng, Cheng Deng, Xing Sun, Xinyang Jiang, Xi- aowei Guo, Zongqiao Yu, Feiyue Huang, and Rongrong Ji. Pyramidal person re-identification via multi-loss dynamic training. In CVPR, pages 8514–8522, 2019. 7
[52] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jing- dong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In ICCV, pages 1116–1124, 2015. 1, 6
[53] Liang Zheng, Yi Yang, and Alexander G Hauptmann. Per- son re-identification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016. 1, 2
[54] Zhedong Zheng, Liang Zheng, and Yi Yang. Unlabeled sam- ples generated by gan improve the person re-identification baseline in vitro. In ICCV, pages 3754–3762, 2017. 1, 2, 6
[55] Yang Zou, Xiaodong Yang, Zhiding Yu, BVK Kumar, and Jan Kautz. Joint disentangling and adaptation for cross- domain person re-identification. In ECCV, pages 87–104, 2020. 1, 2