Stanford, salesforce|maskvit: masked vision pre training for video prediction

title ： Stanford 、Salesforce|MaskViT: Masked Visual Pre-Training for Video Prediction（MaskViT： Masked visual pre training for video prediction ）

author ：Agrim Gupta, Stephen Tian, Li Fei-Fei etc.

brief introduction ： This paper introduces a video pre training method . The ability to predict future visual observation and motor commands based on the past , It enables specific agents to plan and solve various tasks in complex environments . This work shows that , The author can pre train the converter through masking vision to create a good video prediction model . The author's method is named MaskViT, It is based on two simple design decisions . First , For memory and training efficiency , The author uses two types of window attention ： Space and time . secondly , During training , The author masked the variable percentage mark , Instead of a fixed shielding rate . For reasoning ,MaskViT Generate all symbols through iterative refinement , The author gradually reduces the rate of masking following the mask scheduling function . On several datasets shown by the author MaskViT It is better than previous work in video prediction , Efficient parameters , And it can generate high-resolution video （256×256）. Besides , The author proves that due to the use of MaskViT Iterative decoding of planning on a real robot , Reasoning speeds up （ the height is 512 times ）. The author's work shows that , The author can endow agents with powerful prediction ability through general masked vision model and a small amount of domain knowledge .

Paper download ：https://arxiv.org/pdf/2206.11894v1.pdf