How to excavate and research ideas from the paper

1 Patching ：
Hit it right in the face and it will be fatal
With MAE For example ： be based on ViT + BERT,ViT The shortcomings of the article mentioned in the paper are BERT The effect is not good , Based on this MAE Two improvements have been made , First, mask more picture blocks （ The purpose is to make the pictures less integrated , Increase the challenge of the task ）, And when encoding, only the uncovered blocks are processed to speed up ; The second is to use Transformer Block to output, namely decoding . The first idea is actually the process of data enhancement , To prevent the model from over fitting , because ViT The convergence of is very slow , So it's natural to think of the first idea ： Whether it can be used *** Data to enhance *** To ensure the effect of the model at the same time *** Reduce training time ***; Second thought ： Whether it can be used *** New architecture *** Instead of ViT; The third idea ： in the light of BERT*** Loss function *** Whether it can be improved （ such as Contrastive Learning）;

Considering the details of the paper, which point can make it better , After listing your thoughts , We must use experiments to verify , And provide new ideas through the experimental results . When patching, do not simply list , Must pass a *** The story connects the patches ***, That is to test the ability of writing . There is also a lot of technology in choosing the paper to patch , The preferred *** The patch space of the new paper is relatively large ***; Second, consider the actual situation of the hardware ; Of course, you can also consider accelerating training , Reduce the calculation cost as an idea .

Patching is actually aimed at a certain paper （ Preferably a new paper ） Improve the deficiencies in , For the specific details in the paper , Think about whether you can think of a better way to make up , And through experiments to verify and conceive new ideas again .

To be continued ~