Thesis reading_ Chinese NLP_ ELECTRA

• Introduce ：ELECTRA from Manning Jointly released by Google , Later, iFLYTEK Joint Laboratory of Harbin Institute of technology trained the corresponding Chinese model . The reduced model effect and BERT Not so much , The size of the model is only BERT Of 1/10,ELECTRA-small Only 46M.
• Code & Model download & Detailed instructions ：https://github.com/ymcui/Chinese-ELECTRA
• Use ：LTP Use it as the base model .
• principle ： Training natural language models using generative confrontation Networks , Time is short , Less parameters . The model is divided into two parts ： Generators and discriminators , Build implementation MLM, The discriminator is used to identify whether each word is generated by the model .
• effect ： Take Chinese reading comprehension as an example , The effect comparison is as follows , For other experiments, see github