Paddle framework: paddlenlp overview [propeller natural language processing development library]

PaddleNLP It is the development library of natural language processing of flying propeller , Have Easy to use text field API, Application examples of multiple scenarios 、 and High performance distributed training Three characteristics , It aims to improve the efficiency of text domain modeling of propeller developers , It aims to improve the development efficiency of developers in the text field , And provide rich NLP Application example .

• Easy to use text field API

  • Provide rich industrial preset task capabilities Taskflow And the text field of the whole process API： Support loading of rich Chinese data sets Dataset API, It can flexibly and efficiently complete data preprocessing Data API , preset 60+ Pre trained word vector Embedding API , Provide 100+ In the pre training model Transformer API etc. , Can greatly improve NLP Efficiency of task modeling .

• Application examples of multiple scenarios

  • Covering from academic to industrial NLP Application example , cover NLP Basic technology 、NLP System application and related extended applications . Fully based on the core framework of the propeller 2.0 new API System development , Provide developers with best practices in the field of propeller text .

• High performance distributed training

  • Advanced automatic mixing accuracy optimization strategy based on the core framework of the propeller , Combined with distributed Fleet API, Support 4D Hybrid parallel strategy , It can efficiently complete large-scale pre training model training .

project GitHub: https://github.com/PaddlePaddle/PaddleNLP

---

---

---

Reference material ：
PaddleNLP The official tutorial