Tencent cloud won the first place in the cloud natural language understanding classification task

In recent days, , Tencent cloud AI With the acceleration team of Tencent Youtu laboratory in CLUE Experiment on the task of language classification , At one fell swoop in the classification task 1.0 and 1.1 Won the first place in the industry .

HUMAN Mark achievements for human beings , Non model effects , Don't participate in ranking

How to communicate with AI Have an accessible dialogue ？

In recent years, with the development of artificial intelligence ,NLP（ natural language processing ） It has been the focus of attention both inside and outside the industry , The pre training model (Pre-Trained Model,PTM) Technology is the most revolutionary innovation achievement at present , It is becoming the focus of Internet enterprises at home and abroad , It is imperative to build a super large-scale pre training model with Chinese as the core and ecological environment , The major companies are feeding back their own businesses, and at the same time, they are turning to CLUE List initiation “ charge ”.

CLUE It is one of the most authoritative benchmarks in the field of Chinese language understanding , It covers text similarity 、 classification 、 Many semantic analysis and understanding subtasks such as reading comprehension . As “ A sharp weapon to brush the list ” The pre training model of is to gather a large amount of computing power on large-scale texts , Train the big model intensively , The pre training has developed common language features , It can be used by a large number of enterprises , It greatly reduces the threshold of naturallanguageprocessing research and application .

 “ A good workman does his work well , You must sharpen your tools first ”

Tencent cloud TI The platform is a one-stop machine learning ecological service platform based on the powerful computing power of Tencent cloud . It can be used for a variety of data sources 、 Components 、 Algorithm 、 The model and the evaluation module are combined , Algorithm Engineers and data scientists can easily train models on it 、 Evaluate and predict .TI Series of products support public cloud access 、 Privatized deployment and exclusive cloud deployment .

TI-ACC Tencent cloud AI And the latest product released by Youtu laboratory AI Acceleration component products , It can provide AI Model training and reasoning acceleration services , Support a variety of frameworks and scenarios , It can significantly improve the reasoning efficiency of model training 、 cost reduction .

The pre training of this large model completely relies on Tencent cloud TI platform , And USES the TI-ACC Train and speed up . The overall training program is as follows ：

First , The excellent effect of the model is inseparable from the support of a large number of high-quality Chinese pre training corpus behind it . Tencent cloud team is TI The preprocessing of massive corpus is built on the platform 、 Cleaning and evaluation tasks , A collection of novels 、 Journalism 、 Quality content in different areas such as community reviews , And papers on various subjects 、 Application description and other specialized contents , Filter out hundreds GB High quality Chinese corpus , Ensure data “ Wide source ” And “ The quality is fine ”.

On this basis , in the light of NLP The characteristics and existing problems of super large model , Tencent cloud team has conducted in-depth optimization in terms of single machine computing performance and multi machine expansion in combination with the underlying infrastructure . In computing performance optimization ,TI-ACC Yes Transformer The structure model is sparsely calculated 、 Operator fusion 、 Dynamic text length input optimization . On multi machine extensions , Adopted Zero-DP Technology combined with reverse graphics memory saving 、 Multi round communication with large model parameters 、 application layer NCCL Communication optimization, parameter automatic optimization and other optimization means . Final ,TI-ACC Capable of efficiently training hundreds of billions of level parameters NLP Big model , It greatly improves the efficiency of model pre training .

Besides , We are right on the model Transformer The structure has been fine tuned , Plus the progressive course learning and training program , Make large models faster 、 Learn knowledge better .

This summit CLUE The list , On the one hand, it represents that Tencent cloud is NLP The ecological field has reached the leading level in the industry , On the other hand, it indicates TI-ACC Help Chinese pre training model to reach a new level in efficient training and reasoning .

---

Click to learn more Tencent cloud AI Product solutions