当前位置:网站首页>Thesis reading_ Chinese NLP_ LTP
Thesis reading_ Chinese NLP_ LTP
2022-07-05 17:41:00 【xieyan0811】
English title :N-LTP: An Open-source Neural Language Technology Platform for Chinese
Chinese title : Open source Chinese neural network language technology platform N-LTP
Address of thesis :https://arxiv.org/pdf/2009.11616v4.pdf
field : natural language processing
Time of publication :2021
author :Wanxiang Che etc. , Harbin Institute of technology
Source :EMNLP
Quantity cited :18+
Code and data :https://github.com/HIT-SCIR/ltp
Reading time :22.06.20
Journal entry
It is based on Pytorch For chinese Of Offline tools , Take the trained model , Minimum model only 164M. Directly support word segmentation , Six tasks such as named entity recognition , The six tasks basically revolve around word segmentation 、 Determine the composition of the word 、 Relationship .
Actually measured : Better than expected , If used to identify a person's name , The effect is OK , Directly used in vertical fields , Results the general , Further fine tuning may be required .
Article contribution
- Support six Chinese natural language tasks .
- be based on Multitasking framework , Sharing knowledge , Reduce memory usage , Speed up .
- High scalability : Support Introduced by users BERT Class model .
- Easy to use : Support Multilingual interface C++, Python, Java, Rust
- Achieve better results than the previous model
Design and Architecture
chart -2 Shows the software architecture , It consists of a coding layer shared by multiple tasks and a decoding layer implemented by each task .
Shared coding layer
Use pre trained models ELECTRA, The input sequence is s=(s1,s2,…,sn), Add symbols to make it s = ([CLS], s1, s2, . . . , sn, [SEP]), Please see BERT principle , The output is the corresponding hidden layer code
H = (h[CLS],h1, h2, . . . , hn, h[SEP]).
Chinese word segmentation CWS
After coding H Substitute linear decoder , Classify each character :
y Is the probability that each character category is each label .
Position marking POS
Location marking is also NLP An important task in , For further parsing . At present, the mainstream method is to treat it as a sequence annotation problem . It will also be encoded H As input , Label of output position :
y Is the probability that the character in this position belongs to a label , among i It's location information .
Named entity recognition NER
The goal of named entity recognition is to find the starting and ending positions of entities , And the category of the entity . Use in tools Adapted-Transformer Method , Add direction and distance features :
The last step also uses a linear classifier to calculate the category of each word :
among y yes NER The probability of belonging to a tag .
Dependency resolution DEP
Dependency parsing mainly analyzes the semantic structure of sentences ( See online examples for details ), Look for the relationship between words . Double affine neural network and einser Algorithm .
Semantic analysis SDP
Similar to dependency analysis , Semantic dependency analysis also captures the semantic structure of sentences . It parses sentences into a dependency syntax tree , Describe the dependencies between the words . That is to say, it points out the syntactic collocation relationship between words , This collocation is related to semantics . Specific include : Subject predicate relationship SBV, The verb object relationship VOB, Settle China relations ATT etc. , See :
from 0 To 1, Teach you how to use it hand in hand NLP Tools ——PyLTP
The specific method is to find semantically related word pairs , And find the predefined semantic relationship . The implementation also uses the double affine model .
When p>0.5 when , They think that words i And j There is a connection between .
Semantic Role Labeling SRL
The main goal of semantic role tagging is to recognize the predicate centered structure of sentences , The specific method is to use end-to-end SRL Model , It combines double affine neural network and conditional random field as encoder , The conditional random field formula is as follows :
among f Used to calculate from yi,j-1 To yi,j The probability of transfer .
Distillation of knowledge
To compare individual training tasks with multi task training , Introduced BAM Method :
usage
install
$ pip install ltp
On-line demo
http://ltp.ai/demo.html
Sample code
from ltp import LTP
ltp = LTP()
seg, hidden = ltp.seg([" He asked Tom to get his coat ."])
pos = ltp.pos(hidden)
ner = ltp.ner(hidden)
srl = ltp.srl(hidden)
dep = ltp.dep(hidden)
sdp = ltp.sdp(hidden)
among seg Function implements word segmentation , And output the segmentation result , And the vector representation of each word .
Fine tuning model
Download the source code
$ git clone https://github.com/HIT-SCIR/ltp
In its ltp There is... In the catalog task_xx.py, Trainable and tuning model , Usage as shown in the py Internal example . Form like :
python ltp/task_segmention.py --data_dir=data/seg --num_labels=2 --max_epochs=10 --batch_size=16 --gpus=1 --precision=16 --auto_lr_find=lr
experiment
Stanza It supports a multilingual NLP Tools , The comparison of Chinese modeling effect is as follows :
in addition , Experiments also prove that , Faster using federated models , Less memory .
Reference resources
边栏推荐
- Which is more cost-effective, haqu K1 or haqu H1? Who is more worth starting with?
- MYSQL group by 有哪些注意事项
- 中国银河证券开户安全吗 开户后多久能买股票
- Tips for extracting JSON fields from MySQL
- 漫画:如何实现大整数相乘?(下)
- 为什么阳历中平年二月是28天
- MySQL之知识点(七)
- mysql中取出json字段的小技巧
- Zhang Ping'an: accelerate cloud digital innovation and jointly build an industrial smart ecosystem
- ICML 2022 | meta proposes a robust multi-objective Bayesian optimization method to effectively deal with input noise
猜你喜欢
Vulnerability recurrence - 48. Command injection in airflow DAG (cve-2020-11978)
Oracle recovery tools -- Oracle database recovery tool
Abnormal recovery of virtual machine Oracle -- Xi Fenfei
33:第三章:开发通行证服务:16:使用Redis缓存用户信息;(以减轻数据库的压力)
论文阅读_医疗NLP模型_ EMBERT
哈趣K1和哈趣H1哪个性价比更高?谁更值得入手?
Design of electronic clock based on 51 single chip microcomputer
33: Chapter 3: develop pass service: 16: use redis to cache user information; (to reduce the pressure on the database)
Tips for extracting JSON fields from MySQL
In depth understanding of redis memory obsolescence strategy
随机推荐
Design of electronic clock based on 51 single chip microcomputer
7 pratiques devops pour améliorer la performance des applications
一文了解Go语言中的函数与方法的用法
Kafaka technology lesson 1
Disabling and enabling inspections pycharm
Example tutorial of SQL deduplication
哈趣K1和哈趣H1哪个性价比更高?谁更值得入手?
flask接口响应中的中文乱码(unicode)处理
ICML 2022 | Meta提出魯棒的多目標貝葉斯優化方法,有效應對輸入噪聲
Database design in multi tenant mode
解读:如何应对物联网目前面临的安全问题?
Complete solution instance of Oracle shrink table space
Count the running time of PHP program and set the maximum running time of PHP
MySQL queries the latest qualified data rows
使用QT设计师界面类创建2个界面,通过按键从界面1切换到界面2
Tita performance treasure: how to prepare for the mid year examination?
Clickhouse (03) how to install and deploy Clickhouse
Disabling and enabling inspections pycharm
如何保存训练好的神经网络模型(pytorch版本)
漫画:一道数学题引发的血案