It is thought-provoking: is syntax really important? Qiu Xipeng group proposed a powerful baseline for aspect based emotional analysis

Every day I send you NLP Technical dry cargo ！

---

The paper ：Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa  [ACL21]

Company ： Fudan University 、 Carnegie Mellon University

link ：https://aclanthology.org/2021.naacl-main.146.pdf

Extract abstract

Emotional analysis based on aspects （Aspect-Based Sentiment Analysis (ABSA)） Everyone should be familiar with the task , The experiment of this paper is based on this .

A lot of previous work has shown that integrating syntactic information can improve ABSA Performance of . But the recent pre training model also shows a strong performance improvement .

therefore , A question arises ：

• Whether there is a large amount of syntactic information in the pre training model ？
• Is it enough to use only the pre training model ？

For the first time, this paper compares the dependency syntax trees generated based on the pre training model and the syntax parser in ABSA Performance on , Results show ：FT-RoBERTa (finetune RoBERTa) Better than the syntactic structure information generated by the syntactic parser . Through experimental analysis, it is further found that FT-RoBERTa The generated syntax tree is more emotional word oriented , It's good for ABSA Mission . Finally, we achieved SOTA Or something similar SOTA Performance of .

Specific introduction

How to generate syntax tree through pre training model ？

This article USES Perturbed Masking ( Hong Kong University and Huawei ACL20 Put forward ) To use the pre training model to generate the syntax tree . The general process is ：

For calculation token j about token i The degree of influence , First mask fall token i, At this time to get mask Word representation of ：

; then mask fall token j, It will also return a representation token i and j Word representation of

, that j about i The influence degree formula of is calculated as ：

Repeat the above operation , You will get one for each pair token Matrix of influence degree , Then the syntax tree can be obtained through this matrix .

Model

The model used in this paper is very simple , Is in the Aspect Do on MLP Classified finetune RoBERTa that will do .

experiment

Performance comparison of different tree structures on English data sets

Performance comparison of different tree structures on non English data sets

FT-RoBERTa The generated tree is more correct Aspect friendly

And SOTA Let's make a comparison

reflection

So , Really don't need syntactic structure information ？ Is syntax still important ？ This article is only about ABSA I did an experiment on , Other tasks ？ My personal tendency is that the artificially defined syntactic structure must be defective , Future unification is the trend , What do you think? ？ Voice their opinions ~

---

Contribute or exchange learning , remarks ： nickname - School （ company ）- Direction , Get into DL&NLP Communication group .

There are many directions ： machine learning 、 Deep learning ,python, Sentiment analysis 、 Opinion mining 、 Syntactic parsing 、 Machine translation 、 Man-machine dialogue 、 Knowledge map 、 Speech recognition, etc .

Remember the remark

Log in and copy

      
      

       
        Sorting is not easy to , I hope to give you a ！
      
      
      
      

       
       
1.