The 19th - 22nd week of scientific research training - about tnet and memnet

---

tags:

• Paper reading

• NLP

---

1. Paper reading

《Transformation Networks for Target-Oriented Sentiment Classifification》

• [paper]

• [code]

《Aspect Level Sentiment Classification with Deep Memory Network》

• [paper]
• [code]

1.1 introduction

1.1.1 TNet

Put forward Transformation Networks (TNet) solve Attention The mechanism and CNN In itself ABSC A defect in the task （ See for specific defects 1.2）

Main contributions ：

• Adopted CNN Of TNet solve ABSC problem , And the performance has achieved the best results on the benchmark data set .
• A new method based on objective target Conversion components for , To better integrate target information into word representation .
• A context saving mechanism is designed , Forward context information into the deep transformation architecture , therefore , This model can learn more abstract contextual word features from the deeper network .

1.1.2 MemNet

Put forward MemNet Model , utilize Attention Multi layer mechanism to memorize the text information of the context , Its speed ratio LSTM Much faster （cpu）
Main contributions ：

• be based on Attention To capture context information , Get rid of LSTM Time consuming problem
• The efficiency and accuracy are greatly improved

1.2 motivation

1.2.1 TNet

• The defect of attention mechanism itself ：
  • When calculating the attention score of a word , Will introduce noise

explain ： for instance ,“This dish is my favorite and I always get it and never get tired of it.” About the dish When calculating the attention mechanism , Introduce irrelevant words such as “never”“tired”

• CNN Its own defects in text processing ：

  • Can not be like RNN Explore the context information completely

  • Difficult to deal with multi-objective words （ Even if explored, it is difficult to make trade-offs based on context ）

explain ： Like a sentence “great food but the service was dreadful”, For the target word “food” for ,CNN It is difficult to give “good” and “dreadful” A reasonable choice between

1.2.2 MemNet

LSTM Time consuming , Weak parallelism , At the same time, it can not accurately capture the data based on aspect The context of

1.3 model

1.3.1 TNet

Combining with the above , Explain the model part from the bottom up ：

1. $x_i$ Is the original input , The red module is bidirectional LSTM,（ Because previous studies have shown that context based word representation is an effective representation of words in convolution architecture ）, Output $h_i$ It is the result of the word representation in this layer .$h_i=[LST{M}_{left->right}(x_i);LST{M}_{right->left}(x_i)],i\in [1,n]$

2. The grey module is CPT（ Context saving transformation ）, The main function is to introduce target Come to the word expression No traditional attention Calculate the weight to do …… Instead, they used their own design TST modular , Emphasize at the same time CPT It's multi-layered

   TST Components are introduced from bottom to top ：

   • two-way LSTM Get the representation of the target word $h^{\tau }$

   • Take the next step $h^{\tau }$ Dynamically with the words in the sentence $w_i$ Connect , Variation customizes the target representation at the time step KaTeX parse error: Expected 'EOF', got '}' at position 11: r_i^{\tau}}̲

   • Fully connected layer , For the first $i$ The expression of a target word

   • LF/AS：TST after , from Bi-LSTM The context information obtained will be lost , To take advantage of contextual information , Here are two strategies LF and AS

     • LF： Lossless forwarding , The model is red at first Bi-LSTM The generated representation and TST The generated representations are added directly

     • AS： Adaptive scaling , Introduce parameters $W$ and $b$, How much learning model should be kept in red at the beginning Bi-LSTM The generated representation and how many TST Generated representation . It is equivalent to introducing the gating mechanism .

3. CNN： Location code （ Suppose a target Multiple... Are detected around opinion When , Default that the closest contribution to him is the greatest ）

1.3.2 MemNet

Parallelism is better than using LSTM A better model , It's actually multiple attention Stack of layers , Using multiple layers attention Capture relationships between contexts , In order to replace the use of such as LSTM The solution of this time series model .

1.4 Examination

TNet Just tried LF How to do it ,AS There are still problems in debugging , The accuracy does not reach a reasonable state

1.5 Result

Because I didn't find it twitter Of xml data …… So it is only given in laptop as well as restaurant The result on ：

2. Coding The story of

bugs lovely

3. Reference material

1. Tnet Reading notes
2. MemNet Reading notes

twitter ：
Long live the return to work ~~！！！
School starts to prepare for the fight between immortals ~ Go go go go