Paper notes ACL 2020 improving event detection via open domain trigger knowledge

1 brief introduction

Thesis title ：Improving Event Detection via Open-domain Trigger Knowledge
Source of the paper ：ACL 2020
Thesis link ：https://aclanthology.org/2020.acl-main.522.pdf
Code link ：https://github.com/shuaiwa16/ekd

1.1 motivation

• Due to the long tail problem of marked data ( A large number of categories, only a small number of samples ) And the homogeneity of generated data , Previous methods have performed poorly on unseen or sparse data , Over fitting on dense data .

1.2 innovation

• The first one is to improve the performance of event detection by using the trigger word knowledge of open domain .
• Put forward a new teacher-student Model , Learn from tagged and unlabeled data , Reduce the built-in deviation in the tag .

2 Method

2.1 Knowledge collection

from WordNet Collect open domain trigger word knowledge , It is divided into the following two steps ：

1. Disambiguate words ： Use IMS disambiguation , And then use Stanford CoreNLP Get features ( Part of speech tagging 、 Syntactic parsing ).
2. Determine whether the event is triggered ： Use a table lookup method , Determine whether the event is triggered .

2.2 Model

The overall framework of the model is shown in the figure above , It mainly includes the following parts ：

• feature extraction ： Use BERT Code the sentence .
• Event prediction ： For marked data , Predict the event type of each word . The formula is as follows ：

• Distillation of knowledge ： The goal of knowledge distillation is to make teacher The probability of the model is equal to student Probability of model ( The formula is as follows ), Two models share parameters ,teacher The input to the model is $S^{+}$(Knowledge-attending Sentences), Trigger word knowledge through open domain , Use B-TRI and E-TRI Mark the start and end boundaries of the trigger word . If the original sentence is $S=\{w_1,w_2,...,w_i,...,w_n\}$,$w_i$ Trigger words defined for open domain trigger word knowledge ,$S^{+}=\{w_1,w_2,...,B-TRI,w_i,E-TRI,...,w_n\}$.B-TRI and E-TRI Fine tune the sentences of knowledge collection (mask The probability of is 15%).student The input to the model is $S^{-}$(Knowledge-absent Sentences), Random mask Trigger words defined by open domain trigger word knowledge , Such as $S^{-}=\{w_1,w_2,...,[MASK],...,w_n\}$. Use KL Divergence minimizes the difference between probability distributions , The formula is as follows ：

• Joint training ： The goal of optimization is the supervision with labeled data loss And unmarked data KL The divergence loss, The formula is as follows ：
  

3 experiment

stay ACE 2005 The experimental results on the data set are shown in the figure below ：

To evaluate whether knowledge is distilled into the model , Observe the experimental effect of trigger word knowledge with and without open domain on the test set , The results are as follows ：

Experimental results in the case of domain transfer ：

Experimental results of trigger words with different frequencies ：

Use three different kinds of knowledge , Verify whether the model can distill other knowledge types , The experimental results are shown below ：

Case Study: