Raki's notes on reading paper: discontinuous named entity recognition as maximum clique discovery

Abstract & Introduction & Related Work

• Research tasks
  Discontinuous NER
• Existing methods and related work
  1. The composite based model first detects all overlaps span, Then learn to combine these fragments with separate classifiers
  2. The transition based model passes through a series of shifts - The restoring action is not continuous span Make incremental marks
• Facing the challenge
  1. The existing method divides it into several successive stages , In the reasoning stage, the model output depends on the previous stage , The deviation will be accumulated
  2. exposure bias
• Innovative ideas
  Will be discontinuous NER It is transformed into a nonparametric process of finding the largest clique in the graph , Avoid the problem of being exposed to deviations
• The experimental conclusion
  sota, Five times the reasoning speed

Review the largest regiment

The biggest group = Maximal independent set of complementary graphs = Total points − Maximum matching number of complementary graphs

Methodology

Grid Tagging Scheme

Construct a n*n Matrix , To each pair token Between a label , Based on their relationship

Be careful , According to the predefined tag set , A tag pair may have multiple tags

Segment Extraction

This two-dimensional matrix is used to represent

s Represents a continuous entity ,B and I representative beginning and inside

Edge Prediction

1. Head to head (X-H2H) It means that it is located in a place $(t_i,t_j)$, among $t_i$ and $t_j$ They are composed of X The starting tag of two fragments of the same entity of type
2. Tail to tail （X-T2T） And X-H2H be similar , But the point is on the closing tag

Sever shoulder pain

Decoding Workflow

Model Structure

Token Representation

Coding with a linear layer , But I want to ask the original $h_i$ Did you jump out of the crack in the stone ？ I thought you didn't say ？

Segment Extractor

The probability that a pair of markers are boundary markers of a segment can be expressed as ：

Condition layer normalization

A condition vector is used as additional context information to generate gain parameters ,c and x They are the condition vector and the input vector

be based on CLN Mechanism ,token Yes $(t_i,t_j)$ The representation of the boundary of a fragment can be defined as ：

Learn better fragment representation

Edge Predictor

Edge prediction is similar to segment extraction , Because they all need to learn the representation of each tag pair . The key differences are summarized in the following two aspects .

1. The distance between paragraphs is usually uninformative , So the length is embedded $e_{len}^{i:j}$ It is worthless in edge prediction
2. Encoding tags between fragments may bring noise semantics of related tags , And increase the burden of training , therefore i:j There is no need for $h_{i:j}^{in}$. Under such consideration , We represent each marker pair used for edge prediction as
   

Training and Inference

Fully connected layer , Used to predict a pair of token The label of

Cross entropy loss

Total loss

Evaluation

Conclusion

In this paper , We will be discontinuous NER Reformulated as the task of finding the largest clique in the fragment graph , A novel Mac framework . It decomposes the construction of fragment graph into two independent two-dimensional grid marking problems , And solve it together in one stage , It solves the problem of exposure bias in previous studies . Extensive experiments on three benchmark data sets show that ,Mac stay F1 Than before SOTA Method higher than 3.5 percentage , At the same time, the speed is fast 5 times . Further analysis shows that , Our model has the ability to identify discontinuous and overlapping entity references . some time , We hope to explore similar expressions in other information extraction tasks , Such as event extraction and nesting NER

Remark

novelty yes , we have , But it is not beautiful to realize , The experiment was done very well , That's OK