Relationship extraction --r-bert

Relationship extraction –R-BERT

Relation extraction is an important task in naturallanguageprocessing . The relationship extraction task is divided into pipline Type extraction and joint line The formula extraction . among pipline The extraction method generally divides the relationship extraction into two processes , First, extract entities , After classifying entities, we can get the relationship between entities , Transform relation extraction into a classification problem .joint line Formal extraction is to extract entities and relationships at the same time ,joint line Relational extraction will become an important direction of relational extraction . This blog introduces a good model of relationship extraction ideas ——R-BERT Model . The content is roughly divided into the following parts ：

1. R-BERT Model
2. R-BERT Key code Introduction
3. experimental result

R-BERT Model

R-BERT The model is based on pre training BERT The relational classification model of the model , The core of the model is as follows ：
（1） Use pre training BERT To express the characteristics of words
（2） Extract pre training BERT Of special marking symbols in classification tasks "[CLS]" Characteristic information of ,subject Characteristic information of and object The characteristic information of .
R-BERT In the model, the entity information is used for classification , A new relational classification model is proposed .R-BERT The model structure is shown below ：

R-BERT Key code Introduction

R-BERT The authors of the model open source the code , Is in torch and transformers Realized . Let's introduce subject and object Extraction of two entities and classification of relationships between entities .
（1）subject and object Entity extraction

    def entity_average(hidden_output, e_mask):
        """ Average the entity hidden state vectors (H_i ~ H_j) :param hidden_output: [batch_size, j-i+1, dim] :param e_mask: [batch_size, max_seq_len] e.g. e_mask[0] == [0, 0, 0, 1, 1, 1, 0, 0, ... 0] :return: [batch_size, dim] """
        e_mask_unsqueeze = e_mask.unsqueeze(1)  # [b, 1, j-i+1]
        length_tensor = (e_mask != 0).sum(dim=1).unsqueeze(1)  # [batch_size, 1]

        # [b, 1, j-i+1] * [b, j-i+1, dim] = [b, 1, dim] -> [b, dim]
        sum_vector = torch.bmm(e_mask_unsqueeze.float(), hidden_output).squeeze(1)
        avg_vector = sum_vector.float() / length_tensor.float()  # broadcasting
        return avg_vector

Yes subject and object Conduct 0 and 1 Of mask, At entity location mask by 1, Other locations mask by 0, So as to obtain subject and object The representation of pre - training words .
（2） Relationship classification

self.cls_fc_layer = FCLayer(config.hidden_size,
                                    config.hidden_size,
                                    args.dropout_rate)
self.entity_fc_layer = FCLayer(config.hidden_size,
                                       config.hidden_size,
                                       args.dropout_rate)
self.label_classifier = FCLayer(
            config.hidden_size * 3,
            config.num_labels,
            args.dropout_rate,
            use_activation=False,
        )

among FCLayer It's a full connection layer .
（3）R-BERT Yes subject and object Insert special symbols before and after to mark the entity .

experimental result

stay ccks2019 Relation extraction data set ,R-BERT See the figure below for the model effect

The above is right R-BERT Introduction to the model , If there is any inaccuracy , Welcome to correct .