Online text entity extraction capability helps applications analyze massive text data

With the development of information technology , Many valuable knowledge hiding is distributed in massive data , It affects the efficiency of people acquiring knowledge , How to deal with the complicated unstructured text data has become a difficult problem .

In recent days, ,HMS Core​ ​ Machine learning services ​​, This ability can detect whether there is a date in the text 、 full name 、 Proper nouns and other entity information , And extract such entities , That is, the ability to automatically process unstructured natural language text data . for example , The application of the film and television industry often produces a large number of film reviews 、 Information, etc , Using the online text entity extraction capability, we can quickly extract structural information , Help build knowledge map , It is easy for users to understand clearly .

Besides , Text entity extraction capability is more used in question and answer system 、 Information index 、 Knowledge map construction and other fields .

Question answering system

Question answering system is an advanced form of information retrieval system , It can be used accurately 、 Simple natural language to answer user's questions . During the implementation of question answering system , Then we need to use the text entity extraction ability to identify the problem and the entity information in the knowledge base , And then through a variety of algorithm models to match the accurate answer .

Information index

Use online text entity extraction capabilities , You can name specific entity information as indexes and hyperlinks . For example, the proper nouns mentioned by users in their comments , You can generate hyperlinks , It is convenient for other users to search and understand relevant contents .

Knowledge map construction

The map of knowledge is made up of entities 、 A data structure consisting of relationships and attributes , That is, a knowledge base with directed graph structure , Text entity extraction capability is the bottom capability in the process of knowledge map construction , It plays an extremely important role . For example, build a music knowledge map , First, we need to extract singers from a large amount of text data 、 song 、 Lyrics 、 Film and television and other related information , Then build the knowledge map . at present , The online text entity extraction capability of Huawei machine learning service supports a total of person names 、 money 、 Film name 、 Page links 16 Entity categories , It can be applied to different categories according to the actual semantic scenarios App in .

Integration steps

1. The development of preparation

For detailed preparation steps, please refer to ​ ​ Official website of Huawei developer Alliance ​​.

2. Integration and configuration apigateway authentication

be based on apigateway The authentication mechanism of ：

Log in and copy

      
      

       
       "paths": {
       
       
"/entityExtract": { "post": { "operationId": "entityExtract",
       
       
"parameters": [{"in": "body",  "name": "req",  "required": true,
       
       
"schema": { "$ref": "#/definitions/NerEnterReq"  } },  {
       
       
"name": "X-Request-ID", "in": "header",  "required": true,
       
       
"type": "string"
       
       
}, {"name": "X-Package-Name",  "in": "header", "required": true,
       
       
"type": "string"  }, ……],
       
       
 "responses": {"200": {   "description": "response of 200",
       
       
"schema": { "$ref": "#/definitions/ResponseEntityNerBodyVo"}}}}}
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
      
      

3. Create an online text entity constructor

Log in and copy

      
      

       
       //  Create a language detector using a custom parameter configuration .
       
       
 MLRemoteNerSetting setting = new MLRemoteNerSetting.Factory()
       
       
                .setSourceLangCode("zh")
       
       
                .create();
       
       
MLRemoteNer ner = MLNerFactory.getInstance().getRemoteNer(setting);
       
       
 Extract text entities .
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
      
      

4. Get text entity extraction online

Asynchronous method sample code ：

Log in and copy

      
      

       
       ner.asyncEntityExtract(input).addOnSuccessListener(new OnSuccessListener<RemoteNerResultItem[]>() {
       
       
            @Override
       
       
            public void onSuccess(RemoteNerResultItem[] remoteNerResults) {
       
       
                //  Successful processing logic .
       
       
                if(remoteNerResults != null){
       
       
                    //  There are identification results 
       
       
                }else {
       
       
                    //   The recognition result is null 
       
       
                }
       
       
            }
       
       
        }).addOnFailureListener(new OnFailureListener() {
       
       
            @Override
       
       
            public void onFailure(Exception e) {
       
       
                //  Recognition failed , Get relevant exception information .
       
       
                try {
       
       
                    MLException mlException = (MLException) e;
       
       
                    //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .
       
       
                    int errorCode = mlException.getErrCode();
       
       
                    //  Get error information , Developers can combine error codes , Fast location problem .
       
       
                    String errorMessage = mlException.getMessage();
       
       
                } catch (Exception error) {
       
       
                    //  Conversion error handling .
       
       
                }
       
       
            }
       
       
        });
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
       
       
11.
       
       
12.
       
       
13.
       
       
14.
       
       
15.
       
       
16.
       
       
17.
       
       
18.
       
       
19.
       
       
20.
       
       
21.
       
       
22.
       
       
23.
       
       
24.
       
       
25.
      
      

Synchronization method sample code ：

Log in and copy

      
      

       
       try {
       
       
       RemoteNerResultItem[] remoteNerResults = ner.syncEntityExtract(input);
       
       
       //  Identify success logic 
       
       
       if(remoteNerResults != null){
       
       
       //  There are identification results 
       
       
       }else {
       
       
       //   The recognition result is null 
       
       
        }
       
       
   } catch (MLException mlException) {
       
       
       //  Failed processing logic .
       
       
       //  Get error code , Developers can handle error codes , According to the error code for differentiated page prompt .
       
       
       int errorCode = mlException.getErrCode();
       
       
       //  Get error information , Developers can combine error codes , Fast location problem .
       
       
       String errorMessage = mlException.getMessage();
       
       
   }
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
       
       
11.
       
       
12.
       
       
13.
       
       
14.
       
       
15.
      
      

5. After completion , Release resources

Log in and copy

      
      

       
       if (ner != null) {
       
       
    ner.stop();
       
       
}
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
      
      

Learn more >>

visit ​ ​ Official website of Huawei developer Alliance ​​

obtain ​ ​ Development guidance document ​​

Huawei mobile service open source warehouse address ：​ ​GitHub​​

Pay attention to our , The first time to understand HMS Core Latest technical information ~