NLP (natural language processing) natural language processing learning

One ： Noun recognition

   1 Segmentation ( Division ) Full stop Punctuation marks such as commas Separate sentences

   2 Tokenizing ( Tokenization ）

   3 Stop words( Stop words ） such as was are and in And other types of words

   4 Stemming ( Extract the stem of a word ）skipped skipping skips   All from words skip

   5 Lemmatization ( Word pattern reduction ）are am is All are be Verb

   6 speech tagging （ The part of speech ）noum Noun .vreb  Verb . preposition Preposition

   7 Named entity tagging（ Named entity tags ）

Two ：language modeling （n-gram  RNN

      effect ： Predict next word （ For example, Google search After inputting some words It will prompt the content you want to find according to the possibility of the next word ）

     

RNN Advantages and disadvantages of the model ：

Original text （raw  data) __

participle （segmentation)____

cleaning （cleaning) Useless punctuation   Special symbols   Stop words

Standardization （nomalization)  stemming Stem extraction   lemmation Word pattern reduction

feature extraction （feature extraction)  tf-idf word2vec

modeling (modeling)  Similarity algorithm   Classification algorithm

Text preprocessing ：

1: Remove the non text part of the data

Regular expressions remove unnecessary symbolic punctuation ：clearn = re.compile(‘<.*[email protected]>’)

1. participle

english ：split()

chinese ：pip install jieba

2. Remove the stop words

english ： install nltk

chinese ： Construct a Chinese stop list by yourself  1208 individual

3. English words

stemming Stem extraction   lemmation Word pattern reduction

Use nltk Of wordnet

4. English words are converted to lowercase

word=word.lower()

5. Feature handling

bag of words The word bag model (bow ,,  tf-idf)

n-gram Language model (bigram, trigram)

word2vec Distributed model

Two ：RNN stay NLP The use of （Recurrent Neural Network)

Original reference ：https://zhuanlan.zhihu.com/p/40797277

The difference with N-gram RNN You can see the whole sentence before and after , and 2-gram.3-gram  Wait, you can only see a part of a sentence , So the error is relatively large .

Basic cyclic neural network ：

Input layer — Hidden layer — Output layer     You can look forward to any output value

Traditional neural networks （ Include CNN), Input and output are independent of each other , For example, the cat and dog in a picture are separated , But the subsequent output of some tasks is related to the previous content , Local information is not enough to make subsequent tasks continue .RNN It is a neural network that needs the information of the previous or previous sequence to make the task go on .RNN introduce ‘ memory ’ The concept of , loop 2 The word comes from the fact that each of its elements performs the same task , But output depends on input and ‘ memory ’. The structure shown below ：

Two way recurrent neural network （ English Cloze depends not only on the words in front of it, but also on the words behind it ）

Deep loop neural network （ Hidden layers stack more than two hidden layers ）

3、 ... and ： Recursive neural network （Recursive Neural Network)

RNN The actual effect is similar to CNN Not much difference , But the training speed ratio CNN Too slow

Four ： Why is it RNN Add LSTM http:// https://zhuanlan.zhihu.com/p/40797277

As the time interval increases ,RNN Will lose the learning connection to far away information , That is, the memory capacity is limited ,LSTM Our memory cells are transformed , Things to remember （ If you input something new ） Will always be passed on , What should not be remembered will be cut off .

RNN Cell structure ：

LSTM Cell structure ： stay RNN On the basis of

First step ：forget 

The second step ：update

The third step ：output 

 LSTM Several variants of