Chapter7-13_ Dialogue State Tracking (as Question Answering)

This article is for teacher lihongyi 【Dialogue State Tracking (as Question Answering)】 My course notes , Course video youtube Address , spot here ( Need to climb over the wall ).

The pictures used in the following paragraphs are all from Mr. lihongyi PPT, If there is infringement , Must delete .

Article index ：

Part 1 - 7-12 Controllable Chatbot

The next part - nothing

General catalogue

1 What is? Dialogue State Tracking

Dialogue State Tracking yes Task-oriented Dialogue A submodule . Let's talk about it first Task-oriented Dialogue What is it? .Task-oriented Dialogue The input of is the current words spoken by the user and the historical dialogue record , If the user's input is voice , There will be an extra ASR The module converts input into text , There will also be an extra TTS The module converts the output to voice . The user's input goes through an understanding module NLU(Natural language understanding), The output will be given to state tracker, That is the core content of this paper .state tracker It will extract useful information from the current input , preserved . At the same time, historical input will save useful information in the same form . The stored information will be in the form of state To the decision-making module (Policy).Policy Will tell the generation module NLG(natural language generation) What should be output . Last NLG The output responds to the user's words .

Said a lot , In fact, to put it simply Dialogue State Tracking It's up there state tracker, The goal is to simplify what users say into a useful information base , It can also be understood as a summary . The following figure vividly illustrates this process . So many conversations , It turns out to be key-value The key/value pair .

these keys It is the content defined in advance , and value It's all multiple choice questions , It is also a well-defined option . Because of the present Dialogue State There are often many keys, therefore key There will be domain and slot Two parts , Used to distinguish and classify different keys, Convenient for model processing , It is also about the migration of models .

All in all ,Dialogue State Tracking According to the conversation , Fill in the preset keys Of values It's a question of what .

2 Data sets

Dialogue State Tracking Some of the most famous datasets are MultiWOZ 2.0,SGD,CrossWOZ wait . This one here WOZ From the wizard of oz (The Wizard of Oz), The final big in the wizard of oz BOSS An old man was behind the scenes . The data set here is also disguised as AI The data from talking to people , That's right. It's people talking to people ,AAI(Artificial Artificial Intelligence).

3 Two challenges

Dialogue State Tracking Of values There are many options to make because the previous schemes regard this as a classification problem , The output is a category , But like the phone number , Time and other options are hard to enumerate , This method doesn't work , This is a challenge .

Another challenge is to have new keys Come in , The model will be re train too , The data has to be reconstructed , Migration costs are high .

To sum up ：

• There are some value Inexhaustible
• new key Come in , Migration costs are high

These two questions , Both can be used. QA(Question Answering) How to solve this problem .QA The structure of is shown in the figure below , We put Dialogue history As source Input , And then according to key Ask the appropriate questions and let the model answer them . So the model goes back from source One of them span As the answer , It's solved value An inexhaustible question . And then for domain Different ,slot Same question , Change the problem domain that will do , Solved the problem of migration . Of course, the premise is this QA The model is ok Of .

4 Classic models

Dialogue State Tracking The classic model is TRADE, The structure shown below .Dialogue Will pass by a encoder Extracting features , meanwhile key Of domain and slot I'll take it separately embedding, Then add them together as question Part of the input , It will also be with Dialogue Of encoder Output as a attention.attention The result will be a slot gate, It was decided that PTR（ Extract one from the conversation span）, still DONTCARE（ Users don't care ）, still NONE（ I haven't mentioned this yet key）. According to this decision ,state generator The final result will be output .

because key Will be extracted separately domain and slot Of embedding, So this method can be applied to key Of , This also means that it can be solved zero-shot Problem. .

There are some key There will be some. description, Extract these description Of embedding It's also a solution zero-shot The way of the problem .

There are other variations , such as Slot Carryover Prediction I think that every word I say should put all keys It takes too much time and effort to ask again , So I designed a “ Same as above ” Of classifier To help avoid asking questions that have been known and have not been modified keys;DST QA Think some keys Between values It's related , For example, I know which hotel to book , Then the destination of the taxi is probably this hotel , Then a module is designed to solve this problem ;SimpleTOD be based on GPT-2 Designed a end-to-end Methods , The user said a word , The model outputs one sentence , The effect is surprisingly good .