Brief introduction of prompt paradigm

Prompt Brief description of paradigm

Traditional Framework:

• pre-train
• fine-tune

The traditional training framework is , First, pre train the model on a large-scale data set , Then fine tune the data set of the target task .

Prompt Framework

• pre-train
• prompt
• predict

Prompt The framework is divided into three parts , Preliminary training ,Prompt Generate , And prediction

Goal: Let the pertained model itself can be used to predict the desired output without any task-specific training.

Prompt In essence, it is to transform task data , Put the original goal 、 label , Make a conversion , Integrate into the data .

For example, the emotion label of the sentence “I won the game.” is good.

We can also get a longer sentence “I won the game, so I felt good.”

The above example is , Label good Convert to additional statements , Add after input .

Such a transformation makes , We learn through natural self-monitoring , You can achieve the goals required by the task

Prompt Basics

Generally speaking ,Prompt There are three steps ：

1. Prompt Addition
2. Answer Search
3. Answer Mapping

Prompt Addition

This step is actually to modify the input

such as [X] Overall, it was a [Z] movie Form like this

We fill in the input to [X] The location of , Returns an entire statement , Set aside [Z] The location of , Wait for the answer to fill .

Answer Search

$$\hat{z}={\text{search}}_{z\in \mathcal{Z}}P(f_{\text{fill}}(x^{\prime },z);\theta )$$

As shown in the above formula , In this step , Here's what we're trying to do , Fill in the most likely answer .

Answer Mapping

Match the searched answer with the output value

That is, the answer that will be filled , Map to the final output , It could be a label , It may also be the corresponding sentence directly .