Transfer Learning

Labeled target data and source data

Model Fine-tuning

​ Task describe :

​

​ characteristic Target data Very few

be called One-shot Learning

example:

Speech recognition ,

The voice assistant will say a few words and then

Processing mode

Conservation Training

a large number of source data, To initialize another network Parameters of , And then use target data Fine tuning parameters , be prone to overfitting

new network Equivalent to the old regularization

Training limitations ：

1. Just adjust one layer Parameters , Prevent over fitting

2. fine-tune the whole network

Which one layer?

Speech recognition ： Adjust the first general , near input Of layer

IMAGE: Fixed at the front , transfer Output A few nearby Layer The previous basic feature extraction

fine-tune + Transfer The best effect

Multitask Learning

task a & b Share the same feature:

feature all Cannot share ：

Do something in the middle transform

Select the appropriate relevant task

example

Speech recognition , The sound signal is thrown in , Translate into human language ：

Together train

Progressive Neural Network

Study first task A then task B

Will you learn B It will affect task A Well ？

Blue output Enter to green input As another task The input of , But again BP It won't be blue when it is , Blue lock

If more task

---

Unlabeled Target data & Labeled Source Data

For example, handwritten numeral recognition

other image, No, label:

One is train One yes test, It won't work very well , because mismatch

data Of Distribution Is not the same

Domain-adversarial Training

hold source and target Go to the same domain Handle

feature There is no intersection at all

Need feature extractor Try to remove source target Of Different

Cheated domain classifier, It's easy ,Why?

green all output 0 That's it To add more Label predictor Need to meet

Domain classifier fails in the end

It should struggle !

Zero-shot Learning

There may be some target stay source Inside Never happened

Speech recognition ： Using phonemes （ Phonetic symbols ）, Not in words

***Representing each class by its attributes !*** Find unique attributes

Training:

Judgment properties , Not the last direct classification

x1 x2 Through one f Mapping to embedding space then Corresponding properties y1 y2 Also through g Map to the above , If a new one enters X3 The same method can still be used The goal is the result f g As close as possible

But what about x-attributes It is estimated that you may have to rely on database support

modify The farther away from the irrelevant, the better ：

K be called margin max(0, Back ) Rear greater than 0 Just can have loss <0 No, loss, When ？ When didn't loss ?

inner product

There is no attribute ： use word vector

Return to zero Learning:

Carry out a combination, It's in the middle ？ Identify things you've never seen

Unlabeled Source and Labeled Target Data

self-taught learning

Similar to semi supervised learning , There is a big difference ：data It may be irrelevant It's a different kind of

Unlabeled Source and Unlabeled Target Data

Self-taught Clustering

led Target Data

Self-taught Clustering