Speech synthesis: overview [generation task of unequal length sequence relation modeling]

One 、 What is speech synthesis ？

Speech synthesis is a “ Generation task of unequal length sequence relation modeling ”

• Input ：【tex len 】; Input ：【frequency dim, spectrum length】
• Enter shape ： Text token Sequence length ; Shape of the output ：( Frequency dimension , Spectrum sequence length )

“ 739 ”5 individual “token” Corresponding 20 Multiple voices “ frame ”

Cannot model alone “ 7、 ... and ” And X The relationship between frames ,“ hundred ” And Y The relationship between frames ,...., And then put them together , This is against the nature of human pronunciation .

Two 、 Basic training framework of speech synthesis

1、 Introduction to training data

Sampling rate = 16000

2、Token Embedding Layer

To map characters to floating point numbers ,pytorch To take the “ Trainable query table ” The way , Set the... Contained in the dataset token Number &