Matlab sound classification based on short-time neural network

This example shows how to use the deep learning process to classify sounds .

1、 Dataset generation

Generate 1000 A white noise signal 、1000 A brown noise signal and 1000 A pink noise signal . Suppose the sampling rate is 44.1 kHz, Each signal indicates 0.5 Second duration .

fs = 44.1e3;
duration = 0.5;
N = duration*fs;

wNoise = 2*rand([N,1000]) - 1;
wLabels = repelem(categorical("white"),1000,1);

bNoise = filter(1,[1,-0.999],wNoise);
bNoise = bNoise./max(abs(bNoise),[],'all');
bLabels = repelem(categorical("brown"),1000,1);

pNoise = pinknoise([N,1000]);
pLabels = repelem(categorical("pink"),1000,1);

Copy

2、 Data visualization

Listen to the sound signal , And use melSpectrogram Function to visualize it .

sound(wNoise(:,1),fs)
melSpectrogram(wNoise(:,1),fs)
title('White Noise')

Copy

sound(bNoise(:,1),fs)
melSpectrogram(bNoise(:,1),fs)
title('Brown Noise')

Copy

sound(pNoise(:,1),fs)
melSpectrogram(pNoise(:,1),fs)
title('Pink Noise')

Copy

3、 The data set is divided into training set and verification set

Create a by 800 A white noise signal 、800 A brown noise signal and 800 A training set of pink noise signals .

audioTrain = [wNoise(:,1:800),bNoise(:,1:800),pNoise(:,1:800)];
labelsTrain = [wLabels(1:800);bLabels(1:800);pLabels(1:800)];

Copy

Use the rest of 200 A white noise signal 、200 A brown noise signal and 200 A pink noise signal creates a verification set .

audioValidation = [wNoise(:,801:end),bNoise(:,801:end),pNoise(:,801:end)];
labelsValidation = [wLabels(801:end);bLabels(801:end);pLabels(801:end)];

Copy

4、 Signal extraction

Audio data is high dimensional , Usually contains redundant information . By first extracting features , Then use the extracted features to train the model , It can reduce the dimension . establish audioFeatureExtractor Object to extract mel The centroid and slope of the spectrum over time .

aFE = audioFeatureExtractor("SampleRate",fs, ...
    "SpectralDescriptorInput","melSpectrum", ...
    "spectralCentroid",true, ...
    "spectralSlope",true);

Copy

call extract Extract features from audio training data .

featuresTrain = extract(aFE,audioTrain);
[numHopsPerSequence,numFeatures,numSignals] = size(featuresTrain)

Copy

5、 Data preparation

In the next step , You will treat the extracted features as sequences , And use sequenceInputLayer As the first layer of the deep learning model . When using SequenceInputLayers As the first layer in the network ,trainNetwork You want to format the training and validation data into a sequence of cell arrays , Each sequence is composed of eigenvectors over time .sequenceInputLayer The time dimension is required to follow the second dimension .

featuresTrain = permute(featuresTrain,[2,1,3]);
featuresTrain = squeeze(num2cell(featuresTrain,[1,2]));

numSignals = numel(featuresTrain)
numSignals = 2400

[numFeatures,numHopsPerSequence] = size(featuresTrain{1})
numFeatures = 2

numHopsPerSequence = 42

Copy

The extracted features

featuresValidation = extract(aFE,audioValidation);
featuresValidation = permute(featuresValidation,[2,1,3]);
featuresValidation = squeeze(num2cell(featuresValidation,[1,2]));

Copy

6、 Define and train networks

Define network architecture .

layers = [ ...
    sequenceInputLayer(numFeatures)
    lstmLayer(50,"OutputMode","last")
    fullyConnectedLayer(numel(unique(labelsTrain)))
    softmaxLayer
    classificationLayer];

Copy

To define the train Options , Please use option Options （ Deep learning toolbox ）.

options = trainingOptions("adam", ...
    "Shuffle","every-epoch", ...
    "ValidationData",{featuresValidation,labelsValidation}, ...
    "Plots","training-progress", ...
    "Verbose",false);

Copy

To train the network , Please use trainNetwork（ Deep learning toolbox ）.

net = trainNetwork(featuresTrain,labelsTrain,layers,options);

Copy

7、 Verify the network

Use a trained network for new white noise 、 Brown noise and pink noise signals are classified .

wNoiseTest = 2*rand([N,1]) - 1;
classify(net,extract(aFE,wNoiseTest)')
ans = categorical
     white


bNoiseTest = filter(1,[1,-0.999],wNoiseTest);
bNoiseTest= bNoiseTest./max(abs(bNoiseTest),[],'all');
classify(net,extract(aFE,bNoiseTest)')
ans = categorical
     brown


pNoiseTest = pinknoise(N);
classify(net,extract(aFE,pNoiseTest)')
ans = categorical
     pink

Copy