Patent | subject classification method based on graph convolution neural network fusion of multiple human brain maps

Hello,
Hello everyone , This is a brain cloud , I am a Ting Zhang~
Today I will study with you “ A subject classification method based on graph convolution neural network fusing multiple brain maps ” This patent . It is a patent invented by Zhang Xin and Liang Chengbo of South China University of Technology .

This invention discloses a subject classification method based on graph convolution neural network to fuse various brain maps , The human brain atlas is a data structure , It represents the interactive information between different brain regions in the human brain , By identifying five kinds of brain atlas of the subject, we can classify and predict the subject , It belongs to the research field of brain science and deep learning .

The steps of this classification method are as follows ： Obtain the timing signal of human brain fMRI and preprocess ; According to different functional connection strength calculation methods, five types of human brain maps are constructed for each sample to obtain five data sets ; Construct five graph convolution neural network classifiers ; Train on the corresponding human brain atlas data set , So as to obtain the binary classification ability of specific human brain atlas ; Synthesize the prediction results of five graph convolution neural network classifiers , Classify and predict the subjects , That is, predict what kind of people the subjects belong to .

We will start from the background technology 、 The content of the invention 、 Advantages and effects of three aspects to interpret , Let's study together ~

Background technology

Functional magnetic resonance imaging is a fast imaging technology , When a certain part of the brain is active , Increased blood flow , It leads to the enhancement of fMRI signal , So fMRI (fMRI) Technology is widely used to detect blood oxygen activity in the brain , Then test the functional activity changes of relevant brain regions . A large number of experimental results show that some individual characteristics such as age are related to the individual's functional connection network , Therefore, the functional connection network of human brain is closely related to physiological characteristics , Yes fMRI The analysis of brain functional connectivity extracted from has become a popular method to classify individual features .

The picture is from ： Figure worm creative

Many indicators for calculating correlation are used to measure the strength of functional connections between brain regions , For example, correlation coefficient 、 Sparse representation 、 Clustering coefficient 、 Statistical characteristics 、 Causal characteristics, etc , Among them, the correlation coefficient method is simple and effective , It is widely used in the measurement of functional connection strength , So as to build human brain atlas and model human brain functional network .

at present , In the research of ADHD recognition based on functional connection network , Mainly for the use of a single functional connection method , Because various connection measurement indicators have their advantages and disadvantages , For some sample data , Using only a single functional connection strength measurement method is easy to fall into the inherent shortcomings of this measurement method, resulting in identification difficulties . For example, the correlation coefficient method is easily affected by the signal-to-noise ratio , Statistical characteristics lack the connection in time dimension , The parameters of causality are sensitive , It is easy to have a great impact on the calculation results of causality .

lately , In various fields, machine learning methods show excellent fitting ability to complex data , Therefore, machine learning methods are introduced into the research of brain science images , And achieved excellent results . At present, most are based on fMRI The research adopts traditional machine learning algorithm to classify , However, it is difficult to process high-dimensional medical imaging data , So before entering the model , First, the samples are screened for characteristics , This also leads to the problem that the classification performance of the model depends too much on feature screening , Therefore, the generalization performance of traditional machine learning methods in brain science research is limited .

Some studies have used graph convolution neural networks to classify human brain atlas , But the research is still in the process of exploration , The process of extracting node features lacks the connection between different levels .

The picture is from ： Figure worm creative

thus it can be seen , The disadvantages and deficiencies of the existing technology are as follows ：

1、 At present, most of the existing classification studies based on human brain functional network only use a single index of functional connection strength , Susceptible to its inherent shortcomings , It limits the classification performance of the classification model .

2、 Most of the existing human brain functional network classification methods need feature screening process , When the physiological mechanism is not clear , The classification accuracy and generalization performance of the model are easily limited by the feature screening process .

3、 Graph convolution neural network classification model , Lack of consideration of the relationship between nodes at different levels .

The content of the invention
The invention aims to solve the above defects in the prior art , This paper provides a subject classification method based on graph convolution neural network to fuse multiple brain maps , This classification method uses five graph convolution neural network classifiers to classify five kinds of human brain maps of subjects , And vote on all the predicted results , So as to obtain the final prediction result , That is, predict what kind of person the subject belongs to .

The graph convolution neural network classifier in the invention can be used to classify the human brain atlas of adults and children , Classification of men and women , But not limited to the above classification tasks .

The subject classification method based on graph convolution neural network to fuse a variety of human brain maps comprises the following steps ：

Flow chart of subject classification method based on graph convolution neural network fusing multiple brain maps

STRP1 Obtain the data set of human brain fMRI timing signals , Each sample in the data set is a collection of fMRI timing signals in each brain region of a subject's brain , Express it as x, The... In this sample i The time series of brain regions are expressed as xi, Describe the activity state of this brain area within a certain time range , Preprocess the data set , among , The pretreatment is to make sample labels and balance the number of samples between classes ;

STRP2 Construct five brain maps for each sample in the data set , They are the functional network Atlas of low-order human brain LON、 Map of high-order human brain functional network HON、 Hybrid high-order human brain functional network atlas HHON、 be based on KS Tested human brain function network map KSN、 The functional network map of human brain based on Granger causality test GN, The above five brain maps reflect five different types of functional connection information in the human brain , Thus five data sets for training graph convolution neural network classifier are obtained , Expressed as DLON、DHON、DHHON、DKSN、DGN,

among ,LON Used to describe the temporal correlation between brain regions ,HON It is used to describe the structural correlation between low-order subnetworks ,HHON It is used to describe the structural correlation between low-order subnetworks and high-order subnetworks ,KSN According to the statistical distribution characteristics of brain time series, we can measure whether there is a connection between different brain regions ,GN According to the causal relationship between the time series of different brain regions, we can measure whether different brain regions establish connections ;

STRP3 Construct five graph convolution neural network classifiers to classify five kinds of human brain maps , Input the functional connection matrix of a specific human brain map into the graph convolution neural network classifier , So as to obtain the probability of predicting the human brain atlas as positive samples and negative samples ,

among , The five graph convolution neural network classifiers are represented as LON classifier 、HON classifier 、HHON classifier 、KSN classifier 、GN classifier , Each picture volume Jishen meridian classifier is composed of 5 Layer multiscale messaging layer MS_MP、3 Layer by layer node feature extraction layer CLN、2 Layer global node feature extraction layer GNR、1 Layer graph feature extraction layer GFR as well as 1 Layer full connection layer FC constitute , Among them, the multi-scale messaging layer MS_MP Process the input human brain function connection matrix , According to the multi-scale topology information of the nodes at both ends of the connection edge, the weight of the connection edge is updated , After multi-layer iteration, the connecting edge can obtain more abstract function information ;

Cross layer node feature extraction layer CLN Two adjacent multi-scale messaging layers MS_MP Output connection matrix as input , And the eigenvector of the node is aggregated as the output , The use of cross layer feature extraction strategy increases the diversity of node feature information , This vector is called cross layer node eigenvector ;

Global node feature extraction layer GNR Aggregate multiple cross layer node features , After two iterations , Generate the global eigenvector of each node , It is called global node eigenvector ;

Graph feature extraction layer GFR Aggregate the global node eigenvectors , Generate graph eigenvectors ;

Fully connected layer FC Take the graph feature vector as input , Output probability vector , Indicates the probability of predicting the human brain atlas as positive samples and negative samples ;

Figure structure diagram of convolution neural network classifier , The graph convolution neural network classifier is used to classify specific human brain functional networks , Classify it as positive sample or negative sample
STRP4 For steps S3 The five graph convolution neural network classifiers obtained in are trained , take LON classifier 、HON classifier 、HHON classifier 、KSN classifier 、GN The classifiers are in DLON、DHON、DHHON、DKSN、DGN Training on five data sets , Cross entropy is used to measure the error between the predicted output and the training label , The random gradient descent method is used to optimize the cross entropy error , So as to improve the classification ability of each classifier for a specific human brain map ;

STRP5 Follow the steps S4 Five graph convolution neural network classifiers in the output probability vector to vote , Each probability vector contains the prediction probability of positive samples and negative samples , Choose the category with the maximum prediction probability to vote , Finally, the category with the largest number of votes is used as the prediction classification result of the subjects .

A frame diagram for predicting and classifying subjects using a variety of brain maps , Synthesize multiple classifiers to analyze a variety of human brain maps of subjects , Realize the information complementarity of different brain maps , Obtain objective and comprehensive prediction results

Advantages and effects
Compared with the prior art, the invention has the following advantages and effects ：

1. The five human brain maps used by the invention model the human brain functional network , It reflects different types of functional connectivity information in the human brain , Increase the diversity of human brain characteristics , At the same time, analyze the human brain from multiple perspectives , Make various types of functional connectivity information complement each other , Make the classification results more objective and accurate .

2. The graph convolution neural network classifier used in the invention is an end-to-end model , Establish a mapping relationship between the input human brain map and the output tag , Make each component of the classifier closely connected , Compared with artificial features , It can make the classifier model adaptively extract features that are highly relevant to the classification task , It is conducive to improving the accuracy of classification .

3. When the graph convolution neural network classifier used by the invention updates the connection edge of the connection matrix , Use the strategy of multi-scale messaging , Make the updated connection edges aggregate the topology information of different sizes of nodes at both ends , Make the feature information of the connecting edge richer .

4. The graph convolution neural network classifier used in the invention uses the strategy of cross layer node feature extraction in the process of extracting node features , It is beneficial to enrich the information of node characteristics , At the same time, the flow of feature information in the multi-scale messaging layer and cross layer node feature extraction layer is more sufficient , Give better play to the performance of graph convolution neural network classifier model .

That's all for this sharing , Welcome to your attention , Let's study other patents in artificial intelligence ~
Reference material ：
[1] Zhang Xin , Liang Chengbo . A subject classification method based on graph convolution neural network fusing multiple brain maps : China ,CN111563533A. 2020-08-21.

• END -

edit | Ting Zhang
proofreading ｜ Elder sister miaojun