Correlation - intra group correlation coefficient

1、 effect

Intra group correlation coefficient (ICC) It is a reliability coefficient index to measure and evaluate inter observer reliability and retest reliability . It can usually be used in the questionnaire to evaluate the test-retest reliability of an object to multiple samples over a period of time , Or judge the consistency test of a batch of objects on multiple samples .

2、 Input / output description

Input ： At least two or more quantitative variables or ordered categorical variables , Generally, the data required is the scale data
Output ： Whether the samples in the title of the designed questionnaire are consistent

3、 Case example

5 Judges will rate the same group of contestants , Measure whether their scores are consistent . Or measure retest reliability , For example, collect a certain judge's right for many times 100 Scores of contestants （ Or the average score of the same group of judges ）, Measure whether the correlation of each score is consistent .

4、 Modeling steps

First put forward ICC It's transformational Pearson The correlation coefficient , Assume n Data pair (,),(i=1,2,…,n)ICC And Pearson The correlation coefficients are defined as : 

The difference between the two formulas lies in the calculation of mean and standard deviation , r What we use x1 and x2 Respective mean and standard deviation , and ICC What we use x1 and x2 Combined mean and standard deviation . When each subject measured more ,ICC Approximately equal to

It can be explained as the percentage of intergroup variation in the total variation .
There are many different ICC statistic , These statistics do not estimate the same population parameters . Choose the right ICC It depends on the following three aspects , The chosen model is one-way model or two-way model; use single measure or average measure; choice absolute agreement or consistency.
​

1. choice one-way model or two-way model?
(1)one-way radom effects model: Single factor random effect model , Use when considering random individual effects . Each subject is scored by different randomly selected assessors or all subjects are scored by one assessor , It is used to test the hypothesis that the mean values of each subject are all equal , In fact, it cannot be tested x Test-retest reliability . If you are interested in the differences between subjects , This model can be used .
(2)two-way radom effects model: Two factor random effect model , When both individual effect and item effect are random . That is, all effects are independent of each other .
(3)two-way mixed effects model: Two factor mixed effect model , When individual effects are random , When the item effect is fixed, choose .
(2) and (3) At the same time, the effects of row variables and column variables are decomposed , It can be used to test the test-retest reliability . The difference between the two is whether the evaluator effect is a random effect or a fixed effect . If the assessor comes from all possible situations , Statistical inference is also limited to these cases , Then the evaluator effect is a fixed effect ; If evaluator is a sample randomly sampled from a theoretically infinite population , Statistical inference should also be extended to this population , Then the assessor effect is a random effect . Whether the column variable is a fixed effect or a random effect ,ICC The calculation results are the same , The difference is when interpreting the results (2) Medium ICC It can be extended to all possible evaluators , and (3) Medium ICC Limited to a given assessor .
​

2. choice single measure or average measure?
single measure ICC The analysis unit is the score of each evaluator , Give the reliability of a single rater's score , If a single rater is used for further research , Then use this measure ;average measure ICC The analysis unit is k The average of the scores of the evaluators , give k Reliability of raters' scores , If the study design uses the mean value of multiple raters' scores for each item .
​

3. choice absolute agreement or consistency?
absolute agreement and consistency The difference in definition is in the denominator . Yes absolute agreement, Considering the systematic error of evaluator , Column variance, namely evaluator variance, is retained in denominator variance , It is used to measure whether the evaluator gives the same absolute score to the subjects ; Yes consistency, Because the systematic error of evaluator is not considered , The evaluator variance is the column variance removed from the denominator variance , It is used to measure whether the rater's score is highly correlated .

among ,MSR Is the mean square of the row variable ;MSC Is the mean square of the column variable ;MSW Is the mean square of variation within the Group ;MSE Mean square error .
It is generally believed that the reliability coefficient is lower than 0.4 Indicates poor reliability , Greater than 0.75 Indicates good reliability , For quantitative data, higher ICC value .