[data processing] boxplot drawing

1. summary

boxplot Is in 1977 By American statistician John · tukey (John Tukey) Invented , It contains five basic elements ：

• minimum value （Q0, The first 0 The quartile ）： The minimum value after removing outliers
• Maximum （Q4）： The maximum value after removing outliers
• Median ： All values in the sample are arranged from small to large, and then 50% The number of
• First quartile (Q1)： also called “ Lower quartile ”, It is equal to the number of all values in the sample arranged from small to large 25% The number of .
• third quartile (Q3)： also called “ Larger quartile ”, It is equal to the number of all values in the sample arranged from small to large 75% The number of .

In addition to the above 5 Two basic elements , Four minute spacing （InterQuartile Range,IQR） It is also often used for boxplot Construction , It is defined as the difference between the third quartile and the first quartile , namely IQR=Q3-Q1.

boxplot The figure of is as follows , It consists of boxes （box） And whiskers （whiskers ） constitute , among , The upper and lower boundaries of the box are Q3、Q1 constitute , The middle of the box is separated by the median . Must be （whiskers ） There are many variations on the upper and lower boundaries of .

A standard must be defined as ： The maximum and minimum values in the dataset ;
There is also ： Adopt upper limit =Q3+1.5IQR, Lower limit =Q1-1.5IQR, As the upper and lower boundaries of whiskers , Points outside this boundary are considered outliers .
In addition, there are, for example, the following ways to define boundaries ：

• The minimum and the maximum value of the data set
• One standard deviation above and below the mean of the data set
• The 9th percentile and the 91st percentile of the data set
• The 2nd percentile and the 98th percentile of the data set
  

2. Calculation

Premise ： Sort the data

Set the data length as n

1. Median Q2： It is the calculation of the median in the general statistical sense （n Take the middle of odd numbers ,n Average the middle two values for even numbers ）
2. First quartile Q1: Calculate the first quarter position pos, Then calculate the value of the first quartile . among , There is no unified standard for the calculation of the first quartile position , But there are usually two ways to calculate ： Mode one ：$pos=1+\frac{n-1}{4}$; Mode two ：$pos=\frac{n+1}{4}$. The quartile value is calculated as a simple linear interpolation .
3. third quartile Q3: Calculate the third and fourth points pos, Then calculate the value of the third quartile . among , There is no unified standard for the calculation of the third quartile position , But there are usually two ways to calculate ： Mode one ：$pos=1+\frac{3\ast (n-1)}{4}$; Mode two ：$pos=\frac{3(n+1)}{4}$. The quartile value is calculated as a simple linear interpolation .

3. Example （ Take mode 1 as an example ）：

The data is ：num=[1,2,3,4,5,6,7,8], Data length n=8

• Median Q2=(num[4]+num[5])/2=(4+5)/2=4.5
• First quartile Q1: $pos=1+\frac{n-1}{4}=2.75$;$Q1=num[1]+0.75\ast (num[2]-num[1])=1+0.75\ast (2-1)=2.75$
• third quartile Q3：$pos=pos=1+\frac{3\ast (n-1)}{4}=6.25$;$Q3=num[6]+0.25\ast (num[7]-num[6])=6+0.25\ast (7-6)=6.25$

Code ：
Median 、 First quartile Q1、 third quartile Q3, The calculation process is shown above ,pandas The display of Zhongxu is ： utilize Q3+1.5IQR、Q1-1.5IQR Identify outliers , Take the maximum and minimum of the remaining values as the upper and lower bounds ：

import pandas as pd
num =[1,2,3,4,5,6,7,8]
df = pd.DataFrame(num)
boxplot = df.boxplot()
print(df.describe())

---

import pandas as pd
num =[-5,2,3,4,5,6,7,13]
df = pd.DataFrame(num)
boxplot = df.boxplot()
print(df.describe())

Reference resources ：wiki