Introduction to common activation functions

1.sigmoid function

The formula is as follows

The image below .

sigmoid Functional y Always in [0,1] Between , When x Greater than 0 when ,y Greater than 0.5. When doing simple binary tasks , It can be set at [0,1] A certain value between is used as the threshold , The result of exceeding the threshold , We fall into one category , Otherwise they fall into another category .

2.softmax function

if sigmoid The function is to solve the problem of binary classification , that softmax Function is to solve the multi classification problem . As shown in the figure above , Input is [z1,z2,z3], The output is [y1,y2,y3], The sum of the three numbers is 1.

3.relu function

When x<=0 when ,y=0, When x>0,y=x.

4.tanh function

tanh Functions and sigmoid The function is very similar to , however tanh Of y Value range is [-1,1].