The difference between Tansig and logsig. Why does BP like to use Tansig

  Original article , Reprint please indicate from 《 Old cake explains neural networks 》:bp.bbbdata.com

About 《 Old cake explains neural networks 》：

This website structurally explains the knowledge of Neural Networks , Principle and code .

repeat matlab Algorithm of neural network toolbox , It is a good assistant for learning neural networks . 

---

Catalog

01. Formula analysis

02.  Characteristic analysis

03.  Derivative Analysis

My guess

---

Why? BP Neural networks are generally used tansig, I believe this is the confusion of many people .

We might as well analyze tansig and logsig Properties of 、 characteristic 、 Derivative and so on ,

Try to find out why they tend to use tansig Why .

01. Formula analysis

---

The formula

tansig and logsig The formula is as follows ：

analysis

From the formula of both , There is not much difference between the two ,

tansig It's just logsig Perform stretching and translation operation on the basis of .

Both rely on exponential calculations , There is no difference in computational complexity .

therefore , On the formula level , It does not constitute a tendentious choice tansig The reason of .

---

02.  Characteristic analysis

---

characteristic

When tansig When the independent variable is one dimension , It's a S Shape curve .
●  Its value range is  (-1,1)                                           
● tansig The nonlinear part mainly focuses on 【-1.7,1.7】 Between ,
●  stay 【-1.7,1.7】 Outside ,tansig Gradually tend to saturation .          

---

When logsig When the independent variable is one dimension , It's a S Shape curve .
●  Its value range is  (0,1)                                            
● logsig The nonlinear part mainly focuses on 【-1.7,1.7】 Between ,
●  stay 【-1.7,1.7】 Outside ,logsig Gradually tend to saturation .          

analysis

From the comparison of characteristics , We have not found any qualitative difference between the two ,

because tansig Will be logsig Stretch , Translation to 【-1,1】 The value range of .

I didn't find much difference in features ,

The only difference is , The two values are different .

03.  Derivative Analysis

---

derivative

tansig The derivative of is ：

---

logsig The derivative of is ：

analysis

Through the comparison of derivatives ,

Both of them can use their own value to obtain the derivative value ,

The amount of calculation is also consistent ,

therefore , On derivative tansig There is no greater advantage ,

Does not constitute a tendency to use tansig Why

The author's view

---

Through the above analysis , We can hardly see tansig Than logsig What are the advantages of .

Then why use tansig Well ？

The author's view is ,

One 、 Unified input range .                  

Two 、 Make full use of the active interval of the activation function

---

We know , The input of the upper layer is the output of the lower layer ,

and tansig and logsig The active range of is 【-1.7,1.7】 Between ,

In the input layer , We will undoubtedly normalize the input to 【-1,1】,

It is more effective for using the active interval of the activation function of the first hidden layer .

---

And use tansig, In the case of multiple hidden layers ,

Output of each layer , That is, the input of the lower layer is still 【-1,1】

In this way, the input range of each layer is unified ,

And they all make effective use of the active interval of the activation function .

---

Unity is very beneficial ,

At least in theoretical research , It can bring a lot of convenience ,

Otherwise, we need to discuss the input layer and hidden layer respectively .

The above is the author's view , Because there is no literature research , For reference only .