ConvNets Principles

1. Cross-Entropy vs. Hinge

2. Compute the Gradient

3. A Single Hidden Layer Neural Network

A neural network is just a bunch of individual neurons . One can think about the function of activation as an indicator of the existence of certain weighted combinations . then , We can use a combination of these activations to perform classification tasks .

4. Objective Funciton

Let's take the hinge loss as our objective function . If we count scores “true” The tag data is calculated as s, The score is calculated as “false” Mark data as sc. So the optimization goal is ：

5. The Big Picture of Backpropgation

（1） Each edge between two neurons in the network is associated with
Rate factor , It's just a partial derivative of neuron activation with respect to the other side ;
（2） The rate factor of the path is the product
The rate factor along the path ;
（3） And the sum of the rate factors of all possible paths of the total rate of change .