Perceptron model and Application

** If there is a mistake , Thank you for correcting **

If there is a mistake , Thank you for correcting , Please send a private message to the blogger , There is a red envelope for hard work , Worship “ one-word teacher ”.

Please find the paragraphs you need according to the table of contents

Introduction ： This blog is for individuals to sort out the learning records of digital analog Algorithm , If there is a mistake , Thank you for correcting . System learning , Welcome to continue to pay attention , Follow up updates
Java communication qq Group 383245788

order

The purpose of this article is to record the experience of preparing for the personal digital simulation American competition . Reprint please indicate the source .
The perceptron model involved in this paper is only a brief introduction and digital analog usage , And the application scope is small , It's a side door

Perceptron model

perceptron Pereptron) It's a two class linear classification model , The input is the eigenvector of the instance , The output is the category of the instance , take +1 and 1. The perceptron corresponds to the input space ( The feature space ) The instance is divided into positive and negative separated hyperplanes , It's a discriminant model .
The history of machine learning

Perceptron model biological mechanism

Input: The vector of the instance , Each column - - Features
Weight: The weight , Set initial value , And use the gradient descent method to update
Bias: bias , Allow the classifier to move the decision boundary left and right , Contribute to better , Faster training model .
Weighted sum: The weighted sum is Input Vector and weight Product sum of
Output: Output as category , take +1,-1
In short , The perceptron model is After feature extraction , Multiplicative weight , Sum using gradient descent , Training , Add bias to assist classification , The last two points .

Mathematical model of perceptron model

Given the training data set :
T={ (x1,y1) ,(x2,)2)…(x.n,yn) }
among xi∈Rn, yi ∈{+1-1},i=1,2,.,.n,
Input x Represents the eigenvector of an instance , Points corresponding to the input space , Output y Represents the category of the sample ..
Then the perceptron model is :

The symbolic function is ：

Geometric interpretation

linear equation :wx+b=0
Corresponding hyperplane S,w For the normal vector ,b intercept , Separate positive and negative classes

Only two points can be solved , If the difference is not obvious or cross , It's hard to solve . Typical error driven algorithms .

Model solving

Input : Training data set T={(x1,y1),(x2,)2)…(xn,yn)}
among xi∈R, yi∈{-1,+1}, i=12…N
Learning rate η (0<n<=1) ;
Output : w,b
Perceptron model : f(x)=sign(wx+b)
(1) Select the initial value w0,b0
(2) Select data in the training set (xi,yi)
(3) If yi(w*xi+b)<=0
(4) Finding partial derivatives , Update parameters
(5) repeat (2) Until there are no misclassification points in the training set

Divisible

Hard to distinguish

matlab Code

function [W,b] = perceptron(X,y,Maxstep)
% Perceptron learning algorithm 
%W Is the weight vector to be solved ,b For deviation 
%X For input space , This time, take two-dimensional ,y For output space , The value is [-1,1]
[n,m] = size(X);% Solving the matrix X Size 
% Assign initial values to weights and deviations , Specify the learning step size 
W = zeros(m, 1);%W by m Row column vector 
b=0;% Set the initial value of the deviation to 0
mu= 0.5;% Set the learning step to 0.5,0<mu<=1
for step = 1:Maxstep% Iteratively update parameter values 
	miss_flag=true;% Set a flag bit to judge whether there are misclassification points 
	fori=1:n%
		if (i)*(X(;)*W+b))< =0% Judge whether the randomly selected point is a misclassification point 
			miss_ flag = false;% If the randomly selected point is misclassification point , Then set the flag bit to false
			% Update the weight and deviation parameters according to the gradient descent method 
			W = W + mu*y(i)*X(i;); 
			b= b + mu*y(i); .
		end
	end
	if miss_ flag == true% If the flag bit is true, Indicates that the number of misclassification points is 0, The algorithm has achieved correct classification , There is no need to continue iterating and updating , Jump out of circulation 
		break
	end
end

clc
% Specify the input space 
X = [3,3;4,3;1,1];
% Specify the output space 
y=[1,1,-1];
n = size(y,2);% solve y Column length of 
% Draw the instance points in the feature space 
forj = 1:n
	if y(j) == 1% Draw positive instance points 
		plot(X(,1),X(j,2),r*);
	end
	if y(j) == -1% Draw negative instance points 
		plot(XGj,1),X(j,2),b*);
	end
	hold on % This statement can draw a new image based on the original image , If you don't add it, you can only see the last one 
end
% Call the function to update the parameters 
[W,b] = perceptron(X,y,2000);
xlabel =linspace(0,5,500);
ylabel = -(W(1)/W(2))*xlabel -b/W(2);% take X(2) Look at the genetic variables ,X (1) As an independent variable , Therefore, the hyperplane drawn by the algorithm can be obtained 
plot(xlabel,ylabel);