Face recognition using BP neural network of NNET in R language

A few days ago, my girlfriend had a R Language homework , I asked for my help , Record here , I hope I can help you . Part of it comes from her report .

Data sets

Dataset use Yale_32x32.mat As an example , Yale face database , All pixels are 32×32 Grayscale image of .
contain 15 Personal face , Everyone has a different expression 、 Posture and light 11 Face images , common 165 A picture .

Use R Language reading gets ：

fea Is the characteristic , That is, the pixel value ,gnd Is the label , The picture of the person who belongs to .

nnet Use

Here are the official introductions and routines ：

Fit Neural Networks
Description
Fit single-hidden-layer neural network, possibly with skip-layer connections.

Usage
nnet(x, ...)

## S3 method for class 'formula'
nnet(formula, data, weights, ...,
     subset, na.action, contrasts = NULL)

## Default S3 method:
nnet(x, y, weights, size, Wts, mask,
     linout = FALSE, entropy = FALSE, softmax = FALSE,
     censored = FALSE, skip = FALSE, rang = 0.7, decay = 0,
     maxit = 100, Hess = FALSE, trace = TRUE, MaxNWts = 1000,
     abstol = 1.0e-4, reltol = 1.0e-8, ...)
Arguments
formula	
A formula of the form class ~ x1 + x2 + ...

x	
matrix or data frame of x values for examples.

y	
matrix or data frame of target values for examples.

weights	
(case) weights for each example – if missing defaults to 1.

size	
number of units in the hidden layer. Can be zero if there are skip-layer units.

data	
Data frame from which variables specified in formula are preferentially to be taken.

subset	
An index vector specifying the cases to be used in the training sample. (NOTE: If given, this argument must be named.)

na.action	
A function to specify the action to be taken if NAs are found. The default action is for the procedure to fail. An alternative is na.omit, which leads to rejection of cases with missing values on any required variable. (NOTE: If given, this argument must be named.)

contrasts	
a list of contrasts to be used for some or all of the factors appearing as variables in the model formula.

Wts	
initial parameter vector. If missing chosen at random.

mask	
logical vector indicating which parameters should be optimized (default all).

linout	
switch for linear output units. Default logistic output units.

entropy	
switch for entropy (= maximum conditional likelihood) fitting. Default by least-squares.

softmax	
switch for softmax (log-linear model) and maximum conditional likelihood fitting. linout, entropy, softmax and censored are mutually exclusive.

censored	
A variant on softmax, in which non-zero targets mean possible classes. Thus for softmax a row of (0, 1, 1) means one example each of classes 2 and 3, but for censored it means one example whose class is only known to be 2 or 3.

skip	
switch to add skip-layer connections from input to output.

rang	
Initial random weights on [-rang, rang]. Value about 0.5 unless the inputs are large, in which case it should be chosen so that rang * max(|x|) is about 1.

decay	
parameter for weight decay. Default 0.

maxit	
maximum number of iterations. Default 100.

Hess	
If true, the Hessian of the measure of fit at the best set of weights found is returned as component Hessian.

trace	
switch for tracing optimization. Default TRUE.

MaxNWts	
The maximum allowable number of weights. There is no intrinsic limit in the code, but increasing MaxNWts will probably allow fits that are very slow and time-consuming.

abstol	
Stop if the fit criterion falls below abstol, indicating an essentially perfect fit.

reltol	
Stop if the optimizer is unable to reduce the fit criterion by a factor of at least 1 - reltol.

...	
arguments passed to or from other methods.

Details
If the response in formula is a factor, an appropriate classification network is constructed; this has one output and entropy fit if the number of levels is two, and a number of outputs equal to the number of classes and a softmax output stage for more levels. If the response is not a factor, it is passed on unchanged to nnet.default.

Optimization is done via the BFGS method of optim.

Value
object of class "nnet" or "nnet.formula". Mostly internal structure, but has components

wts	
the best set of weights found

value	
value of fitting criterion plus weight decay term.

fitted.values	
the fitted values for the training data.

residuals	
the residuals for the training data.

convergence	
1 if the maximum number of iterations was reached, otherwise 0.

References
Ripley, B. D. (1996) Pattern Recognition and Neural Networks. Cambridge.

Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth edition. Springer.

See Also
predict.nnet, nnetHess

Examples
# use half the iris data
ir <- rbind(iris3[,,1],iris3[,,2],iris3[,,3])
targets <- class.ind( c(rep("s", 50), rep("c", 50), rep("v", 50)) )
samp <- c(sample(1:50,25), sample(51:100,25), sample(101:150,25))
ir1 <- nnet(ir[samp,], targets[samp,], size = 2, rang = 0.1,
            decay = 5e-4, maxit = 200)
test.cl <- function(true, pred) {
    true <- max.col(true)
    cres <- max.col(pred)
    table(true, cres)
}
test.cl(targets[-samp,], predict(ir1, ir[-samp,]))


# or
ird <- data.frame(rbind(iris3[,,1], iris3[,,2], iris3[,,3]),
        species = factor(c(rep("s",50), rep("c", 50), rep("v", 50))))
ir.nn2 <- nnet(species ~ ., data = ird, subset = samp, size = 2, rang = 0.1,
               decay = 5e-4, maxit = 200)
table(ird$species[-samp], predict(ir.nn2, ird[-samp,], type = "class"))

Next, I will introduce my usage ：

nnet <- nnet(train, targets.train, size = 10, rang = 1, decay = 1e-3,
                maxit = 200, MaxNWts=30000) 

Among them train It refers to the training data , That is, the normalized pixel value , targets.train Is with the train Corresponding unique heat code label ,size Is the number of hidden layers ,rang Is the data value range of the training set ,decay Is the weight attenuation coefficient ,maxit Is the maximum number of iterations ,MaxNWts It seems to be the maximum parameter quantity .

Example of single hot code conversion ：

Complete code

The complete code is as follows ：

library("R.matlab")


path<-("F:\\R_language")
pathname <- file.path(path, "Yale_32x32.mat")
datatr <- readMat(pathname)

feattr=datatr[[1]]
feattr = feattr / 127.5 - 1 # normalization 
datatrnum=datatr[[2]]
result <- array(c(0),dim = c(165,15))#  Into a single hot code 
cnt <- 1
repeat {
  result[cnt, datatrnum[cnt, 1]] <- 1
  cnt <- cnt+1
  
  if(cnt > 165) {
    break
  }
}


set.seed(521)
##70% As a training model 
samp <- c(sample(1:165,135))
train <- feattr[samp,]
targets.train <- result[samp,]
validation <- feattr[-samp,]
targets.validation <- result[-samp,]
library(nnet)
nnet <- nnet(train, targets.train, size = 10, rang = 1,decay = 5e-4,
                maxit = 200, MaxNWts=30000)
summary(nnet)

true = max.col(true)
cres = max.col(pred)
table(true, cres)
}
test.cl(targets.validation, predict(nnet, validation))
cres = max.col(predict(nnet, validation))
true = max.col(targets.validation)
plot(x=cres,y=true,col="red",xlab=" Predictive value ",ylab=" True value ")
cnt =1
correct = 0
repeat {
if(cres[cnt]==true[cnt])
{
correct <- correct + 1
}
cnt <- cnt+1
if(cnt > length(cres)) {
break
}
}
correct/length(cres)# Accuracy rate 

Training process and results

Because the data is randomly selected 70% As a training set , So each training set and test set are different , So the result will be different .
Training 200 individual iter, During the training loss as follows ：

 weights:  10415
initial  value 432.348356 
iter  10 value 135.657129
iter  20 value 131.287453
iter  30 value 127.033601
iter  40 value 118.414307
iter  50 value 111.639129
iter  60 value 106.634401
iter  70 value 99.618272
iter  80 value 95.067623
iter  90 value 91.453289
iter 100 value 88.281143
iter 110 value 77.366638
iter 120 value 57.478618
iter 130 value 50.865985
iter 140 value 46.196341
iter 150 value 44.470719
iter 160 value 41.549106
iter 170 value 36.746992
iter 180 value 35.089741
iter 190 value 33.031845
iter 200 value 31.085787
final  value 31.085787 
stopped after 200 iterations

The confusion matrix obtained after the test is as follows ：

Dot graph ：

And current accuracy ：

 >correct/length(cres)# Accuracy rate 
[1] 0.5

Conclusion

nnet Compared with the use of pytorch It's much easier to get started , But it is also destined to deal with simple data , For higher dimensional data （ picture ） The learning ability of the model is certainly not good , Less parameters can be set , Can't be used GPU Training , Training is slow . The effect should be good when used in simple data fitting , It is also easy to use .