OpenCV DNN blogFromImage() detailed explanation

OpenCV3.x版本,增加了DNN模块,Supports direct loading of models trained by deep learning frameworks.How to pass data into the model,This is to seeblogFromImage()函数.

 

1. 背景知识：

To get correct predictions from deep neural networks,The data needs to be preprocessed first.Process the image data into the data format required by the model.These preprocessing usually include：

A. 均值减法(Mean Subtraction).That is the overall pixel minus the mean.

B. 缩放(Scaling).通过缩放系数(scalefactor)Scales image pixel values.

C. 把数据shapeAdjust to the format required by the model.

均值减法：

Mean subtraction is used to adapt to lighting transformations（combat illumination changes）.因此,We can think of mean subtraction as a technique to assist convolutional neural networks.Before starting to train our own deep neural network,Need to count the images we want to train onRGBAverage pixels of the three channels（average pixel）

2.具体使用：

Python:

cv.dnn.blobFromImage(image[, scalefactor[, size[, mean[, swapRB[, crop[, ddepth]]]]]]) ->retval

C++:

Mat cv::dnn::blobFromImage	(	InputArray	image,
		double	scalefactor = 1.0,
		const Size &	size = Size(),
		const Scalar &	mean = Scalar(),
		bool	swapRB = false,
		bool	crop = false,
		int	ddepth = CV_32F
	)

参数：

image	input image (with 1-, 3- or 4-channels).The image that will be fed into the neural network for processing or classification
size	spatial size for output image 这个参数是我们神经网络在训练的时候要求输入的图片尺寸
mean	scalar with mean values which are subtracted from channels. Values are intended to be in (mean-R, mean-G, mean-B) order if image has BGR ordering and swapRB is true. 需要将图片整体减去的平均值,如果我们需要对RGB图片的三个通道分别减去不同的值,那么可以使用3组平均值,如果只使用一组,那么就默认对三个通道减去一样的值.减去平均值（mean）：为了消除同一场景下不同光照的图片,对我们最终的分类或者神经网络的影响,我们常常对图片的R、G、B通道的像素求一个平均值,然后将每个像素值减去我们的平均值,这样就可以得到像素之间的相对值,就可以排除光照的影响.
scalefactor	multiplier for image values. 当我们将图片减去平均值之后,还可以对剩下的像素值进行一定的尺度缩放,它的默认值是1,如果希望减去平均像素之后的值,全部缩小一半,那么可以将scalefactor设为1/2
swapRB	flag which indicates that swap first and last channels in 3-channel image is necessary. OpenCVThe picture channel order in is BGR,But sometimes the order of the mean assumptions isRGB,所以如果需要交换R和G,那么就要使swapRB=true
crop	flag which indicates whether image will be cropped after resize or not
ddepth	Depth of output blob. Choose CV_32F or CV_8U.

输出：

Returns

4-dimensional Mat with NCHW dimensions order.

例子：

import cv2
import numpy as np
import sys

#Sam add it. load mobilenet model.

image = cv2.imread("E:\\current\\svn_Python_home\\OpenCV_Study\\Example\\DNN_Sample\\Images\\voc_boat_pi.jpg")
if(image is None):
    sys.exit(0)

net =cv2.dnn.readNetFromTensorflow("E:\current\svn_Python_home\OpenCV_Study\Example\DNN_Sample\models\mobilenet_v1_1.0_224_frozen.pb")

image = cv2.resize(image, (300,300))
blob = cv2.dnn.blobFromImage(image, 1.0, (300, 300), (104.0, 177.0, 123.0))

print("blob shape:", blob.shape)

print("Success")

blob shape: (1, 3, 300, 300)