Chapter_ 02 how to scan and view images, query tables and time metrics in opencv

One . The objectives of this chapter

We will answer the following questions :

• How to traverse every pixel of the image ?
• How to store Opencv The matrix value of ?
• How to measure the performance of our algorithm ?
• What is a lookup table ? Why use them ?

Two . Our test cases

Let's consider a simple color restoration method . By using unsigned char Type of C/C++ To store the matrix , A pixel channel can have 256 Different values . For a three channel image , This can form too many colors ( exactly 1600 ten thousand ). Handling so much chromaticity may bring a heavy blow to the performance of our algorithm . However , Sometimes a smaller number is enough to achieve the same end result .

under these circumstances , We usually reduce the color space . This means that we split the current value of the color space with a new input value , To get less color . for example ,0 To 9 Each value between takes a new value 0,10 To 19 Each value between takes a new value 10, And so on .

When you use one int Type divided by one uchar(unsigned char - aka The value of the 0 To 255 Between ) When it's worth it , The result is also char type . These values can only be char value . So any fraction will be rounded down . Take advantage of this fact ,uchar The upper level operation of the domain can be expressed as :

A simple color space reduction algorithm involves passing through each pixel of the image matrix and applying this formula . It is worth noting that , We did division and multiplication . These operations are very cumbersome for the system . If possible , It's worth using some easier operations , For example, some subtractions , Add , Or, at best, use simple assignment operations to avoid them . Besides , Please note that , For the above operation , There are only a limited number of input values . stay uchar In the system , Exactly 256.

therefore , For larger images , It is best to calculate all possible values in advance , Use the lookup table to assign values during the assignment process . A query table is a simple array ( Having one or more dimensions ), For a given input variable , It saves the final output value . Its advantage is that we do not need to calculate , Just read the results .

Our test case program ( And the following code example ) Do the following : Read the image passed as a command function parameter ( It can be a color or grayscale image ), And apply the reduction using the integer value of the given command line parameter . stay Opencv in , At present, there are three main pixel by pixel image processing methods . To make things more interesting , We will use each method to process images , And take the time you need .
Add a knowledge point to this , How to be in visul_sutio Middle configuration main() The parameters of the function
Click item , Try -> Property page -> debugging -> Command parameter

The following is a program example :

#include <opencv2/core.hpp>
#include <opencv2/core/utility.hpp>
#include "opencv2/imgcodecs.hpp"
#include <opencv2/highgui.hpp>
#include <iostream>
#include <sstream>

using namespace std;
using namespace cv;

static void help()
{
    
    cout
        << "\n--------------------------------------------------------------------------" << endl
        << "This program shows how to scan image objects in OpenCV (cv::Mat). As use case"
        << " we take an input image and divide the native color palette (255) with the " << endl
        << "input. Shows C operator[] method, iterators and at function for on-the-fly item address calculation."
         << endl
        << "Usage:" << endl
        << "./how_to_scan_images <imageNameToUse> <divideWith> [G]" << endl
        << "if you add a G parameter the image is processed in gray scale" << endl
        << "--------------------------------------------------------------------------" << endl
        << endl;
}

Mat &ScanImageAndReduceC(Mat &I, const uchar *table);
Mat &ScanImageAndReduceIterator(Mat &I, const uchar *table);
Mat &ScanImageAndReduceRandomAccess(Mat &I, const uchar *table);


//! [scan-c]
Mat &ScanImageAndReduceC(Mat &I, const uchar *table)
{
    
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);
    int channels = I.channels();

    int nRows = I.rows;
    int nCols = I.cols * channels;
    if (I.isContinuous())
    {
    
        nCols *= nRows;
        nRows = 1;
    }
    int i, j;
    uchar *p;
    for (int i = 0; i < nRows; i++)
    {
    
        p = I.ptr<uchar>(i);
        for (j = 0; j < nCols; j++)
        {
    
            p[j] = table[p[j]];
        }
    }
    return I;
}
//! [scan-c]


//! [scan-iterator]
Mat &ScanImageAndReduceIterator(Mat &I, const uchar * table)
{
    
    // accept only char type matrices
    CV_Assert(I.depth() == CV_8U);
    const int channels = I.channels();
    switch (channels)
    {
    
    case 1:
    {
    
        MatIterator_<uchar> it, end;
        for (it = I.begin<uchar>(), end = I.end<uchar>(); it != end; it++)
        {
    
            *it = table[*it];
        }
        break;
    }
    case 3:
    {
    
        MatIterator_<Vec3b> it, end;
        for (it = I.begin<Vec3b>(), end = I.end<Vec3b>(); it != end; it++)
        {
    
            (*it)[0] = table[(*it)[0]];
            (*it)[1] = table[(*it)[1]];
            (*it)[2] = table[(*it)[2]];
        }
    }
    }
    return I;

}
//! [scan-iterator]

//! [scan-random]
Mat &ScanImageAndReduceRandomAccess(Mat &I, const uchar *table)
{
    
    CV_Assert(I.depth() == CV_8U);
    const int channels = I.channels();
    switch (channels)
    {
    
    case 1:
        {
    
            for (int i = 0; i < I.rows; i++)
            {
    
                for (int j = 0; j < I.cols; j++)
                {
    
                    I.at<uchar>(i, j) = table[I.at<uchar>(i, j)];
                }
            }
            break;
        }
    case 3:
        {
    
            Mat_<Vec3b> _I = I;
            for (int i = 0; i < I.rows; i++)
            {
    
                for (int j = 0; j < I.cols; j++)
                {
    
                    _I(i, j)[0] = table[_I(i, j)[0]];
                    _I(i, j)[1] = table[_I(i, j)[1]];
                    _I(i, j)[2] = table[_I(i, j)[2]];
                }
            }
            I = _I;
            break;
        }
    }
    return I;
}
//! [scan-random]

int main(int argc,char* argv[])
{
    
    help();
    if (argc < 3)
    {
    
        cout << "Not enough parameters" << endl;
        return -1;
    }

    Mat I, J;
    if (argc == 4 && !strcmp(argv[3], "G"))
    {
    
        I = imread(argv[1], IMREAD_GRAYSCALE);
    }
    else
    {
    
        I = imread(argv[1], IMREAD_COLOR);
    }

    if (I.empty())
    {
    
        cout << "The image" << argv[1] << "could not be loaded." << endl;
        return -1;
    }

    //! [dividewith]
    int divideWith = 0; // convert our input string to number
    stringstream s;
    s << argv[2];
    s >> divideWith;
    if (!s || !divideWith)
    {
    
        cout << "Invalid number entered for dividing." << endl;
        return -1;
    }

    uchar table[256];
    for (int i = 0; i < 256; i++)
    {
    
        table[i] = (uchar)(divideWith * (i / divideWith));
    }
    // ![dividewith]
    const int times = 100;
    double t;

    t = (double)getTickCount();
    for (int i = 0; i < times; i++)
    {
    
        cv::Mat clone_i = I.clone();
        J = ScanImageAndReduceC(clone_i, table);
    }
    // getTickCount() function , The value returned by this function is from a certain time ( For example, the computer starts up ) Start , The total amount of time that the computer has passed tick The number of times .
    //  It needs to be combined with getTickFrequency() Function USES ,getTickFrequency() The return is CPU It will be emitted in one second tick The number of times .
    //  So the result of the two functions here is the number of seconds from the start of the count to the current position , Multiplied by 1000 Is the number of milliseconds , And then divided by 100
    //, It means the time of a single time   The final unit is also ms.
    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;

    cout << "Time of reducing with the C operator [](averaged for" << times << " runs): " << 
        t << " milliseconds." << endl;

    t = (double)getTickCount();

    for (int i = 0; i < times; i++)
    {
    
        cv::Mat clone_i = I.clone();
        J = ScanImageAndReduceIterator(clone_i, table);
    }
    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;
    cout << "Time of reducing with the iterator (averated for" << times << " runs): " <<
        t << "milliseconds." << endl;

    t = (double)getCPUTickCount();
    for (int i = 0; i < times; i++)
    {
    
        cv::Mat clone_i = I.clone();
        ScanImageAndReduceRandomAccess(clone_i, table);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;
    cout << "Time of reducing with the on-the-fly address generation - at function 
    (averaged for" << times << " runs): " << t << "milliseconds." << endl;


    //! [table-init]
    Mat lookUpTable(1, 256, CV_8U);
    uchar *p = lookUpTable.ptr();
    for (int i = 0; i < 256; i++)
    {
    
        p[i] = table[i];
    }
    //! [table-init]

    t = (double)getTickCount();
    for (int i = 0; i < times; i++)
    {
    
        //! [table-use]
        LUT(I, lookUpTable, J);
    }

    t = 1000 * ((double)getTickCount() - t) / getTickFrequency();
    t /= times;
    cout << "Time of reducing with the LUT function (averaged for" << times << " runs): " 
        << t << "milliseconds." << endl;
    return 0;
}

result :

You can run the program in the following way :

how_to_scan_images imageName.jpg intValueToReduce G

The last parameter is optional . If a given image is loaded in grayscale format , No in use BGR Color space . First, we need to calculate the lookup table

int divideWith = 0; // convert our input string to number - C++ type
stringstream s;
s << argv[2];
s >> dividewith;
if(!s || !dividewith)
{
    
	cout << "Invalid number entered for dividing. " << endl;
	return -1;
}
uchar table[256];
for(int i = 0; i < 256;i++)
{
    
	table[i] = (uchar)(dividewith * (i / dividewith));
}

here , We use C++ stringstream Class converts the third command line argument from text to integer format . Then we use a simple table and the above formula to calculate the lookup table . Nothing here Opencv Something specific .

Another problem is how we measure time ?Opencv Two simple functions are provided to implement ,cv::getTickCount() and cv::getTickFrequency(). The first function returns the system CPU In an event ( For example, after the system is started ) Medium Tick. The second function returns CPU In a second tick Count . Therefore, it is easy to measure the time interval between two operations :

double t = (double)getTickCount();
// do something
t = ((double)getTickCount - t) / getTickFrequency();
cout << "Times passed in seconds: " << t << endl;

3、 ... and . How do we store the image matrix in memory

We can do it in Mat- Basic image of the container tutorial read , The size of the matrix depends on the color system used . To be more precise , It depends on the amount of channel data used . In the case of grayscale images , We have such things :

For multi-channel images , Column contains the same number of child columns as the number of channels . With BGR Take the color system as an example :

Be careful , The order of channels is the opposite :BGR instead of RGB. Because in most cases , The memory is large enough to store rows in a continuous manner , So lines may follow one another , Create a long line . Because everything is in the same place , One by one , This may help speed up the scanning process . We use cv::Mat::isContinuous() Function to determine whether this condition exists in the matrix . We will illustrate with examples in the next section :

Four . An effective way to scan an image

In terms of performance , You can't beat the classic C Style operators []( The pointer ) visit , therefore , The most effective assignment method we can recommend is :

Mat& ScanImageAndReduceC(Mat& I,const uchar* const table)
{
    
	// accept only char type matrices
	CV_Assert(I.depth() == CV_8U);
	int channels = I.channels();
	
	int nRows = I.rows;
	int nCols = I.cols;
	if(I.isContinuous())
	{
    
		nCols *= nRows;
		nRows = 1;
	}
	int i,j;
	uchar* p;
	for(int i = 0; i < nRows;i++)
	{
    
		p = I.ptr<uchar>(i);
		for(j = 0;j < nCols;j++)
		{
    
			p[j] = table[[pj]]
		}
	}
}

Here we just need to get a pointer to the beginning of each line , And then traverse it , Until it's over . In the special case where the matrix is stored in a continuous manner , We only need to request the pointer once , And then to the end . We need to pay attention to color images : We have three channels , So we need three times as many items in the row .

There is another way .Mat Object's data The member data returns a pointer to the first row and the first column . If the pointer is empty , Then there is no valid input in the object . This check method is the simplest way to check whether the image is loaded successfully . If the storage is continuous , We can use this to traverse the entire data pointer . In the case of grayscale images , In the following way :

uchar* p = I.data;
for(unsigned int i = 0;i < ncols * nrows; i++)
{
    
	*p++ = talbe[*p];
}

You will get the same result . however , This code will be much more difficult to read in the future . If you have some more advanced technology here , It will become very difficult . Besides , In practice , You will find that you will get the same performance results ( Because most modern compilers may automatically implement this little optimization technique for you )

5、 ... and . iterator ( Security ) Method

Compared to ensuring that the correct amount of uchar Fields and skip possible gaps between rows is your responsibility .itrator Method is considered a safer method , Because it takes over these tasks from the user . All you need to do is ask about the beginning and end of the image matrix , Then add the start iterator , Until the end . To get the value pointed to by the iterator , Use * The operator ( Add it to the front of the iterator )

Mat& ScanImageAndReduceInterator(Mat& I,const uchar* const table)
{
    
	// accept only char type matrices
	CV_Assert(I.depth() == CV_8U);
	const int channels = I.chanels();
	switch(channels)
	{
    
		case 1:
			{
    
				MatItrator_<uchar> it,end;
				for(it = I.begin<uchar>(),end=I.end<uchar>();it != end;it++)
				{
    
					*it = table[*it];
				}
				break;
			}
		case 3:
			{
    
				MatIterator_<Vec3b> it ,end;
				for(it = I.begin<Vec3b>(),end = I.end<Vec3b>();it != end;it++)
				{
    
					(*it)[0] = table[(*it)[0]];
					(*it)[1] = talbe[(*it)[1]];
					(*it)[2] = talbe[(*it)[2]];
				}
			}
	}
	return I;
}

For color images , There are three in each column uchar term . This can be thought of as uchar The small vector of the term , stay Opencv Named Vec3b. To visit the n Child column , We use simple operator[] visit . It's important to remember ,Opencv The iterator arranges the columns and automatically jumps to write a row . therefore , For color images , If you use simple uchar iterator , It can only access the values of the blue channel .

6、 ... and . Use the dynamic address returned by the reference to calculate

The last method is not recommended for traversal . It is used to obtain or modify random elements in the image . Its basic usage is to specify the row and column numbers of the items to be accessed . In our early traversal methods , You may have noticed , What type of image we are working on is very important . There's no difference here , Because you need to find the type you want to use automatically . Line and surface source code (cv::Mat::at() Function usage ) This can be observed in the grayscale image of :

Mat& ScanImageAndReduceRandomAccess(Mat& I,const uchar* const table)
{
    
	// accept only char type matries
	CV_Assert(I.depth() == CV_8U);
	const int channels = I.channels();
	switch(channels)
	{
    
		case 1:
			{
    
				for(int i = 0;i < I.rows;i++)
				{
    
					for(int j = 0;j < I.cols;j++)
					{
    
						I.at<uchar>(i,j) = table[I.at<uchar>(i,j)];
					}
				}
				break;
			}
		case 3:
			{
    
				Mat_<Vec3b> _I = I;
				for(int i = 0; i < I.rows;i++)
				{
    
					for(int j = 0; j < I.cols;j++)
					{
    
						_I(i,j)[0] = table[_I(i,j)[0]];
						_I(i,j)[1] = table[_I(i,j)[1]];
						_I(i,j)[2] = table[_I(i,j)[2]];
					}
				}
				I = _I;
				break;
			}
	}
	return I;
}

This function receives the type and coordinates you enter , And calculate the address of the query item . Then return a reference to the object . When you get the value , It could be a constant , And when you set the value , It may be an extraordinary number . As a safe step in debug mode ,* The No. operator performs a check , To ensure that the coordinates you enter are valid , And it does exist . If this is not the case , You will get a good output message on the standard error output stream . And release Compared with the effective methods in the mode , The only difference in using this method is , For each element in the image , You will get a new row pointer , Second, we use C The operator [] To get column elements .

If you need to use this method to find an image many times , Then enter... For each access type and at Keywords can be cumbersome and time consuming . To solve this problem ,Opencv There is one cv::Mat_ data type . It is associated with Mat identical , Only when defining, you need to specify the data type by viewing the data matrix , But in return , You can use the operator () To quickly access items . What's better is , It can be easily converted to cv::Mat data type . You can see the example usage of this function in the color image above . However , It should be noted that , Same operation ( At the same running speed ) It can also be used. cv::Mat::at Function completion . This is just a tip for lazy programmers .

7、 ... and . The core approach

This is an additional way to implement lookup table modification in images . In image processing , It is common to change all given image values to other values .Opencv It provides the function of modifying image value , Second, there is no need to learn the traversal logic of the image . We use the core module cv::LUT() function , First , Let's build a Mat Type lookup table :

Mat lookUpTable(1,256,CV_8U);
uchar* p = lookUpTable.ptr();
for(int i = 0;i < 256;i++)
{
    
	p[i] = table[i];
}

Finally, call this method (I Is our input image matrix ,J Is the transformed output matrix )

LUT(I,lookUpTable,J);

8、 ... and . Performance differences

In order to get the best result , Please compile and run the program yourself . In order to make the difference more obvious , We used a fairly large (2560*1600) Images . The performance described here is for color images . To get a more accurate value , We get 100 The average number of function calls .

We can draw a few conclusions . If possible , Use Opencv The existing functions ( Instead of reinventing these functions ). The fastest way is LUT function . This is because Opencv The library is multithreaded through the Intel thread builder . However , If you need to write a simple image scanning preferred pointer method . Iterators are a safer option , But it's quite slow . In debug mode , Full image scanning using the dynamic reference access method is the most time-consuming . stay release In mode , It may defeat iterator methods , Maybe not , But it certainly sacrifices the safety features of iterators .