VOT toolkit environment configuration and use

   This article is mainly for use C/C++ Students who design tracker Algorithm , build VOT Test environment for , And some experience of using .
  VOT challenge round It is recognized as the most authoritative target tracking game in the world , Papers related to the competition will be published in ICCV On , Even years will happen in ECCV On . On the official website of the event, you can download the competition results and data sets over the years ,VOT The algorithm performance evaluation tool used is based on TraX agreement , The algorithm to be evaluated needs to follow TraX Write the agreement .

TraX agreement

   Reference resources Luka Čehovin 2014 Year paper ：TraX: Visual Tracking eXchange Protocol, There is a detailed introduction based on TraX The way of protocol test tracker .

   The counterintuitive thing is , Our own tracker is called “Server”, The process of sending pictures to our tracker is called “Client”.Client Will call our tracker to generate a Seerver process , The two processes communicate in a certain message format .

compile OpenCV

  TraX The compilation of is dependent on OpenCV Of , The following tracker as an example also needs OpenCV. Although you can find others' compiled on the Internet OpenCV, But we will inevitably encounter incompatibility with our local compiler . If there is a problem with the library file compiled by others , It's going to be a headache , So the best way is to compile by ourselves .
   Another advantage of our own compilation is that we get it OpenCV contain OpenCVConfig.cmake and OpenCVConfig-version.cmake, Later, we will use OpenCV The place of , As long as CMakeLists.txt Pass through find_package You can find OpenCV The relevant path . You don't need to set it yourself in the project include and lib Path to file .
   First, in the OpenCV Its official website Download the required source file , And unzip it .

   My customary practice is , Create a new one under the unzipped directory “build” Folder as the path to generate the project . Then open the CMake GUI, Choose the path of source code and output project . In turn, click Config、Generate and Open Project Just go .BUILD Below, you can choose which library files need to be compiled , Some unnecessary items can be left unchecked , such as opencv_python If I don't plan to use it, I don't need to tick .

   There may be some problems in the process , Like this one down here Warning, By removing OPENCV_GENERATE_SETUPVARS Check this option to eliminate .

   It is also possible that some files cannot be downloaded due to some network problems , You can use some scientific methods to surf the Internet , Go to GitHub Download it manually on the Internet . in addition Python2.7 It also needs to be installed ,OpenCV I'll go to VTK, look for BLAS, But it doesn't matter if you can't find it , It hasn't affected my use yet , The big deal is to compile OpenCV Compile first OpenBLAS and VTK, The steps are similar .
   open CMake Generated VS After the project , choice Release edition .

   And then in Class View Right click inside “INSTALL”, choice “Build”.

  INSTALL It's dependence ALL_BUILD Of , So I choose to compile INSTALL When , It compiles ALL_BUILD. compile INSTALL The results of the previous compilation will be installed in a directory . This directory is in CMake As determined in , stay CMake GUI When configuring ,CMAKE_INSTALL_PREFIX Specifies the prefix of the installation path .

   After compiling , take bin Path to PATH Environment variables . You can also add a “OpenCV_DIR” Environment variables of , hold OpenCVConfig.cmake As the value of this variable . In this way, every time we meet the need OpenCV Of CMake Engineering time ,CMake You can find it automatically OpenCV.

compile TraX

   Successfully compiled OpenCV after , compile TraX There's no difficulty . from GitHub On clone Source code . Then pass in the same way CMake Generate VS engineering , Compile and INSTALL Then get the following files ：

   You can set trax_DIR This variable , The value of the variable is share Path to folder , Like mine “C:\Tools\TraX\share”. And then put bin Directory Settings to PATH In the environment variable .

VOT Toolkit Environment configuration

   Using the latest python Version of vot-toolkit Before , I also tried to toss the old version Matlab Version of . But I met Matlab The problem of collapse , Later on GitHub I found the same thing I met problem People who . Probably a new version Matlab Are not compatible , So the author developed python Version of toolkit, It's really better to use .

A virtual environment

   In the installation vot-toolkit Before , It is suggested to build a separate virtual environment first , This can play a role of isolation , image Matlab The problem of running crash after version upgrade will not occur .

py -0

   I installed 3.9 and 2.7 Version of python, The specific operation python You can rely on python launcher Make a selection .“py -0” The command can display which versions are installed in the current system python.

   I use python Official venv modular , To use the module, you need to add “-m” The option to . For example, I want to create a new virtual environment under the current path , It can be used ：

py -3.9 -m venv .

   If you want to create in another path , Just put “.” Just change to another pathname .

   There is one under the path of the newly created virtual environment Scripts Folder , Through the inside activate.bat（cmd） perhaps Activate.ps1（PS） To activate the virtual environment . If you use CMD, Still need to use Scripts In the catalog deactivate.bat Exit virtual environment , And if it is PS, You just need to use it deactivate Command can exit .
   You can refer to later The official tutorial install vot-toolkit.

initialization

   Refer to the official document initialization workspace When , You may meet someone who can't find stack The problem of . Tips “Experiment stack None not found”, This is because pip Download the installed vot/stack The path is missing stack Configuration file for .

   find GitHub Directory in the source code of , You can find these configuration files above , Just copy the files under this directory to vot/stack Just in the directory .
   Downloading datasets may be slow , Baidu SkyDrive , Extraction code ：4id2. I downloaded several data sets in recent years and put them on the online disk . You can add “–nodownload” Options , Then manually download the dataset , Then put the downloaded data on workspace Of sequence Just under the directory .

Compile tracker

   stay GitHub You can also find the tracker on Examples , stay native Use and compile under the directory OpenCV and TraX Compile in the same way NCC tracker , Get one ncc.exe Executable file .
  GitHub You can also find KCF Source code , Again clone After arriving here , We need to make some changes . Because this online version KCF No will TraX Embedded , We need to make some changes .
   open CMakeLists.txt, Mainly for reference NCC, Add pair TraX Of include：

cmake_minimum_required(VERSION 3.3)
project(kcf)

find_package(OpenCV REQUIRED)

FIND_PACKAGE(trax REQUIRED COMPONENTS core)
LINK_DIRECTORIES(${TRAX_LIBRARY_DIRS})
LINK_LIBRARIES(${TRAX_LIBRARIES})
INCLUDE_DIRECTORIES(AFTER ${TRAX_INCLUDE_DIRS})

if(NOT WIN32)
ADD_DEFINITIONS("-std=c++0x -O3")
endif(NOT WIN32)

include_directories(src) 
FILE(GLOB_RECURSE sourcefiles "src/*.cpp")
add_executable( KCF ${sourcefiles} )
target_link_libraries( KCF ${OpenCV_LIBS})

   And then put NCC Source code vot.h Also copied to KCF Of src Under the path of . use CMake structure VS After the project , open runtracker.cpp, hold main Change the function to and NCC Similar format ：

#include <iostream>
#include <fstream>
#include <sstream>
#include <algorithm>

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>

#include "kcftracker.hpp"
#include <dirent.h>
#include "vot.h"

int main(int argc, char* argv[]){
    

#if __USE_VOT

    // Create KCFTracker object
    KCFTracker tracker;
    VOT vot;

    cv::Rect initialization;
    initialization << vot.region();
    cv::Mat image = cv::imread(vot.frame());
    tracker.init(initialization, image);

    while (!vot.end()) {
    
        string imagepath = vot.frame();
        if (imagepath.empty()) break;
        cv::Mat image = cv::imread(imagepath);
        float confidence = 0.0;
        cv::Rect rect = tracker.update(image);
        vot.report(rect);
    }
}

   Did you find it when compiling dirent.h This header file , Need to download online , Reference resources This article . After compilation, you will also get a KCF.exe The executable of .

Result analysis

   Refer to the tutorial on the official website , Before testing, you need to write trackers.ini The configuration file . Now I have NCC and KCF Two trackers .

# trackers.ini
[KCF]  # <tracker-name>
label = KCF
protocol = trax
command = D:\Algorithm\Trackers\KCFCpp\build\Release\KCF.exe
# Additional environment paths
env_PATH = ${PATH}

[NCC]
label = NCC
protocol = trax
command = D:\Algorithm\Trackers\integration\native\build\Release\ncc.exe
env_PATH = ${PATH}

   Just put it before OpenCV and TraX Of bin The path is added to PATH In the environment variable , The setting here is no problem . It can be used "vot test <tracker>" Advanced line test .

   Like this , And the last hint “Test concluded successfully”, That's it TraX Communication is ok . Unless there is something wrong with the tracker's tracking code , There is evaluate I will make a mistake .

   stay VOT Of workspace Under the directory, two trackers evaluate:

   Proceed again analysis, Generate a report , The report will be generated in workspace Of analysis Under the table of contents , The default is html Format .

VOT experiment

   I used stack yes vot2019, It is mainly divided into three experiments .
  baseline yes VOT Characteristic experiment of , It is to reinitialize the trace after the trace fails . As early as VOT2015 It is proposed that , In tracking continuous 5 Reinitialize after frame failure , Before reinitialization 10 Frame ignore .
  realtime Is in VOT2018 Proposed in , It mainly focuses on the real-time performance of the algorithm .vot-toolkit Will give the tracker 20fps Send pictures at the same speed , If the tracker does not give the results in time , that vot-toolkit Will take the last result as the new result , Similar to zero order preservation .
  unsupervised There is no supervision , Whether the tracker is lost or not ,toolkit Will not reinitialize the tracker . So there's only one AUC Test indicators of .

Accuracy

   Accuracy is actually easy to understand , It's the tracking box and grandtruth The proportion of coincidence range . It is worth noting that , Here, only those graphs with effective tracking are added to the calculation . For example, before reinitialization 10 Frames are not valid .

Robustness

   Stability is measured by the number of lost frames , The more frames are lost , The less stable . stay Visual object tracking performance measures revisited In this paper , In order to intuitively display accuracy and stability , An exponential function is introduced to measure the stability ：

$$e^{-S\frac{F_0}{N}}$$

   among $F_0$ Is the number of times lost ,N Is the total number of frames .S It's a coefficient , The meaning of this coefficient is the number of frames that the tracker can track continuously and stably . The result of this calculation is that the tracker is tracking continuously S After the frame , Probability of stable tracking . After this calculation ,A and R Can be unified to 0~1 The scope of the , Then show it on a picture like this , The closer the point to the upper right corner, the better the tracker performance .

Expected Average Overlap

   Now? VOT Our ranking mainly depends on EAO Indicators of ,EAO It is an index that combines accuracy and stability .EAO Also in VOT2015 It was put forward when , This index is proposed mainly because considering that the length of different image sequences in the data set is different .

   In a dataset , Count the number of sequences with different lengths , You can draw a probability density function about the length of the sequence . Take from it 50% Sequence , Give Way $N_{lo}$ The number of sequences and $N_{hi}$ The number of sequences is the same . stay $N_{lo}$ and $N_{hi}$ Sequence calculation within the length range EAO.
   The accuracy of a tracking can be calculated for sequences of different lengths , The longer the sequence , The lower the accuracy , There is one EAO curve , As shown in the upper left corner of the figure above . And then in $N_{lo}$ and $N_{hi}$ Calculate the average to get EAO Indicators of .

Custom Settings

   How to customize this vot-toolkit？ After all, it's open source , You can change it any way you want .stack Inside yaml Files are configuration files , With vot2019 For example .

title: VOT-ST2019 challenge
dataset: vot:vot-st2019
url: http://www.votchallenge.net/vot2019/
experiments:
  baseline:
    type: supervised
    repetitions: 15
    skip_initialize: 5
    analyses:
      - type: supervised_average_ar
        sensitivity: 30
      - type: supervised_eao_score
        low: 46
        high: 291
      - type: supervised_eao_curve
  realtime:
    type: supervised
    realtime:
      grace: 3
    repetitions: 1
    skip_initialize: 5
    analyses:
      - type: supervised_average_ar
        sensitivity: 30
      - type: supervised_eao_score
        low: 46
        high: 291
      - type: supervised_eao_curve
  unsupervised:
    type: unsupervised
    repetitions: 1
    analyses:
      - type: average_accuracy

  python I don't use much , Whole vot-toolkit I haven't quite understood the working mechanism of . But look at this configuration file , There is already a lot of information in it . There are three experiments listed , For each experiment type yes supervised, It can correspond to analysis/supervised.py This file ;sensitivity It is in the stability measurement index S;repetitions Is the number of repetitions of the experiment ;skip_initialize Is the number of frames to skip initialization ;grace Is the number of consecutive frames that are judged as failed after tracking failure .
   In addition to changing the configuration here , In addition, it can also be changed vot In the catalog python Script , Of course, it can only be changed after understanding .