[target tracking] | single target tracking indicator

VOT

VOT Think , Only large-scale datasets are completely useless , A reliable data set should test tracker Performance under different conditions

VOT Put forward , Each sequence should be marked with its visual properties （visual attributes）, To correspond to the different conditions mentioned above ,VOT2013 Six visual attributes are proposed ：

Camera movement （camera motion, That is, jitter blur ）
Light variation （illumination change）
Target size changes （object size change）
Target action changes （object motion change, Similar to camera jitter , It's all fuzzy ）
Not degraded （non-degraded）

stay VOT Before I put it forward , The more popular evaluation system is to let tracker Initialize at the first frame of the sequence , Then let tracker Run to the last frame . However tracker It may be lost at the beginning of some frames due to oneortwo factors （fail）, So the final evaluation system only uses a small part of the sequence , wasteful . and VOT Put forward , The evaluation system should be in tracker An error is detected when losing （failure）, And in failure the 5 Right after the frame tracker Reinitialize （reinitialize）, This makes full use of data sets . The reason is 5 Frame instead of initializing immediately , Because failure Initialization immediately after that is likely to fail tracking again , And because the occlusion in the video generally does not exceed 5 frame , So there will be such a setting .

VOT A characteristic mechanism of , The restart （reset/reinitialize）. But some frames after restart cannot be used for evaluation , These frames are called burn-in period, A lot of experimental results show that ,burn-in period About after initialization （ Including the initialization of the first frame and all restarts ） Of 10 frame .

Accuracy(A)

Accuracy Used to evaluate tracker Accuracy of tracking target , The greater the numerical , The more accurate .

Number of a sequence t The frame of accuracy Defined as ： Every frame of IOU value

The average accuracy is the average of all effective frames

Robustness

Robustness Used to evaluate tracker Tracking target stability , The greater the numerical , The less stable .

F In order to repeat the test N_rep Number of failures in

AR rank( comprehensive A R)

take tracker The performance on different attribute sequences is in accordance with accuracy（A） and robustness（R） Rank separately , Then average , Get it tracker Comprehensive ranking of , According to the number of this comprehensive ranking tracker Sort to get the final ranking . This ranking is called AR rank.

The specific operation is ： First let's tracker Test under the sequence of the same attribute , For the data obtained （average accuracy/average robustness） Make a weighted average , The weight of each data is the length of the corresponding sequence , Thus, a single tracker Data on this attribute sequence , Then for different tracker Rank under this attribute sequence . Get a single tracker Rank behind all attribute sequences , Find the average （ Unweighted ） obtain AR rank.

EAO(Expected Average Overlap)

VOT2015 Put forward , be based on AR rank The evaluation method of is not fully utilized accuracy and robustness Raw data （raw data）, So we created a new evaluation index ——EAO(Expected Average Overlap). As it literally means , This evaluation index is only based on overlap Defined accuracy.

EAO curve , For sequence tests with different sequence lengths , The horizontal axis is the sequence length , The vertical axis is A,

EFO( Comparison of different hardware speeds )

EFO（Equivalent Filter Operations ） yes VOT2014 Proposed a measure tracking A new unit of speed , Using vot-toolkit evaluation tracker Before , First, I will measure in a 600600 Used on grayscale images 3030 Time for maximum filter to filter , So we can get a reference unit , And then measured in this basic unit tracker The speed of , So as to reduce the external factors such as hardware platform and programming language tracker The effect of speed .

Accuracy of normalization （Norm. Prec）

source ——TrackingNet
in consideration of Ground Truth The size of the box , take Precision Normalize , obtain Norm. Prec, Its value is in [0, 0.5] Between . That is, judge the prediction box and Ground Truth The Euclidean distance between the center point of the frame and Ground Truth The scale of the bezel of the box .

The success rate Success Rate/IOU Rate/AOS

source ——OTB2013
The success rate calculation is to calculate the prediction box and Ground Truth The intersection and union ratio of pixels in the area of the truth box of , That is, the ratio of the red box to the blue beveled area . The formula is shown in the figure above S.
Usually we will see one in the paper AUC（Area under curve） fraction , This score is actually the area under the success rate curve , The effect achieved is equivalent to taking into account the success rate scores under different thresholds . Some papers will also directly specify the threshold （ Such as 0.5）. In fact, when the success rate curve is smooth enough , take 0.5 The corresponding success rate score and the calculation of success rate AUC The scores are the same 【 The mean value theorem 】

ref
https://blog.csdn.net/Dr_destiny/article/details/80108255