Detection and tracking evaluation index

Expected model ：

Fast , Small memory , High precision

Detection The evaluation index

Accuracy index ：

• MAP Average accuracy mean

Speed index ：
The speed evaluation index must be carried out on the same hardware .

• FPS ：
  frames per second Frame rate
  The number of pictures processed per second or the time required to process each picture （ Compare under the same hardware ）
  Influencing factors ： Model parameters , Activation function , Loss function
• FLOPS：
  floating point operations per second. Floating point operations per second represent hardware performance
• FLOPs：
  Amount of computation , It can be used to measure the algorithm / The complexity of the model .

understand ：
FLOPs：floating point operations Refers to the number of floating-point operations , It means the amount of calculation , It can be used to measure the algorithm / The complexity of the model . Distinguish here FLOPS（ All capitals ）,FLOPS Refers to the floating-point number of operations per second , It means computing speed , The standard to measure a hardware . What we want is an index to measure the complexity of the model , So choose FLOPs.
Each image FLOPs Is different , So what the same hardware needs to process the same image FLOPS The smaller it is , More pictures can be processed in the same time , The faster

• mAP：mean Average Precision , All categories AP Average value . All categories in the dataset AP The average of the values
• AP：PR The area under the curve
• PR：precision-recall curve
• Precision：TP/TP+FP
• Recall：TP/(TP+FN)
  P=TP/TP+FP
  R=TP/TP+FN

understand ：
mAP It's for the whole data set
AP For a category in the dataset
P and R It is to predict a certain category for a single picture

MOT Problem evaluation index

principle ：

• All emerging goals should be found in time ;
• The target location should be consistent with the real target location as far as possible ;
• Each goal should be assigned a unique ID, And the target assigned this ID Remain unchanged throughout the sequence .

Accuracy：

• Accuracy . Use error to express . Not the wrong proportion .

Presicion：

• The right is the right probability

MOTA：

• Multi target tracking accuracy (Multiple Object Tracking Accuracy, MOTA) .
• An indicator to measure the accuracy of multi-target tracking with a single camera

MOTP：

• Multi target tracking accuracy (Multiple Object Tracking Precision, MOTP)
• An index to measure the position error of single camera multi-target tracking

“ Accuracy refers to the degree to which the average value measured many times under certain experimental conditions is consistent with the true value , Expressed as error . It is used to express the magnitude of the systematic error . Accuracy refers to the degree of consistency between measured values and their “ Truth value ” The proximity of , That is, the comprehensive concept of precision and accuracy .

Some of the concepts

• SOTA : state-of-the-art

• SOTA-modle: In this research task , best / State of the art models .

• SOTA-result: In this task , The results of the best model so far / performance / performance .

• pipeline and baseline What is it? ？
  pipeline and baseline What is it? ？
  baseline:
  Basic model , Data preprocessing can be realized , Characteristic engineering of foundation 、 Model establishment and result output and evaluation , Then through in-depth data processing , feature extraction , Model tuning or fusion , bring baseline Can improve . Equivalent to a basic model , It can be used as a benchmark to compare whether the improvement of the model is effective .
  pipeline：
  data fetch \ Data preprocessing \ Creating models \ Model evaluation results \ Model parameter adjustment .
  The application of pipeline mechanism in machine learning algorithm is rooted in , The parameter set is in the new data set （ For example, test sets ） Reuse on .

• IOU： Occurring simultaneously than .bbox GT Overlap
  NMS：（Non-Maximum Suppression,NMS） Non maximum suppression . As the name implies, inhibition is not a maximum element , It can be understood as local maximum search , Suppress windows with low scores .
  IOU + NMS: Get multiple candidate boxes by classifier , And the probability value of belonging to the category in the candidate box , Select a probability maximum box , By way of IOU As a threshold , Merge some boxes , Iterate this process .

Multitarget tracking algorithm （MOT） The evaluation index
Multi target tracking evaluation index