The onnx model is exported as a TRT model

linux Similar environment , This article aims at Windows Environmental Science

Configure it according to this article TensorRT Environment ：
TensorRT windows10 Installation process record

Method 1 ： Code conversion

It can be used directly TensorRT Python API Medium onnx Parser To read and serialize the model and save

however , In the process , In fact, it has been built , Or just say , Namely build The process , So we need to convert the accuracy to TF32、FP32 perhaps INT8, It needs to be done before saving .

Before conversion , First, check whether your platform supports the corresponding accuracy ：

1. Or according to your graphics card model , Check your... Here CUDA Computing power version —> CUDA Calculation capacity table , Then according to this version , To check whether the corresponding precision is supported --> Precision mode supported by hardware
   for instance , I am a GTX1650Ti The graphics card ：
   
   The computing power is 7.5, Then look up this table ：
   
   You can see , This version of the card does not support TF32 and DLA

2. or , according to Of this blog API To create a builder, Then use these three API Let's see if the corresponding precision is supported ：

import tensorrt as trt
logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)

>>> builder.platform_has_fast_fp16
True

>>> builder.platform_has_fast_int8
True

>>> builder.platform_has_tf32
False

Above is GTX 1650Ti Final accuracy support

Know the accuracy support , Next , According to the content of this blog ：Python API analysis In the first part of

In the process of transformation , How to specify precision , You can refer to this blog ：
https://blog.csdn.net/HaoZiHuang/article/details/125830788

A little hint , If you don't know Config Where is the object , He is in builder It is instantiated only after it has been created ：

config = builder.create_builder_config()

forehead , Still refer to this article Python API analysis

Method 2 ： Command line conversion

I still remember in this article Blog Mentioned in trtexec Do you

Command line conversion , It's not so fancy ：

trtexec --onnx=output.onnx --saveEngine=outfp32.engine --workspace=2048 --minShapes=x:1x3x224x224 --optShapes=x:1x3x224x224 --maxShapes=x:1x3x224x224

trtexec --onnx=output.onnx --saveEngine=outfp16.engine --workspace=2048 --minShapes=x:1x3x224x224 --optShapes=x:1x3x224x224 --maxShapes=x:1x3x224x224 --fp16

trtexec --onnx=output.onnx --saveEngine=outfpi8.engine --workspace=2048 --minShapes=x:1x3x224x224 --optShapes=x:1x3x224x224 --maxShapes=x:1x3x224x224  --int8

trtexec --onnx=output.onnx --saveEngine=outfpbest.engine --workspace=2048 --minShapes=x:1x3x224x224 --optShapes=x:1x3x224x224 --maxShapes=x:1x3x224x224  --best

There are 4 That's ok , Namely fp32、fp16、int8 and The best strategy

the last one --best , Generally, no brain chooses this OK 了 ,TensorRT Our algorithm will automatically find the fastest

But in fact , You didn't specify --int8 He must be int8 Of , Only the preference is --int8, For details, please refer to this introduction about accuracy ：TensorRT in Reduced Precision, If you don't understand it, read the original English …

A little bit about other parameters ：

--onnx=output.onnx                               #  To be transformed  onnx  Model 
--saveEngine=outfp32.engine                      #  Output position 
--workspace=2048                                 #  Set the workspace size , The unit is  MB
--minShapes=x:1x3x224x224                        #  This is the minimum input to calibrate the model , The input tensor name is  `x`
--optShapes=x:1x3x224x224 
--maxShapes=x:1x3x224x224

Actually , I don't know how to translate these three below ,hxd Make do with it ：

  --minShapes=spec            Build with dynamic shapes using a profile with the min shapes provided
  --optShapes=spec            Build with dynamic shapes using a profile with the opt shapes provided
  --maxShapes=spec            Build with dynamic shapes using a profile with the max shapes provided

Mainly the first few parameters , The last few are not important hh