OpenMMLab Model Deployment Toolset

I am using Dyhead to train an image detection model: https://github.com/open-mmlab/mmdetection/blob/master/configs/dyhead/atss_swin-l-p4-w12_fpn_dyhead_mstrain_2x_coco.py

Using GPU docker, convert to tensorrt with tools/deploy.py success: python3 tools/deploy.py /workdir/detection_onnx_static_1024x1024.py /workdir/atss_swin-l-p4-w12_fpn_dyhead_mstrain_2x.py /workdir/latest.pth /workdir/test-deploy-img-1024.jpg --device cuda --dump-info

Although the conversion have a lot warning like below:

/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py:24: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/backbones.py:202: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  slice_w = (W + self.window_size - 1) // self.window_size * self.window_size
WARNING: The shape inference of mmdeploy::MMCVModulatedDeformConv2d type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
[09/01/2022-01:33:56] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +809, GPU +348, now: CPU 3460, GPU 785 (MiB)
[09/01/2022-01:33:56] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +126, GPU +60, now: CPU 3586, GPU 845 (MiB)
[09/01/2022-01:33:56] [TRT] [W] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.3.2
[09/01/2022-01:33:56] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/01/2022-01:35:06] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size will enable more tactics, please check verbose output for requested sizes.
[09/01/2022-01:36:46] [TRT] [I] Detected 1 inputs and 2 output network tensors.
[09/01/2022-01:36:48] [TRT] [I] Total Host Persistent Memory: 500480
[09/01/2022-01:36:48] [TRT] [I] Total Device Persistent Memory: 393728
[09/01/2022-01:36:48] [TRT] [I] Total Scratch Memory: 1918222336
[09/01/2022-01:36:48] [TRT] [I] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 0 MiB
[09/01/2022-01:36:53] [TRT] [I] [BlockAssignment] Algorithm ShiftNTopDown took 5282.29ms to assign 35 blocks to 1075 nodes requiring 2039239680 bytes.
[09/01/2022-01:36:53] [TRT] [I] Total Activation Memory: 2039239680
[09/01/2022-01:36:53] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 5632, GPU 2311 (MiB)
[09/01/2022-01:36:53] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 5632, GPU 2319 (MiB)
[09/01/2022-01:36:53] [TRT] [W] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.3.2
[09/01/2022-01:36:53] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[09/01/2022-01:36:53] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[09/01/2022-01:36:53] [TRT] [W] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
2022-09-01 01:36:54,590 - mmdeploy - INFO - Finish pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt
2022-09-01 01:36:55,388 - mmdeploy - WARNING - "visualize_model" has been skipped may be because it's             running on a headless device.
2022-09-01 01:36:55,388 - mmdeploy - INFO - All process success.

I know after conversion the result is not exactly the same so I am okey with bounding box value difference(although its off quite a bit too), but the score is kind of drop too much! Below is the trt engine result (x1,y1,x2,y2,score) [8.8506012, 358.41714, 149.80162, 495.56137, 0.081301391] And below is the original predict with mmdet, the bbox have been round down [0, 328, 165, 526, 0.53286]

Here is my env with python3 tools/check_env.py

2022-09-01 02:08:19,488 - mmdeploy - INFO - 

2022-09-01 02:08:19,489 - mmdeploy - INFO - **********Environmental information**********
2022-09-01 02:08:19,681 - mmdeploy - INFO - sys.platform: linux
2022-09-01 02:08:19,681 - mmdeploy - INFO - Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0]
2022-09-01 02:08:19,681 - mmdeploy - INFO - CUDA available: True
2022-09-01 02:08:19,681 - mmdeploy - INFO - GPU 0: NVIDIA RTX A4000
2022-09-01 02:08:19,681 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda
2022-09-01 02:08:19,681 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
2022-09-01 02:08:19,681 - mmdeploy - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
2022-09-01 02:08:19,681 - mmdeploy - INFO - PyTorch: 1.12.0
2022-09-01 02:08:19,681 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2022.0-Product Build 20211112 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.6
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

2022-09-01 02:08:19,681 - mmdeploy - INFO - TorchVision: 0.13.0
2022-09-01 02:08:19,681 - mmdeploy - INFO - OpenCV: 4.6.0
2022-09-01 02:08:19,681 - mmdeploy - INFO - MMCV: 1.6.1
2022-09-01 02:08:19,681 - mmdeploy - INFO - MMCV Compiler: GCC 9.3
2022-09-01 02:08:19,681 - mmdeploy - INFO - MMCV CUDA Compiler: 11.6
2022-09-01 02:08:19,681 - mmdeploy - INFO - MMDeploy: 0.7.0+21775ce
2022-09-01 02:08:19,681 - mmdeploy - INFO - 

2022-09-01 02:08:19,681 - mmdeploy - INFO - **********Backend information**********
2022-09-01 02:08:20,066 - mmdeploy - INFO - onnxruntime: 1.8.1	ops_is_avaliable : True
2022-09-01 02:08:20,091 - mmdeploy - INFO - tensorrt: 8.4.3.1	ops_is_avaliable : True
2022-09-01 02:08:20,105 - mmdeploy - INFO - ncnn: None	ops_is_avaliable : False
2022-09-01 02:08:20,106 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-09-01 02:08:20,107 - mmdeploy - INFO - openvino_is_avaliable: False
2022-09-01 02:08:20,121 - mmdeploy - INFO - snpe_is_available: False
2022-09-01 02:08:20,121 - mmdeploy - INFO - 

2022-09-01 02:08:20,121 - mmdeploy - INFO - **********Codebase information**********
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmdet:	2.25.1
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmseg:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmcls:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmocr:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmedit:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmdet3d:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmpose:	None
2022-09-01 02:08:20,122 - mmdeploy - INFO - mmrotate:	None

And here is my deploy config (/workdir/detection_onnx_static_1024x1024.py):

codebase_config = dict(
    type='mmdet',
    task='ObjectDetection',
    model_type='end2end',
    post_processing=dict(
        score_threshold=0.05,
        confidence_threshold=0.005,  # for YOLOv3
        iou_threshold=0.5,
        max_output_boxes_per_class=200,
        pre_top_k=5000,
        keep_top_k=100,
        background_label_id=-1,
    )
)

onnx_config = dict(
    type='onnx',
    export_params=True,
    keep_initializers_as_inputs=False,
    opset_version=11,
    save_file='dyhead_swin_1024.onnx',
    input_names=['input'],
    output_names=['dets', 'labels'],
    input_shape=[1024, 1024],
    optimize=True
)

backend_config = dict(
        type='tensorrt',
        common_config=dict(fp16_mode=False, max_workspace_size=8 << 30),
        model_inputs=[
            dict(
                input_shapes=dict(
                    input=dict(min_shape=[1, 3, 1024, 1024],
                        opt_shape=[1, 3, 1024, 1024],
                        max_shape=[1, 3, 1024, 1024]
                    )
                )
            )
        ]
)

I also try with the default tensorrt version 8.2.x but no success

Could someone please help? Thanks a lot!!

Describe the bug 使用代码调用模型转换，将pytorch模型转换为tensorRT模型时执行失败 Using code to call model conversion, the execution fails when converting a pytorch model to a tensorRT model

运行流程为fastapi接收到模型转换请求 下发到huey队列 huey队列代码与deploy.py代码基本一致 The running process is that fastapi receives the model conversion request and sends it to the huey queue. The huey queue code is basically the same as the deploy.py code.

当我放弃tensorRT转为onnx后 可以正常转换 但是在实例化Detector后 接口层被阻塞在那里 并没有往下执行 When I gave up tensorRT and switched to onnx, I could convert normally, but after instantiating the Detector, the interface layer was blocked there and did not go down.

也没有错误信息 除了几个config的info日志 并没有任何其他的反馈 There is no error message, except for a few config info logs and no other feedback

Reproduction

1. What command or script did you run? 我没有运行命令去执行模型转换 而是通过代码调起deploy代码去执行转换 实际上和执行命令转换没有什么区别 可以尝试以下列命令复现 I did not run the command to perform the model transformation, but invoked the deploy code to perform the transformation through the code. In fact, it is no different from executing the command transformation. You can try the following command to reproduce

python tools/deploy.py \
    configs/mmdeploy/mmdet/detection/detection_tensorrt-fp16_dynamic-320x320-1344x1344.py \
    /static/work_dirs/bcccd9e0-41a1-408d-9dfa-f4e634e9608c/yolox_l_8x8_300e_coco.py \
    /static/work_dirs/bcccd9e0-41a1-408d-9dfa-f4e634e9608c/best_bbox_mAP_epoch_149.pth \
    /dataset/car-damage-coco/images/val/1.jpg \
    --work-dir /static/work_dirs/bcccd9e0-41a1-408d-9dfa-f4e634e9608c \
    --device cuda:0 \
    --log-level DEBUG \

2. Did you make any modifications on the code or config? Did you understand what you have modified?

Environment

2022-07-21 09:23:24,423 - mmdeploy - INFO - 

2022-07-21 09:23:24,424 - mmdeploy - INFO - **********Environmental information**********
2022-07-21 09:23:32,096 - mmdeploy - INFO - sys.platform: win32
2022-07-21 09:23:32,096 - mmdeploy - INFO - Python: 3.9.0 (default, Nov 15 2020, 08:30:55) [MSC v.1916 64 bit (AMD64)]
2022-07-21 09:23:32,096 - mmdeploy - INFO - CUDA available: True
2022-07-21 09:23:32,096 - mmdeploy - INFO - GPU 0: NVIDIA GeForce GTX 1060 6GB
2022-07-21 09:23:32,096 - mmdeploy - INFO - CUDA_HOME: D:\CUDAToolkit
2022-07-21 09:23:32,096 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.3, V11.3.109
2022-07-21 09:23:32,096 - mmdeploy - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.29.30145 版
2022-07-21 09:23:32,097 - mmdeploy - INFO - GCC: n/a
2022-07-21 09:23:32,097 - mmdeploy - INFO - PyTorch: 1.11.0+cu113
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.2
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=C:/actions-runner/_work/pytorch/pytorch/builder/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/actions-runner/_work/pytorch/pytorch/builder/windows/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

2022-07-21 09:23:32,098 - mmdeploy - INFO - TorchVision: 0.12.0+cu113
2022-07-21 09:23:32,098 - mmdeploy - INFO - OpenCV: 4.6.0
2022-07-21 09:23:32,098 - mmdeploy - INFO - MMCV: 1.6.0
2022-07-21 09:23:32,099 - mmdeploy - INFO - MMCV Compiler: MSVC 192930140
2022-07-21 09:23:32,099 - mmdeploy - INFO - MMCV CUDA Compiler: 11.3
2022-07-21 09:23:32,099 - mmdeploy - INFO - MMDeploy: 0.6.0+1d6437c
2022-07-21 09:23:32,099 - mmdeploy - INFO -

2022-07-21 09:23:32,099 - mmdeploy - INFO - **********Backend information**********
2022-07-21 09:23:36,070 - mmdeploy - INFO - onnxruntime: 1.11.1 ops_is_avaliable : True
2022-07-21 09:23:36,168 - mmdeploy - INFO - tensorrt: 8.4.1.5   ops_is_avaliable : True
2022-07-21 09:23:36,208 - mmdeploy - INFO - ncnn: None  ops_is_avaliable : False
2022-07-21 09:23:36,210 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-07-21 09:23:36,212 - mmdeploy - INFO - openvino_is_avaliable: False
2022-07-21 09:23:36,212 - mmdeploy - INFO - 

2022-07-21 09:23:36,212 - mmdeploy - INFO - **********Codebase information**********
2022-07-21 09:23:36,232 - mmdeploy - INFO - mmdet:      2.25.0
2022-07-21 09:23:36,232 - mmdeploy - INFO - mmseg:      None
2022-07-21 09:23:36,232 - mmdeploy - INFO - mmcls:      None
2022-07-21 09:23:36,232 - mmdeploy - INFO - mmocr:      None
2022-07-21 09:23:36,233 - mmdeploy - INFO - mmedit:     None
2022-07-21 09:23:36,233 - mmdeploy - INFO - mmdet3d:    None
2022-07-21 09:23:36,233 - mmdeploy - INFO - mmpose:     None
2022-07-21 09:23:36,233 - mmdeploy - INFO - mmrotate:   None

Error traceback 这是pytorch转tensorRT模型的日志 This is the log of pytorch to tensorRT model

2022-07-21 09:57:35,423 - mmdeploy - INFO - 当前任务ID：bcccd9e0-41a1-408d-9dfa-f4e634e9608c
Registry：{'input_size': (640, 640), 'random_size_range': (15, 25), 'random_size_interval': 10, 'backbone': {'type': 'CSPDarknet', 'deepen_factor': 1.0, 'widen_factor': 1.0}, 'neck': {'type': 'YOLOXPAFPN', 'in_channels': [256, 512, 1024], 'out_channels': 256, 'num_csp_blocks': 3}, 'bbox_head': {'type': 'YOLOXHead', 'num_classes': 5, 'in_channels': 256, 'feat_channels': 256}, 'train_cfg': None, 'test_cfg': {'score_thr':0.01, 'nms': {'type': 'nms', 'iou_threshold': 0.65}}}
Registry：{'deepen_factor': 1.0, 'widen_factor': 1.0}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{'in_channels': [256, 512, 1024], 'out_channels': 256, 'num_csp_blocks': 3}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{'num_classes': 5, 'in_channels': 256, 'feat_channels': 256, 'train_cfg': None, 'test_cfg': {'score_thr': 0.01, 'nms': {'type': 'nms', 'iou_threshold': 0.65}}}
Registry：{'use_sigmoid': True, 'reduction': 'sum', 'loss_weight': 1.0}
Registry：{'mode': 'square', 'eps': 1e-16, 'reduction': 'sum', 'loss_weight': 5.0}
Registry：{'use_sigmoid': True, 'reduction': 'sum', 'loss_weight': 1.0}
Registry：{'reduction': 'sum', 'loss_weight': 1.0}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
Registry：{}
load checkpoint from local path: D:\static\work_dirs\bcccd9e0-41a1-408d-9dfa-f4e634e9608c\best_bbox_mAP_epoch_149.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: ema_backbone_stem_conv_conv_weight, ema_backbone_stem_conv_bn_weight, ema_backbone_stem_conv_bn_bias, ema_backbone_stem_conv_bn_running_mean, ema_backbone_stem_conv_bn_running_var, ema_backbone_stem_conv_bn_num_batches_tracked, ema_backbone_stage1_0_conv_weight, ema_backbone_stage1_0_bn_weight, ema_backbone_stage1_0_bn_bias, ema_backbone_stage1_0_bn_running_mean, ema_backbone_stage1
_0_bn_running_var, ema_backbone_stage1_0_bn_num_batches_tracked, ema_backbone_stage1_1_main_conv_conv_weight, ema_backbone_stage1_1_main_conv_bn_weight, ema_backbone_stage1_1_main_conv_bn_bias, ema_backbone_stage1_1_main_conv_bn_running_mean, ema_backbone_stage1_1_main_conv_bn_running_var, ema_backbone_stage1_1_main_conv_bn_num_batches_tracked, ema_backbone_stage1_1_short_conv_conv_weight, ema_backbone_stage1_1_short_c
onv_bn_weight, ema_backbone_stage1_1_short_conv_bn_bias, ema_backbone_stage1_1_short_conv_bn_running_mean, ema_backbone_stage1_1_short_conv_bn_running_var, ema_backbone_stage1_1_short_conv_bn_num_batches_tracked, ema_backbone_stage1_1_final_conv_conv_weight, ema_backbone_stage1_1_final_conv_bn_weight, ema_backbone_stage1_1_final_conv_bn_bias, ema_backbone_stage1_1_final_conv_bn_running_mean, ema_backbone_stage1_1_final
_conv_bn_running_var, ema_backbone_stage1_1_final_conv_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_conv1_conv_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_weight, ema_backbone_stage1_1_blocks_0_conv1_bn_bias, ema_backbone_stage1_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv1_bn_running_var, ema_backbone_stage1_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_0_
conv2_conv_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_weight, ema_backbone_stage1_1_blocks_0_conv2_bn_bias, ema_backbone_stage1_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage1_1_blocks_0_conv2_bn_running_var, ema_backbone_stage1_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_1_conv1_conv_weight, ema_backbone_stage1_1_blocks_1_conv1_bn_weight, ema_backbone_stage1_1_blocks_1_conv1_bn_bia
s, ema_backbone_stage1_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage1_1_blocks_1_conv1_bn_running_var, ema_backbone_stage1_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_1_conv2_conv_weight, ema_backbone_stage1_1_blocks_1_conv2_bn_weight, ema_backbone_stage1_1_blocks_1_conv2_bn_bias, ema_backbone_stage1_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage1_1_blocks_1_conv2_bn_running_var, 
ema_backbone_stage1_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_2_conv1_conv_weight, ema_backbone_stage1_1_blocks_2_conv1_bn_weight, ema_backbone_stage1_1_blocks_2_conv1_bn_bias, ema_backbone_stage1_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage1_1_blocks_2_conv1_bn_running_var, ema_backbone_stage1_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage1_1_blocks_2_conv2_conv_weight,
 ema_backbone_stage1_1_blocks_2_conv2_bn_weight, ema_backbone_stage1_1_blocks_2_conv2_bn_bias, ema_backbone_stage1_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage1_1_blocks_2_conv2_bn_running_var, ema_backbone_stage1_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage2_0_conv_weight, ema_backbone_stage2_0_bn_weight, ema_backbone_stage2_0_bn_bias, ema_backbone_stage2_0_bn_running_mean, ema_backbone_stage2_0
_bn_running_var, ema_backbone_stage2_0_bn_num_batches_tracked, ema_backbone_stage2_1_main_conv_conv_weight, ema_backbone_stage2_1_main_conv_bn_weight, ema_backbone_stage2_1_main_conv_bn_bias, ema_backbone_stage2_1_main_conv_bn_running_mean, ema_backbone_stage2_1_main_conv_bn_running_var, ema_backbone_stage2_1_main_conv_bn_num_batches_tracked, ema_backbone_stage2_1_short_conv_conv_weight, ema_backbone_stage2_1_short_con
v_bn_weight, ema_backbone_stage2_1_short_conv_bn_bias, ema_backbone_stage2_1_short_conv_bn_running_mean, ema_backbone_stage2_1_short_conv_bn_running_var, ema_backbone_stage2_1_short_conv_bn_num_batches_tracked, ema_backbone_stage2_1_final_conv_conv_weight, ema_backbone_stage2_1_final_conv_bn_weight, ema_backbone_stage2_1_final_conv_bn_bias, ema_backbone_stage2_1_final_conv_bn_running_mean, ema_backbone_stage2_1_final_c
onv_bn_running_var, ema_backbone_stage2_1_final_conv_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_conv1_conv_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_weight, ema_backbone_stage2_1_blocks_0_conv1_bn_bias, ema_backbone_stage2_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv1_bn_running_var, ema_backbone_stage2_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_0_co
nv2_conv_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_weight, ema_backbone_stage2_1_blocks_0_conv2_bn_bias, ema_backbone_stage2_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_0_conv2_bn_running_var, ema_backbone_stage2_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv1_conv_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_weight, ema_backbone_stage2_1_blocks_1_conv1_bn_bias,
 ema_backbone_stage2_1_blocks_1_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv1_bn_running_var, ema_backbone_stage2_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_1_conv2_conv_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_weight, ema_backbone_stage2_1_blocks_1_conv2_bn_bias, ema_backbone_stage2_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_1_conv2_bn_running_var, em
a_backbone_stage2_1_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv1_conv_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_weight, ema_backbone_stage2_1_blocks_2_conv1_bn_bias, ema_backbone_stage2_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv1_bn_running_var, ema_backbone_stage2_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_2_conv2_conv_weight, e
ma_backbone_stage2_1_blocks_2_conv2_bn_weight, ema_backbone_stage2_1_blocks_2_conv2_bn_bias, ema_backbone_stage2_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_2_conv2_bn_running_var, ema_backbone_stage2_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_3_conv1_conv_weight, ema_backbone_stage2_1_blocks_3_conv1_bn_weight, ema_backbone_stage2_1_blocks_3_conv1_bn_bias, ema_backbone_stag
e2_1_blocks_3_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_3_conv1_bn_running_var, ema_backbone_stage2_1_blocks_3_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_3_conv2_conv_weight, ema_backbone_stage2_1_blocks_3_conv2_bn_weight, ema_backbone_stage2_1_blocks_3_conv2_bn_bias, ema_backbone_stage2_1_blocks_3_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_3_conv2_bn_running_var, ema_backbone_stage2_
1_blocks_3_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_4_conv1_conv_weight, ema_backbone_stage2_1_blocks_4_conv1_bn_weight, ema_backbone_stage2_1_blocks_4_conv1_bn_bias, ema_backbone_stage2_1_blocks_4_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_4_conv1_bn_running_var, ema_backbone_stage2_1_blocks_4_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_4_conv2_conv_weight, ema_backbone_stage2
_1_blocks_4_conv2_bn_weight, ema_backbone_stage2_1_blocks_4_conv2_bn_bias, ema_backbone_stage2_1_blocks_4_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_4_conv2_bn_running_var, ema_backbone_stage2_1_blocks_4_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_5_conv1_conv_weight, ema_backbone_stage2_1_blocks_5_conv1_bn_weight, ema_backbone_stage2_1_blocks_5_conv1_bn_bias, ema_backbone_stage2_1_blocks_5_conv
1_bn_running_mean, ema_backbone_stage2_1_blocks_5_conv1_bn_running_var, ema_backbone_stage2_1_blocks_5_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_5_conv2_conv_weight, ema_backbone_stage2_1_blocks_5_conv2_bn_weight, ema_backbone_stage2_1_blocks_5_conv2_bn_bias, ema_backbone_stage2_1_blocks_5_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_5_conv2_bn_running_var, ema_backbone_stage2_1_blocks_5_conv2_b
n_num_batches_tracked, ema_backbone_stage2_1_blocks_6_conv1_conv_weight, ema_backbone_stage2_1_blocks_6_conv1_bn_weight, ema_backbone_stage2_1_blocks_6_conv1_bn_bias, ema_backbone_stage2_1_blocks_6_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_6_conv1_bn_running_var, ema_backbone_stage2_1_blocks_6_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_6_conv2_conv_weight, ema_backbone_stage2_1_blocks_6_conv2_
bn_weight, ema_backbone_stage2_1_blocks_6_conv2_bn_bias, ema_backbone_stage2_1_blocks_6_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_6_conv2_bn_running_var, ema_backbone_stage2_1_blocks_6_conv2_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_7_conv1_conv_weight, ema_backbone_stage2_1_blocks_7_conv1_bn_weight, ema_backbone_stage2_1_blocks_7_conv1_bn_bias, ema_backbone_stage2_1_blocks_7_conv1_bn_running_mean,
 ema_backbone_stage2_1_blocks_7_conv1_bn_running_var, ema_backbone_stage2_1_blocks_7_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_7_conv2_conv_weight, ema_backbone_stage2_1_blocks_7_conv2_bn_weight, ema_backbone_stage2_1_blocks_7_conv2_bn_bias, ema_backbone_stage2_1_blocks_7_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_7_conv2_bn_running_var, ema_backbone_stage2_1_blocks_7_conv2_bn_num_batches_trac
ked, ema_backbone_stage2_1_blocks_8_conv1_conv_weight, ema_backbone_stage2_1_blocks_8_conv1_bn_weight, ema_backbone_stage2_1_blocks_8_conv1_bn_bias, ema_backbone_stage2_1_blocks_8_conv1_bn_running_mean, ema_backbone_stage2_1_blocks_8_conv1_bn_running_var, ema_backbone_stage2_1_blocks_8_conv1_bn_num_batches_tracked, ema_backbone_stage2_1_blocks_8_conv2_conv_weight, ema_backbone_stage2_1_blocks_8_conv2_bn_weight, ema_bac
kbone_stage2_1_blocks_8_conv2_bn_bias, ema_backbone_stage2_1_blocks_8_conv2_bn_running_mean, ema_backbone_stage2_1_blocks_8_conv2_bn_running_var, ema_backbone_stage2_1_blocks_8_conv2_bn_num_batches_tracked, ema_backbone_stage3_0_conv_weight, ema_backbone_stage3_0_bn_weight, ema_backbone_stage3_0_bn_bias, ema_backbone_stage3_0_bn_running_mean, ema_backbone_stage3_0_bn_running_var, ema_backbone_stage3_0_bn_num_batches_tr
acked, ema_backbone_stage3_1_main_conv_conv_weight, ema_backbone_stage3_1_main_conv_bn_weight, ema_backbone_stage3_1_main_conv_bn_bias, ema_backbone_stage3_1_main_conv_bn_running_mean, ema_backbone_stage3_1_main_conv_bn_running_var, ema_backbone_stage3_1_main_conv_bn_num_batches_tracked, ema_backbone_stage3_1_short_conv_conv_weight, ema_backbone_stage3_1_short_conv_bn_weight, ema_backbone_stage3_1_short_conv_bn_bias, e
ma_backbone_stage3_1_short_conv_bn_running_mean, ema_backbone_stage3_1_short_conv_bn_running_var, ema_backbone_stage3_1_short_conv_bn_num_batches_tracked, ema_backbone_stage3_1_final_conv_conv_weight, ema_backbone_stage3_1_final_conv_bn_weight, ema_backbone_stage3_1_final_conv_bn_bias, ema_backbone_stage3_1_final_conv_bn_running_mean, ema_backbone_stage3_1_final_conv_bn_running_var, ema_backbone_stage3_1_final_conv_bn_
num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv1_conv_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_weight, ema_backbone_stage3_1_blocks_0_conv1_bn_bias, ema_backbone_stage3_1_blocks_0_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv1_bn_running_var, ema_backbone_stage3_1_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_0_conv2_conv_weight, ema_backbone_stage3_1_blocks_0_conv2_bn
_weight, ema_backbone_stage3_1_blocks_0_conv2_bn_bias, ema_backbone_stage3_1_blocks_0_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_0_conv2_bn_running_var, ema_backbone_stage3_1_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv1_conv_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_weight, ema_backbone_stage3_1_blocks_1_conv1_bn_bias, ema_backbone_stage3_1_blocks_1_conv1_bn_running_mean, e
ma_backbone_stage3_1_blocks_1_conv1_bn_running_var, ema_backbone_stage3_1_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_1_conv2_conv_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_weight, ema_backbone_stage3_1_blocks_1_conv2_bn_bias, ema_backbone_stage3_1_blocks_1_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_1_conv2_bn_running_var, ema_backbone_stage3_1_blocks_1_conv2_bn_num_batches_tracke
d, ema_backbone_stage3_1_blocks_2_conv1_conv_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_weight, ema_backbone_stage3_1_blocks_2_conv1_bn_bias, ema_backbone_stage3_1_blocks_2_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv1_bn_running_var, ema_backbone_stage3_1_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_2_conv2_conv_weight, ema_backbone_stage3_1_blocks_2_conv2_bn_weight, ema_backb
one_stage3_1_blocks_2_conv2_bn_bias, ema_backbone_stage3_1_blocks_2_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_2_conv2_bn_running_var, ema_backbone_stage3_1_blocks_2_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_3_conv1_conv_weight, ema_backbone_stage3_1_blocks_3_conv1_bn_weight, ema_backbone_stage3_1_blocks_3_conv1_bn_bias, ema_backbone_stage3_1_blocks_3_conv1_bn_running_mean, ema_backbone_stage3
_1_blocks_3_conv1_bn_running_var, ema_backbone_stage3_1_blocks_3_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_3_conv2_conv_weight, ema_backbone_stage3_1_blocks_3_conv2_bn_weight, ema_backbone_stage3_1_blocks_3_conv2_bn_bias, ema_backbone_stage3_1_blocks_3_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_3_conv2_bn_running_var, ema_backbone_stage3_1_blocks_3_conv2_bn_num_batches_tracked, ema_backbone_st
age3_1_blocks_4_conv1_conv_weight, ema_backbone_stage3_1_blocks_4_conv1_bn_weight, ema_backbone_stage3_1_blocks_4_conv1_bn_bias, ema_backbone_stage3_1_blocks_4_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_4_conv1_bn_running_var, ema_backbone_stage3_1_blocks_4_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_4_conv2_conv_weight, ema_backbone_stage3_1_blocks_4_conv2_bn_weight, ema_backbone_stage3_1_block
s_4_conv2_bn_bias, ema_backbone_stage3_1_blocks_4_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_4_conv2_bn_running_var, ema_backbone_stage3_1_blocks_4_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_5_conv1_conv_weight, ema_backbone_stage3_1_blocks_5_conv1_bn_weight, ema_backbone_stage3_1_blocks_5_conv1_bn_bias, ema_backbone_stage3_1_blocks_5_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_5_conv1_
bn_running_var, ema_backbone_stage3_1_blocks_5_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_5_conv2_conv_weight, ema_backbone_stage3_1_blocks_5_conv2_bn_weight, ema_backbone_stage3_1_blocks_5_conv2_bn_bias, ema_backbone_stage3_1_blocks_5_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_5_conv2_bn_running_var, ema_backbone_stage3_1_blocks_5_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_6_co
nv1_conv_weight, ema_backbone_stage3_1_blocks_6_conv1_bn_weight, ema_backbone_stage3_1_blocks_6_conv1_bn_bias, ema_backbone_stage3_1_blocks_6_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_6_conv1_bn_running_var, ema_backbone_stage3_1_blocks_6_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_6_conv2_conv_weight, ema_backbone_stage3_1_blocks_6_conv2_bn_weight, ema_backbone_stage3_1_blocks_6_conv2_bn_bias,
 ema_backbone_stage3_1_blocks_6_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_6_conv2_bn_running_var, ema_backbone_stage3_1_blocks_6_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_7_conv1_conv_weight, ema_backbone_stage3_1_blocks_7_conv1_bn_weight, ema_backbone_stage3_1_blocks_7_conv1_bn_bias, ema_backbone_stage3_1_blocks_7_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_7_conv1_bn_running_var, em
a_backbone_stage3_1_blocks_7_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_7_conv2_conv_weight, ema_backbone_stage3_1_blocks_7_conv2_bn_weight, ema_backbone_stage3_1_blocks_7_conv2_bn_bias, ema_backbone_stage3_1_blocks_7_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_7_conv2_bn_running_var, ema_backbone_stage3_1_blocks_7_conv2_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_8_conv1_conv_weight, e
ma_backbone_stage3_1_blocks_8_conv1_bn_weight, ema_backbone_stage3_1_blocks_8_conv1_bn_bias, ema_backbone_stage3_1_blocks_8_conv1_bn_running_mean, ema_backbone_stage3_1_blocks_8_conv1_bn_running_var, ema_backbone_stage3_1_blocks_8_conv1_bn_num_batches_tracked, ema_backbone_stage3_1_blocks_8_conv2_conv_weight, ema_backbone_stage3_1_blocks_8_conv2_bn_weight, ema_backbone_stage3_1_blocks_8_conv2_bn_bias, ema_backbone_stag
e3_1_blocks_8_conv2_bn_running_mean, ema_backbone_stage3_1_blocks_8_conv2_bn_running_var, ema_backbone_stage3_1_blocks_8_conv2_bn_num_batches_tracked, ema_backbone_stage4_0_conv_weight, ema_backbone_stage4_0_bn_weight, ema_backbone_stage4_0_bn_bias, ema_backbone_stage4_0_bn_running_mean, ema_backbone_stage4_0_bn_running_var, ema_backbone_stage4_0_bn_num_batches_tracked, ema_backbone_stage4_1_conv1_conv_weight, ema_back
bone_stage4_1_conv1_bn_weight, ema_backbone_stage4_1_conv1_bn_bias, ema_backbone_stage4_1_conv1_bn_running_mean, ema_backbone_stage4_1_conv1_bn_running_var, ema_backbone_stage4_1_conv1_bn_num_batches_tracked, ema_backbone_stage4_1_conv2_conv_weight, ema_backbone_stage4_1_conv2_bn_weight, ema_backbone_stage4_1_conv2_bn_bias, ema_backbone_stage4_1_conv2_bn_running_mean, ema_backbone_stage4_1_conv2_bn_running_var, ema_bac
kbone_stage4_1_conv2_bn_num_batches_tracked, ema_backbone_stage4_2_main_conv_conv_weight, ema_backbone_stage4_2_main_conv_bn_weight, ema_backbone_stage4_2_main_conv_bn_bias, ema_backbone_stage4_2_main_conv_bn_running_mean, ema_backbone_stage4_2_main_conv_bn_running_var, ema_backbone_stage4_2_main_conv_bn_num_batches_tracked, ema_backbone_stage4_2_short_conv_conv_weight, ema_backbone_stage4_2_short_conv_bn_weight, ema_b
ackbone_stage4_2_short_conv_bn_bias, ema_backbone_stage4_2_short_conv_bn_running_mean, ema_backbone_stage4_2_short_conv_bn_running_var, ema_backbone_stage4_2_short_conv_bn_num_batches_tracked, ema_backbone_stage4_2_final_conv_conv_weight, ema_backbone_stage4_2_final_conv_bn_weight, ema_backbone_stage4_2_final_conv_bn_bias, ema_backbone_stage4_2_final_conv_bn_running_mean, ema_backbone_stage4_2_final_conv_bn_running_var
, ema_backbone_stage4_2_final_conv_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv1_conv_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_weight, ema_backbone_stage4_2_blocks_0_conv1_bn_bias, ema_backbone_stage4_2_blocks_0_conv1_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv1_bn_running_var, ema_backbone_stage4_2_blocks_0_conv1_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_0_conv2_conv_weight, e
ma_backbone_stage4_2_blocks_0_conv2_bn_weight, ema_backbone_stage4_2_blocks_0_conv2_bn_bias, ema_backbone_stage4_2_blocks_0_conv2_bn_running_mean, ema_backbone_stage4_2_blocks_0_conv2_bn_running_var, ema_backbone_stage4_2_blocks_0_conv2_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_1_conv1_conv_weight, ema_backbone_stage4_2_blocks_1_conv1_bn_weight, ema_backbone_stage4_2_blocks_1_conv1_bn_bias, ema_backbone_stag
e4_2_blocks_1_conv1_bn_running_mean, ema_backbone_stage4_2_blocks_1_conv1_bn_running_var, ema_backbone_stage4_2_blocks_1_conv1_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_1_conv2_conv_weight, ema_backbone_stage4_2_blocks_1_conv2_bn_weight, ema_backbone_stage4_2_blocks_1_conv2_bn_bias, ema_backbone_stage4_2_blocks_1_conv2_bn_running_mean, ema_backbone_stage4_2_blocks_1_conv2_bn_running_var, ema_backbone_stage4_
2_blocks_1_conv2_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_2_conv1_conv_weight, ema_backbone_stage4_2_blocks_2_conv1_bn_weight, ema_backbone_stage4_2_blocks_2_conv1_bn_bias, ema_backbone_stage4_2_blocks_2_conv1_bn_running_mean, ema_backbone_stage4_2_blocks_2_conv1_bn_running_var, ema_backbone_stage4_2_blocks_2_conv1_bn_num_batches_tracked, ema_backbone_stage4_2_blocks_2_conv2_conv_weight, ema_backbone_stage4
_2_blocks_2_conv2_bn_weight, ema_backbone_stage4_2_blocks_2_conv2_bn_bias, ema_backbone_stage4_2_blocks_2_conv2_bn_running_mean, ema_backbone_stage4_2_blocks_2_conv2_bn_running_var, ema_backbone_stage4_2_blocks_2_conv2_bn_num_batches_tracked, ema_neck_reduce_layers_0_conv_weight, ema_neck_reduce_layers_0_bn_weight, ema_neck_reduce_layers_0_bn_bias, ema_neck_reduce_layers_0_bn_running_mean, ema_neck_reduce_layers_0_bn_r
unning_var, ema_neck_reduce_layers_0_bn_num_batches_tracked, ema_neck_reduce_layers_1_conv_weight, ema_neck_reduce_layers_1_bn_weight, ema_neck_reduce_layers_1_bn_bias, ema_neck_reduce_layers_1_bn_running_mean, ema_neck_reduce_layers_1_bn_running_var, ema_neck_reduce_layers_1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_main_conv_conv_weight, ema_neck_top_down_blocks_0_main_conv_bn_weight, ema_neck_top_down_block
s_0_main_conv_bn_bias, ema_neck_top_down_blocks_0_main_conv_bn_running_mean, ema_neck_top_down_blocks_0_main_conv_bn_running_var, ema_neck_top_down_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_short_conv_conv_weight, ema_neck_top_down_blocks_0_short_conv_bn_weight, ema_neck_top_down_blocks_0_short_conv_bn_bias, ema_neck_top_down_blocks_0_short_conv_bn_running_mean, ema_neck_top_down_blocks_0_sh
ort_conv_bn_running_var, ema_neck_top_down_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_0_final_conv_conv_weight, ema_neck_top_down_blocks_0_final_conv_bn_weight, ema_neck_top_down_blocks_0_final_conv_bn_bias, ema_neck_top_down_blocks_0_final_conv_bn_running_mean, ema_neck_top_down_blocks_0_final_conv_bn_running_var, ema_neck_top_down_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_top_down
_blocks_0_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_0_conv2_conv_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_
bn_weight, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_1_conv1_conv_weight, ema_neck_top_down_blocks_0_blocks_1_conv1_bn_weight, ema_neck_top_down_blocks_0_blocks_1_conv1_bn_bias, ema_neck_top_down_
blocks_0_blocks_1_conv1_bn_running_mean, ema_neck_top_down_blocks_0_blocks_1_conv1_bn_running_var, ema_neck_top_down_blocks_0_blocks_1_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_1_conv2_conv_weight, ema_neck_top_down_blocks_0_blocks_1_conv2_bn_weight, ema_neck_top_down_blocks_0_blocks_1_conv2_bn_bias, ema_neck_top_down_blocks_0_blocks_1_conv2_bn_running_mean, ema_neck_top_down_blocks_0_blocks_1_con
v2_bn_running_var, ema_neck_top_down_blocks_0_blocks_1_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_0_blocks_2_conv1_conv_weight, ema_neck_top_down_blocks_0_blocks_2_conv1_bn_weight, ema_neck_top_down_blocks_0_blocks_2_conv1_bn_bias, ema_neck_top_down_blocks_0_blocks_2_conv1_bn_running_mean, ema_neck_top_down_blocks_0_blocks_2_conv1_bn_running_var, ema_neck_top_down_blocks_0_blocks_2_conv1_bn_num_batches_trac
ked, ema_neck_top_down_blocks_0_blocks_2_conv2_conv_weight, ema_neck_top_down_blocks_0_blocks_2_conv2_bn_weight, ema_neck_top_down_blocks_0_blocks_2_conv2_bn_bias, ema_neck_top_down_blocks_0_blocks_2_conv2_bn_running_mean, ema_neck_top_down_blocks_0_blocks_2_conv2_bn_running_var, ema_neck_top_down_blocks_0_blocks_2_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_1_main_conv_conv_weight, ema_neck_top_down_blocks_
1_main_conv_bn_weight, ema_neck_top_down_blocks_1_main_conv_bn_bias, ema_neck_top_down_blocks_1_main_conv_bn_running_mean, ema_neck_top_down_blocks_1_main_conv_bn_running_var, ema_neck_top_down_blocks_1_main_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_short_conv_conv_weight, ema_neck_top_down_blocks_1_short_conv_bn_weight, ema_neck_top_down_blocks_1_short_conv_bn_bias, ema_neck_top_down_blocks_1_short_conv_
bn_running_mean, ema_neck_top_down_blocks_1_short_conv_bn_running_var, ema_neck_top_down_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_final_conv_conv_weight, ema_neck_top_down_blocks_1_final_conv_bn_weight, ema_neck_top_down_blocks_1_final_conv_bn_bias, ema_neck_top_down_blocks_1_final_conv_bn_running_mean, ema_neck_top_down_blocks_1_final_conv_bn_running_var, ema_neck_top_down_blocks_1_final_
conv_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv1_conv_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_0_conv2_conv_weig
ht, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_weight, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_bias, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_running_var, ema_neck_top_down_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_1_conv1_conv_weight, ema_neck_top_down_blocks_1_blocks_1_conv1_bn_weight, ema_neck_top_down_block
s_1_blocks_1_conv1_bn_bias, ema_neck_top_down_blocks_1_blocks_1_conv1_bn_running_mean, ema_neck_top_down_blocks_1_blocks_1_conv1_bn_running_var, ema_neck_top_down_blocks_1_blocks_1_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_1_conv2_conv_weight, ema_neck_top_down_blocks_1_blocks_1_conv2_bn_weight, ema_neck_top_down_blocks_1_blocks_1_conv2_bn_bias, ema_neck_top_down_blocks_1_blocks_1_conv2_bn_running
_mean, ema_neck_top_down_blocks_1_blocks_1_conv2_bn_running_var, ema_neck_top_down_blocks_1_blocks_1_conv2_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_2_conv1_conv_weight, ema_neck_top_down_blocks_1_blocks_2_conv1_bn_weight, ema_neck_top_down_blocks_1_blocks_2_conv1_bn_bias, ema_neck_top_down_blocks_1_blocks_2_conv1_bn_running_mean, ema_neck_top_down_blocks_1_blocks_2_conv1_bn_running_var, ema_neck_top_do
wn_blocks_1_blocks_2_conv1_bn_num_batches_tracked, ema_neck_top_down_blocks_1_blocks_2_conv2_conv_weight, ema_neck_top_down_blocks_1_blocks_2_conv2_bn_weight, ema_neck_top_down_blocks_1_blocks_2_conv2_bn_bias, ema_neck_top_down_blocks_1_blocks_2_conv2_bn_running_mean, ema_neck_top_down_blocks_1_blocks_2_conv2_bn_running_var, ema_neck_top_down_blocks_1_blocks_2_conv2_bn_num_batches_tracked, ema_neck_downsamples_0_conv_w
eight, ema_neck_downsamples_0_bn_weight, ema_neck_downsamples_0_bn_bias, ema_neck_downsamples_0_bn_running_mean, ema_neck_downsamples_0_bn_running_var, ema_neck_downsamples_0_bn_num_batches_tracked, ema_neck_downsamples_1_conv_weight, ema_neck_downsamples_1_bn_weight, ema_neck_downsamples_1_bn_bias, ema_neck_downsamples_1_bn_running_mean, ema_neck_downsamples_1_bn_running_var, ema_neck_downsamples_1_bn_num_batches_trac
ked, ema_neck_bottom_up_blocks_0_main_conv_conv_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_weight, ema_neck_bottom_up_blocks_0_main_conv_bn_bias, ema_neck_bottom_up_blocks_0_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_main_conv_bn_running_var, ema_neck_bottom_up_blocks_0_main_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_short_conv_conv_weight, ema_neck_bottom_up_blocks_0_short_conv_bn_weigh
t, ema_neck_bottom_up_blocks_0_short_conv_bn_bias, ema_neck_bottom_up_blocks_0_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_0_short_conv_bn_running_var, ema_neck_bottom_up_blocks_0_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_final_conv_conv_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_weight, ema_neck_bottom_up_blocks_0_final_conv_bn_bias, ema_neck_bottom_up_blocks_0_final_conv_bn_runni
ng_mean, ema_neck_bottom_up_blocks_0_final_conv_bn_running_var, ema_neck_bottom_up_blocks_0_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv1_bn_running_var, ema_neck_botto
m_up_blocks_0_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_0_conv2_bn_num_batches_tracked, ema_neck_bottom_up_bl
ocks_0_blocks_1_conv1_conv_weight, ema_neck_bottom_up_blocks_0_blocks_1_conv1_bn_weight, ema_neck_bottom_up_blocks_0_blocks_1_conv1_bn_bias, ema_neck_bottom_up_blocks_0_blocks_1_conv1_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_1_conv1_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_1_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_1_conv2_conv_weight, ema_neck_bottom_up_blocks_0_blocks_1_co
nv2_bn_weight, ema_neck_bottom_up_blocks_0_blocks_1_conv2_bn_bias, ema_neck_bottom_up_blocks_0_blocks_1_conv2_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_1_conv2_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_1_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_2_conv1_conv_weight, ema_neck_bottom_up_blocks_0_blocks_2_conv1_bn_weight, ema_neck_bottom_up_blocks_0_blocks_2_conv1_bn_bias, ema_nec
k_bottom_up_blocks_0_blocks_2_conv1_bn_running_mean, ema_neck_bottom_up_blocks_0_blocks_2_conv1_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_2_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_0_blocks_2_conv2_conv_weight, ema_neck_bottom_up_blocks_0_blocks_2_conv2_bn_weight, ema_neck_bottom_up_blocks_0_blocks_2_conv2_bn_bias, ema_neck_bottom_up_blocks_0_blocks_2_conv2_bn_running_mean, ema_neck_bottom_up_bl
ocks_0_blocks_2_conv2_bn_running_var, ema_neck_bottom_up_blocks_0_blocks_2_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_main_conv_conv_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_weight, ema_neck_bottom_up_blocks_1_main_conv_bn_bias, ema_neck_bottom_up_blocks_1_main_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_main_conv_bn_running_var, ema_neck_bottom_up_blocks_1_main_conv_bn_num_batches_tracked,
 ema_neck_bottom_up_blocks_1_short_conv_conv_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_weight, ema_neck_bottom_up_blocks_1_short_conv_bn_bias, ema_neck_bottom_up_blocks_1_short_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_short_conv_bn_running_var, ema_neck_bottom_up_blocks_1_short_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_final_conv_conv_weight, ema_neck_bottom_up_blocks_1_final_conv_bn_wei
ght, ema_neck_bottom_up_blocks_1_final_conv_bn_bias, ema_neck_bottom_up_blocks_1_final_conv_bn_running_mean, ema_neck_bottom_up_blocks_1_final_conv_bn_running_var, ema_neck_bottom_up_blocks_1_final_conv_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv1_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_bias, ema_neck_bottom_up_blocks_1_block
s_0_conv1_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_0_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_0_conv2_conv_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_weight, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_bias, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_r
unning_var, ema_neck_bottom_up_blocks_1_blocks_0_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_1_conv1_conv_weight, ema_neck_bottom_up_blocks_1_blocks_1_conv1_bn_weight, ema_neck_bottom_up_blocks_1_blocks_1_conv1_bn_bias, ema_neck_bottom_up_blocks_1_blocks_1_conv1_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_1_conv1_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_1_conv1_bn_num_batches_trac
ked, ema_neck_bottom_up_blocks_1_blocks_1_conv2_conv_weight, ema_neck_bottom_up_blocks_1_blocks_1_conv2_bn_weight, ema_neck_bottom_up_blocks_1_blocks_1_conv2_bn_bias, ema_neck_bottom_up_blocks_1_blocks_1_conv2_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_1_conv2_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_1_conv2_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_2_conv1_conv_weight, ema_neck_bott
om_up_blocks_1_blocks_2_conv1_bn_weight, ema_neck_bottom_up_blocks_1_blocks_2_conv1_bn_bias, ema_neck_bottom_up_blocks_1_blocks_2_conv1_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_2_conv1_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_2_conv1_bn_num_batches_tracked, ema_neck_bottom_up_blocks_1_blocks_2_conv2_conv_weight, ema_neck_bottom_up_blocks_1_blocks_2_conv2_bn_weight, ema_neck_bottom_up_blocks_1_block
s_2_conv2_bn_bias, ema_neck_bottom_up_blocks_1_blocks_2_conv2_bn_running_mean, ema_neck_bottom_up_blocks_1_blocks_2_conv2_bn_running_var, ema_neck_bottom_up_blocks_1_blocks_2_conv2_bn_num_batches_tracked, ema_neck_out_convs_0_conv_weight, ema_neck_out_convs_0_bn_weight, ema_neck_out_convs_0_bn_bias, ema_neck_out_convs_0_bn_running_mean, ema_neck_out_convs_0_bn_running_var, ema_neck_out_convs_0_bn_num_batches_tracked, e
ma_neck_out_convs_1_conv_weight, ema_neck_out_convs_1_bn_weight, ema_neck_out_convs_1_bn_bias, ema_neck_out_convs_1_bn_running_mean, ema_neck_out_convs_1_bn_running_var, ema_neck_out_convs_1_bn_num_batches_tracked, ema_neck_out_convs_2_conv_weight, ema_neck_out_convs_2_bn_weight, ema_neck_out_convs_2_bn_bias, ema_neck_out_convs_2_bn_running_mean, ema_neck_out_convs_2_bn_running_var, ema_neck_out_convs_2_bn_num_batches_
tracked, ema_bbox_head_multi_level_cls_convs_0_0_conv_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_weight, ema_bbox_head_multi_level_cls_convs_0_0_bn_bias, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_0_1_conv_weight, ema_bbox_head_multi_level_cls_
convs_0_1_bn_weight, ema_bbox_head_multi_level_cls_convs_0_1_bn_bias, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_0_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_0_conv_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_weight, ema_bbox_head_multi_level_cls_convs_1_0_bn_bias, ema_bbox_head_multi_le
vel_cls_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_1_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_1_1_conv_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_weight, ema_bbox_head_multi_level_cls_convs_1_1_bn_bias, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_1_1_bn_running_var, e
ma_bbox_head_multi_level_cls_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_cls_convs_2_0_conv_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_weight, ema_bbox_head_multi_level_cls_convs_2_0_bn_bias, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_0_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_cls
_convs_2_1_conv_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_weight, ema_bbox_head_multi_level_cls_convs_2_1_bn_bias, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_cls_convs_2_1_bn_running_var, ema_bbox_head_multi_level_cls_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_0_conv_weight, ema_bbox_head_multi_level_reg_convs_0_0_bn_weight, ema_bbox_head_mul
ti_level_reg_convs_0_0_bn_bias, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_0_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_0_1_conv_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_weight, ema_bbox_head_multi_level_reg_convs_0_1_bn_bias, ema_bbox_head_multi_level_reg_convs_0_1_bn_running_mean, ema
_bbox_head_multi_level_reg_convs_0_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_0_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_0_conv_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_weight, ema_bbox_head_multi_level_reg_convs_1_0_bn_bias, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_0
_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_1_1_conv_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_weight, ema_bbox_head_multi_level_reg_convs_1_1_bn_bias, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_1_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_1_1_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_0_conv_weight, ema_bbox_head_
multi_level_reg_convs_2_0_bn_weight, ema_bbox_head_multi_level_reg_convs_2_0_bn_bias, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_0_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_0_bn_num_batches_tracked, ema_bbox_head_multi_level_reg_convs_2_1_conv_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_weight, ema_bbox_head_multi_level_reg_convs_2_1_bn_bias, ema_bb
ox_head_multi_level_reg_convs_2_1_bn_running_mean, ema_bbox_head_multi_level_reg_convs_2_1_bn_running_var, ema_bbox_head_multi_level_reg_convs_2_1_bn_num_batches_tracked, ema_bbox_head_multi_level_conv_cls_0_weight, ema_bbox_head_multi_level_conv_cls_0_bias, ema_bbox_head_multi_level_conv_cls_1_weight, ema_bbox_head_multi_level_conv_cls_1_bias, ema_bbox_head_multi_level_conv_cls_2_weight, ema_bbox_head_multi_level_conv
_cls_2_bias, ema_bbox_head_multi_level_conv_reg_0_weight, ema_bbox_head_multi_level_conv_reg_0_bias, ema_bbox_head_multi_level_conv_reg_1_weight, ema_bbox_head_multi_level_conv_reg_1_bias, ema_bbox_head_multi_level_conv_reg_2_weight, ema_bbox_head_multi_level_conv_reg_2_bias, ema_bbox_head_multi_level_conv_obj_0_weight, ema_bbox_head_multi_level_conv_obj_0_bias, ema_bbox_head_multi_level_conv_obj_1_weight, ema_bbox_head_multi_level_conv_obj_1_bias, ema_bbox_head_multi_level_conv_obj_2_weight, ema_bbox_head_multi_level_conv_obj_2_bias

Registry：{}
Registry：{'img_scale': (640, 640), 'flip': False, 'transforms': [{'type': 'Resize', 'keep_ratio': True}, {'type': 'RandomFlip'}, {'type': 'Pad', 'pad_to_square': True, 'pad_val': {'img': (114.0, 114.0, 114.0)}}, {'type': 'DefaultFormatBundle'}, {'type': 'Collect', 'keys': ['img']}]}
Registry：{'keep_ratio': True}
Registry：{}
Registry：{'pad_to_square': True, 'pad_val': {'img': (114.0, 114.0, 114.0)}}
Registry：{}
Registry：{'keys': ['img']}
2022-07-21 09:57:54,053 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2022-07-21 09:57:54,053 - mmdeploy - INFO - Export PyTorch model to ONNX: /static/work_dirs/bcccd9e0-41a1-408d-9dfa-f4e634e9608c\end2end.onnx.
D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
D:\Anaconda3\envs\aoc\lib\site-packages\torch\functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:2228.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\codebase\mmdet\core\post_processing\bbox_nms.py:259: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\mmcv\ops\nms.py:178: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-07-21 09:58:17,235 - mmdeploy - INFO - Execute onnx optimize passes.
2022-07-21 09:58:17,235 - mmdeploy - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/blob/master/docs/en/experimental/onnx_optimizer.md
2022-07-21 09:58:21,554 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess
2022-07-21 09:58:22,324 - mmdeploy - INFO - Successfully loaded tensorrt plugins from D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\lib\mmdeploy_tensorrt_ops.dll
[07/21/2022-09:58:25] [TRT] [I] [MemUsageChange] Init CUDA: CPU +198, GPU +0, now: CPU 10828, GPU 990 (MiB)
[07/21/2022-09:58:27] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +6, GPU +2, now: CPU 11015, GPU 992 (MiB)
[07/21/2022-09:58:27] [TRT] [W] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[07/21/2022-09:58:28] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[07/21/2022-09:58:28] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[07/21/2022-09:58:28] [TRT] [W] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[07/21/2022-09:58:28] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[07/21/2022-09:58:28] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[07/21/2022-09:58:28] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[07/21/2022-09:58:28] [TRT] [W] FP16 support requested on hardware without native FP16 support, performance will be negatively affected.
[07/21/2022-09:58:30] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +211, GPU +74, now: CPU 11542, GPU 1066 (MiB)
[07/21/2022-09:58:31] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +171, GPU +80, now: CPU 11713, GPU 1146 (MiB)
[07/21/2022-09:58:31] [TRT] [W] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.2.0
[07/21/2022-09:58:31] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[07/21/2022-09:58:31] [TRT] [E] 4: [shapeCompiler.cpp::nvinfer1::builder::DynamicSlotBuilder::evaluateShapeChecks::911] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: condition '==' violated. 6400 != 16800. Concat_505: dimensions not compatible for concatenation)
Process Process-3:
Traceback (most recent call last):
  File "D:\Anaconda3\envs\aoc\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "D:\Anaconda3\envs\aoc\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\backend\tensorrt\onnx2tensorrt.py", line 79, in onnx2tensorrt
    from_onnx(
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\backend\tensorrt\utils.py", line 153, in from_onnx
    assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-07-21 09:58:31,918 - mmdeploy - ERROR - `mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt` with Call id: 1 failed. exit.
[2022-07-21 09:58:31,919] ERROR:huey.consumer:Worker-1:Process Worker-1 died!
Traceback (most recent call last):
  File "D:\Anaconda3\envs\aoc\lib\site-packages\huey\consumer.py", line 356, in _run
    process.loop()
  File "D:\Anaconda3\envs\aoc\lib\site-packages\huey\consumer.py", line 117, in loop
    self.huey.execute(task, now)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\huey\api.py", line 362, in execute
    return self._execute(task, timestamp)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\huey\api.py", line 379, in _execute
    task_value = task.execute()
  File "D:\Anaconda3\envs\aoc\lib\site-packages\huey\api.py", line 772, in execute
    return func(*args, **kwargs)
  File "D:\workspace\python\ai-online-core\publish.py", line 153, in deploy
    onnx2tensorrt(
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 324, in call_function
    return self.get_result_sync(call_id)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 305, in get_result_sync
    ret = self.get_caller(func_name).pop_mp_output(call_id)
  File "D:\Anaconda3\envs\aoc\lib\site-packages\mmdeploy\apis\core\pipeline_manager.py", line 82, in pop_mp_output
    exit()
  File "D:\Anaconda3\envs\aoc\lib\_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
SystemExit: None
[2022-07-21 09:58:34,199] WARNING:huey.consumer:MainThread:Worker 1 died, restarting.

Hello, I run mmdeploy/tools/profiler.py script to get the inference speed both of tensorrt engine and .pth model. FPS of tensorrt inference is higher than .pth,but the inference total time is longer. Is it normal or something wrong? Here is how I get the total inference time: Here is the speed of tensorrt engine: Here is the speed of .pth model:

Here is my environment: Looking forward to your reply！

I convert Swin-Transformer from Pytorch to ONNX and TensorRT and got slow speed: Pytorch: 0.6s TensorRT:0.59s ONNX: 0.92s Here is my configure:

_base_ = [
    '../_base_/base_instance-seg_dynamic.py',
    '../../_base_/backends/tensorrt-int8.py'
]

backend_config = dict(
    common_config=dict(max_workspace_size=1 << 60),
    model_inputs=[
        dict(
            input_shapes=dict(
                input=dict(
                    min_shape=[1, 3, 320, 320],
                    opt_shape=[1, 3, 800, 1344],
                    max_shape=[3, 3, 1344, 1344])))
    ])

python3 ./tools/deploy.py ./configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py ../mmdetection/configs/insurance/cascade_mask_rcnn_swinB.py ../mmdetection/pretrained/cascade_mask_rcnn_swinB.pth ./demo/demo.jpg --device cuda:0

When I call the function mmdeploy_detector_create_by_path, setting model_path by the ONNX model path, a problem occured: no ModelImpl can read sdk_model. Is model_path not the ONNX model path?What the model_path should be?

Motivation

Currently, many documents seem to suppose that the user installed TRT >= 8.*. But, Dockerfile does not follow it, therefore related issues happen when using a given Dockerfile.

Modification

• Upgrade the base image to use TensorRT 8.2.4.2. See release note.
  • Fix wrong environment variable for LD_LIBRARY_PATH (the image does not include /usr/local/cuda-11.3/).
• Upgrade torch & torchvision
• Upgrade MMCV

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

Hello, I am trying to convert a CascadeNet pth model into onnx and getting the same error:

[email protected]:~/workspace/mmdeploy# python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_dynamic.py cascade_mask_rcnn_hrnetv2p_w32_20e.py General.Model.table.detection.v2.pth demo.png [2022-05-19 16:21:23.549] [mmdeploy] [info] Register 'DirectoryModel' 2022-05-19 16:21:23,618 - mmdeploy - INFO - torch2onnx start. [2022-05-19 16:21:24.866] [mmdeploy] [info] Register 'DirectoryModel' /opt/conda/lib/python3.7/site-packages/mmdet/models/builder.py:53: UserWarning: train_cfg and test_cfg is deprecated, please specify them in model 'please specify them in model', UserWarning) load checkpoint from local path: General.Model.table.detection.v2.pth /opt/conda/lib/python3.7/site-packages/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file. 'data pipeline in your config file.', UserWarning) 2022-05-19 16:21:32,430 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 2022-05-19:16:21:32,mmdeploy WARNING [utils.py:92] DeprecationWarning: get_onnx_config will be deprecated in the future. /root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! ys_shape = tuple(int(s) for s in ys.shape) /opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3455: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode) /opt/conda/lib/python3.7/site-packages/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead warnings.warn('DeprecationWarning: anchor_generator is deprecated, ' /opt/conda/lib/python3.7/site-packages/mmdet/core/anchor/anchor_generator.py:333: UserWarning: grid_anchors would be deprecated soon. Please use grid_priors warnings.warn('grid_anchors would be deprecated soon. ' /opt/conda/lib/python3.7/site-packages/mmdet/core/anchor/anchor_generator.py:370: UserWarning: single_level_grid_anchors would be deprecated soon. Please use single_level_grid_priors 'single_level_grid_anchors would be deprecated soon. ' /root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py:78: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert cls_score.size()[-2:] == bbox_pred.size()[-2:] /root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. k = torch.tensor(k, device=input.device, dtype=torch.long) /root/workspace/mmdeploy/mmdeploy/pytorch/functions/topk.py:33: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! return ctx.origin_func(input, k, dim=dim, largest=largest, sorted=sorted) /opt/conda/lib/python3.7/site-packages/mmdet/core/bbox/coder/legacy_delta_xywh_bbox_coder.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert pred_bboxes.size(0) == bboxes.size(0) 2022-05-19:16:21:35,root ERROR [utils.py:43] The size of tensor a (4) must match the size of tensor b (4432) at non-singleton dimension 2 Traceback (most recent call last): File "/root/workspace/mmdeploy/mmdeploy/utils/utils.py", line 38, in target_wrapper result = target(*args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 113, in torch2onnx output_file=output_file) File "/root/workspace/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 55, in torch2onnx_impl verbose=verbose) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/init.py", line 276, in export custom_opsets, enable_onnx_checker, use_external_data_format) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 94, in export use_external_data_format=use_external_data_format) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 698, in _export dynamic_axes=dynamic_axes) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 456, in _model_to_graph use_new_jit_passes) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 417, in _create_jit_graph graph, torch_out = _trace_and_get_graph_from_model(model, args) File "/opt/conda/lib/python3.7/site-packages/torch/onnx/utils.py", line 377, in _trace_and_get_graph_from_model torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True) File "/opt/conda/lib/python3.7/site-packages/torch/jit/_trace.py", line 1139, in _get_trace_graph outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/opt/conda/lib/python3.7/site-packages/torch/jit/_trace.py", line 130, in forward self._force_outplace, File "/opt/conda/lib/python3.7/site-packages/torch/jit/_trace.py", line 116, in wrapper outs.append(self.inner(*trace_inputs)) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 887, in _call_impl result = self._slow_forward(*input, **kwargs) File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 860, in _slow_forward result = self.forward(*input, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper return self.func(self, *args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 69, in base_detector__forward return __forward_impl(ctx, self, img, img_metas=img_metas, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py", line 261, in g rets = f(*args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 28, in __forward_impl return self.simple_test(img, img_metas, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper return self.func(self, *args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/two_stage.py", line 58, in two_stage_detector__simple_test proposals, _ = self.rpn_head.simple_test_rpn(x, img_metas) File "/opt/conda/lib/python3.7/site-packages/mmdet/models/dense_heads/dense_test_mixins.py", line 130, in simple_test_rpn proposal_list = self.get_bboxes(*rpn_outs, img_metas=img_metas) File "/root/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper return self.func(self, *args, **kwargs) File "/root/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py", line 122, in rpn_head__get_bboxes max_shape=img_metas[0]['img_shape']) File "/opt/conda/lib/python3.7/site-packages/mmdet/core/bbox/coder/legacy_delta_xywh_bbox_coder.py", line 79, in decode self.stds, max_shape, wh_ratio_clip) File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/parrots_jit.py", line 22, in wrapper_inner return func(*args, **kargs) File "/opt/conda/lib/python3.7/site-packages/mmdet/core/bbox/coder/legacy_delta_xywh_bbox_coder.py", line 181, in legacy_delta2bbox denorm_deltas = deltas * stds + means RuntimeError: The size of tensor a (4) must match the size of tensor b (4432) at non-singleton dimension 2 2022-05-19 16:21:36,282 - mmdeploy - ERROR - torch2onnx failed.

Does anyone know what the problem might be? Thanks in advance!

Hello,

Over the last couple of months, I have been working on a bash script for building and installing MMDeploy and prerequisites to ease the install procedure when we use MMDetection, MMDeploy tools. Maybe this script can serve as a starting point for an easier install procedure when building and installing MMDeploy from source.

Naturally, this script is a work in progress. There are probably parts that can be cleaned up - and maybe some dependencies are not needed to install anymore.

Let me know if there are any suggestions and ideas on how to integrate this script with the MMDeploy

Note:

1. Only tested with "standard" python virtual environment, as I do not use Conda
2. Only tested on linux
3. Only supports TensorRT build at this point.
4. The script should work on both x86_64 and aarch64 (jetson) platforms.

Hi: I have converted faster-rcnn model downloaded from mmdetection zoo to trt engine sucessfully, but when I run inference_model the error happened: [2022-04-22 07:27:52.715] [mmdeploy] [info] [model.cpp:95] Register 'DirectoryModel' 2022-04-22 07:27:57,889 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so 2022-04-22 07:27:57,889 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /root/workspace/mmdeploy/build/lib/libmmdeploy_tensorrt_ops.so /opt/conda/lib/python3.8/site-packages/mmdet-2.22.0-py3.8.egg/mmdet/datasets/utils.py:66: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recomm ended to manually replace it in the test data pipeline in your config file. warnings.warn( #assertion/root/workspace/mmdeploy/csrc/backend_ops/tensorrt/batched_nms/trt_batched_nms.cpp,98 Aborted (core dumped)

Could you please tell me why it happend and how to deal with it? Thank you.

When i run the demo on the jetson: python ./tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py /home/jetson/mmdetection/configs/retinanet/retinanet_r18_fpn_1x_coco.py /home/jetson/mmdetection/checkpoints/retinanet_r18_fpn_1x_coco_20220407_171055-614fd399.pth /home/jetson/mmdetection/demo/demo.jpg --work-dir work_dir --show --device cuda:0 --dump-info

it occur this problem:

[2022-09-17 14:18:01.468] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
[2022-09-17 14:18:07.267] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
/home/jetson/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
[2022-09-17 14:18:30.458] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
2022-09-17 14:18:30,474 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /home/jetson/mmdetection/checkpoints/retinanet_r18_fpn_1x_coco_20220407_171055-614fd399.pth
/home/jetson/mmdetection/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2022-09-17 14:18:53,964 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2022-09-17 14:18:53,967 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir/end2end.onnx.
2022-09-17 14:18:54,050 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied
/home/jetson/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/home/jetson/mmdeploy/mmdeploy/codebase/mmdet/models/dense_heads/base_dense_head.py:96: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/jetson/mmdeploy/mmdeploy/pytorch/functions/topk.py:57: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/home/jetson/mmdeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(0) == bboxes.size(0)
/home/jetson/mmdeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert pred_bboxes.size(1) == bboxes.size(1)
/home/jetson/mmdeploy/mmdeploy/codebase/mmdet/deploy/utils.py:93: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  assert len(max_shape) == 2, '`max_shape` should be [h, w]'
/home/jetson/mmdeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:260: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/home/jetson/mmdeploy/mmdeploy/mmcv/ops/nms.py:178: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  out_boxes = min(num_boxes, after_topk)
/home/jetson/mmdeploy/mmdeploy/mmcv/ops/nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  (batch_size, out_boxes)).to(scores.device))
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
/home/jetson/archiconda3/envs/mmdeploy/lib/python3.6/site-packages/torch/onnx/symbolic_opset9.py:2819: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::GridPriorsTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTBatchedNMS type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
2022-09-17 14:19:09,799 - mmdeploy - INFO - Execute onnx optimize passes.
2022-09-17 14:19:10,942 - mmdeploy - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
[2022-09-17 14:19:18.405] [mmdeploy] [info] [model.cpp:98] Register 'DirectoryModel'
2022-09-17 14:19:18,421 - mmdeploy - INFO - Start pipeline mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt in subprocess
2022-09-17 14:19:18,731 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/jetson/mmdeploy/mmdeploy/lib/libmmdeploy_tensorrt_ops.so
[09/17/2022-14:19:20] [TRT] [I] [MemUsageChange] Init CUDA: CPU +355, GPU +0, now: CPU 441, GPU 7993 (MiB)
[09/17/2022-14:19:20] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 441 MiB, GPU 8022 MiB
[09/17/2022-14:19:21] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 546 MiB, GPU 8128 MiB
[09/17/2022-14:19:21] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[09/17/2022-14:19:21] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[09/17/2022-14:19:21] [TRT] [I] No importer registered for op: GridPriorsTRT. Attempting to import as plugin.
[09/17/2022-14:19:21] [TRT] [I] Searching for plugin: GridPriorsTRT, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:21] [TRT] [I] Successfully created plugin: GridPriorsTRT
[09/17/2022-14:19:21] [TRT] [I] No importer registered for op: GridPriorsTRT. Attempting to import as plugin.
[09/17/2022-14:19:21] [TRT] [I] Searching for plugin: GridPriorsTRT, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:21] [TRT] [I] Successfully created plugin: GridPriorsTRT
[09/17/2022-14:19:21] [TRT] [I] No importer registered for op: GridPriorsTRT. Attempting to import as plugin.
[09/17/2022-14:19:21] [TRT] [I] Searching for plugin: GridPriorsTRT, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:21] [TRT] [I] Successfully created plugin: GridPriorsTRT
[09/17/2022-14:19:21] [TRT] [I] No importer registered for op: GridPriorsTRT. Attempting to import as plugin.
[09/17/2022-14:19:21] [TRT] [I] Searching for plugin: GridPriorsTRT, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:21] [TRT] [I] Successfully created plugin: GridPriorsTRT
[09/17/2022-14:19:21] [TRT] [I] No importer registered for op: GridPriorsTRT. Attempting to import as plugin.
[09/17/2022-14:19:21] [TRT] [I] Searching for plugin: GridPriorsTRT, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:21] [TRT] [I] Successfully created plugin: GridPriorsTRT
[09/17/2022-14:19:24] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[09/17/2022-14:19:24] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[09/17/2022-14:19:24] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[09/17/2022-14:19:24] [TRT] [W] DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU
[09/17/2022-14:19:24] [TRT] [I] ---------- Layers Running on DLA ----------
[09/17/2022-14:19:24] [TRT] [I] ---------- Layers Running on GPU ----------
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Range_255
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_8 + Relu_9
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] [HostToDeviceCopy]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] MaxPool_10
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_11 + Relu_12
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_13 + Add_14 + Relu_15
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_16 + Relu_17
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_18 + Add_19 + Relu_20
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_21 + Relu_22
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_23
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_24 + Add_25 + Relu_26
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_27 + Relu_28
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_29 + Add_30 + Relu_31
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_32 + Relu_33
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_34
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_35 + Add_36 + Relu_37
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_38 + Relu_39
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_40 + Add_41 + Relu_42
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_43 + Relu_44
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_45
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_46 + Add_47 + Relu_48
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_49 + Relu_50
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_51 + Add_52 + Relu_53
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_77
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_141 + Relu_142 || Conv_133 + Relu_134
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_78
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_159 + Relu_160 || Conv_151 + Relu_152
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_56
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Resize_64
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_76
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_123 + Relu_124 || Conv_115 + Relu_116
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_55 + Add_65
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_117 + Relu_118
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_125 + Relu_126
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Resize_72
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_75
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_105 + Relu_106 || Conv_97 + Relu_98
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_153 + Relu_154
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_161 + Relu_162
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_135 + Relu_136
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_143 + Relu_144
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_54 + Add_73
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_145 + Relu_146
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_137 + Relu_138
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_163 + Relu_164
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_155 + Relu_156
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_99 + Relu_100
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_107 + Relu_108
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_74
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_87 + Relu_88 || Conv_79 + Relu_80
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_127 + Relu_128
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_119 + Relu_120
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_121 + Relu_122
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_129 + Relu_130
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_81 + Relu_82
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_89 + Relu_90
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_109 + Relu_110
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_101 + Relu_102
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_157 + Relu_158
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_165 + Relu_166
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_139 + Relu_140
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_147 + Relu_148
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_150
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_401 + Reshape_402
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_149
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_398 + Reshape_399
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_168
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_456 + Reshape_457
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_167
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_453 + Reshape_454
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_103 + Relu_104
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_111 + Relu_112
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_91 + Relu_92
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_83 + Relu_84
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_132
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_346 + Reshape_347
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_131
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_343 + Reshape_344
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 520) [Constant] + (Unnamed Layer* 521) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_355
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Sigmoid_345)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 534) [Constant] + (Unnamed Layer* 535) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_363
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(479_41 + (Unnamed Layer* 539) [Shuffle], Add_364)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 621) [Constant] + (Unnamed Layer* 622) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_410
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Sigmoid_400)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 635) [Constant] + (Unnamed Layer* 636) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_418
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(479_56 + (Unnamed Layer* 640) [Shuffle], Add_419)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 722) [Constant] + (Unnamed Layer* 723) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_465
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Sigmoid_455)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 736) [Constant] + (Unnamed Layer* 737) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_473
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(479_71 + (Unnamed Layer* 741) [Shuffle], Add_474)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_85 + Relu_86
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_93 + Relu_94
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_114
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_291 + Reshape_292
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_113
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_288 + Reshape_289
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 859 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 896 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 867 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 881 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 755 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 792 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 763 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 777 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 651 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 688 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 659 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 673 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 419) [Constant] + (Unnamed Layer* 420) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_300
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Sigmoid_290)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 433) [Constant] + (Unnamed Layer* 434) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_308
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(479_26 + (Unnamed Layer* 438) [Shuffle], Add_309)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ReduceMax_476
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ReduceMax_421
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ReduceMax_366
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TopK_367
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TopK_422
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TopK_477
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_96
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_228 + Reshape_230
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Conv_95
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Transpose_223 + Reshape_226
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 547 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 584 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 555 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 569 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 304) [Constant] + (Unnamed Layer* 305) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_239
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Sigmoid_227)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 318) [Constant] + (Unnamed Layer* 319) [Shuffle]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_247
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(479 + (Unnamed Layer* 323) [Shuffle], Add_249)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ReduceMax_311
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TopK_312
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 443 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 480 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 451 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 465 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ReduceMax_251
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TopK_252
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] {ForeignNode[Unsqueeze_256...Slice_524]}
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 397
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 196) [Shuffle]_output[Constant]
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_189
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_190
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 403
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_195
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_196
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 421
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_213
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_214
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 409
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_201
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_202
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 415
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_207
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] ConstantOfShape_208
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] GridPriorsTRT_209
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] GridPriorsTRT_203
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] GridPriorsTRT_215
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] GridPriorsTRT_197
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] GridPriorsTRT_191
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 452
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Unsqueeze_216
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 452_19
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Unsqueeze_217
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 452_34
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Unsqueeze_218
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 452_49
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Unsqueeze_219
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 452_64
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] Unsqueeze_220
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 431 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 711) [Constant]_output copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 430 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 610) [Constant]_output copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 429 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 509) [Constant]_output copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 428 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 408) [Constant]_output copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] 427 copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] (Unnamed Layer* 293) [Constant]_output copy
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] PWN(Clip_533)
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] {ForeignNode[Flatten_280...Concat_558]}
[09/17/2022-14:19:24] [TRT] [I] [GpuLayer] TRTBatchedNMS_563
[09/17/2022-14:19:25] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +226, GPU +285, now: CPU 962, GPU 8608 (MiB)
[09/17/2022-14:19:25] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[09/17/2022-14:19:25] [TRT] [E] 2: [utils.cpp::checkMemLimit::380] Error Code 2: Internal Error (Assertion upperBound != 0 failed. Unknown embedded device detected. Please update the table with the entry: {{1794, 6, 16}, 12653},)
Process Process-3:
Traceback (most recent call last):
  File "/home/jetson/archiconda3/envs/mmdeploy/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/jetson/archiconda3/envs/mmdeploy/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/jetson/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/jetson/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 88, in onnx2tensorrt
    device_id=device_id)
  File "/home/jetson/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 215, in from_onnx
    assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-09-17 14:19:26,458 - mmdeploy - ERROR - `mmdeploy.backend.tensorrt.onnx2tensorrt.onnx2tensorrt` with Call id: 1 failed. exit.

And i run python /tools/check.env

2022-09-17 14:40:59,038 - mmdeploy - INFO -

2022-09-17 14:40:59,039 - mmdeploy - INFO - **********Environmental information**********
fatal: not a git repository (or any of the parent directories): .git
2022-09-17 14:41:04,289 - mmdeploy - INFO - sys.platform: linux
2022-09-17 14:41:04,290 - mmdeploy - INFO - Python: 3.6.15 | packaged by conda-forge | (default, Dec  3 2021, 19:12:04) [GCC 9.4.0]
2022-09-17 14:41:04,291 - mmdeploy - INFO - CUDA available: True
2022-09-17 14:41:04,291 - mmdeploy - INFO - GPU 0: Xavier
2022-09-17 14:41:04,291 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda-10.2
2022-09-17 14:41:04,291 - mmdeploy - INFO - NVCC: Build cuda_10.2_r440.TC440_70.29663091_0
2022-09-17 14:41:04,291 - mmdeploy - INFO - GCC: gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
2022-09-17 14:41:04,291 - mmdeploy - INFO - PyTorch: 1.10.0
2022-09-17 14:41:04,291 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 7.5
  - C++ Version: 201402
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
  - CuDNN 8.2.1
    - Built with CuDNN 8.0
  - Build settings: BLAS_INFO=open, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=8.0.0, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -W                                        no-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_XNNPACK -DSYMBOLICA                                        TE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initiali                                        zers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-fu                                        nction -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-s                                        tringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligne                                        d-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -DMISSING_ARM_VST1 -DMISSI                                        NG_ARM_VLD1 -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=open, TORCH_VERSION=1.10.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EI                                        GEN_FOR_BLAS=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=ON, USE_NCCL=0, USE_NNPACK=ON,                                         USE_OPENMP=ON,

2022-09-17 14:41:04,292 - mmdeploy - INFO - TorchVision: 0.11.1
2022-09-17 14:41:04,292 - mmdeploy - INFO - OpenCV: 4.6.0
2022-09-17 14:41:04,292 - mmdeploy - INFO - MMCV: 1.4.0
2022-09-17 14:41:04,292 - mmdeploy - INFO - MMCV Compiler: GCC 7.5
2022-09-17 14:41:04,292 - mmdeploy - INFO - MMCV CUDA Compiler: 10.2
2022-09-17 14:41:04,292 - mmdeploy - INFO - MMDeploy: 0.8.0+
2022-09-17 14:41:04,292 - mmdeploy - INFO -

2022-09-17 14:41:04,293 - mmdeploy - INFO - **********Backend information**********
2022-09-17 14:41:06,396 - mmdeploy - INFO - onnxruntime: None   ops_is_avaliable : False
2022-09-17 14:41:06,491 - mmdeploy - INFO - tensorrt: 8.2.1.8   ops_is_avaliable : True
2022-09-17 14:41:06,547 - mmdeploy - INFO - ncnn: None  ops_is_avaliable : False
2022-09-17 14:41:06,550 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-09-17 14:41:06,554 - mmdeploy - INFO - openvino_is_avaliable: False
2022-09-17 14:41:06,607 - mmdeploy - INFO - snpe_is_available: False
2022-09-17 14:41:06,613 - mmdeploy - INFO - ascend_is_available: False
2022-09-17 14:41:06,616 - mmdeploy - INFO - coreml_is_available: False
2022-09-17 14:41:06,616 - mmdeploy - INFO -

2022-09-17 14:41:06,616 - mmdeploy - INFO - **********Codebase information**********
2022-09-17 14:41:06,621 - mmdeploy - INFO - mmdet:      2.25.2
2022-09-17 14:41:06,621 - mmdeploy - INFO - mmseg:      None
2022-09-17 14:41:06,621 - mmdeploy - INFO - mmcls:      None
2022-09-17 14:41:06,621 - mmdeploy - INFO - mmocr:      None
2022-09-17 14:41:06,622 - mmdeploy - INFO - mmedit:     None
2022-09-17 14:41:06,622 - mmdeploy - INFO - mmdet3d:    None
2022-09-17 14:41:06,622 - mmdeploy - INFO - mmpose:     None
2022-09-17 14:41:06,622 - mmdeploy - INFO - mmrotate:   None

Thank you!!!!!

Hello, this is my first time to use the mmdeploy, i follow the get_started.md to convert model with Linux-x86_64, CUDA 11.x, TensorRT 8.2.5.1, just like the md show

but it occur the problem: ValueError: cpu is invalid for the backend tensort.

By the way , my env as follow: