Raspberry pie 4B uses Intel movidius NCS 2 for inference acceleration

Generally, only model inference is performed on embedded devices , That is to say PC You can train the model first , Then take it to the embedded device for use .OpenVINO You can refer to the previous series of blogs of bloggers for the use of

OpenVINO Introduction ( One ）_ green bamboo snake lvye The blog of -CSDN Blog _openvino

 OpenVINO Sample introduction _ green bamboo snake lvye The blog of -CSDN Blog

Intel Movidius Neural Computer Stick 2 Use (PC-Based Ubuntu）_ green bamboo snake lvye The blog of -CSDN Blog

  The blogger will introduce how to configure raspberry pie OpenVINO, And combine Intel Movidius Neural Computer Stick 2 To speed up .

One . With raspberry pie OpenVINO

Please refer to the official website

Install OpenVINO toolkit for Raspbian OS — OpenVINO documentationhttps://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_raspbian.html Bloggers here use source code compilation to compile

BuildingForRaspbianStretchOS · openvinotoolkit/openvino Wiki · GitHub

 1. Download resources

git clone https://github.com/openvinotoolkit/openvino

  The resources downloaded by bloggers are here

2. Initialize the submodules in the download directory (submodule)

cd To openvino After directory , The terminal executes the following command

git submodule update --init --recursive

This will automatically download some files , Pay attention to the network problems during the process , It may be incomplete , You can download it manually . Or below cmake Which files are incomplete will be prompted when compiling in , In this way, you can download files manually through error messages . For example, the blue folder in the red box below . Note that there are also some sub modules that need to be downloaded manually .

You can also get it from .gitmodules It can be seen from the file that additional sub modules need to be downloaded

3. Compile and install

notes ： Here, let's compile the specified python The version is the version of the blogger's blog

Raspberry pie 4B install pytorch_ green bamboo snake lvye The blog of -CSDN Blog _ Raspberry pie 4b install pytorch

Create a new build Folder ,cd After going in , Execute the following command statement

cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=SEQ -DENABLE_PYTHON=ON -DPYTHON_EXECUTABLE=/home/pi/python3.7/bin/python3.7 -DPYTHON_LIBRARY=/home/pi/python3.7/lib/libpython3.7m.so -DPYTHON_INCLUDE_DIR=/home/pi/python3.7/include/python3.7m -DCMAKE_INSTALL_PREFIX=/home/pi/openvino/install ..

 cmake Success will show the following message ：

  Re execution make command , This process will probably perform 1 To 2 Hours , Wait patiently

Finally, execute the following statement to install

sudo make install

Because the blogger compiled it before , Installation path specified , So they are all installed here

After the completion of , stay ~./bashrc Add the following 3 Environment variables （ Combine your own path ）

source /home/pi/openvino/install/setupvars.sh
export LD_LIBRARY_PATH=/home/pi/opencv-3.4.4/install/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export LD_LIBRARY_PATH=/home/pi/openvino/install/runtime/lib/armv71${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

 

 

Two . test

1. Put the front install Under the folder openvino Copy the folder to the corresponding version of python Of site-packages Next （ Combine your own path ）

2. cd To the following directory , Execute the following command to install NCS 2 drive ( Combine your own path ）

./install_NCS_udev_rules.sh

3. Here we use the... In the blog openvino IR Model

Intel Movidius Neural Computer Stick 2 Use (PC-Based Ubuntu）_ green bamboo snake lvye The blog of -CSDN Blog

The code is as follows ：

import numpy
import openvino.runtime as ov
import numpy as np
import time
from PIL import Image
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions
from tensorflow.keras.preprocessing import image
import openvino.runtime as ov

core = ov.Core()
for device in core.available_devices:
    print(device)

core = ov.Core()
model = core.read_model("./saved_model.xml")
model.reshape([1, 224, 224, 3])

compiled_model = core.compile_model(model, "MYRIAD")
infer_request = compiled_model.create_infer_request()

img = image.load_img('2008_002682.jpg', target_size=(224, 224))
img = image.img_to_array(img)
img = preprocess_input(img)
img_np = np.expand_dims(img, axis=0)

# Create tensor from external memory
input_tensor = ov.Tensor(array=img_np, shared_memory=False)

infer_request.set_input_tensor(input_tensor)

t_model = time.perf_counter()
infer_request.start_async()
infer_request.wait()
print(f'do inference cost:{time.perf_counter() - t_model:.8f}s')

# Get output tensor for model with one output
output = infer_request.get_output_tensor()
output_buffer = output.data

# output_buffer[] - accessing output tensor data
print(output_buffer.shape)
print('Predicted:', decode_predictions(output_buffer, top=5)[0])

print("ok")

Don't forget to put... On the raspberry pie NCS2

The operation results are as follows ： 

/home/pi/PythonProject/openvinoTest/bin/python3 /home/pi/PycharmProjects/pythonProject/test_openvino.py
MYRIAD
do inference cost:0.05369431s
(1, 1000)
Predicted: [('n02123597', 'Siamese_cat', 0.16870117), ('n02108915', 'French_bulldog', 0.14099121), ('n04409515', 'tennis_ball', 0.08093262), ('n02095314', 'wire-haired_fox_terrier', 0.05609131), ('n02123045', 'tabby', 0.049865723)]
ok

Process finished with exit code 0

  Compare the results in the blog to see

Raspberry pie 4B install Tensorflow2.4.0_ green bamboo snake lvye The blog of -CSDN Blog _ Raspberry pie installation tensorflow

  The prediction results are consistent , Speed from 23s Ascended to 0.05s, You can see that after using the nerve Accelerator , The acceleration is obvious .