003. Torchserve calls LSTM model prediction

Preface

Introduction to general steps , Execute the whole process , There may be many small problems in each step

Own model , Yes, it is LSTM The model predicts , front 12 One data predicts the next data ,

Reference article https://blog.csdn.net/weixin_45492560/article/details/121316540

One 、 The overall steps

1、 Save the model as .pt perhaps .pth Format

import torch
from ... import 

#  Training models 
model
model.fit()

#  Transformation model 
model.eval()

traced_script_module = torch.jit.script(model)

#  preservation  TorchScript model
traced_script_module.save("xxx.pt")

2、 To .mar Format
stay serve/model-archiver The input

torch-model-archiver -f --model-name  xxx \   # mar Format model name xxx
--version 1.0 \ #  edition 
--serialized-file xxx.pt \   # pt file 
--handler handlers/xxx_handler.py \    #  Data preprocessing before model input , Output result processing , Display processing and other functions , This is very important 
--export-path targets/  #  Output mar  The location of the file  , It's usually serve/model-archiver/model-store  Next 

3、docker start-up
stay serve/model-archiver The input

docker run --rm -it -p 8080:8080 -p 8081:8081 --name mar -v $(pwd)/model-store:/home/model-server/model-store -v $(pwd)/examples:/home/model-server/examples pytorch/torchserve:latest

-v Directory mapping

4、 Register a model

curl --location --request POST 'http://localhost:8081/models' \
--header 'Content-Type: application/json' \
--data-raw '{
"url": "xxx.mar",
"batch_size": 64,
"initial_workers": 1
}'

After registration , curl http://127.0.0.1:8081/models , Query whether the registration is successful

5、 Forecast data

curl --location --request POST 'http://localhost:8080/predictions/xxx  -T   #  Data to be predicted 

Show forecast data

Two 、 Pit Guide

There is no error in saving the model .

1、 encounter OSError: [Errno 13] Permission denied:

ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: ‘/home/venv/lib/python3.8/site-packages/threadpoolctl.py’

Check the permissions.

It should be a package that cannot be installed

resolvent ：

1、 Add one –requirements-file , Plus dependence , Reference link https://pip.pypa.io/en/stable/user_guide/#requirements-files

2、 According to this package information , Do not call , Write your own code

2、 In the second step ,handle Documentation . crucial

2.1 preprocess , Data preprocessing

First of all, the data data yes list Format , Need to get data、body Data and information , I'm going to use csv file

def preprocess(self, data):
    # processed = []
    print("data The original data ",data)
    for row in data:
        print("row:",row)
        values_ori = row.get("data") or row.get("body") #  Byte array 
        values_str = values_ori.decode('utf-8')  # str
        values_str_split = values_str.split("\r\n")  # split
        values_str_split.pop() #  Delete the last empty string 
        values = list(map(float, values_str_split))
        print("values data ：",values)
        print("values data type ",type(values))
        tensor = super().preprocess(values)  #  To Tensor  type 
        print("tensor  data ：",tensor)

        #  normalization 
        max_values = torch.max(tensor)
        min_values = torch.min(tensor)
        normalized_values = (tensor - min_values) / \
                            (max_values - min_values + EPSILON)
        #  After processing , Data format transformation , Input into the model 
        expanded_values = torch.unsqueeze(normalized_values, dim=0).t()
        print("expanded_values All data ：", expanded_values)
        print("expanded_valuesshape:", (np.array(expanded_values)).shape)
        # processed.append(expanded_values) #  get data 
        return expanded_values

2.2 postprocess, Data processing after prediction

def postprocess(self, data):

    pred = []
    with torch.no_grad():
        print("data:",data)
        print("data type:",type(data))
        # tensor  To  numpy
        data_num = data.numpy()
        print("data_num:", data_num)
        print("data_num type:", type(data_num))

        data_num = data_num * 100
        pred.append(data_num.tolist())
    print("pred:",pred)

    return pred #  To  list

3、 encounter TypeError, How much is in preprocess Stage , There is a problem with data parsing , When you first started writing code , You can print out more , What is the positioning problem .