Gradio quickly builds ml/dl Web Services

Gradio Quickly build ML/DL Web End service

Preface

When we have trained a certain model and the effect is good , The first thought should be Deploy . Deployment can be divided into online Web Services and edge modules ; In order to report, online deployment is often selected , After all, the box has to be configured with the corresponding hardware input after it is deployed, which is not convenient to display . Try using before this column fastapi Set the Web service , And deploy some algorithm models to api Interface , But because we have to design some pretreatment work by ourselves , route , Additional functions and pages UI It's a lot of trouble . So today I will introduce this one based on fastapi Build a very convenient and powerful Web frame —Gradio

The link to the official website is here , You can go to know about Gradio Although there are no Chinese documents at present , But I think the whole document is easy to read , So it's not difficult to imitate document operation step by step .

So much for that , Let's start demo. The plan is like this

1. A simple one with fewer layers ResNet, stay cifar10 Train for a while , Get a better model ( There is no need to migrate here, mainly because of the previous re parameterization for experiments in ResNet The structural effect is not good, and then I wrote a simple one and used it by the way )
2. Use Gradio Write a Web service , In fact, as long as we implement the reasoning function , Others only instantiate interfaces , It is fast .

Start

model training

The most basic residual modules are piled up , Then you can train after a full connection layer classification . Here, the training has specially tried what has been proposed in recent years Ranger The optimizer replaces the previous sgd,adam And so on. , The effect will indeed be improved .

Take a look at the effect comparison given on the official website , After my training results, it is true .

The simplest way to use is to put ranger21.py download , Then you can quote it during training, or use it according to the official website pip install pip install git+https://github.com/lessw2020/Ranger21.git

The final effect of the model with only a few layers is also good

In this way, we get a model with good effect , The next step is to use Gradio Deploy

Deploy

You can see from the introductory tutorial on the official website , We need to write our own reasoning function , And in gr.Interface() Register your own functions and input and output some additional functions . I have to say that this framework is really easy to develop after encapsulation , Many functions have been realized , such as : Result cache , The explanation of the model even takes into account the use of queues instead of parallel response when the number of visits is particularly large

First load the model and then write the inference function

if os.path.exists("best.pt"):
    ResNet.load_state_dict(torch.load('best.pt'))
ResNet.eval()
labels=['airplane','automobile','bird','cat','deer','dog','frog','horse','ship','truck']

def classify_image(img):
    img = transforms.ToTensor()(img).unsqueeze(0).cuda()
    with torch.no_grad():
        predict=torch.nn.functional.softmax(ResNet(img)[0],dim=0).cpu()
        confidence={
    labels[i]:float(predict[i]) for i in range(10)}
    return confidence

Then register to the interface instantiation and start

gr.Interface(
    fn=classify_image,
    inputs=gr.Image(type="pil",shape=(320,320)),
    outputs=gr.Label(num_top_classes=10),
    examples=['./data/airplane.jpeg'],
    interpretation="default",cache_examples=True,title="Shelgi The classification of Demo"
).launch(enable_queue=True)

The above explanation is based on the default ``SHAP`, You can also set your own interpretation function . Let's look at other results

Here, the basic deployment part is realized , But as usual , Or take a look at the source code .

The source code section

I said at the beginning that this is based on fastapi Encapsulated framework , Let's take a good look at some of its implementations . The most basic thing is Web frame , Let's first look at its routing part .

Obviously , Is the use fastapi To set up routing .App Inherited FastAPI, Realized its own related routing (login,config,file wait )

And then in blocks There are more functions in , Including pretreatment , Run the registration function and post-processing , This is also an important code to call prediction

Let's take a look at the basic settings , Be similar to url, The port numbers are all in networking.py in , The settings are as follows

Of course , We can also do some of our own diy Design

You can see that the static page is from templates Read under folder , So we can make changes to the page . Or we want to simply add some content , Try imitating the title and description , call Markdown() Add

Of course, if you want to create more complex pages , You need to use Blocks Little build pages , For more detailed content, if you are interested, you can check it in the official website document .