Pytorch Profiler+ Tensorboard + VS Code

Pytorch Profiler

Actually, it's not very troublesome to use , Just insert some in the original code , for example

with torch.profiler.profile(
    activities=[
        torch.profiler.ProfilerActivity.CPU,
        torch.profiler.ProfilerActivity.CUDA],
    schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),
    on_trace_ready=torch.profiler.tensorboard_trace_handler('./visual/bs3000'),
    record_shapes=True,
    profile_memory=True,  # This will take 1 to 2 minutes. Setting it to False could greatly speedup.
    with_stack=True
) as prof:
    for step, batch_data in enumerate(train_loader):
        if step >= (1 + 1 + 3) * 2:
            break
        train(batch_data)
        prof.step()  # Need call this at the end of each step to notify profiler of steps' boundary.

The... Used here on_trace It uses tensorboard Of trace_handle, Generated in a while .json Documents can be used tensorboard open
Very good English blog, Strongly recommend

tensorboard

It can be opened on the server tensorboard, Then connect the local port and the Server Of tensorboard The port can be opened locally after binding
For details, please refer to this blog

Vs Code open Tensorboard

This is simpler , stay

import torch.profiler

There will be a launch tensorboard The button , Basically, you can operate foolishly with one click , At the same time, you can put log Put the directory of in a large directory , This allows you to view multiple profiler, stay tensorboard One more Diff Button , Can compare different profile The similarities and differences of documents , Very easy to use ！