VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

Last update: Jan 08, 2023

Overview

VizTracer

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

The front-end UI is powered by Perfetto. Use "AWSD" to zoom/navigate. More help can be found in "Support - Controls".

Highlights

Detailed function entry/exit information on timeline with source code
Super easy to use, no source code change for most features, no package dependency
Supports threading, multiprocessing, subprocess and async
Logs arbitrary function/variable using RegEx without code change
Powerful front-end, able to render GB-level trace smoothly
Works on Linux/MacOS/Windows

Install

The prefered way to install VizTracer is via pip

pip install viztracer

Basic Usage

Command Line

Assume you have a python script to run:

python3 my_script.py arg1 arg2

You can simply use VizTracer by

viztracer my_script.py arg1 arg2

A result.json file will be generated, which you can open with vizviewer

vizviewer will host an HTTP server on http://localhost:9001. You can also open your browser and use that address.

If you do not want vizviewer to open the webbrowser automatically, you can use

vizviewer --server_only result.json

If you just need to bring up the trace report once, and do not want the persistent server, use

vizviewer --once result.json

vizviewer result.json

You can also generate standalone html file

viztracer -o result.html my_script.py arg1 arg2

The standalone HTML file is powered by catapult trace viewer which is an old tool Google made and is being replaced by Perfetto gradually.

Catapult trace viewer is sluggish with larger traces and is not actively maintained. It is recommended to use Perfetto instead.

However, if you really need a standalone HTML file, this is the only option. Perfetto does not support standalone files.

You can use vizviewer to open the html file as well, just to make the interface consistent

vizviewer result.html

Or add --open to open the reports right after tracing

viztracer --open my_scripy.py arg1 arg2
viztracer -o result.html --open my_script.py arg1 arg2

modules and console scripts(like flask) are supported as well

viztracer -m your_module

viztracer flask run

Inline

You can also manually start/stop VizTracer in your script as well.

from viztracer import VizTracer

tracer = VizTracer()
tracer.start()
# Something happens here
tracer.stop()
tracer.save() # also takes output_file as an optional argument

Or, you can do it with with statement

with VizTracer(output_file="optional.json") as tracer:
    # Something happens here

Jupyter

If you are using Jupyter, you can use viztracer cell magics.

# You need to load the extension first
%load_ext viztracer

%%viztracer
# Your code after

A VizTracer Report button will appear after the cell and you can click it to view the results

Advanced Usage

Trace Filter

VizTracer can filter out the data you don't want to reduce overhead and keep info of a longer time period before you dump the log.

Extra Logs without Code Change

VizTracer can log extra information without changing your source code

Add Custom Event

VizTracer supports inserting custom events while the program is running. This works like a print debug, but you can know when this print happens while looking at trace data.

Misc

Multi Thread Support

VizTracer supports python native threading module without the need to do any modification to your code. Just start VizTracer before you create threads and it will just work.

Multi Process Support

VizTracer supports subprocess, multiprocessing and os.fork() out of the box.

For more general multi-process cases, VizTracer can support with some extra steps.

Refer to multi process docs for details

Async Support

VizTracer supports asyncio natively, but could enhance the report by using --log_async.

Refer to async docs for details

Flamegraph

VizTracer can show flamegraph of traced data.

vizviewer --flamegraph result.json

Remote attach

VizTracer supports remote attach to a process as long as you installed VizTracer on that process.

Refer to remote attach docs

JSON alternative

VizTracer needs to dump the internal data to json format. It is recommended for the users to install orjson, which is much faster than the builtin json library. VizTracer will try to import orjson and fall back to the builtin json library if orjson does not exist.

Virtual Debug

You can virtually debug your program with you saved json report. The interface is very similar to pdb. Even better, you can go back in time because VizTracer has all the info recorded for you.

vdb

Refer to the docs for detailed commands

Performance

VizTracer will introduce 2x to 3x overhead in the worst case. The overhead is much better if there are less function calls or if filters are applied correctly.

An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM

fib:
0.000678067(1.00)[origin]
0.019880272(29.32)[py] 0.011103901(16.38)[parse] 0.021165599(31.21)[json]
0.001344933(1.98)[c] 0.008181911(12.07)[parse] 0.015789866(23.29)[json]
0.001472846(2.17)[cProfile]

hanoi     (6148, 4100):
0.000550255(1.00)[origin]
0.016343521(29.70)[py] 0.007299123(13.26)[parse] 0.016779364(30.49)[json]
0.001062505(1.93)[c] 0.006416136(11.66)[parse] 0.011463236(20.83)[json]
0.001144914(2.08)[cProfile]

qsort     (8289, 5377):
0.002817679(1.00)[origin]
0.052747431(18.72)[py] 0.011339725(4.02)[parse] 0.023644345(8.39)[json]
0.004767673(1.69)[c] 0.008735166(3.10)[parse] 0.017173703(6.09)[json]
0.007248019(2.57)[cProfile]

slow_fib  (1135, 758):
0.028759652(1.00)[origin]
0.033994071(1.18)[py] 0.001630461(0.06)[parse] 0.003386635(0.12)[json]
0.029481623(1.03)[c] 0.001152415(0.04)[parse] 0.002191417(0.08)[json]
0.028289305(0.98)[cProfile]

Documentation

For full documentation, please see https://viztracer.readthedocs.io/en/stable

Bugs/Requests

Please send bug reports and feature requests through github issue tracker. VizTracer is currently under development now and it's open to any constructive suggestions.

License

Distributed under the terms of the Apache 2.0 license.

Comments

Feature request: magic command to use viztracer from within jupyter notebook
Note sure what would be the best way to implement this, but the usability of viztracer could be greatly improved for datascientists and other Python users who work in interactive envrionments based on Jupyter such as Jupyter Lab and VS Code notebooks.

The setup use case would work as follow in jupyter notebook with the following cells:

# Notebook cell to enable the viztracer <=> jupyter extension %load_ext viztracer

# Here is a notebook biggish_data = prepare_big_datastructure_or_load_from_disk()

# Here is a notebook cell with some code to interactively # define some complex data processing function. def some_function(some_data, some_param=0.): # do complicated stuff ...

# Here is a cell that profiles on %%viztracer some_function(biggish_data, some_param=42)

The last cell would execute some_function(biggish_data, some_param=42) while recording the profiling events to a temp file on the disk and then automatically display an iframe with the perfetto viewer of the recorded events right below the code cell with the %%viztracer statement.

If instead one uses %%viztracer --new-tab would open a new browser tab to the viewer instead of using an iframe.

Finally %%viztracer --static-html would embed the legacy static HTML viewer in the output of the cell of the notebok document to make it easier to publish and share the notebook (e.g. on https://nbviewer.jupyter.org) so that other can browse the profiling output without having to run a (local or remote) server (for jupyter and vizviewer).

It would also be possible to pass any viztracer commandline options to the magic command itself, for instance %%viztracer --log_gc --log_subprocess.

This setup is really nice, because prepare_big_datastructure_or_load_from_disk can be very slow and the user might want to iterate quickly by editing the code of some_function in the previous cell based on the result of the previous profiling output without calling prepare_big_datastructure_or_load_from_disk() again each time.

The last 2 cells could be even be collapsed into one for even faster coding iterations:

# Here is a notebook cell with some code to interactively # define some complex data processing function and profile # it on the fly: def some_function(some_data, some_param=0.): # do complicated stuff ... %viztracer some_function(biggish_data, some_param=42)

Related jupyter extensions for profilers that can be used for inspiration:

jupyter supports the %%prun profiler magic based on cProfile from the standard library by default.

line_profiler can register the %%lprun magic command.

snakeviz can register the %%snakeviz magic command. It has the --new-tab option I mentioned before.
opened by ogrisel 23

VCompressor Benchmark

As #249 mentioned, VCompressor is designed for a decent compress ratio. Here provides a benchmark framework to easily calculate performance improvement space-wise and time-wise on specific test cases.

Note: Here, I chose to use LZMA, famous for its high compress ratio on average files, as a baseline in terms of space-wise benchmark.

Main Features:

[x] Space-wise Benchmark
[x] Time-wise Benchmark
[x] Display compress ratio in the vcompressor benchmark
[x] Support multiple other compressors for comparison

Other Important Change Log:

[x] FIX: custom repeat(previously loop_time) in _benchmark is now implemented correctly
[x] STYLE: All related functions and types are now placed in a single class, without affecting global namespace

Output Example of this Benchmark Test:

2022-09-05 23:01:24,684 INFO: tests.test_vcompressor.TestVCompressorPerformance.test_benchmark_basic start
2022-09-05 23:01:26,302 INFO: 1. On file "vdb_basic.json":
2022-09-05 23:01:26,302 INFO:     [Space]
2022-09-05 23:01:26,302 INFO:       Uncompressed:       2.95KB
2022-09-05 23:01:26,302 INFO:       VCompressor:      531.00B(1.000) [CR: 17.57%]
2022-09-05 23:01:26,302 INFO:       LZMA:             504.00B(0.949) [CR: 16.68%]
2022-09-05 23:01:26,303 INFO:     [Time]
2022-09-05 23:01:26,303 INFO:       VCompressor:        0.405s(1.000)
2022-09-05 23:01:26,303 INFO:       LZMA:               0.004s(0.010)
2022-09-05 23:01:27,757 INFO: 2. On file "multithread.json":
2022-09-05 23:01:27,757 INFO:     [Space]
2022-09-05 23:01:27,757 INFO:       Uncompressed:     105.22KB
2022-09-05 23:01:27,757 INFO:       VCompressor:        9.05KB(1.000) [CR:  8.60%]
2022-09-05 23:01:27,757 INFO:       LZMA:              16.66KB(1.840) [CR: 15.83%]
2022-09-05 23:01:27,757 INFO:     [Time]
2022-09-05 23:01:27,757 INFO:       VCompressor:        0.330s(1.000)
2022-09-05 23:01:27,757 INFO:       LZMA:               0.023s(0.071)
2022-09-05 23:01:27,757 INFO: tests.test_vcompressor.TestVCompressorPerformance.test_benchmark_basic finish

opened by Sefank 18

Fix test viewer
This is a pull request to solve issue #269. Summary:

Add _find_a_free_port into TestViewer

Add URL method to Viewer

Change cases use 9001 port to a free dynamic port found before creating a viewer.
opened by acehinnnqru 14

Trace with concurrent.futures.ProcessPoolExecutor

Hi,

New to python multi-processing work and tried to profile an old code with a code multi-process/thread dispatcher code way like below

def multi_process_dispatcher(single_worker_func, data_list_to_dispatch,  job_num):
    """ multi  process dispatcher """
    output = []
    with concurrent.futures.ProcessPoolExecutor(job_num) as executor:
        for _, data_collected in zip(check_list, executor.map(single_worker_func, data_list_to_dispatch)):
            if data_collected is not None:
                output += data_collected
        output.sort()
    return output

def multi_thread_dispatcher(single_worker_func, data_list_to_dispatch, job_num):
    """ multi thread dispatcher """
    output = []
    # max_workers=job_num
    with concurrent.futures.ThreadPoolExecutor(job_num) as executor:
        for _, data_collected in zip(check_list, executor.map(single_worker_func, data_list_to_dispatch)):
            if data_collected is not None:
                output += data_collected
        output.sort()
    return output

in doc saying：

pass this argument to your sub-process, and instantiate a VizTracer object with it

# init_kwargs is the argument from main process
tracer = VizTracer(**init_kwargs)
tracer.register_exit()
tracer.start()

any good idea of embeding trace with ProcessPoolExecutor ? the data_list is dispatched with map actually I think

Thanks

bug

opened by butterl 14

`json_tokenizer_failure` in Perfetto
Importing the JSON output fails in Perfetto, showing something like this:

Here's the JSON produced by viztracer -o test.json vt-test.py: https://gist.github.com/gjoseph92/f5c21ad1b328c8bed3e472be92a32342

And the contents of vt-test.py (tried a few different scripts; doesn't seem to matter):

x = 0 for i in range(10000): x += 1

Note that the JSON loads fine in catapult.

I've replicated this with viztracer==0.11.4, viztracer==0.12.0, viztracer==0.12.1, viztracer==0.12.2, viztracer==0.12.3.

The fact that it replicates even with older versions makes me think it's a Perfetto issue, but I wanted to open the issue here first in case anyone has seen this before, or it's an issue with viztracer's formatting of the JSON. If it's not viztracer's problem, I'm happy to move this to https://github.com/google/perfetto/issues

System:

macOS 11.2 (x86)

Python 3.8.8
opened by gjoseph92 13
enhancement: support docker

seems Vizviewer is same with Nvidia NSight, the logger can run in container but cannot forward the visualisation to host. i have to host the viewer on main OS. this probably will help developer to benchmark on cloud. thank you for paying attention to my comment.
enhancement

opened by royinx 12

Fix All flake8 Errors and Add a dedicated flake8 Github Action

Fix flake8 errors and add a dedicated flake8 Github action. Also, add a flake8 badge to README.

Two errors are ignored:

C901: Function is too complex (too much branching logic). This PR is meant to fix the style issues.
W503: line break before binary operator. It's not compatible with PEP8.

Original flake8 errors:

$ flake8 . --count --ignore=C901,W503 --max-complexity=10 --max-line-length=127 --statistics
./setup.py:28:20: E231 missing whitespace after ':'
./setup.py:39:20: E251 unexpected spaces around keyword / parameter equals
./setup.py:39:22: E251 unexpected spaces around keyword / parameter equals
./example/generate_examples.py:13:45: E225 missing whitespace around operator
./example/generate_examples.py:18:35: W291 trailing whitespace
./example/generate_examples.py:26:39: W292 no newline at end of file
./example/src/different_sorts.py:77:1: E302 expected 2 blank lines, found 1
./example/src/different_sorts.py:126:20: W292 no newline at end of file
./example/src/gradient_descent.py:120:34: E226 missing whitespace around arithmetic operator
./example/src/logging_integration.py:10:17: E226 missing whitespace around arithmetic operator
./example/src/logging_integration.py:10:28: E226 missing whitespace around arithmetic operator
./example/src/logging_integration.py:17:7: W292 no newline at end of file
./example/src/function_args_return.py:8:17: E226 missing whitespace around arithmetic operator
./example/src/function_args_return.py:8:28: E226 missing whitespace around arithmetic operator
./example/src/function_args_return.py:11:35: W291 trailing whitespace
./example/src/function_args_return.py:12:37: W291 trailing whitespace
./example/src/multithread.py:11:17: E226 missing whitespace around arithmetic operator
./example/src/multithread.py:11:28: E226 missing whitespace around arithmetic operator
./example/src/multithread.py:25:1: W293 blank line contains whitespace
./example/src/multithread.py:30:1: W293 blank line contains whitespace
./example/src/multithread.py:32:1: W293 blank line contains whitespace
./example/src/mcts_game.py:80:48: W292 no newline at end of file
./tests/test_multiprocess.py:12:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:19:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:28:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:42:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:65:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:93:1: E122 continuation line missing indentation or outdented
./tests/test_multiprocess.py:134:128: E501 line too long (175 > 127 characters)
./tests/test_multiprocess.py:145:128: E501 line too long (179 > 127 characters)
./tests/test_multiprocess.py:155:128: E501 line too long (221 > 127 characters)
./tests/test_multiprocess.py:159:128: E501 line too long (214 > 127 characters)
./tests/test_multiprocess.py:161:128: E501 line too long (173 > 127 characters)
./tests/test_multiprocess.py:168:128: E501 line too long (233 > 127 characters)
./tests/cmdline_tmpl.py:12:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:12:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:19:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:27:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:34:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:42:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:50:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:79:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:100:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:120:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:129:1: E122 continuation line missing indentation or outdented
./tests/test_cmdline.py:154:128: E501 line too long (128 > 127 characters)
./tests/test_cmdline.py:155:128: E501 line too long (134 > 127 characters)
./tests/test_cmdline.py:157:128: E501 line too long (139 > 127 characters)
./tests/test_cmdline.py:158:128: E501 line too long (145 > 127 characters)
./tests/test_cmdline.py:174:128: E501 line too long (136 > 127 characters)
./tests/test_cmdline.py:176:128: E501 line too long (183 > 127 characters)
./tests/test_cmdline.py:177:128: E501 line too long (180 > 127 characters)
./tests/test_cmdline.py:182:128: E501 line too long (136 > 127 characters)
./tests/test_cmdline.py:184:128: E501 line too long (173 > 127 characters)
./tests/test_cmdline.py:201:128: E501 line too long (156 > 127 characters)
./tests/test_cmdline.py:206:17: E128 continuation line under-indented for visual indent
./tests/test_cmdline.py:207:128: E501 line too long (140 > 127 characters)
./tests/test_cmdline.py:208:17: E128 continuation line under-indented for visual indent
./tests/test_cmdline.py:215:128: E501 line too long (156 > 127 characters)
./tests/test_cmdline.py:223:128: E501 line too long (150 > 127 characters)
./tests/test_cmdline.py:224:128: E501 line too long (182 > 127 characters)
./tests/test_cmdline.py:236:128: E501 line too long (176 > 127 characters)
./tests/test_cmdline.py:237:128: E501 line too long (174 > 127 characters)
./tests/test_cmdline.py:238:128: E501 line too long (180 > 127 characters)
./tests/test_cmdline.py:241:128: E501 line too long (175 > 127 characters)
./tests/test_cmdline.py:249:128: E501 line too long (188 > 127 characters)
./tests/test_cmdline.py:252:128: E501 line too long (186 > 127 characters)
./tests/test_cmdline.py:255:128: E501 line too long (178 > 127 characters)
./tests/test_cmdline.py:257:128: E501 line too long (177 > 127 characters)
./tests/test_cmdline.py:263:128: E501 line too long (164 > 127 characters)
./tests/test_cmdline.py:276:128: E501 line too long (162 > 127 characters)
./tests/test_performance.py:77:128: E501 line too long (174 > 127 characters)
./tests/test_performance.py:78:128: E501 line too long (150 > 127 characters)
./tests/test_logsparse.py:11:1: E122 continuation line missing indentation or outdented
./tests/test_logsparse.py:26:1: E122 continuation line missing indentation or outdented
./tests/test_logsparse.py:61:128: E501 line too long (167 > 127 characters)
./tests/test_logsparse.py:62:128: E501 line too long (154 > 127 characters)
./tests/test_logsparse.py:72:128: E501 line too long (228 > 127 characters)
./tests/test_prog_snapshot.py:9:128: E501 line too long (1405 > 127 characters)
./tests/test_prog_snapshot.py:76:128: E501 line too long (210 > 127 characters)
./tests/test_prog_snapshot.py:80:128: E501 line too long (319 > 127 characters)
./tests/test_prog_snapshot.py:94:128: E501 line too long (318 > 127 characters)
./tests/test_prog_snapshot.py:102:128: E501 line too long (375 > 127 characters)
./tests/test_regression.py:94:1: E122 continuation line missing indentation or outdented
./tests/test_regression.py:111:128: E501 line too long (128 > 127 characters)
./tests/test_regression.py:121:1: E122 continuation line missing indentation or outdented
./tests/test_regression.py:164:1: E122 continuation line missing indentation or outdented
./tests/test_regression.py:183:128: E501 line too long (153 > 127 characters)
./tests/test_regression.py:189:128: E501 line too long (129 > 127 characters)
./tests/data/vdb_multithread.py:10:16: E226 missing whitespace around arithmetic operator
./tests/data/vdb_multithread.py:10:26: E226 missing whitespace around arithmetic operator
./tests/data/vdb_multithread.py:17:17: E226 missing whitespace around arithmetic operator
./tests/data/vdb_multithread.py:17:28: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:1:1: E265 block comment should start with '# '
./tests/data/vdb_basic.py:2:1: F401 'time' imported but unused
./tests/data/vdb_basic.py:4:1: E265 block comment should start with '# '
./tests/data/vdb_basic.py:10:13: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:10:16: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:12:1: E302 expected 2 blank lines, found 1
./tests/data/vdb_basic.py:15:5: E265 block comment should start with '# '
./tests/data/vdb_basic.py:17:1: E302 expected 2 blank lines, found 1
./tests/data/vdb_basic.py:18:5: E265 block comment should start with '# '
./tests/data/vdb_basic.py:19:10: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:21:8: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:21:13: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:22:5: E265 block comment should start with '# '
./tests/data/vdb_basic.py:25:1: E302 expected 2 blank lines, found 1
./tests/data/vdb_basic.py:26:8: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:26:13: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:28:8: E226 missing whitespace around arithmetic operator
./tests/data/vdb_basic.py:30:1: E305 expected 2 blank lines after class or function definition, found 1
./tests/data/vdb_basic.py:36:30: W292 no newline at end of file
./tests/data/fib.py:4:17: E226 missing whitespace around arithmetic operator
./tests/data/fib.py:4:28: E226 missing whitespace around arithmetic operator
./docs/source/conf.py:1:1: F401 'sphinx_rtd_theme' imported but unused
./docs/source/conf.py:65:31: W292 no newline at end of file
./src/viztracer/report_builder.py:63:21: E127 continuation line over-indented for visual indent
./src/viztracer/report_builder.py:67:128: E501 line too long (144 > 127 characters)
./src/viztracer/viztracer.py:36:17: E126 continuation line over-indented for hanging indent
./src/viztracer/functree.py:46:17: E126 continuation line over-indented for hanging indent
./src/viztracer/main.py:51:128: E501 line too long (135 > 127 characters)
./src/viztracer/main.py:53:128: E501 line too long (140 > 127 characters)
./src/viztracer/main.py:89:128: E501 line too long (162 > 127 characters)
./src/viztracer/main.py:97:128: E501 line too long (129 > 127 characters)
./src/viztracer/main.py:137:128: E501 line too long (145 > 127 characters)
./src/viztracer/main.py:354:128: E501 line too long (138 > 127 characters)
./src/viztracer/tracer.py:296:128: E501 line too long (133 > 127 characters)
./src/viztracer/tracer.py:297:128: E501 line too long (161 > 127 characters)
./src/viztracer/code_monkey.py:160:21: E126 continuation line over-indented for hanging indent
./src/viztracer/code_monkey.py:166:21: E126 continuation line over-indented for hanging indent
./src/viztracer/code_monkey.py:172:21: E126 continuation line over-indented for hanging indent
./src/viztracer/code_monkey.py:181:17: E126 continuation line over-indented for hanging indent
22    E122 continuation line missing indentation or outdented
6     E126 continuation line over-indented for hanging indent
1     E127 continuation line over-indented for visual indent
2     E128 continuation line under-indented for visual indent
1     E225 missing whitespace around operator
21    E226 missing whitespace around arithmetic operator
1     E231 missing whitespace after ':'
2     E251 unexpected spaces around keyword / parameter equals
5     E265 block comment should start with '# '
4     E302 expected 2 blank lines, found 1
1     E305 expected 2 blank lines after class or function definition, found 1
52    E501 line too long (135 > 127 characters)
2     F401 'sphinx_rtd_theme' imported but unused
3     W291 trailing whitespace
6     W292 no newline at end of file
3     W293 blank line contains whitespace
132

opened by luoos 11

Make vizviewer test more stable on Mac on github actions

Now vizviewer test sometimes fails on nightly validations(example). It's not a real failure, just the port is somehow occupied. As vizviewer does support custom ports, we can maybe check if the port works first, then try to test on it, which will reduce the fake positives in nightly validation.
good first issue deficiency

opened by gaogaotiantian 9
Add Linux AArch64 wheel build support

Added linux aarch64 wheel build support. Related to https://github.com/gaogaotiantian/viztracer/issues/190, @gaogaotiantian Could you please review this PR?

opened by odidev 9

Subprocess injection mechanism gaps make it impossible to try tracing pip's behaviour

I think the title covers my usecase. :)

In a clean virtualenv, install viztracer, and then run:

viztracer -m pip install --debug requests --no-binary :all:

You'll notice that all of pip's subprocess calls will fail:

  ERROR: Command errored out with exit status 2:
   command: /Users/pradyunsg/Developer/pip/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-install-z2tj4fa8/requests_82de87b5220c4e10809d513d90ca41a5/setup.py'"'"'; __file__='"'"'/private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-install-z2tj4fa8/requests_82de87b5220c4e10809d513d90ca41a5/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-pip-egg-info-0gaf3f7n
       cwd: /private/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/pip-install-z2tj4fa8/requests_82de87b5220c4e10809d513d90ca41a5/
  Complete output (5 lines):
  
  Usage:
    /Users/pradyunsg/Developer/pip/.venv/bin/python -m pip <command> [options]
  
  no such option: -c
  ----------------------------------------

bug

opened by pradyunsg 9

viztracer reports twice and does not show the subprocesses tracks on a loky-based multiprocessing program

loky is an extension of concurrent.futures ProcessPoolExecutor with the spawn start method to keep a running process pool in the background to avoid paying the process start time each time a process pool is needed. Furthermore it uses cloudpickle to make it possible to call interactively defined functions (in the __main__ module) or lambda closures and such.

Here is the reproducer:

pip install loky

from loky import get_reusable_executor
import time
import random
import os


def my_function(*args):
   pid = os.getpid()
   duration = random.random() * 2
   print(f"[{pid}] sleeping {duration:.3f} s")
   time.sleep(duration)
   print(f"[{pid}] done")



e = get_reusable_executor(max_workers=4)

print("[main] start")
e.map(my_function, range(8))

e = get_reusable_executor(max_workers=4)
e.map(my_function, range(8))

print("[main] done")

❯ viztracer --log_subprocess --log_multiprocess bench_loky.py
[main] start
[main] done
[26109] sleeping 1.471 s
[26106] sleeping 1.364 s
[26107] sleeping 0.170 s
[26108] sleeping 1.002 s
[26107] done
[26107] sleeping 0.877 s
[26108] done
[26108] sleeping 1.297 s
[26107] done
[26107] sleeping 1.635 s
[26106] done
[26106] sleeping 1.079 s
[26109] done
[26109] sleeping 1.809 s
[26108] done
[26108] sleeping 1.962 s
[26106] done
[26106] sleeping 1.377 s
[26107] done
[26107] sleeping 0.973 s
[26109] done
[26109] sleeping 1.528 s
[26107] done
[26107] sleeping 0.615 s
[26106] done
[26106] sleeping 0.256 s
[26106] done
[26106] sleeping 1.554 s
[26108] done
[26107] done
[26109] done
[26106] done
Loading finish                                        
==================================================
== Starting from version 0.13.0, VizTracer will ==
== use json as the default report file. You can ==
== generate HTML report with "-o result.html"   ==
==================================================
Dumping trace data, total entries: 31753
Saving report to /Users/ogrisel/viztracer_multiprocess_tmp_26103_1629359788/result_26103.json ...
Use "vizviewer <your_report>" to open the report
==================================================
== Starting from version 0.13.0, VizTracer will ==
== use json as the default report file. You can ==
== generate HTML report with "-o result.html"   ==
==================================================
Dumping trace data, total entries: 31753
Saving report to /Users/ogrisel/result.json ...
Use "vizviewer <your_report>" to open the report

The reports does not show the subprocesses.

Python: 3.9.2 on macOS.

opened by ogrisel 9

multiprocessing profiling does not work with inline viztracer

version 0.15.6 python 3.8.10

I am trying to integrate viztracer with bazel but run into the following blocking issue that inline viztracer can not generate profiling reports for subprocess.

It works well via command line:

from multiprocessing import Pool
import os
import time

from viztracer import VizTracer, get_tracer


def init_worker():
    """Init worker to print pid."""
    pid = os.getpid()
    print(f'pid={pid}')


def main():

    # pylint: disable=unused-variable
    def afterfork_callback(tracer):
        print('afterfork_callback')

    get_tracer().set_afterfork(afterfork_callback)
    with Pool(5, initializer=init_worker) as pool:
        pool.map(time.sleep, [5] * 5)


if __name__ == "__main__":
    main()

Running results:

$ viztracer test.py 
afterfork_callback
pid=2169334
afterfork_callback
pid=2169335
afterfork_callback
pid=2169336
afterfork_callback
pid=2169337
afterfork_callback
pid=2169338
Total Entries: 4280                                                             
Use the following command to open the report:

Screen Shot 2022-12-29 at 1 09 46 PM

But all subprocess profiling info will disappear if we switch to use with inline code:

from multiprocessing import Pool
import os
import time

from viztracer import VizTracer


def init_worker():
    """Init worker to print pid."""
    pid = os.getpid()
    print(f'pid={pid}')


def main():

    # pylint: disable=unused-variable
    def afterfork_callback(tracer):
        print('afterfork_callback')

    with VizTracer() as tracer:
        tracer.set_afterfork(afterfork_callback)
        with Pool(5, initializer=init_worker) as pool:
            pool.map(time.sleep, [5] * 5)

Running results:

$ python3 test.py 
pid=2169028
pid=2169029
pid=2169030
pid=2169031
pid=2169032
Loading finish                                        
Total Entries: 3673                                                             
Use the following command to open the report:

Note that the total entries are much less and also there is no log printed in afterfork_callback.

Screen Shot 2022-12-29 at 1 06 00 PM

Is it a bug?

opened by speakless86 1

Add instant event compressor
This is compressor for instant events.

I've learned about compress recently. Actually, we did not apply any algorithm on the compressor. What we are doing is to rearrange to reduce the redundancy of the data. And it's helpful for compressing. I did some simple tests, here's the result: I tested different_sorts.py with file_info=False and used different algorithms to compress result.json and result.cvf, here's the results:

zlib compress result.json, origin size is 3563190, compressed size is 127722 zlib compress result.cvf, origin size is 448242, compressed size is 97918 lzma compress result.json, origin size is 3563190, compressed size is 83900 lzma compress result.cvf, origin size is 448242, compressed size is 69148
opened by TTianshun 2

Installation failed on Mac OS Monterey M1 Chip

I tried to install viztracer using pip install viztracer but it failed yielding the following error. I also have gcc installed in my system.

Collecting viztracer
  Using cached viztracer-0.15.3.tar.gz (8.9 MB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: objprint>=0.1.3 in /Users/abdullahalimran/opt/miniforge3/envs/test/lib/python3.8/site-packages (from viztracer) (0.2.2)
Building wheels for collected packages: viztracer
  Building wheel for viztracer (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/abdullahalimran/opt/miniforge3/envs/test/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"'; __file__='"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-wheel-ug1q7ob0
       cwd: /private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/
  Complete output (109 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11.0-arm64-3.8
  creating build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/report_builder.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/patch.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/viewer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/vizevent.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/vizplugin.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/util.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/decorator.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/attach.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/viztracer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/__init__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/vizcounter.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/functree.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/vizlogging.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/vizobject.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/flamegraph.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/simulator.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/cellmagic.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/prog_snapshot.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/event_base.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/main.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/__main__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/tracer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  copying src/viztracer/code_monkey.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
  copying src/viztracer/attach_process/add_code_to_python_process.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
  copying src/viztracer/attach_process/__init__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/html
  copying src/viztracer/html/flamegraph.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
  copying src/viztracer/html/trace_viewer_embedder.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
  copying src/viztracer/html/trace_viewer_full.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  copying src/viztracer/web_dist/index.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  copying src/viztracer/web_dist/LICENSE -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  copying src/viztracer/web_dist/trace_processor -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  copying src/viztracer/web_dist/service_worker.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  copying src/viztracer/web_dist/service_worker.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/index.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/controller_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/engine_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/traceconv_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/trace_to_text.wasm -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/controller_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/traceconv_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/trace_processor.wasm -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/frontend_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/frontend_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/perfetto.css -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/manifest.json -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/engine_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/logo-128.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_lmk.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_voltage.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ftrace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoMono-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_syscalls.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_one_shot.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_atrace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-500.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Raleway-Thin.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_long_trace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/catapult_trace_viewer.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-400.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_board_voltage.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoCondensed-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ps_stats.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_battery_counters.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_mem_hifreq.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/MaterialIcons.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/favicon.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-100.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_native_heap_profiler.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_fine.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoCondensed-Light.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_frame_timeline.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/catapult_trace_viewer.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_vmstat.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_gpu_mem_total.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/logo-3d.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_java_heap_dump.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_logcat.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_meminfo.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_coarse.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Raleway-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_freq.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/brand.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-300.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ring_buf.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
  copying src/viztracer/attach_process/LICENSE -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
  copying src/viztracer/attach_process/attach_x86_64.dylib -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
  creating build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process/linux_and_mac
  copying src/viztracer/attach_process/linux_and_mac/lldb_prepare.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process/linux_and_mac
  running build_ext
  building 'viztracer.snaptrace' extension
  creating build/temp.macosx-11.0-arm64-3.8
  creating build/temp.macosx-11.0-arm64-3.8/src
  creating build/temp.macosx-11.0-arm64-3.8/src/viztracer
  creating build/temp.macosx-11.0-arm64-3.8/src/viztracer/modules
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/abdullahalimran/opt/miniforge3/envs/test/include -arch arm64 -I/Users/abdullahalimran/opt/miniforge3/envs/test/include -arch arm64 -I/Users/abdullahalimran/opt/miniforge3/envs/test/include/python3.8 -c src/viztracer/modules/util.c -o build/temp.macosx-11.0-arm64-3.8/src/viztracer/modules/util.o -Werror -std=c99
  gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for viztracer
  Running setup.py clean for viztracer
Failed to build viztracer
Installing collected packages: viztracer
    Running setup.py install for viztracer ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/abdullahalimran/opt/miniforge3/envs/test/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"'; __file__='"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-record-f8l4fizf/install-record.txt --single-version-externally-managed --compile --install-headers /Users/abdullahalimran/opt/miniforge3/envs/test/include/python3.8/viztracer
         cwd: /private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/
    Complete output (111 lines):
    running install
    /Users/abdullahalimran/opt/miniforge3/envs/test/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build/lib.macosx-11.0-arm64-3.8
    creating build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/report_builder.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/patch.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/viewer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/vizevent.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/vizplugin.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/util.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/decorator.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/attach.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/viztracer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/__init__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/vizcounter.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/functree.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/vizlogging.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/vizobject.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/flamegraph.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/simulator.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/cellmagic.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/prog_snapshot.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/event_base.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/main.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/__main__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/tracer.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    copying src/viztracer/code_monkey.py -> build/lib.macosx-11.0-arm64-3.8/viztracer
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
    copying src/viztracer/attach_process/add_code_to_python_process.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
    copying src/viztracer/attach_process/__init__.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/html
    copying src/viztracer/html/flamegraph.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
    copying src/viztracer/html/trace_viewer_embedder.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
    copying src/viztracer/html/trace_viewer_full.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/html
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    copying src/viztracer/web_dist/index.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    copying src/viztracer/web_dist/LICENSE -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    copying src/viztracer/web_dist/trace_processor -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    copying src/viztracer/web_dist/service_worker.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    copying src/viztracer/web_dist/service_worker.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/index.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/controller_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/engine_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/traceconv_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/trace_to_text.wasm -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/controller_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/traceconv_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/trace_processor.wasm -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/frontend_bundle.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/frontend_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/perfetto.css -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/manifest.json -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/engine_bundle.js.map -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/logo-128.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_lmk.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_voltage.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ftrace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoMono-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_syscalls.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_one_shot.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_atrace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-500.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Raleway-Thin.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_long_trace.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/catapult_trace_viewer.js -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-400.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_board_voltage.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoCondensed-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ps_stats.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_battery_counters.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_mem_hifreq.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/MaterialIcons.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/favicon.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-100.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_native_heap_profiler.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_fine.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/RobotoCondensed-Light.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_frame_timeline.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/catapult_trace_viewer.html -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_vmstat.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_gpu_mem_total.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/logo-3d.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_java_heap_dump.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_logcat.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_meminfo.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_coarse.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Raleway-Regular.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_cpu_freq.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/brand.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/Roboto-300.woff2 -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/web_dist/v25.0-cca3e0ce7/assets/rec_ring_buf.png -> build/lib.macosx-11.0-arm64-3.8/viztracer/web_dist/v25.0-cca3e0ce7/assets
    copying src/viztracer/attach_process/LICENSE -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
    copying src/viztracer/attach_process/attach_x86_64.dylib -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process
    creating build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process/linux_and_mac
    copying src/viztracer/attach_process/linux_and_mac/lldb_prepare.py -> build/lib.macosx-11.0-arm64-3.8/viztracer/attach_process/linux_and_mac
    running build_ext
    building 'viztracer.snaptrace' extension
    creating build/temp.macosx-11.0-arm64-3.8
    creating build/temp.macosx-11.0-arm64-3.8/src
    creating build/temp.macosx-11.0-arm64-3.8/src/viztracer
    creating build/temp.macosx-11.0-arm64-3.8/src/viztracer/modules
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/abdullahalimran/opt/miniforge3/envs/test/include -arch arm64 -I/Users/abdullahalimran/opt/miniforge3/envs/test/include -arch arm64 -I/Users/abdullahalimran/opt/miniforge3/envs/test/include/python3.8 -c src/viztracer/modules/util.c -o build/temp.macosx-11.0-arm64-3.8/src/viztracer/modules/util.o -Werror -std=c99
    gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/abdullahalimran/opt/miniforge3/envs/test/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"'; __file__='"'"'/private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-install-7htfqewb/viztracer_ca1d85fcb0b042e88c496dcb89cf340f/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/sy/5gyrfs415dnbjfyt7xgjdhj00000gn/T/pip-record-f8l4fizf/install-record.txt --single-version-externally-managed --compile --install-headers /Users/abdullahalimran/opt/miniforge3/envs/test/include/python3.8/viztracer Check the logs for full command output.

opened by abdalimran 2

Write a compressor/decompressor for the trace log file

Now the trace log file is huge, which is okay on local machines. However, it makes it difficult to share the trace file through network, or to store it somewhere in the cloud.

Most of the info in the trace file is duplicated and we should be able to get a very decent compress ratio for the trace file.
enhancement

opened by gaogaotiantian 18
multiprocessing.Pool related tests occasionally fail in github actions
For now, they are:

tests.test_multiprocess.TestMultiprocessing.test_multiprocess_pool

tests.test_regression.TestIssue58.test_issue58

I can't reproduce this locally. It's probably a racing issue somewhere. We have not gotten any bug report from the users yet and I've put some efforts in fixing it but failed(I think I improved it a bit).

Anyone is welcome to give it another try, but if you found your tests failed on these tests, it's okay, that's not introduced by your changes.
unstable-test
opened by gaogaotiantian 0

Releases(0.15.6)

0.15.6(Nov 13, 2022)
Fixed vizplugins

Source code(tar.gz)
Source code(zip)
0.15.5(Oct 31, 2022)
Added support for 3.11 and dropped support for 3.6

Fixed escape character issue in dump_raw

Source code(tar.gz)
Source code(zip)
0.15.4(Aug 15, 2022)
Nested multi-process worked

Supported function name error ignore

vizviewer now exits when the port is occupied

Source code(tar.gz)
Source code(zip)
0.15.3(Jun 10, 2022)
Thread safe now

Supported external processor

Stack for log_sparse

Source code(tar.gz)
Source code(zip)
0.15.2(May 13, 2022)
Enabled color on Windows terminal without default VT

Fixed vizviewer Ctrl+C bug on Windows

Enforced C99 for C code compile

Added manual thread aware feature

Source code(tar.gz)
Source code(zip)
0.15.1(Mar 14, 2022)
Fix jupyter

Add py3.6 wheel

Add functools.wraps for decorators

Deprecate vdb

Source code(tar.gz)
Source code(zip)
0.15.0(Feb 21, 2022)
Supported REAL attach

Source code(tar.gz)
Source code(zip)
0.14.5(Jan 23, 2022)
aarch64 wheels

Fix help command

add trio support

Source code(tar.gz)
Source code(zip)
0.14.4(Dec 13, 2021)
Added log_audit feature

Accelerated report generation speed with raw dump

Added directory viewer feature

Source code(tar.gz)
Source code(zip)
0.14.3(Nov 29, 2021)
Fixed an encoding bug in vizviewer

Saved trace even with exception with with statement

Added .viztracerrc rcfile for configuration

Cleared the exit routine, less failure, faster dump, better messages

Added conditional magic comment

Source code(tar.gz)
Source code(zip)
0.14.2(Oct 31, 2021)
Fixed duplicated timestamp calculation

Made multiprocessing pool more stable

Source code(tar.gz)
Source code(zip)
0.14.1(Oct 24, 2021)
Supported python 3.10

Added fast log feature - log_var and log_instant

Added magic comment feature

Enabled vizviewer to use arbitrary port

Fixed subprocess and exit routine bugs

Source code(tar.gz)
Source code(zip)
0.14.0(Aug 28, 2021)
removed --log_multiprocess and --log_subprocess

fix multiprocess bugs

upgrade perfetto version

add min_duration filter

Source code(tar.gz)
Source code(zip)
0.13.4(Jun 12, 2021)
Optimized flamegraph and its output

Upgrade perfetto to support counter events

Fixed instant event format

Bug fix

Source code(tar.gz)
Source code(zip)
0.13.3(May 29, 2021)
Support flamegraph

Optimize memory usage

Source code(tar.gz)
Source code(zip)
0.13.2(May 25, 2021)

Support Instant and Counter event in UI
Source code(tar.gz)
Source code(zip)
0.13.1(May 20, 2021)
Updated perfetto

Fixed timestamp unit

Source code(tar.gz)
Source code(zip)
0.13.0(May 15, 2021)
Use Perfetto as default UI

Changed vizviewer

Fixed timestamp bug

Source code(tar.gz)
Source code(zip)
0.12.3(Apr 5, 2021)
Support jupyter cell magics

Add column number

Some performance improvement

Source code(tar.gz)
Source code(zip)
0.12.2(Mar 6, 2021)

Bug fix for vizviewer and timestamps
Source code(tar.gz)
Source code(zip)
0.12.1(Mar 6, 2021)
support perfetto

add vizviewer

unique name for tmp folder so multi-multi-processing program can work

Source code(tar.gz)
Source code(zip)
0.12.0(Feb 22, 2021)
log_async is supported

log_event is supported

log_sparse can show source code now

Source code(tar.gz)
Source code(zip)
0.11.4(Jan 31, 2021)
add remote attach feature

fix source code display bug

fix spawn process bug

Source code(tar.gz)
Source code(zip)
0.11.3(Jan 22, 2021)

Now --log_multiprocess supports spawn
Source code(tar.gz)
Source code(zip)
0.11.2(Jan 14, 2021)

Fix __name__ issue with -m
Source code(tar.gz)
Source code(zip)
0.11.1(Jan 7, 2021)

Renamed ignore_non_file to ignore_frozen and changed the implementation to reflect that. Now it will only ignore frozen functions, not <string> ones.
Source code(tar.gz)
Source code(zip)
0.11.0(Dec 31, 2020)
thread and process name supported

subprocess works in more cases now

multithread works better now

plugin feature is added

Source code(tar.gz)
Source code(zip)
0.10.5(Dec 8, 2020)
Fixed run_module issue to locate the attributes

Fixed log_sparse when --log_sparse is not passed in

Source code(tar.gz)
Source code(zip)
0.10.4(Nov 26, 2020)
set_afterfork is added to customize tracer after fork()

forked process will clear the buffer and reset current stack depth with multiprocessing

--max_stack_depth works properly with multiprocessing now

Source code(tar.gz)
Source code(zip)
0.10.3(Nov 21, 2020)
Added space after file path print on console to make it clickable

Make log_var work with function and for loop

Increment timestamp if we had the same timestamp to fix same ts issue

Source code(tar.gz)
Source code(zip)

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.

Related tags

Overview

VizTracer

Highlights

Install

Basic Usage

Command Line

Inline

Jupyter

Advanced Usage

Trace Filter

Extra Logs without Code Change

Add Custom Event

Misc

Multi Thread Support

Multi Process Support

Async Support

Flamegraph

Remote attach

JSON alternative

Virtual Debug

Performance

Documentation

Bugs/Requests

License

Comments

Releases(0.15.6)

0.15.6(Nov 13, 2022)

0.15.5(Oct 31, 2022)

0.15.4(Aug 15, 2022)

0.15.3(Jun 10, 2022)

0.15.2(May 13, 2022)

0.15.1(Mar 14, 2022)

0.15.0(Feb 21, 2022)

0.14.5(Jan 23, 2022)

0.14.4(Dec 13, 2021)

0.14.3(Nov 29, 2021)

0.14.2(Oct 31, 2021)

0.14.1(Oct 24, 2021)

0.14.0(Aug 28, 2021)

0.13.4(Jun 12, 2021)

0.13.3(May 29, 2021)

0.13.2(May 25, 2021)

0.13.1(May 20, 2021)

0.13.0(May 15, 2021)

0.12.3(Apr 5, 2021)

0.12.2(Mar 6, 2021)

0.12.1(Mar 6, 2021)

0.12.0(Feb 22, 2021)

0.11.4(Jan 31, 2021)

0.11.3(Jan 22, 2021)

0.11.2(Jan 14, 2021)

0.11.1(Jan 7, 2021)

0.11.0(Dec 31, 2020)

0.10.5(Dec 8, 2020)

0.10.4(Nov 26, 2020)

0.10.3(Nov 21, 2020)

Owner

Run-time type checker for Python

Full featured multi arch/os debugger built on top of PyQt5 and frida

Visual Interaction with Code - A portable visual debugger for python

AryaBota: An app to teach Python coding via gradual programming and visual output

Silky smooth profiling for Django

Hunter is a flexible code tracing toolkit.

Debugging manhole for python applications.

pdb++, a drop-in replacement for pdb (the Python debugger)

Tracing instruction in lldb debugger.Just a python-script for lldb.

Dahua Console, access internal debug console and/or other researched functions in Dahua devices.

Automated bug/error reporting for napari

A drop-in replacement for Django's runserver.

Debugger capable of attaching to and injecting code into python processes.

A configurable set of panels that display various debug information about the current request/response.

Middleware that Prints the number of DB queries to the runserver console.

NoPdb: Non-interactive Python Debugger

A gdb-like Python3 Debugger in the Trepan family

Trace any Python program, anywhere!

Sampling profiler for Python programs

GDB plugin for streaming defmt messages over RTT from e.g. JLinkGDBServer