NoPdb: Non-interactive Python Debugger

Overview

NoPdb: Non-interactive Python Debugger

PyPI Package Documentation Status Lint Status Lint Status

NoPdb is a programmatic (non-interactive) debugger for Python. This means it gives you access to debugger-like superpowers directly from your code. With NoPdb, you can:

  • capture function calls, including arguments, local variables, return values and stack traces
  • set "breakpoints" that trigger user-defined actions when hit, namely:
    • evaluate expressions to retrieve their values later
    • execute arbitrary code, including modifying local variables
    • enter an interactive debugger like pdb

NoPdb is also a convenient tool for inspecting machine learning model internals. For example, this notebook shows how to use it to visualize Transformer attention in PyTorch.

NoPdb should run at least under CPython and PyPy. Most features should work under any implementation as long as it has sys.settrace().

Note: This project is in its early development stage. Contributions and improvement ideas are welcome.

Capturing function calls

The functions capture_call() and capture_calls() allow capturing useful information about calls to a given function. They are typically used as context managers, e.g.:

with nopdb.capture_calls(fn) as calls:
    some_code_that_calls_fn()

    print(calls)  # see details about how fn() was called

The information we can retrieve includes the function's arguments, return value, local variables and stack trace. For example:

>>> with nopdb.capture_call(f) as call:
...     g(1)
>>> call
CallCapture(name='f', args=OrderedDict(x=1, y=1), return_value=4)
>>> call.print_stack()
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 2, in g
  File "<stdin>", line 1, in f
>>> call.args['x']
1
>>> call.return_value
4
>>> call.locals
{'y': 1, 'x': 1, 'z': 2}

Setting breakpoints

Like conventional debuggers, NoPdb can set breakpoints. However, because NoPdb is a non-interactive debugger, its breakpoints do not actually stop the execution of the program. Instead, they allow executing actions scheduled in advance, such as evaluating expressions.

To set a breakpoint, call the breakpoint() function. A breakpoint object is returned, allowing to schedule actions using its methods such as eval() and exec(). For example:

# Break at line 3 of the file or notebook cell where f is defined
with nopdb.breakpoint(function=f, line=3) as bp:
    x = bp.eval("x")             # Schedule an expression
    type_y = bp.eval("type(y)")  # Another one
    bp.exec("print(y)")          # Schedule a print statement

    some_code_that_calls_f()

print(x, type_y)  # Retrieve the captured values

There are other ways to specify the breakpoint location. For example:

# Break at any line with the given source code in the given file
with nopdb.breakpoint(file="pathlib.py", line="return obj") as bp:
    ...

# Break as soon as any function with the given name is called
with nopdb.breakpoint(function="myfunc") as bp:
    ...

Not only can we capture values, we can also modify them!

>>> with nopdb.breakpoint(function=f, line=3) as bp:
...     # Get the value of x, then increment it, then get the new value
...     x_before = bp.eval('x')
...     bp.exec('x += 1')
...     x_after = bp.eval('x')
...
...     some_code_that_calls_f()
>>> x_before
[2]
>>> x_after
[3]

Planned features

Functionalities that do not exist, but could be added in the future:

  • Breakpoint.callback() for calling a given callback function, passing information about the current frame as an argument.
  • Breakpoint.jump() for jumping to a different line in the same function.
  • A way to disable breakpoints.

Limitations

  • Like Pdb, NoPdb only works with pure-Python functions. Calls to built-ins and C extensions cannot be captured. This also applies to ML frameworks that compile models into static graphs; for NoPdb to work, this feature needs to be disabled, e.g. with tf.config.run_functions_eagerly(True) in TensorFlow and with the jax.disable_jit() context manager in JAX.
  • Local variable assignment in Breakpoint.exec() is only supported under CPython and PyPy.
You might also like...
PINCE is a front-end/reverse engineering tool for the GNU Project Debugger (GDB), focused on games.
PINCE is a front-end/reverse engineering tool for the GNU Project Debugger (GDB), focused on games.

PINCE is a front-end/reverse engineering tool for the GNU Project Debugger (GDB), focused on games. However, it can be used for any reverse-engi

Little helper to run Steam apps under Proton with a GDB debugger

protongdb A small little helper for running games with Proton and debugging with GDB Requirements At least Python 3.5 protontricks pip package and its

Full featured multi arch/os debugger built on top of PyQt5 and frida

Full featured multi arch/os debugger built on top of PyQt5 and frida

A simple rubber duck debugger

Rubber Duck Debugger I found myself many times asking a question on StackOverflow or to one of my colleagues just for finding the solution simply by d

Hdbg - Historical Debugger
Hdbg - Historical Debugger

hdbg - Historical Debugger This is in no way a finished product. Do not use this

Trashdbg - TrashDBG the world's worse debugger
Trashdbg - TrashDBG the world's worse debugger

The world's worse debugger Over the course of multiple OALABS Twitch streams we

Trace any Python program, anywhere!

lptrace lptrace is strace for Python programs. It lets you see in real-time what functions a Python program is running. It's particularly useful to de

Debugging manhole for python applications.

Overview docs tests package Manhole is in-process service that will accept unix domain socket connections and present the stacktraces for all threads

(OLD REPO) Line-by-line profiling for Python - Current repo -

line_profiler and kernprof line_profiler is a module for doing line-by-line profiling of functions. kernprof is a convenient script for running either

Comments
  • Functionality questions

    Functionality questions

    Thanks for making this project. A non-interactive python debugger is exactly what I'm looking for. I'd like to use your project to support the Python probe for Source++. I've been able to successfully implement some portions of the functionality, but I'm having some trouble understanding some Python debugging relating things. I think some of this trouble might be alleviated with new functionality for nopdb and I'm more than willing to implement that functionality. My background is with JVM languages so I'm just looking for a bit of guidance on how things work the Python way.

    Breakpoint doesn't hit on Flask app

    I don't think this issue is a nopdb issue as I'm unable to hit the breakpoint using import pdb; pdb.set_trace() in the main method either. I believe this issue is because of either compiled code within Flask or pdb is not hitting because of separate threads. Either way, I'm able to set breakpoints via PyCharm so it appears to be possible in some way. Hopefully, it's just a threading thing and breakpoints can be placed for every thread. Do you know how this functionality could be implemented with nopdb?

    Here is some code that shows a breakpoint that's never hit:

    import os
    
    from flask import Flask
    from nopdb import nopdb
    
    app = Flask(__name__)
    
    
    @app.route('/')
    def hello():
        provider = str(os.environ.get('PROVIDER', 'world'))
        return 'Hello ' + provider + '!'
    
    
    if __name__ == '__main__':
        with nopdb.breakpoint(file="FlaskTest.py", line=12) as bp:
            bp.exec("print(\"provider: \" + provider)")
            port = int(os.environ.get('PORT', 5000))
            app.run(host='0.0.0.0', port=port)
    
    

    Ability to add breakpoints without calling code via context manager

    This might be related to the issue above, but how do you add breakpoints to code that you don't want to call via the context manager? As in, code that is already running or that will run by itself sometime in the future? I have a small example here where I want to set a breakpoint before a function is called in the future via a delay. Can nopdb address this type of scenario?

    Code which shows a breakpoint that's never hit:

    import threading
    
    from nopdb import nopdb
    
    
    def fun():
        x = 5
        y = x
    
    
    if __name__ == '__main__':
        start_time = threading.Timer(10, fun)
        start_time.start()
    
       # I'd like to set the breakpoint ahead of time and just output x once it's hit
        with nopdb.breakpoint(file="SeparateThread.py", line=8) as bp:
            bp.exec("print(\"x: \" + str(x))")
    
    

    Does cpdb make sense to extend?

    This might be a bit off-topic, but while looking into Python debuggers I ran across https://pypi.org/project/CPdb/. Does this project make sense to extend for nopdb? The reason I ask is because of the claim cpdb makes about "long-running" applications running better with it. I would like to use nopdb in "long-running" applications so I'm curious if it makes sense to integrate the technologies.

    opened by BFergerson 0
Releases(v0.2.0)
Owner
Ondřej Cífka
PhD candidate @ Télécom Paris. Working on music style transfer
Ondřej Cífka
OpenCodeBlocks an open-source tool for modular visual programing in python

OpenCodeBlocks OpenCodeBlocks is an open-source tool for modular visual programing in python ! Although for now the tool is in Beta and features are c

Mathïs Fédérico 1.1k Jan 06, 2023
Hunter is a flexible code tracing toolkit.

Overview docs tests package Hunter is a flexible code tracing toolkit, not for measuring coverage, but for debugging, logging, inspection and other ne

Ionel Cristian Mărieș 705 Dec 08, 2022
Tracing instruction in lldb debugger.Just a python-script for lldb.

lldb-trace Tracing instruction in lldb debugger. just a python-script for lldb. How to use it? Break at an address where you want to begin tracing. Im

156 Jan 01, 2023
Visual Interaction with Code - A portable visual debugger for python

VIC Visual Interaction with Code A simple tool for debugging and interacting with running python code. This tool is designed to make it easy to inspec

Nathan Blank 1 Nov 16, 2021
Cyberbrain: Python debugging, redefined.

Cyberbrain1(电子脑) aims to free programmers from debugging.

laike9m 2.3k Jan 07, 2023
A toolbar overlay for debugging Flask applications

Flask Debug-toolbar This is a port of the excellent django-debug-toolbar for Flask applications. Installation Installing is simple with pip: $ pip ins

863 Dec 29, 2022
pdb++, a drop-in replacement for pdb (the Python debugger)

pdb++, a drop-in replacement for pdb What is it? This module is an extension of the pdb module of the standard library. It is meant to be fully compat

1k Jan 02, 2023
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 7.3k Dec 29, 2022
Code2flow generates call graphs for dynamic programming language. Code2flow supports Python, Javascript, Ruby, and PHP.

Code2flow generates call graphs for dynamic programming language. Code2flow supports Python, Javascript, Ruby, and PHP.

Scott Rogowski 3k Jan 01, 2023
一个小脚本,用于trace so中native函数的调用。

trace_natives 一个IDA小脚本,获取SO代码段中所有函数的偏移地址,再使用frida-trace 批量trace so函数的调用。 使用方法 1.将traceNatives.py丢进IDA plugins目录中 2.IDA中,Edit-Plugins-traceNatives IDA输

296 Dec 28, 2022
Visual profiler for Python

vprof vprof is a Python package providing rich and interactive visualizations for various Python program characteristics such as running time and memo

Nick Volynets 3.9k Jan 01, 2023
Run-time type checker for Python

This library provides run-time type checking for functions defined with PEP 484 argument (and return) type annotations. Four principal ways to do type

Alex Grönholm 1.1k Jan 05, 2023
Debugger capable of attaching to and injecting code into python processes.

DISCLAIMER: This is not an official google project, this is just something I wrote while at Google. Pyringe What this is Pyringe is a python debugger

Google 1.6k Dec 15, 2022
Django package to log request values such as device, IP address, user CPU time, system CPU time, No of queries, SQL time, no of cache calls, missing, setting data cache calls for a particular URL with a basic UI.

django-web-profiler's documentation: Introduction: django-web-profiler is a django profiling tool which logs, stores debug toolbar statistics and also

MicroPyramid 77 Oct 29, 2022
The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.

LM-Debugger is an open-source interactive tool for inspection and intervention in transformer-based language models. This repository includes the code

Mor Geva 110 Dec 28, 2022
Little helper to run Steam apps under Proton with a GDB debugger

protongdb A small little helper for running games with Proton and debugging with GDB Requirements At least Python 3.5 protontricks pip package and its

Joshie 21 Nov 27, 2022
A powerful set of Python debugging tools, based on PySnooper

snoop snoop is a powerful set of Python debugging tools. It's primarily meant to be a more featureful and refined version of PySnooper. It also includ

Alex Hall 874 Jan 08, 2023
A simple rubber duck debugger

Rubber Duck Debugger I found myself many times asking a question on StackOverflow or to one of my colleagues just for finding the solution simply by d

1 Nov 10, 2021
Middleware that Prints the number of DB queries to the runserver console.

Django Querycount Inspired by this post by David Szotten, this project gives you a middleware that prints DB query counts in Django's runserver consol

Brad Montgomery 332 Dec 23, 2022
Sampling profiler for Python programs

py-spy: Sampling profiler for Python programs py-spy is a sampling profiler for Python programs. It lets you visualize what your Python program is spe

Ben Frederickson 9.5k Jan 08, 2023