dev_utils

Classes and functions used in development phase.

Tracing operations

Classes and decorators to work with sys.settrace(tracefunc)

Technical note:

tracefunc(the trace function) should have three arguments: frame, event, and arg:

  • frame is the current stack frame.
  • event is a string: 'call', 'line', 'return', 'exception' or 'opcode'.
  • arg depends on the event type.

frame has many attributes, including those below which are used in the tracing classes below:

Type Attribute Description
frame f_back next outer frame object (this frame’s caller)
f_code code object being executed in this frame
f_lineno current line number in Python source code
code co_code string of raw compiled bytecode
co_filename name of file in which this code object was created
co_name name with which this code object was defined
co_names tuple of names other than arguments and function locals
co_stacksize virtual machine stack space required
co_varnames tuple of names of arguments and local variables

See full documentation in sys and inspect built-in modules.

Experiments with frame and its attributes

def fn(a):
    return a + 1

b = fn(1)

frames = sys._current_frames()
framesids = list(frames.keys())

for id in framesids:
    fr = frames[id]
    print(f"{fr.f_lineno:5d}   {fr.f_code.co_name:20s}   {fr.f_code.co_filename:50s}")
   37   run                    /home/vtec/miniconda3/envs/ecutils/lib/python3.10/site-packages/ipykernel/parentpoller.py
  330   wait                   /home/vtec/miniconda3/envs/ecutils/lib/python3.10/threading.py
  482   select                 /home/vtec/miniconda3/envs/ecutils/lib/python3.10/selectors.py
  327   _watch_pipe_fd         /home/vtec/miniconda3/envs/ecutils/lib/python3.10/site-packages/ipykernel/iostream.py
  327   _watch_pipe_fd         /home/vtec/miniconda3/envs/ecutils/lib/python3.10/site-packages/ipykernel/iostream.py
  103   run                    /home/vtec/miniconda3/envs/ecutils/lib/python3.10/site-packages/ipykernel/heartbeat.py
  482   select                 /home/vtec/miniconda3/envs/ecutils/lib/python3.10/selectors.py
    6   <cell line: 6>         /tmp/ipykernel_7075/1808055364.py                 
frame = frames[framesids[-1]]

print(frame.f_code.co_names)
print(frame.f_code.co_stacksize)
print(frame.f_code.co_varnames)

print(inspect.getsource(frame))
('sys', '_current_frames', 'frames')
2
()
def fn(a):
    return a + 1

Tracing classes

source

StackTrace

 StackTrace (with_call:bool=True, with_return:bool=True,
             with_exception:bool=True, max_depth:int=-1)

Callable class acting as tracefunc to capture and print information on all stack frame being run

Type Default Details
with_call bool True when True, call events are traced
with_return bool True when True, return events are traced
with_exception bool True when True, exceptions events are traced
max_depth int -1 maximum depth of the trace, default is full depth

source

StackTrace.__call__

 StackTrace.__call__ (frame:inspect.FrameInfo, event:str, arg:Any)

tracefuncused in sys.settrace(tracefunc)

Type Details
frame inspect.FrameInfo frame argument in tracefunc
event str event argument in tracefunc
arg Any arg argument in tracefunc

source

StackTrace.print_stack_info

 StackTrace.print_stack_info (co_filename:str|pathlib.Path, ret:bool,
                              depth:int)

This methods can be overloaded to customize what is printed out

Type Details
co_filename str | Path code file name
ret bool
depth int depth

source

StackTraceJupyter

 StackTraceJupyter (with_call:bool=True, with_return:bool=True,
                    with_exception:bool=True, max_depth:int=-1)

Print stack frame information in Jupyter notebook context (filters out jupyter overhead)

Type Default Details
with_call bool True when True, call events are traced
with_return bool True when True, return events are traced
with_exception bool True when True, exceptions events are traced
max_depth int -1 maximum depth of the trace, default is full depth

Tracing decorators

source

stack_trace

 stack_trace (**kw)

stack_trace decorator function

source

stack_trace_jupyter

 stack_trace_jupyter (**kw)

stack_trace_jupyter decorator function

Usage:

Several functions, some of them nested and some of them with errors.

def empty_func():
        pass

def call_empty_and_return_zero():
    empty_func()
    return 0

def divide_by_zero_error():
    1/0

def decrement_recursion(i):
    if i == 0:
        return
    decrement_recursion(i-1)

Using the @stack_trace or @stack_trace_jupyter decorator allows a detailled trace, function by function and identify where it fails.

@stack_trace(with_return=True, with_exception=True, max_depth=10)
def function_to_trace():
    call_empty_and_return_zero()
    decrement_recursion(5)
    divide_by_zero_error()
test_fail(
    function_to_trace,
    msg='Should raise a div by 0 exception',
    contains='division by zero'
)
function_to_trace   [call]  in /tmp/ipykernel_7075/661382525.py line:1
  call_empty_and_return_zero    [call]  in /tmp/ipykernel_7075/2653264264.py line:4
    empty_func  [call]  in /tmp/ipykernel_7075/2653264264.py line:1
    empty_func  [return]    None    in /tmp/ipykernel_7075/2653264264.py line:2
  call_empty_and_return_zero    [return]    0   in /tmp/ipykernel_7075/2653264264.py line:6
  decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
    decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
      decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
        decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
          decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
            decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
            decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:13
          decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
        decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
      decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
    decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
  decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
  divide_by_zero_error  [call]  in /tmp/ipykernel_7075/2653264264.py line:8
  divide_by_zero_error  [exception] <class 'ZeroDivisionError'> in /tmp/ipykernel_7075/2653264264.py line:9
  divide_by_zero_error  [return]    None    in /tmp/ipykernel_7075/2653264264.py line:9
function_to_trace   [exception] <class 'ZeroDivisionError'> in /tmp/ipykernel_7075/661382525.py line:5
function_to_trace   [return]    None    in /tmp/ipykernel_7075/661382525.py line:5
@stack_trace_jupyter(with_return=True, with_exception=True, max_depth=15)
def function_to_trace_jupyter():
    call_empty_and_return_zero()
    decrement_recursion(5)
    divide_by_zero_error()
test_fail(
    function_to_trace_jupyter,
    msg='Should raise a div by 0 exception',
    contains='division by zero'
)
function_to_trace_jupyter   [call]  in /tmp/ipykernel_7075/2571630784.py line:1
  call_empty_and_return_zero    [call]  in /tmp/ipykernel_7075/2653264264.py line:4
    empty_func  [call]  in /tmp/ipykernel_7075/2653264264.py line:1
    empty_func  [return]    None    in /tmp/ipykernel_7075/2653264264.py line:2
  call_empty_and_return_zero    [return]    0   in /tmp/ipykernel_7075/2653264264.py line:6
  decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
    decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
      decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
        decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
          decrement_recursion   [call]  in /tmp/ipykernel_7075/2653264264.py line:11
            decrement_recursion [call]  in /tmp/ipykernel_7075/2653264264.py line:11
            decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:13
          decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
        decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
      decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
    decrement_recursion [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
  decrement_recursion   [return]    None    in /tmp/ipykernel_7075/2653264264.py line:14
  divide_by_zero_error  [call]  in /tmp/ipykernel_7075/2653264264.py line:8
  divide_by_zero_error  [exception] <class 'ZeroDivisionError'> in /tmp/ipykernel_7075/2653264264.py line:9
  divide_by_zero_error  [return]    None    in /tmp/ipykernel_7075/2653264264.py line:9
function_to_trace_jupyter   [exception] <class 'ZeroDivisionError'> in /tmp/ipykernel_7075/2571630784.py line:5
function_to_trace_jupyter   [return]    None    in /tmp/ipykernel_7075/2571630784.py line:5