How to follow the method execution in python

Question

I am trying to read some source code from some open source python project, like ipython. I often find it hard to follow the execution flow of methods in different classes, even using eclipse's debug tool and execute the code step by step. I don't quite know why the code jumps to certain methods in a distantly related class.

I know that it must be the inheritance hierarchy, but I find it hard to follow. Is there any tools that help to understand how the code executes? Like visualizing the execution order of different methods? Hope it's not a entirely naive question.

Thanks.

Answer 1

The jumps are caused by your functions calling another function which in tern calls another function.

Answer 2

MIT website has a program that traces executions and displays them in picture that you may find useful: http://people.csail.mit.edu/pgbovine/python/tutor.html#mode=edit

Answer 3

I was in the same boat as you. In the end I created my own thingy. Namely there are many built-in ways you can use for this but it takes some effort. One such way is setting a tracer - some sort of analyzer for the code while it executes.

Source:

import sys, inspect

class Tracer(object):

    def __init__(self):
        self.tracing_packages = []
        self.whitespace = '    '
        self.indent_lvl = 0

    def trace(self, frame, event, arg):
        # Module info
        mod = inspect.getmodule(frame)
        if mod:
            modpath = mod.__name__
        else:
            modpath = '<no module>'
        # Just return if not interested in package
        for to_trace in self.tracing_packages:
            if not modpath.startswith(to_trace):
                return self.trace
        # Other info
        fn_name = frame.f_code.co_name
        src_lines = inspect.getsource(frame).split('\n')
        src_line_start = src_lines[0]
        src_line_end = src_lines[-1]
        lineno = frame.f_lineno
        ws = self.whitespace
        # Printing
        if event == 'call':
            self.indent_lvl += 1
            print('%scallin: %s %s %s' % (self.indent_lvl*ws, modpath, fn_name, str(arg)))
        elif event == 'return':
            if isinstance(arg, object):
                ret = type(arg)
            else:
                ret = str(arg)
            print('%sreturn: %s' % (self.indent_lvl*ws, ret))
            self.indent_lvl -= 1
        return self.trace

    def watch_package(self, packname):
        self.tracing_packages.append(packname)

Usage

For your case you just type:

tracer = Tracer()
tracer.watch_package('IPython')
sys.settrace(tracer.trace)

Then if you try to run a function you will get printed the whole calling chain including any other functions from the same package being called.

Other example

I use this when I want to understand the flow of specific functions or of the whole package. If you want an overhead view of the whole package you can also use pyreverse from pylint in order to create UML diagrams.

Anyhow here is an example of tracing the PyOCD package:

>>> import sys, inspect, pyOCD
>>> tracer = Tracer()
>>> tracer.watch_package('pyOCD')
>>> sys.settrace(tracer.trace)
>>> pyOCD.board.MbedBoard.listConnectedBoards()
    callin: pyOCD.board.mbed_board listConnectedBoards None
        callin: pyOCD.board.mbed_board getAllConnectedBoards None
            callin: pyOCD.interface.pyusb_backend getAllConnectedInterface
                callin: pyOCD.interface.pyusb_backend __init__ None
                    callin: pyOCD.interface.interface __init__ None
                    return: <type 'NoneType'>
                return: <type 'NoneType'>
                callin: pyOCD.interface.pyusb_backend start_rx None
                return: <type 'NoneType'>
            return: <type 'list'>

            ...