Given an object, how to get a list of bound methods which are called during runtime

Question

The question comes from this scenario:

We've built a framework to test northbound apis of our project, based on pytest. And now we want to have a coverage report about how many apis are tested(basically, are called) by running the test scripts we have.

We already have a well-defined client manager, which makes every api as a python method. For example, say we have an api GET /user,then we will have a method get_user which sends the actual request to server.

We are now using python-coverage (actually pytest-cov but it's the same thing) which will give us a report of code coverage, and based on that, we will have some sort of concept about how many apis are called. However, it's not accurate and clear enough.

So the key question is, is there a way to get a list of bound methods which are called during runtime.

I'd like to give an example to illustrate the question.

class Client(object):
    def __init__(self):
        pass

    def get_user(self, user):
        pass

    def post_user(self, user):
        pass

    def delete_user(self, user):
        pass

def test_get_user():
    Client().get_user("user")

def test_post_user():
    Client().post_user("user")

After running the two tests, how can I get a report saying that get_user and post_user was called and delete_user was not called during last run.

One possible solution is, since running tests with coverage.py will give us enough information about line numbers and files being executed, maybe I can analyze what I need from there.

Another possible solution is to track which method is called in Client I defined. Actually I solved my question in my particular case where all apis method will call a common method, in my case, call_api. I use inspect in method call_api, and I am able to get the method name because say get_user is always the caller of call_api. However, this solution only works for this special case and is not a general solution.

I also looked at module trace, it looks close to what I want, but unfortunately, you have to use tracer(Trace instance) to run command or function.

What I want to achieve eventually is:

tracer = Trace(obj_to_trace)
tracer.start_trace()
# using obj_to_trace 
tracer.end_trace()
tracer.get_called_methods()

Answer 1

I don't know if you can get this from coverage library itself, but you can definitely get this information by processing files yourself.

Have a look at the AST module. It allows you to parse the all the files reported by coverage and generate ranges of lines for each function. From there you just need to check which functions executed any lines at all.

But... since coverage needs to do something similar, it may be easier to just modify coverage to do report the local function instead.

Answer 2

Coverage.py does not have this feature. You'll need to use the trace module, or post-process the coverage results to get the method names.