Capture output of autograded answer for subsequent autograder tests in nbgrader

Question

I am trying to automate autograding with nbgrader. Usually the student writes a piece of code (like a variable or a function) in the autograded answer cell that the instructor uses (by referring to the variable or function) to write autograder tests.

But some times the input code from the autograded answer cell can be just a print(...) which outputs something to the screen (rather than a variable or function). In that case how is it possible to capture the printed output so that we can use it the next cell for writing autograder tests on it ?

Answer 1

The nbgrader docs include an example demonstrating how to patch the builtin function print so you can capture and test printed output:

Problem: verify that a function call results in the printing of a certain result

def foo()
    #...
    print('right result')

The test code for this can be written as

from unittest.mock import patch
with patch('__main__.print') as mock_print:
    foo()

mock_print.assert_called_once_with('right_result')

This test passes silently if the print is correct, but if the print is wrong:

def foo()
    #...
    print('wrong result')

an assertion error is raised with output of the form

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
...

AssertionError: Expected call: print('right result')
Actual call: print('wrong result')