Is there some standard or well supported error reporting/log tool for Scons?

Question

Is there some kind of tool that exists to manage and report tasks failures generated by Scons?

I know it's not exactly the intended use case for Scons, but in my team it is used to do things like test run Jupyter notebooks in a CI, and it would be nice to have a better way to analyse the output when things go wrong. I can, for example, tell Scons to keep going regardless of errors using -k, which lets me run all the notebook tests even if some failures are encountered, but then it is a headache to dig through the logs and figure out which notebooks had what problems. I'd really like some nice report at the end that can at least quickly tell me which notebooks failed, then I can dig through the full logs for more details.

I tried to convince the team to just run these "tests" via pytest, which would then automatically do this kind of thing very nicely (and which we already use for more normal unit tests and such), but they were kind of stubborn and said "why do we need pytest? Scons already runs the notebooks for us." I can probably push back harder on this, but I might succeed better by just improving what Scons does. I think there are some things I can do to capture certain errors from Scons into a log and then cat that at the end of the CI run or something, but I'm also just wondering if there are some pre-built tools to do things like this, maybe even integrated into Scons already? I'm not a Scons expert and haven't been using it long so I'm not too sure what exists in this ecosystem. But I guess Scons must internally have some idea what scheduled tasks have failed, so maybe it can be convinced to just output this information somehow?

Any help much appreciated!

Answer 1

So I'm finding that the GetBuildFailures example in the scons docs is not bad for this:

https://scons.org/doc/production/HTML/scons-user.html#idp105549032384328

I'm running the "more complete" example there (with some simple Python 2 -> 3 fixes...)

    def bf_to_str(bf):
        """Convert an element of GetBuildFailures() to a string
        in a useful way."""
        import SCons.Errors
        if bf is None: # unknown targets product None in list
            return '(unknown tgt)'
        elif isinstance(bf, SCons.Errors.StopError):
            return str(bf)
        elif bf.node:
            return str(bf.node) + ': ' + bf.errstr
        elif bf.filename:
            return bf.filename + ': ' + bf.errstr
        return 'unknown failure: ' + bf.errstr
    import atexit

    def build_status():
        """Convert the build status to a 2-tuple, (status, msg)."""
        from SCons.Script import GetBuildFailures
        bf = GetBuildFailures()
        if bf:
            # bf is normally a list of build failures; if an element is None,
            # it's because of a target that scons doesn't know anything about.
            status = 'failed'
            failures_message = "\n".join(["Failed building %s" % bf_to_str(x)
                               for x in bf if x is not None])
        else:
            # if bf is None, the build completed successfully.
            status = 'ok'
            failures_message = ''
        return (status, failures_message)

    def display_build_status():
        """Display the build status.  Called by atexit.
        Here you could do all kinds of complicated things."""
        status, failures_message = build_status()
        if status == 'failed':
           print "FAILED!!!!"  # could display alert, ring bell, etc.
        elif status == 'ok':
           print "Build succeeded."
        print failures_message

    atexit.register(display_build_status)

And this gets me some kind of reasonable output at the end of the build:

Failed building build/python/notebooks/<stuff>.log-nb: Error 1
Failed building build/python/notebooks/<stuff2>.log-nb: Error 1
Failed building build/python/notebooks/<stuff2>.log-nb: Error 1
...
etc.

There is probably something fancier that can be built like this, perhaps customised better for these particular tasks, but this is a decent start.