Is it possible to check for a given python file whether it's pyc file is valid and up to date?

Question

For a project which gets deployed to a RO filesystem I want to also deploy pre-compiled .pyc files for all of the Python source.

Thanks to the compileall module this already works, but due to whatever reasons caused by a complex build machinery, those .pyc files can become invalid accidentally (because of unwanted file modifications, source relocation, etc)

So what I'd like to do is to check if there's an already existing pyc file for each with would have been created, if it's being found and if it's valid (i.e. correct source path embedded, correct bytecode, etc).

Is there an integrated way to check "precompilation validity" (like python3 -m compileall --check)? Or what would be a pragmatic way to do this by hand? (e.g. by inspecting some trace or whatever).

The most reliable way I can imagine would be to compare a readily deployed tree to a copy where I deleted and re-created all files using compileall..

Answer 1

You can look at bytecode metadata using that disassembler called pydisasm that comes with xdis using option --format (or -F) option with header:

$ pydisasm --format header __pycache__/five.cpython-38.pyc 
# Python bytecode 3.8.0 (3413)
# Disassembled from Python 3.8.18 (default, Sep  4 2023, 13:19:52) 
# [GCC 12.3.0]
# Timestamp in code: 1708217267 (2024-02-17 19:47:47)
# Source code size mod 2**32: 148 bytes
# Embedded file name: five.py

If you want to get the information from Python, the internal routine that handles the header is called show_module_header().

Be warned though that the last release of xdis on PyPI is over a year old, so use code from github. In fact, I just fixed a bug for this option that prevented more header information from showing.

I will be talking at BlackHat Asia mid April 2024, so I plan to have a new xdis release sometime a little before that.