matplotlib pgf backend – adjusting preamble

Question

Consider the following piece of code:

import matplotlib as mpl
mpl.use("pgf")
preamble = r'\usepackage{newtxtext,newtxmath}'
mpl.rcParams.update({
    "pgf.texsystem": 'lualatex',
    "pgf.preamble": preamble,
    'text.latex.preamble': preamble,
    'text.usetex': True,
    'font.size': 9,
    'lines.markersize' : 3
})

import matplotlib.pyplot as plt
fig = plt.figure(figsize=(4.5, 2.5))
ax = fig.gca()
ax.plot(range(5))
ax.set_xlabel(u"µ is not $\\mu$")
fig.tight_layout()
fig.savefig("test.pdf")

The code fails to run with the lualatex error:

Package fontspec Error: The font "DejaVuSerif" cannot be found.

This is odd, I did neither load fontspec nor did I specify DejaVuSerif as the font to use in my preamble. If I issue "pgf.texsystem": 'pdflatex' instead, the code produces a pdf with the expected output. Compiling a tex document with just that preamble and some random text and math with lualatex also gives the expected output. This is why I think the pgf backend of matplotlib messes something up here.