Irregular font using Matplotlib with pgf backend

Question

I am getting these irregular fonts when plotting with pgf backend.

Irregular Fonts

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(-3,3,0.1)

def f(x):
    return x**3

plt.style.use(['seaborn-v0_8-darkgrid', 'seaborn-v0_8-dark-palette'])
plt.rcParams.update({
    'font.family': 'sans-serif',
    'font.size': 12,
    'pgf.texsystem': 'lualatex',
    'pgf.preamble': r'\usepackage{unicode-math,fontspec}\setmathfont{STIX Two Math}\setsansfont{Inter}[Scale=0.88]',
    'pgf.rcfonts' : False
})

plt.plot(x,f(x))
plt.title('Cubic Function')
plt.xlabel('Input data ($x$)')
plt.ylabel('Output data ($f(x)$)')
plt.tight_layout()
plt.savefig('plot_style.pdf', backend='pgf')

The Inter font is installed at my ~/.fonts directoy and I am using Fedora 37.

Why is this happening and what should I do to get them right?

PS: if I change the pgf.texsystem parameter to pdflatex and change the pgf.preamble to \usepackage{inter} the font goes right, but I want to use lualatex texsystem.

I updated the font files already and it still goes like this. When using LaTeX files, the fonts are normal.