Call to python class constructor from rust with pyo3 spawns new processes

110 Views Asked by At

I'm writing code in Rust but using the crate pyo3 to call Python libraries (specifically tensorflow and tensorboardX). However, when I call the class initializer for tensorboardX.SummaryWriter from Rust it starts a new process that restarts the whole program (starts executing the main function from the start), which in turn will try to initialize a new SummaryWriter, thus spawning an endless sequence of processes.

Here is a minimal working example :

use pyo3::prelude::*;

fn main() {
    println!("Start");
    pyo3::prepare_freethreaded_python();
    let x: PyResult<PyObject> = Python::with_gil(|py| {
        let tbx = PyModule::import(py, "tensorboardX")?;
        let x = tbx.getattr("SummaryWriter")?.call0()?;
        Ok(x.to_object(py))
    });
    let x = x.unwrap();
    println!("End ({:?})", x);
}

When I run this example it prints an infinite loop of

Start
End (Py(0x101025e50))
Start
End (Py(0x1011b9e50))
Start
End (Py(0x1008fde50))
...

and each is printed by a different process (which in turn makes it difficult to kill, I had to use the killall command to kill the process by name since each is quite short lived).

Note: It is a bit tricky to run because it needs to have the path to a Python environment that has tensorboardX installed (I had to add a .cargo/config.toml file at the root of my project with the environment variable PYTHONPATH pointing to the environment :

[env]
PYTHONPATH = "/path/to/venv/lib/python3.11/site-packages"

Anybody knows why it spawns processes instead of simply returning a SummaryWriter object ? How can I fix it ?

0

There are 0 best solutions below