Catching Bosch BSEC Output from BME680

Question

Hello everyone!

What I did

I am using the Bosch BSEC Library to get IAQ Data from an I2C attached BME680, by running it as a subprocess in python:

proc = subprocess.Popen(['path_to_bsec_bme680'],stdout=subprocess.PIPE, stderr=subprocess.STDOUT,shell=False)

In order not to get stuck on this forever-running process, I set the file properties to non-blocking:

flags = fcntl(proc.stdout, F_GETFL)
fcntl(proc.stdout, F_SETFL, flags | O_NONBLOCK)

Then I am checking the output at regular intervalls to process whatever the subprocess might have written to stdout:

output = proc.stdout.readline().decode()

This works reliably as intended.

The Problem and my suspicion

The BSEC program does some internal calibration and saves its status to a binary file called bsec_iaq.state. I suspect the way I am running it, I am somehow preventing the process to ever write to that file. The calibration cycle is quite long (4 Days) and the file usually gets written to about every 8 hours so debugging what actually happens is a bit of a hassle.

A bad workaround

After checking everything trivial like correct filepaths and so on, I decided to simply run the process twice, once from my python program and once manually. I figured the subprocess should at least be able to read the state file. To my suprise, this worked, the statefile is now updated regularily and the calibration is stable.
This is obviously not a good idea and after some days, I noticed the sensor would start to twitch a bit every now and then (abrupt jumps in the different values). I suppose this is a result of running two instances of bsec at the same time.

Am I right about my assumption that I am blocking the subprocess from writing to the state file?
How could I catch the output and still allow it to write its state? Did I go entirely wrong?

Answer 1

Hello everyone!

I solved it myself, maybe my solution is useful to anyone out there, repeating mistakes I have made previously.

The bsec_iaq.state is expected to be somewhere relative to the location of the compiled binary bsec program.

When this program is called by subprocess.Popen, the current working directory is set to the location of the python program it was called from.

The pathes do not match, the bsec_iaq.state file cannot be found the calibration subsequently fails.

I could have either changed the filepaths in the compiled bsec binary or set a different working directory for the subprocess. I decided for the latter and added the parameter cwd.

proc = subprocess.Popen(['path_to_bsec_bme680'],cwd='path_to_bsec_working_dir',stdout=subprocess.PIPE, stderr=subprocess.STDOUT,shell=False)