I'm trying to pass Minio.get_object to subprocess.Popen, which calls age
crypto. Unfortunatelly, it seems the content gets somehow mangled and thus age
fails.
Can't understand what am I doing wrong. I have the following code:
with Minio(
endpoint="endpoint",
access_key="key",
secret_key="secret",
).get_object(bucket_name="bucket", object_name="enc.py") as pipeIn:
with subprocess.Popen(
["age", "--decrypt", "--identity", "./.age-ident", "--output", "-"],
stdin=pipeIn,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
) as pipeCrypto:
pipeIn.auto_close = True
pipeIn.decode_content = False
retCrypto = pipeCrypto.wait()
if retCrypto != 0:
print(pipeCrypto.stderr.read().decode())
And I get age: error: failed to read header: parsing age header: unexpected intro
and a bunch of hex
Doing things manually mc cat bucket/path | age --decrypt --identity ./.age-ident --output -
works fine.
Using Popen.communicate
also works fine:
with Minio(
endpoint="endpoint",
access_key="key",
secret_key="secret",
).get_object(bucket_name="bucket", object_name="enc.py") as pipeIn:
with subprocess.Popen(
["age", "--decrypt", "--identity", "./.age-ident", "--output", "-"],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
) as pipeCrypto:
pipeIn.auto_close = True
pipeIn.decode_content = False
cryptoOut, cryptoErr = pipeCrypto.communicate(pipeIn.read())
print(cryptoOut.decode())
HTTPResponse
has read
method so it should be OK to pass it as stdin
to Popen
or so I though, unfortunately it doesn't seem to work.
P.S. I don't want to use communicate
because
- I'll have to store content in memory and the file can be quite large
- I still need to pass output from
age
down to two more pipes, and so, those pipes need to use communicate too