I'm trying to test my Luigi pipelines inside a vagrant machine using FakeS3 to simulate my S3 endpoints. For boto to be able to interact with FakeS3 the connection must be setup with the OrdinaryCallingFormat
as in:
from boto.s3.connection import S3Connection, OrdinaryCallingFormat
conn = S3Connection('XXX', 'XXX', is_secure=False,
port=4567, host='localhost',
calling_format=OrdinaryCallingFormat())
but when using Luigi this connection is buried in the s3 module. I was able to pass most of the options by modifying my luigi.cfg
and adding an s3 section as in
[s3]
host=127.0.0.1
port=4567
aws_access_key_id=XXX
aws_secret_access_key=XXXXXX
is_secure=0
but I don't know how to pass the required object for the calling_format
.
Now I'm stuck and don't know how to proceed. Options I can think of:
- Figure out how to pass the
OrdinaryCallingFormat
toS3Connection
throughluigi.cfg
- Figure out how to force
boto
to always use this calling format in my Vagrant machine, by setting an unknown option to me either in.aws/config
orboto.cfg
- Make
FakeS3
to accept the defaultcalling_format
used byboto
that happens to beSubdomainCallingFormat
(whatever it means).
Any ideas about how to fix this?
Can you not pass it into the constructor as kwargs for the S3Client?