Riak CS how to get list of all buckets and all files

Question

I was surprised to not find in documentation the answer on really simple questions: How can I get the list of all buckets and all files in my Riak CS installation? Also I need to have access to all those files in all buckets with some "root" account - how can I do that?

Anyway according to documentation: http://docs.basho.com/riakcs/latest/cookbooks/Account-Management/ A sample URL for a user listing request looks like this:

GET http://data.example.com/riak-cs/users -
or
s3curl --id admin -- http://data.mystorage.me/riak-cs/users

So, I've edited .s3crul:

%awsSecretAccessKeys = (
adminname => {
id => 'key here',
key => 'secret here',
},
);

and point s3curl directly to riak cs:

s3curl.pl --id adminname -- http://192.168.0.20:8080/riak-cs/users

Getting answer:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message><Resource>/riak-cs/users</Resource><RequestId></RequestId></Error>

I'm sure that "adminname" is Admin account in Riak CS, I can see it in riac-cs-control panel.

Answer 1

I seems that you use custom host base, so what you have to do is to tell it to s3curl.pl. As you know, because S3 (or AWS v2) authentication can be path style or subdomain style, s3curl.pl can not use host part of URL or host header as is. Therefore, it has some hardcoded endpoints [1] for indentifing host base.

Some forks of s3curl.pl can specify the custom endpoint by option argument [2]. If you setting cs_root_host as data.mystorage.me, then the example command line looks like:

s3curl --id admin --endpoint data.mystorage.me \
    -- http://data.mystorage.me/riak-cs/users

or by subdomain style,

s3curl --id admin --endpoint data.mystorage.me \
    -- http://riak-cs.data.mystorage.me/users

Once you get all user information, you can do anything, e.g listing buckets for certain user or listing all objects in certain bucket.

[1] https://github.com/ksauzz/s3curl/blob/master/s3curl.pl#L30-L34 [2] https://github.com/ksauzz/s3curl/blob/master/s3curl.pl#L121

Answer 2

For the listing part, try s3cmd la -r s3://

I don't think there is an account that has access to everything by default, you would have to add the account to each bucket's ACL