Is there a way to list all the objects in a bucket non-recursively in libcloud

Question

I have a basic container that may point to s3 or any other storage service - and I'm trying to list the objects in a certain folder - but I can't find a way to do it non-recursively - all I can do is to pass it a prefix and ex_prefix (which is deprecated) but not a suffix/delimiter/depth - it would always list all the children's objects as well.

Is there a way to pass it a suffix/delimiter/depth?

Answer 1

According to the Apache libcloud documentation there is no way to accomplish that using only libcloud. There is however a pythonic way to do this using list comprehensions. Let's print the entire content of the bucket:

driver.list_container_objects(bucket)

We get:

    `[<Object: name=A_dirty_chubby_anthropomorphic_badger, size=410938, hash=59643edb7beb48e8904d6557ab4d3d8e, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=A_dirty_chubby_anthropomorphic_badger2, size=379704, hash=6e7f3f1608638be56cecfbd66c1ac902, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=LFS-BOOK-7.9.pdf, size=1736489, hash=e6802cdbec8f4b50d43c3d9cd710bfc2, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=badgers/A_dirty_chubby_anthropomorphic_badger, size=410938, hash=59643edb7beb48e8904d6557ab4d3d8e, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=badgers/A_dirty_chubby_anthropomorphic_badger2, size=379704, hash=6e7f3f1608638be56cecfbd66c1ac902, provider=Amazon S3 (eu-central-1) ...>]`

If we want to limit by depth we can do:

[obj for obj in driver.list_container_objects(bucket) if (obj.name.count('/') < 1)]

Where we will get:

    `[<Object: name=A_dirty_chubby_anthropomorphic_badger, size=410938, hash=59643edb7beb48e8904d6557ab4d3d8e, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=A_dirty_chubby_anthropomorphic_badger2, size=379704, hash=6e7f3f1608638be56cecfbd66c1ac902, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=LFS-BOOK-7.9.pdf, size=1736489, hash=e6802cdbec8f4b50d43c3d9cd710bfc2, provider=Amazon S3 (eu-central-1) ...>]

And if we want to filter by suffix we can just do:

[obj for obj in driver.list_container_objects(bucket) if (obj.name.endswith("badger"))]

Where we end up getting:

    `[<Object: name=A_dirty_chubby_anthropomorphic_badger, size=410938, hash=59643edb7beb48e8904d6557ab4d3d8e, provider=Amazon S3 (eu-central-1) ...>,
<Object: name=badgers/A_dirty_chubby_anthropomorphic_badger, size=410938, hash=59643edb7beb48e8904d6557ab4d3d8e, provider=Amazon S3 (eu-central-1) ...>]`