AWS S3 Object Access and Editing

Question

Let's say I wanted to store a hashset object into a bucket on AWS S3. Can I frequently update that object(adding new strings to the hashset) with ease, or would I have to actually pull the hashset out of the bucket before I can make changes to it?

Answer 1

S3 is a key -> (Object+Metadata) store. Recently, (Dec 2020), S3 has gained strong consistency on update, delete, list and get. Only buckets have eventual consistency constrain. This means you should be able to store serialized objects on S3 with ease and use basic CRUD operation to read, update, delete and list stored objects.

So, yes, in theory you will be able to store a hashset on S3. You can possibly map individual value-hashes of the set to s3 key suffixes and store values in S3 directly. You will not have to load the entire hashset into memory as you can map most of the operations directly to S3.

1. HashSet::Put -> S3::PutObject. O(1)
2. HashSet::isPresent -> S3::GetObjectMetaData. O(1)
3. HashSet::Remove -> S3::DeleteObject. O(1)
4. HashSet::ListAllObjects -> S3::ListObjects. O(n)

You might be able to implement union, intersection operations as well. Though, I am not sure how efficient it might be compared to usual implementations.