Migrate ceph buckets to new user with zero downtime

Question

I have a ceph/radosgw bucket with several million objects in it, and I need to change the ownership of the bucket to another user.

Normally, this is done by linking the bucket to the new user, then chowning all of the files in it, like this:

radosgw-admin bucket unlink --uid=user1 --bucket=bigbucket
radosgw-admin bucket link --uid=user2 --bucket=bigbucket
radosgw-admin bucket chown --uid=user2 --bucket=bigbucket

Unfortunately, the chown operation has to loop over every single object in the bucket in order to update metadata. This results in an extended downtime window (sometimes 1 hour per million objects apparently) where neither the old user nor the new user can access the full contents of the bucket.

Is there any way to change bucket ownership that doesn't require downtime? Some ideas:

• Is it possible for a bucket or specific objects to be owned by two users at the same time?
• Could we create the new user, then just change their uid or some other piece of metadata that grants them access to the old user's bucket?
• Could the problem be solved client-side, or maybe with a proxy?

Answer 1

You can add a bucket policy to the bucket to get access to both users until the migration gets done by chown command:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/user1", "arn:aws:iam:::user/user2"]},
    "Action": "*",
    "Resource": [
      "arn:aws:s3:::bigbucket/*"
    ]
  }]
}