S3 radosgw-admin operations

radosgw-admin is used to manage users, quotas, buckets, indexes, and all other aspects of the radosgw service.

Create a user

End-users get S3 quota from OpenStack (see Object Storage).

In special cases (e.g., Atlas Event Index, CVMFS Stratum 0s, GitLab, Indico, ...), we create users that exist only in Ceph and are not managed by OpenStack. To create a new user of this kind, it is needed to know user_id, email address, display name, quota (optional).

Create the user with:

radosgw-admin user create --uid=<user_id> --email=<email_address> --display-name=<display_name>

To set a quota for the user:

radosgw-admin quota set --quota-scope=user --uid=<user_id> --max-size=<quota>
radosgw-admin quota enable --quota-scope=user --uid=<user_id>

Example:

radosgw-admin user create --uid=myuser --email="myuser@cern.ch" --display-name="myuser"
radosgw-admin quota set --quota-scope=user --uid=myuser --max-size=500G
radosgw-admin quota enable --quota-scope=user --uid=myuser

Change user quota

It is sufficient to set the updated quota value for the user:

radosgw-admin quota set --quota-scope=user --uid=<user_id> --max-size=<quota>

Bucket resharding

RGW shards bucket indices over several objects. The default number of shards per index is 32 in our clusters. It is best practice to keep the number of objects per shard below 100000. You can check the compliance across all buckets with radosgw-admin bucket limit check.

If there is a bucket with "fill_status": "OVER 100.000000%" then it should be resharded. E.g.

> radosgw-admin bucket reshard --bucket=lhcbdev-test --num-shards=128
tenant: 
bucket name: lhcbdev-test
old bucket instance id: 61c59385-085d-4caa-9070-63a3868dccb6.24333603.1
new bucket instance id: 61c59385-085d-4caa-9070-63a3868dccb6.76824996.1
total entries: 1000 2000 ... 8599000 8599603
2019-06-17 09:27:47.718979 7f2b7665adc0  1 execute INFO: reshard of bucket "lhcbdev-test" from "lhcbdev-test:61c59385-085d-4caa-9070-63a3868dccb6.24333603.1" to "lhcbdev-test:61c59385-085d-4caa-9070-63a3868dccb6.76824996.1" completed successfully

SWIFT protocol for quota information

It is convenient to use the SWIFT protocol to retrieve quota information.

  1. Create the SWIFT user as a subuser:
radosgw-admin subuser create --uid=<user_id> --subuser=<user_id>:swift --access=full

This generates a secret key that can be used on the client side to authenticate with SWIFT.

  1. On clients, install the swift package (provided in the OpenStack Repo on linuxsoft) and retrieve quota information with
swift \
    -V 1 \
    -A https://s3.cern.ch/auth/v1.0 \
    -U <user_id>:swift \
    -K <secret_key> \
    stat
Improve me !