Tempo HA docs - How to use ceph-backed s3 storage for HA charms
Say you need, or you already have, a ceph deployment (for example as part of an openstack cloud), and you want to use it to give an S3 bucket to an HA charm such as Tempo, Mimir or Loki HA.
What you will need:
- a ceph deployment
- a Juju kubernetes model
If you do not have access to a ceph cluster already, you can deploy one by following this guide.
Provision a bucket on openstack-managed ceph
You’ll need access to the ceph-mon application on the ceph cluster (openstack model). Run:
juju ssh ceph-mon/0 'sudo radosgw-admin user create --uid="ubuntu" --display-name="Charmed Ceph"'
This will return an access key and a secret key. Note them down as you’ll need them soon.
There are various ways in which you can create a bucket. We found using the minio-mc-nsg snap to be straightforward.
On the host (or anywhere you have access to the rados gateway), run:
sudo snap install minio-mc-nsg
RADOSGW_VIP=$(juju config ceph-radosgw vip)
minio-mc-nsg config host add ceph-radosgw https://$RADOSGW_VIP:443 <access_key> <secret_key> --insecure
minio-mc-nsg mb ceph-radosgw/tempo --insecure
Deploy and configure the s3 integrator
We usually deploy the s3-integrator charm with the s3 alias.
juju deploy s3-integrator s3 --channel edge --config region=PartnerCloud6a --config endpoint=$2 --config bucket=tempo
juju run -m s3/0 sync-s3-credentials access-key=<access_key> secret-key=<secret_key>
It’s unclear at the moment whether the ‘region’ config option has any effect/meaning. The Tempo HA process seems to error out if the s3 integrator does not provide it, but its value seems to be irrelevant.
Using TLS
If ceph is behind a CA, you will need to obtain the certificate so that the charms using the bucket will be able to use HTTPS. If you forget this step, you will get some errors because the charmed application will be attempting to talk HTTP to an HTTPS endpoint.
On openstack, ceph obtains its credentials from the vault charm. To obtain the root ca you can run:
juju run vault/leader get-root-ca
put the root ca in a file called `vault-root.ca`.
And to load it into the s3 integrator:
VAULT_CA=$(cat /path/to/vault.root-ca | base64)
juju config s3 tls-ca-chain="$VAULT_CA"
The s3 integrator will then forward it to the HA charm over the s3 relation.
Last updated 2 months ago.