Create CEPH Cluster

Prepare the hostgroups
  • Log to Foreman and create the following hostgroups:
    • <my_cluster> and select ceph as the parent hostgroup.
    • For the monitors, create host group mon and select ceph/<my_cluster> as the parent group.
    • For the osd, create hostgroup osd and select ceph/<my_cluster> as the parent group.
    • For the metadata servers, create the hostgroup mds and select ceph/<my_cluster> as the parent.
    • Do the puppet configuration:
      • Clone the repo it-puppet-hostgroup-ceph
      • Create the manifests and data files accordingly for the new cluster (use the configuration of other cluster as a base)
      • Remember to create a new uuid for the cluster and put it in /code/hostgroup/ceph/<my_cluster>.yaml
      • Commit, push, do merge request, etc ...

First Monitor Configuration

  • 1 ) Create one virtual machine for the first monitor following this guide

  • 2 ) Create a mon bootstrap key (from any previous ceph cluster):

      ssh root@ceph<existing-cluster>-mon-XXXX
  ceph-authtool --create-keyring /tmp/keyring.mon --gen-key -n mon. --cap mon 'allow *'
    • From aiadm: (maybe you need to ask for permissions to get access to the tbag folder)
      mkdir ~/private/tbag/<my_cluster>
  cd ~/private/tbag/<my_cluster>
  scp root@ceph<existing_cluster>-mon-XXXX:/tmp/keyring.mon .
  tbag set --hg ceph/<my_cluster>/mon keyring.mon --file keyring.mon

  • 3 ) Now run puppet on the first mon.

    puppet agent -t -v

  • 4 ) Now copy the admin keyring to tbag (from aiadm):

      scp root@<first_mon>:/etc/ceph/keyring . 
  tbag set --hg ceph/<my_cluster> keyring --file keyring

  • 5 ) Now create an MGR bootstrap key on the first mon:

      ceph auth get-or-create-key client.bootstrap-mgr mon 'allow profile bootstrap-mgr'
  ceph auth get client.bootstrap-mgr > /tmp/keyring.bootstrap-mgr
    • From aiadm:
      scp root@<first_mon>:/tmp/keyring.bootstrap-mgr .
  tbag set --hg ceph/<my_cluster> keyring.bootstrap-mgr --file keyring.bootstrap-mgr

  • 6 ) Now create an OSD bootstrap key on the first mon:

       ceph auth get-or-create-key client.bootstrap-osd mon 'allow profile bootstrap-osd'
   ceph auth get client.bootstrap-osd > /tmp/keyring.bootstrap-osd
    • From aiadm:
      scp root@<first_mon>:/tmp/keyring.bootstrap-osd .
  tbag set --hg ceph/<my_cluster> keyring.bootstrap-osd --file keyring.bootstrap-osd

Add more Monitors and OSD's
  • Follow the step 1) to add more mons and osds. Everything should install correctly.

  • Prepare and activate the OSD

    /root/ceph-scripts/ceph-disk/ceph-disk-prepare-all

NOTE: To setup a OSD in the same machine as the monitor.

  • mkdir /data/a (for example)
  • chown ceph:ceph -R /data
  • ceph-disk prepare --filestore /data/a (ignore the Deprecate warnings)
  • ceph-disk activate /data/a
Improve me !