QuintupleO Success!

Yep, that's right. I've successfully deployed a cloud in a cloud using a third cloud. I have a video, but I'm not sharing it just yet because it was done using some slightly broken pre-release overcloud images. On the plus side they had exactly the same issues as the non-QuintupleO environments so I'm declaring success. :-) As soon as I have a chance to re-record with a fully functioning overcloud I'll post it here.

Since my previous posts have been a little light on details of how someone might try doing this themselves, I thought I'd do a how-to post. Be forewarned that this is a brand spanking new thing and changing all the time. Even since I started writing this post some important details have changed, so keep that in mind if you're trying to follow the process outlined below.

Environment

I'm currently using a single-node devstack installation with the Nova and Neutron patches from my first QuintupleO update. I've updated that post with the latest diffs of my changes, but be aware that they both changed since it was originally written so they may get out of date again. The actual changes are fairly simple though, so hopefully you can figure out what to do even if the diffs don't apply cleanly.

Baremetal Instance Setup

In my current QuintupleO setup, each fake baremetal instance has a corresponding OpenStack BMC instance. This is rather like a real baremetal environment with a BMC managing a baremetal box.

Baremetal Image

First, I create an empty image to start the baremetal instances with. This can be done with the following command: qemu-img create -f qcow2 empty.qcow2 40G Then I uploaded the image to Glance and called it, creatively enough, empty. :-)

BMC Image

Before you can deploy anything, you'll need a BMC image. These can be created with my openstack-bmc diskimage-builder element. When deployed using the Heat templates below, this image will automatically configure itself to manage a specified baremetal instance. Do note that it's still hard-coded to my devstack installation, so it may need minor tweaking for other environments. Eventually I'll make it configurable.

I called this image openstack-bmc.

Networking

You can see a diagram of the networking layout I'm using in my BMC Post. public is essentially my home network where floating ips are allocated, private is the network all of the instances get attached to, and undercloud is the provisioning network where DHCP is disabled so the undercloud Neutron can handle that.

Deploying a QuintupleO Environment

I've written a couple of tools to help with setting the rest of the environment up. It's hardly a foolproof end-to-end system, but it automates a lot of the most tedious bits.

Deploying through Heat

The first major piece is a Heat template for deploying the images created above. That can be found here: OpenStack BMC Template, and used with Heat as follows: heat stack-create -f ./openstack-bmc.yaml -e ./resource-registry.yaml -P node_count=[baremetal nodes to deploy] baremetal Again, the defaults are hard-coded for my environment. There are some parameters that can be configured, but as of this writing I'm fairly certain not everything that should be configurable is. They're fairly simple templates so it shouldn't be too difficult to tweak them to your liking though.

The template now generates a separate flavor for each baremetal instance so Ironic can set the boot method on a per-instance basis. Because of this, you will need to enable the nova-flavor contrib extension in Heat: https://github.com/openstack/heat/tree/master/contrib/nova_flavor This was not available by default in my Devstack installation.

Collecting IPMI Details

Now you should have a bunch of baremetal_* and bmc_* instances running, but you still need to collect all of their IP and MAC details for Ironic so it can manage them. Luckily, there's a script for that too. It is also configurable in case you named things differently, and will dump out a file called nodes.json containing the details of your baremetal instances. This file is suitable for the os-cloud-config register-nodes command. Note that the IPMI credentials are currently hard-coded the same as the openstack-bmc image element, so if you made changes there you'll need to here as well.

Deploying to a QuintupleO Environment

My testing lately has been using instack-undercloud to install an undercloud instance. In theory devtest should be able to do essentially the same thing, but I've been neck-deep in instack-undercloud lately so that's what I used. :-)

If everything is set up correctly, you should be able to do a baremetal-style deployment to the instances deployed via Heat earlier. Note that unlike a virt-based setup in either devtest or instack-undercloud, this is using the ipmitool driver in Ironic so there's no need to copy around ssh keys for power control.

That's pretty much it for the major steps, at least as I remember them. I realize everything above is assuming a fair amount of OpenStack experience, but if you're too new to know how to set up your own Neutron networks and such you probably want to start somewhere simpler than QuintupleO anyway. ;-)

Hope this was useful and as always feel free to contact me with any questions or comments.