DevelopmentEnvironment

Updated 2020-08-10, see edit below

I recently had to upgrade a CentOS 7 server to CentOS 8. I foolishly expected this to be easy since it wasn't a heavily customized install in the first place and most of the stuff I cared about was in my home directory. I was, as you may have guessed, wrong.

The problem was that CentOS 8 removed the driver for the Dell H700 RAID controller in my server. Setting aside the wisdom of that decision, here's how to get around it:

I have complained extensively over the past couple of years about the over-automation of developer environments in TripleO. But wait, you say, isn't automation a good thing? And yes, it is, but the automation needs to happen in the right places (feel free to append "IMHO" to anything in this post ;-). The problem with a bunch of developer-specific automation is that it hides very real user experience problems because the developers just use their simplified interface and never touch the regular user interface. Essentially it's the opposite of dogfooding, which is something I feel is critical to writing good software. This is also known as the "Works in Devstack" problem for OpenStack as a whole (I will not be tackling that problem here though).

So if I don't like what most people are doing today, what would I prefer? That will be the topic of this post. I'll discuss what I do, and possible areas for improvement.

There are times where it can be handy to test an undercloud install in a standalone VM. For example, if you're working on installing a new service, or upgrading the undercloud, many times you can make progress in a lighter weight single VM environment. Obviously the undercloud won't be usable for anything if it's not part of a full virt environment, but you can at least check that a service is running at the end of the install and do some basic sanity checks.

I did a post in the past about using caching and local mirrors to speed up TripleO development, but at this point TripleO has changed so much that pretty much everything in that post is completely outdated, so I thought it was time for another one about my current caching setup.

Configs updated 2017/2/28 to reflect what I'm using now.

As part of my continuing investigation into QuintupleO, I've been playing around with multi-node devstack to see how QuintupleO works across multiple compute hosts. The good news is that the answer is "quite well" (another blog post on that topic is probably needed as well), and since some of the devstack multi-host documentation I found through Google was a bit out of date, I thought I'd go ahead and post what I've been using for my devstack setup.

Edit: Updated 2015/3/19 with more current diffs of my changes.

At the Atlanta OpenStack Summit we had a session on something called QuintupleO, otherwise known as "TripleO wasn't confusing enough, let's add another layer" :-) Barring a few specific concerns from other teams, which I believe have now been addressed to their satisfaction, everyone seemed to be on board with the idea. But what exactly is QuintupleO, and where does it stand today? Read on to find out.

As I mentioned in a previous post, about a year ago I picked up a 1U server from EBay to use as a local single-node OpenStack environment. In general I was quite happy with it, but at some point I got tired of paying for electricity to run a fairly power-hungry server that sits idle, or close to it, about 95% of the time. The fans also picked up an annoying whine somewhere along the line, so once I discovered how much more efficient a modern desktop processor would be, while actually performing better than the old dual server ones, I decided it was time for a new box. This post will be the story of my journey to get Tempest running in an acceptable fashion on it, and what I learned along the way.

Most OpenStack projects use a combination of tox, testr, and subunit to run their unit tests. This works fine as long as you don't have any simple errors in your code, such as a typo in an import or a missing closing paren. In those cases testr or subunit may swallow the error and you won't know exactly where it is (I hear this may be fixed in some version of the tools, but the one I'm currently using still has this limitation). To fix this, we need another way to run unit tests that will expose the actual error to us.

One of the unintended consequences of Feature Freeze in OpenStack is that a lot of new features get merged in a hurry at the end of the cycle. That's obviously not ideal, but it's a reality we all have to live with for now. TripleO has been bitten by this a few times recently, and the long gate queue that also goes along with Feature Freeze means sometimes fixes for problems take a long time (days) to merge in the other projects. In the gate we have tempreverts and cherry-picks to mitigate that problem, but for local devtest those features are not available. Fortunately there's another solution.

I said in a previous post that I would write something up about my devtest workflow once I had it nailed down a bit more, and although it's an always-evolving thing, I think I've got a pretty good setup at this point so I figured I'd go ahead and write it up. Instead of including scripts and such here, I'm just going to link to my Github repo where all of this stuff is stored and reference those scripts in this post.