OpenStack Development System Setup

I've been meaning to write something up about how I set up my OpenStack development environment because there are some handy things you can do before you even get started that will likely save you some hassle down the line. I've recently discovered that there are rather varied ways other people are doing this sort of thing, so keep in mind that this isn't the "right" way, it isn't the best way, but it's what works for me.

The first thing I do on every new system (or VM) is apply the Puppet manifest below. Hopefully I've remembered to remove my ssh keys before posting it here (Note: I realize there are almost certainly better ways to handle that, but I'm a Puppet newbie and this works fine :-). I used Puppet for this mostly because that's what Red Hat seems to have based most of their installation tools around so it made sense for me to learn a little more about it. Hopefully even if you don't know Puppet you can figure out the basics of what it is doing.

Once the Puppet manifest has been applied it's time to get started on the OpenStack-specific stuff. For the moment I'm going to focus on devstack because my TripleO devtest workflow isn't quite as well-defined at this point. Maybe that would be a topic for a future post. ;-)

On the shiny new system do:

$ git clone https://git.openstack.org/openstack-dev/devstack

cd into the newly-created devstack directory and create a localrc that looks something like the one below. It includes some options that I've found helpful when working with devstack and it's nice to not have to go look them up when I want to change something.

Now you should be ready to run

$ ./stack.sh

and when it finishes you should have a fully functional devstack installation running. In order to use it, you'll need to source an rc file. This is an area I need to make some updates because the hard-coded rc file I had been using no longer works as well. I've included it below for completeness, but you will probably want to replace most of it with a source of the devstack openrc file that should be included in your local devstack git repo. I stick a call to the rc file in my .bashrc so I don't have to remember to do it.

$ echo "source ~/keystonerc" >> ~/.bashrc

Now if you source it

$ source ~/keystonerc

and run something like

$ keystone user-list

you should get a list of users that devstack configured for you.

You're now ready to start hacking on OpenStack (though see this post for a discussion of why you maybe shouldn't start messing around directly in /opt/stack). For reference, I do most of these steps in a VM that I then snapshot and use as a template for my future instances. That means I can basically boot an instance of the template, update the devstack git repo (just a "git pull" in ~/devstack) and run stack.sh out of the box and have a sane environment running.

Questions, corrections, additions? Please feel free to leave a comment below and I'll try to update the post. Thanks.

Puppet manifest

define line($file, $line, $ensure = 'present') {
    case $ensure {
        default : { err ( "unknown ensure value ${ensure}" ) }
        present: {
            exec { "/bin/echo '${line}' >> '${file}'":
                unless => "/bin/grep -qFx '${line}' '${file}'"
            }
        }
        absent: {
            exec { "/usr/bin/perl -ni -e 'print unless /^\\Q${line}\\E\$/' '${file}'":
                onlyif => "/bin/grep -qFx '${line}' '${file}'"
            }
        }
    }
}

user {'bnemec':
   ensure => present,
   managehome => true,
}

file {'bnemec':
   ensure => directory,
   path => '/home/bnemec',
   owner => 'bnemec',
   group => 'bnemec',
   require => User['bnemec'],
}

file {'.ssh':
   path => '/home/bnemec/.ssh',
   ensure => directory,
   mode => 0600,
   owner => 'bnemec',
   group => 'bnemec',
   require => File['bnemec'],
}

file {'authorized_keys':
   path => '/home/bnemec/.ssh/authorized_keys',
   ensure => present,
   mode => 0600,
   owner => 'bnemec',
   group => 'bnemec',
   content => "[put your public key here]",
   require => File['.ssh'],
}

file {'id_rsa.pub':
   path => '/home/bnemec/.ssh/id_rsa.pub',
   ensure => present,
   mode => 0600,
   owner => 'bnemec',
   group => 'bnemec',
   content => "[put your public key here]",
   require => File['.ssh'],
}

file {'id_rsa':
   path => '/home/bnemec/.ssh/id_rsa',
   ensure => present,
   mode => 0600,
   owner => 'bnemec',
   group => 'bnemec',
   content => "-----BEGIN RSA PRIVATE KEY-----
[Private key stuff goes here]
-----END RSA PRIVATE KEY-----",
   require => File['.ssh'],
}

file {'sudoers':
   path => '/etc/sudoers.d/bnemec',
   ensure => present,
   mode => 0440,
   content => "bnemec ALL=(root) NOPASSWD:ALL",
}

file {'.pip':
   path => '/home/bnemec/.pip',
   ensure => directory,
   mode => 0700,
   owner => 'bnemec',
   group => 'bnemec',
   require => File['bnemec'],
}

file {'pip.conf':
   path => '/home/bnemec/.pip/pip.conf',
   ensure => present,
   mode => 0700,
   owner => 'bnemec',
   group => 'bnemec',
   content => "[global]
index-url = http://pypi.openstack.org/openstack
log = /home/bnemec/.pip/pip.log
proxy = http://roxy.torment:3128",
   require => File['.pip'],
}
# Before anyone starts thinking it's creepy that I named my local network "Torment", understand that my router is called "NamelessOne".  Google it if you don't get the reference. ;-)
line {'yum proxy':
   file => '/etc/yum.conf',
   line => 'proxy=http://roxy.torment:3128',
}

exec {'update':
   command => 'yum -y upgrade',
   path => '/usr/bin',
   require => Line['yum proxy'],
}

exec {'prereqs':
   command => 'yum -y install git ccache libxslt-devel mariadb-devel zeromq-devel postgresql-devel python-devel libyaml-devel iftop sysstat python-pip',
   path => '/usr/bin',
   require => Line['yum proxy'],
}

exec {'tox':
   command => 'pip install tox',
   path => '/usr/bin',
   require => Exec['prereqs']
}

exec {'git name':
   command => 'git config --global user.name "Ben Nemec"',
   path => '/usr/bin',
   environment => 'HOME=/home/bnemec',
   require => [Exec['prereqs'], File['bnemec']],
}

exec {'git email':
   command => 'git config --global user.email "bnemec@redhat.com"',
   path => '/usr/bin',
   environment => 'HOME=/home/bnemec',
   require => [Exec['prereqs'], File['bnemec']],
}

# The git commands are run as root, so the .gitconfig owner is wrong
file {'.gitconfig':
   path => '/home/bnemec/.gitconfig',
   ensure => present,
   owner => 'bnemec',
   group => 'bnemec',
   mode => '0664',
   require => [Exec['git name'], Exec['git email']],
}

Sample localrc

DATABASE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_TOKEN=token
SERVICE_PASSWORD=password
ADMIN_PASSWORD=password
ENABLE_DEBUG_LOG_LEVEL=false
INSTALL_TESTONLY_PACKAGES=true
disable_service horizon
# Enable all of cinder like in the gate
enable_service cinder c-api c-vol c-sch c-bak

#HOST_IP=11.2.2.3

RECLONE=yes

# Quantum (Neutron)
#disable_service n-net
#enable_service quantum q-svc q-agt q-dhcp q-l3 q-meta q-lbaas q-vpn q-fwaas q-metering
# Qpid
#enable_service qpid
#disable_service rabbit

Sample rc file

source ~/devstack/openrc admin admin
export OS_NO_CACHE=1
export PIP_DOWNLOAD_CACHE=~/.cache/pip
# I actually don't use these much anymore, but I hated typing "quantum" back when that was the name of OpenStack Networking ;-)
alias u='neutron'
alias n='nova'
alias g='glance'
alias k='keystone'
alias c='cinder'