User login
Tags
Syndicate
Network Isolation in TripleO for Home Networks
Edit ( 2015-10-07): Some changes have happened recently in TripleO that have obsoleted the templates below. I've created a Git repo for the templates so it is easier to keep them up to date: https://github.com/cybertron/tripleo-network-templates The discussion below is still relevant, but the actual template snippets won't work in a current TripleO setup so they need to be replaced with the ones from Github.
During my time working on TripleO, I have accumulated a modest collection of enterprisey hardware that I use to do development. A few months ago I picked up a couple of new servers and switches that allowed me to do actual baremetal deployments using RDO Manager. However, up until recently I was only able to do toy deployments where everything was running on the PXE provisioning network, which in my case is a non-routable network. Furthermore, because most of the example configs in the documentation are oriented toward big enterprise deployments, there was no example for a simple home user such as myself to work from. This post intends to change that.
Environment
First, a bit about my environment. I'm currently deploying with 3 Dell 1/2950 systems. It is handy to have mostly matching hardware as it makes figuring out network device names much easier. All of these systems have at least two gigabit ethernet ports. The first is attached to my home network, so we'll call that the "public" port. The other is attached to an unrouted switch that I use for provisioning. Thus, we'll call that the "provisioning" nic. :-)
These are all connected to simple, unmanaged switches which means I can't use the VLAN isolation method provided as an example in the TripleO Heat templates. Fortunately, it is possible to set up the environment so the public API services listen on my home network and everything else continues to use the provisioning network. Here's how:
Configuration Files
First, the nic configs. These define how the interfaces on the various types of nodes will be assigned. In this case, it's pretty simple: We just need to add a nic to an OVS bridge.
controller.yaml
On the controller we add the public interface (nic1 in os-net-config nomenclature) to the OVS bridge so the undercloud Neutron can assign it an IP for the API services to listen on. We don't need to specify anything for the provisioning network (and in fact it likely wouldn't work if we tried) because by default TripleO attempts to DHCP all interfaces on the system, so that one will handle getting an IP without any interference, and that's all we need it to do.
Note: I'm only including the resources section from the nic config files. The other sections of the files remain the same as in the examples shipped with tripleo-heat-templates. Those files can be used as templates for this configuration, even though by default they are set up to use VLANs. In my environment I only have hardware for a single control and single compute node, so those are the only nic configs I need.
...
resources:
OsNetConfigImpl:
type: OS::Heat::StructuredConfig
properties:
group: os-apply-config
config:
os_net_config:
network_config:
-
type: ovs_bridge
name: {get_input: bridge_name}
use_dhcp: true
members:
-
type: interface
name: nic1
# force the MAC address of the bridge to this interface
primary: true
addresses:
-
ip_netmask: {get_param: ExternalIpSubnet}
routes:
-
ip_netmask: 0.0.0.0/0
next_hop: {get_param: ExternalInterfaceDefaultRoute}
...
compute.yaml
On the compute nodes, we actually add the provisioning nic (nic2) to the OVS bridge. This is because compute nodes don't have any public API endpoints, so all of their traffic is going to travel over the internal API network. In this case, that's also the provisioning network. This time, we let the public nic handle DHCP itself, although that isn't strictly necessary. It is handy to be able to log in to the system over the public network in a dev environment though. In a production environment you wouldn't expose a compute node to the public, but in a production environment you probably aren't using this configuration anyway. :-)
...
resources:
OsNetConfigImpl:
type: OS::Heat::StructuredConfig
properties:
group: os-apply-config
config:
os_net_config:
network_config:
-
type: ovs_bridge
name: {get_input: bridge_name}
use_dhcp: true
members:
-
type: interface
name: nic2
# force the MAC address of the bridge to this interface
primary: true
...
network-environment.yaml
These files are then referenced from network-environment.yaml, which I just leave in the undercloud home directory for convenience.
11.0.0.0/8 is the CIDR of my home network (yeah, I know, but it's easy to type and how likely am I to run into a conflict with a DoD IP?). Note that in this case having a single digit netmask is actually a problem. More on that later.
The other params are the address of my home router and an IP range that isn't currently assigned on my network. While the example network-environment.yaml in tripleo-heat-templates has a lot more parameters, in this simple setup we don't need any of them.
resource_registry:
OS::TripleO::Compute::Net::SoftwareConfig: /home/bnemec/nic-configs/compute.yaml
OS::TripleO::Controller::Net::SoftwareConfig: /home/bnemec/nic-configs/controller.yaml
parameter_defaults:
ExternalNetCidr: 11.0.0.0/8
ExternalAllocationPools: [{'start': '11.1.1.111', 'end': '11.1.1.123'}]
ExternalInterfaceDefaultRoute: 11.1.1.1
network-isolation.yaml
The environments/network-isolation.yaml file also needs to be edited. I suggest making a copy of the entire /usr/share/openstack-tripleo-heat-templates directory and editing it there. Then pass the custom templates directory to the deploy command (I'll provide an example later).
Here again, the configuration is very simple because we only have one extra network.
resource_registry: OS::TripleO::Network::External: ../network/external.yaml # Port assignments for the controller role OS::TripleO::Controller::Ports::ExternalPort: ../network/ports/external.yaml
Single Digit CIDR Mask Fix
As I mentioned earlier, having a single digit netmask for my home network is actually a problem in this case because the current implementation of network isolation can only handle two digit masks. This will be addressed at some point in the future, but for the moment a tweak to the Heat template is required. The file that requires changes is network/ports/external.yaml in the tripleo-heat-templates directory (this is another good reason to make a copy for editing). The change required is to remove/comment the next to last line in the file, which looks like - {get_attr: [ExternalPort, subnets, 0, cidr, -2]}. With that change the templates will properly parse a CIDR with a single digit mask.
Deployment
With all of those changes in place, it's time to actually do the deployment. A sample deployment command is included below, with a color-coded explanation of the options:
openstack overcloud deploy --templates custom --libvirt-type kvm --neutron-network-type vxlan --neutron-tunnel-types vxlan -e network-environment.yaml -e custom/environments/network-isolation.yaml
Templates path: We want to deploy with our customized tripleo-heat-templates. In this example, I copied them to a directory called custom in the undercloud user's home directory.
Virt type: We're deploying to baremetal, so we want to use accelerated virt.
Neutron tunneling (optional): Defaults to GRE for historical reasons, but as I understand it we recommend VXLAN instead.
Network isolation environment files: This is the part where we tell the overcloud to use the network configurations described above. Both files need to be passed as -e parameters in order for it to take effect.
Running that command should result in an OpenStack installation with the public endpoints on the public network and everything else running on the provisioning network, which is reasonably sane for a small home lab like mine. Hope this is helpful and as always my contact information is on the About page if you have any questions or comments. Also, user registration is fixed so you can leave comments if you want.
- Log in to post comments