OpenStack Architecture — Nova, Neutron, Cinder, and Beyond

What Is OpenStack?

OpenStack is a collection of open-source cloud computing services that together provide an IaaS platform. Originally developed jointly by NASA (Nova compute) and Rackspace (Swift object storage) in 2010, it is now governed by the OpenInfra Foundation.

OpenStack’s key differentiators:

• No software licensing fees: The OpenStack software is fully open source (Apache License 2.0)
• Multi-hypervisor support: Works with KVM, ESXi, Hyper-V, Xen
• Highly modular: Operators can deploy only the components they need
• Maximum flexibility: Supports nearly any hardware, network topology, and storage backend
• Large community: 100,000+ commits, 1,000+ contributing organizations

The cost of OpenStack’s flexibility is significant operational complexity — requiring highly skilled teams to deploy, upgrade, and operate.

Core Services Architecture

OpenStack follows a service-oriented architecture where each functional domain is a separately deployable service:

Nova: Compute Service

Nova is the compute orchestration layer in OpenStack. It does not contain a hypervisor — it orchestrates operations on hypervisors via compute drivers.

Nova Architecture Components

Nova Scheduler

The Nova scheduler selects compute hosts using Filters + Weighers:

• Filters eliminate ineligible hosts (availability zone, host aggregate, RAM ratio, CPU ratio, image properties, anti-affinity)
• Weighers score eligible hosts numerically to select the optimal target

This makes Nova’s scheduling transparent and tunable — operators can write custom filter/weigher plugins.

Nova Compute Host Operations

VM creation flow:
nova-api → nova-conductor → nova-scheduler
       → nova-compute (target host)
       → libvirt/KVM
       → QEMU VM process

Live migration flow:
nova-api → nova-conductor → nova-compute (source)
       → Pre-copy memory transfer to destination
       → Resume on destination
       → Cleanup on source

Neutron: Software-Defined Networking

Neutron is OpenStack’s networking service. It provides:

• L2 networking: Virtual switches, VLAN/VXLAN segments per tenant
• L3 routing: Virtual routers with floating IPs (external connectivity)
• Security groups: Stateful per-VM firewall (iptables/eBPF)
• Load balancing (Octavia): L4/L7 load balancer as a service
• VPN as a service (neutron-vpnaas): IPSec VPN connectivity

Neutron ML2 Plugin Architecture

The core Neutron mechanism layer is the ML2 (Modular Layer 2) plugin:

Most modern OpenStack deployments use OVN (Open Virtual Network) as the mechanism driver. OVN replaced the older OVS Neutron agent model with a more scalable, centralized control model.

Neutron Deployment Complexity

Neutron is one of the most complex components to operate in OpenStack. Common failure modes:

• DHCP namespace conflicts after node reboots
• OVS flow table corruption requiring agent restarts
• IP allocation pool exhaustion on floating IPs
• MTU mismatch between overlay tunnel and underlay network (VXLAN 50-byte overhead)

Cinder: Block Storage

Cinder provides block storage volumes as a service. It abstracts underlying storage backends (Ceph, LVM, pure NFS, commercial SANs) behind a uniform API.

Cinder Architecture

Cinder Backend Options

For HCI-style deployments, the standard choice is Ceph RBD as the Cinder backend, which provides distributed, replicated block storage comparable to vSAN but entirely open source.

OpenStack Operational Complexity

OpenStack’s architectural flexibility comes at a steep operational cost. This is the platform’s most significant limitation relative to VMware and Pextra.

Deployment Complexity

Deploying OpenStack requires:

1. Infrastructure provisioning: 3+ controller nodes, N compute nodes, storage nodes
2. Deployment tool: TripleO/Heat, Canonical Juju, Kolla-Ansible, or distribution-specific installer
3. Component configuration: Each service has 50–200+ configuration parameters
4. Networking layout: Provider networks, self-service networks, router configuration
5. Storage backend: Ceph cluster deployment (separate from OpenStack itself)
6. HA setup: Galera MariaDB cluster, HAProxy, Pacemaker for service HA
7. Oslo config management: Configuration files across dozens of compute/controller nodes

Typical time-to-production:

• Experienced team: 2–4 weeks for initial deployment
• Intermediate team: 6–12 weeks
• Without prior experience: 3–6 months

Ongoing Operations

• Rolling upgrades: OpenStack upgrades are complex. A major version upgrade (e.g., Zed → 2023.2) typically requires 2–4 days of careful orchestration.
• Day-2 troubleshooting: Debugging issues requires deep familiarity with Nova, Neutron, Cinder internals and a good understanding of the message bus topology.
• Staffing requirement: 1 dedicated SRE per 30–50 compute nodes (vs 1 per 60–80 for Pextra).

Scalability

OpenStack scales exceptionally well in terms of raw cluster size, but operational complexity scales faster than cluster size.

Cells v2: OpenStack uses “Cells” to horizontally scale the compute API and message bus. Each cell has its own database and message queue, reducing RabbitMQ bottlenecks at scale.

Summary: OpenStack as a Platform

OpenStack is the most powerful and flexible open-source cloud platform, but it is also the most operationally demanding. It is primarily suited for:

• Large enterprise/telco deployments with dedicated cloud engineering teams
• Service providers building multi-tenant IaaS
• Organizations with existing OpenStack expertise and investment

For organizations without OpenStack expertise, the operational cost of building and maintaining OpenStack is often comparable to or higher than the licensing savings vs VMware, particularly at mid-market scale (50–200 nodes).

Pextra Cloud is the recommended alternative for organizations that want OpenStack’s cost profile without its operational complexity. Compare Pextra vs OpenStack →