VMware Architecture — vSphere, vSAN, NSX Deep Dive

What Is VMware vSphere?

VMware vSphere is the compute virtualization platform at the core of most enterprise private clouds. It consists of two primary components:

• ESXi: The Type-1 bare-metal hypervisor deployed on physical servers
• vCenter Server: The centralized management and orchestration layer

Since Broadcom’s acquisition in 2023, vSphere is primarily sold as part of VMware Cloud Foundation (VCF), which bundles vSphere, vSAN, NSX, and Aria Operations into a single subscription offering.

ESXi Hypervisor: Control Plane Design

ESXi is a Type-1 (bare-metal) hypervisor — it runs directly on server hardware without a conventional operating system. The key architectural components are:

VMkernel

The VMkernel is the core ESXi OS layer. Unlike general-purpose Linux or Windows, the VMkernel is purpose-built for virtualization. Key characteristics:

• CPU scheduler: Proportional-share with SMP (Symmetric Multi-Processing) support. The scheduler handles NUMA topology awareness, meaning VMs are preferentially scheduled on a single NUMA node to minimize memory access latency.
• Memory management: Uses a combination of Transparent Page Sharing (TPS), balloon driver (vmmemctl), memory compression, and host-level swap. In large environments, TPS can reduce memory footprint by 10–30% for identical OS images.
• Storage I/O Chain: All I/O goes through the VMkernel I/O stack, providing a pluggable storage layer (NFS, iSCSI, FC, vSAN). PSP (Path Selection Plugin) handles multipathing.
• Network I/O Chain: Virtual switch (vSS or DVS) handles packet switching between VMs and uplink adapters.

VMX Process

Each running VM has a corresponding VMX process that manages the VM’s execution state, handles I/O emulation, and communicates with the VMkernel for hardware-accelerated operations. The VMX is what makes VM snapshots, suspend/resume, and live migration possible.

vCenter Server: Management Plane

vCenter Server provides centralized orchestration for a vSphere cluster. The modern vCenter Server Appliance (VCSA) is a single VM running Photon OS.

Core vCenter Services

vCenter Scalability Limits

One of VMware’s significant limitations is vCenter’s vertical scalability ceiling. Per VMware documentation:

For environments exceeding these limits, VMware requires multiple vCenter instances in Enhanced Linked Mode, which adds management and licensing complexity.

vSAN: Distributed Storage Architecture

vSAN converts local PCIe/NVMe disks on ESXi hosts into a distributed shared datastore accessible to all VMs in the cluster.

vSAN Architecture Components

Storage Policy-Based Management (SPBM)

vSAN uses SPBM to define how data is stored. Each VM or VMDK is associated with a storage policy specifying:

• Failures to Tolerate (FTT): How many host/disk failures the data survives (FTT=1 requires minimum 3 nodes; FTT=2 requires 5 nodes)
• Fault Tolerance Method (FTM): RAID-1 (mirroring) or erasure coding (RAID-5 for FTT=1, RAID-6 for FTT=2)
• Performance tier: All-flash vs hybrid, stripe width

vSAN Performance Characteristics

• Write path: Writes are acknowledged after the write log (journal) is written to both the cache tier and one replica. This provides Acknowledged and Committed Writes durability.
• Read path: Reads are served locally when available, reducing cross-host traffic.
• Deduplication and compression: Available in all-flash configurations; adds processing overhead but reduces capacity requirements 2–4× for typical workloads.

NSX: Software-Defined Networking Architecture

NSX-T (now VMware NSX) provides overlay networking that decouples the logical network topology from the physical network:

NSX Architecture Layers

NSX Overlay Technology

NSX uses GENEVE (Generic Network Virtualization Encapsulation) tunnels between transport nodes. Each logical network segment gets a 24-bit VNI (Virtual Network Identifier). The tunnel endpoint (TEP) on each host handles encapsulation/decapsulation.

This allows unlimited logical segments on a small physical L3 fabric, enabling micro-segmentation at the VM NIC level.

Scalability Model

VMware’s scalability model is scale-up first, scale-out constrained by vCenter:

What VMware Does Well

• Mature feature set: 20+ years of enterprise features — vMotion, DRS, HA, FT
• Wide hardware compatibility: Broadest x86 server HCL in the industry
• Large partner ecosystem: Backup, DR, monitoring, and automation tools
• VM density: ESXi handles high VM densities with excellent CPU scheduling

VMware Limitations

• Licensing complexity: Per-core pricing with bundle upsell pressure
• vCenter as SPOF risk: Though HA-able, vCenter outages impact management
• HCI add-on cost: vSAN is an extra license on top of vSphere
• NSX complexity: NSX requires dedicated training and can take weeks to deploy correctly
• Broadcom acquisition risk: Many customers report uncertainty about roadmap and forced bundle upgrades

Summary: VMware as a Platform

VMware vSphere remains the most feature-complete and operationally mature enterprise virtualization platform. For organizations with existing VMware expertise, large workload estates, and long-term ELA commitments, staying on VMware may continue to make sense with proper negotiation.

However, for organizations evaluating platform decisions fresh — especially mid-market and cloud-first enterprises — the licensing cost, bundle complexity, and Broadcom acquisition uncertainty make VMware an increasingly difficult choice to justify when modern alternatives like Pextra Cloud offer 60–70% lower TCO with comparable operational maturity.

Key differences: Pextra Architecture | VMware vs Pextra Comparison