The Dawn of Virtualization: Hardware Partitioning and Early Mainframes
Before the term “virtual machine” became synonymous with software-defined compute, the concept of resource sharing and isolation was born out of necessity within the realm of mainframe computing. Early mainframes, behemoths of their time, were incredibly expensive resources. Maximizing their utilization was paramount, and the idea of partitioning the machine into independent, isolated environments arose as a natural solution.
IBM’s CP/CMS (Control Program/Cambridge Monitor System), developed in the late 1960s, is widely considered a pioneering example of VM architecture. CP acted as the hypervisor, directly managing the physical hardware. CMS, running on top of CP, provided each user with an isolated environment, effectively a virtual machine. This allowed multiple users to interact with the system simultaneously, each believing they had dedicated access to a complete machine. The key innovation was time-sharing. CP intelligently scheduled processing time across different CMS instances, creating the illusion of concurrency. This was a crucial step away from batch processing, where jobs had to be submitted and processed sequentially.
The architecture revolved around privilege rings, a hardware security feature that allowed the hypervisor (CP) to operate at the most privileged level, directly controlling the CPU, memory, and I/O devices. Guest operating systems (CMS) ran at a less privileged level, requiring them to make system calls through the hypervisor. This hierarchical structure ensured that one VM could not directly access or interfere with the resources allocated to another.
CP/CMS utilized paravirtualization techniques, although the term wasn’t coined until much later. CMS, designed to run on top of CP, was aware that it was running in a virtualized environment. It was modified to explicitly call CP for certain operations, improving performance and stability compared to a scenario where the guest OS remained entirely unaware of virtualization. This collaborative approach between the hypervisor and the guest OS was a defining characteristic of early VM architectures.
The benefits were clear: increased resource utilization, improved security through isolation, and the ability to run multiple operating systems on a single physical machine. However, the complexity of managing these systems and the performance overhead associated with virtualization remained significant challenges.
The Rise of Commercial Virtualization: IBM VM/370 and the S/390 Era
Building upon the success of CP/CMS, IBM introduced VM/370, a commercially supported version of their virtualization technology for the System/370 mainframe architecture. VM/370 solidified the role of virtualization in enterprise computing, offering enhanced features, better performance, and improved management capabilities.
VM/370 introduced virtual addressing, a critical component of modern VM architecture. Each VM was given its own isolated address space, preventing accidental or malicious access to other VMs’ memory. The hypervisor managed the translation between virtual addresses and physical addresses, ensuring that each VM operated within its allocated memory region. This significantly enhanced security and stability.
The control program (CP) component of VM/370 was responsible for managing the physical hardware and scheduling resources among the different VMs. It also provided a set of virtual devices to the guest operating systems, allowing them to interact with the hardware in a consistent and predictable manner. The conversational monitor system (CMS) provided a user interface for interacting with the virtual machines, allowing users to create, manage, and operate their virtual environments.
VM/370 also introduced storage virtualization. The hypervisor could present a virtual disk to the guest operating system, which was actually backed by a larger physical storage pool. This allowed for more efficient storage allocation and management, as VMs could be allocated storage on demand without requiring dedicated physical disks. This was a precursor to modern storage virtualization technologies like SANs and NAS.
The subsequent System/390 architecture further refined and extended the capabilities of VM/370. Features like logical partitioning (LPAR) were introduced, allowing a single physical machine to be divided into multiple independent logical partitions, each running its own operating system and applications. LPARs provided a more robust and secure form of virtualization compared to the earlier VM/370 architecture.
The System/390 era also saw the development of virtualization management tools that simplified the administration of complex virtualized environments. These tools allowed administrators to monitor the performance of VMs, allocate resources, and manage the overall virtual infrastructure.
The Shift to x86: VMware and the Democratization of Virtualization
While mainframe virtualization provided a foundation for the technology, it remained largely confined to large enterprises with significant IT budgets. The real breakthrough came with the rise of the x86 architecture and the introduction of VMware. VMware revolutionized virtualization by bringing it to commodity hardware, making it accessible to a wider range of organizations.
VMware’s initial product, VMware Workstation, allowed users to run multiple operating systems concurrently on a single x86 machine. This was achieved through binary translation, a technique that involved dynamically rewriting the instructions of the guest operating system to make them compatible with the underlying hardware. This approach allowed VMware to run unmodified operating systems within virtual machines, eliminating the need for paravirtualization.
VMware ESX, a hypervisor designed for server environments, further advanced the technology. ESX ran directly on the hardware, providing a thin layer of abstraction between the physical resources and the virtual machines. This approach minimized the overhead associated with virtualization and delivered near-native performance.
VMware’s architecture relied on virtual machine monitors (VMMs), which were responsible for managing the execution of the guest operating systems. The VMMs provided a virtualized view of the hardware to the guest operating systems, allowing them to run without modification. The hypervisor managed the allocation of resources, such as CPU, memory, and I/O, to the different VMs.
VMware also introduced virtual networking, allowing VMs to communicate with each other and with the external network. Virtual switches and virtual network adapters were used to create virtual networks within the physical infrastructure. This allowed for the creation of complex network topologies without requiring physical network hardware.
The impact of VMware was profound. It democratized virtualization, making it accessible to a wider range of organizations and individuals. It also spurred innovation in the virtualization space, leading to the development of new hypervisors and virtualization technologies.
The Rise of Open-Source Virtualization: Xen and KVM
While VMware dominated the early x86 virtualization market, the open-source community quickly recognized the potential of virtualization and began developing their own solutions. Xen and KVM (Kernel-based Virtual Machine) emerged as leading open-source hypervisors, offering alternatives to VMware’s proprietary technologies.
Xen pioneered the use of paravirtualization in the x86 architecture. Xen-enabled guest operating systems were modified to explicitly call the hypervisor for certain operations, such as memory management and I/O access. This approach significantly improved performance compared to binary translation, but it required modifications to the guest operating systems.
KVM took a different approach, leveraging the virtualization extensions built into modern x86 processors. KVM ran as a module within the Linux kernel, allowing it to take advantage of the kernel’s existing capabilities for scheduling, memory management, and I/O. KVM supported both paravirtualization and hardware-assisted virtualization, allowing it to run both modified and unmodified guest operating systems.
The architecture of Xen consisted of a hypervisor that ran directly on the hardware and a dom0 domain that provided management and control functions. Guest operating systems ran in domUs domains. The hypervisor was responsible for managing the physical resources and scheduling the execution of the different domains.
KVM’s architecture was simpler, as it ran as a module within the Linux kernel. The kernel provided the hypervisor functionality, and guest operating systems ran as processes within the kernel. This integration with the kernel allowed KVM to leverage the kernel’s existing features and performance optimizations.
The emergence of Xen and KVM significantly expanded the virtualization landscape, offering open-source alternatives to VMware’s proprietary solutions. These open-source hypervisors played a crucial role in the development of cloud computing and other virtualization-based technologies. They also fostered innovation and competition in the virtualization market.
