Introduction to SR-IOV

The SR-IOV technology is a hardware based virtualization solution that improves both performance and scalability. The SR-IOV standard enables efficient sharing of PCIe (Peripheral Component Interconnect) Express devices among virtual machines and is implemented in the hardware to achieve I/O performance which is comparable to native performance. The SR-IOV specification defines a new standard wherein the new devices that are created will enable the virtual machine to be directly connected to the I/O device.

The SR-IOV specification is defined and maintained by PCI-SIG at http://www.pcisig.com

A single I/O resource can be shared by many virtual machines. The shared devices will provide dedicated resources and also utilize shared common resources. In this way, each virtual machine will have access to unique resources. Therefore, a PCIe device, such as an Ethernet Port, that is SR-IOV enabled with appropriate hardware and OS support can appear as multiple, separate physical devices, each with its own configuration space.

The following figure illustrates the SR-IOV technology for PCIe hardware.

Figure 21-1 SR-IOV Technology

Two new function types in SR-IOV are:

Physical Function (PF)

A PCI Function that supports the SR-IOV capabilities as defined in SR-IOV specification. A PF contains the SR-IOV capability structure and is used to manage the SR-IOV functionality. PFs are fully featured PCIe functions that can be discovered, managed, and manipulated like any other PCIe device. PFs have full configuration resources, and can be used to configure or control the PCIe device.

Virtual Function (VF)

A function that is associated with a Physical Function. A VF is a lightweight PCIe function that shares one or more physical resources with the Physical Function and with other VFs that are associated with the same PF. VFs are only allowed to have configuration resources for its own behavior.

Each SR-IOV device can have a Physical Function (PF) and each PF can have upto 64K Virtual Functions (VFs) associated with the PF. The VFs can be created by the PF through registers designed with properties specifically for this purpose.

Once SR-IOV is enabled in the PF, the PCI configuration space of each VF can be accessed by the bus, device, and function number (Routing ID) of the PF. Each VF also has a PCI Memory Space, which is used to map its register set. The device drivers of the VFs operate on the register set to enable its functionality and appear as a real existing PCI device. Once VFs are created, they can be directly assigned to IO guest domains or individual applications such as Oracle Solaris Zones on bare-metal platforms. This capability enables them to share the physical device and perform I/O without CPU and Hypervisor software overhead.

Benefits of SR-IOV

The SR-IOV standard allows efficient sharing of PCIe devices among IO Guest Domains. An SR-IOV device can have hundreds of Virtual Functions (VFs) associating with a Physical Function (PF). The creation of VFs can be dynamically controlled by the PF through registers designed to turn on the SR-IOV capability. By default, the SR-IOV capability is turned off, and the PF behaves as traditional PCIe device.

Devices that have SR-IOV capability can take advantage of the following benefits:

• Performance – Direct access to hardware from virtual machines environment.

• Cost Reduction – Capital and operational expenditure savings include:

  • Power savings

  • Reduced adapter count

  • Less cabling

  • Fewer switch ports