| Skip Navigation Links | |
| Exit Print View | |
|
Using Virtual Networks in Oracle Solaris 11.1 Oracle Solaris 11.1 Information Library |
| Skip Navigation Links | |
| Exit Print View | |
|
|
Using Virtual Networks in Oracle Solaris 11.1 Oracle Solaris 11.1 Information Library |
1. Network Virtualization and Resource Management in Oracle Solaris
2. Creating and Administering Virtual Networks in Oracle Solaris
3. Managing Network Resources in Oracle Solaris
Working With Clients, Transmit Rings, and Receive Rings
MAC Clients and Ring Allocation
Datalink Properties for Ring Allocation
Commands for Working With Receive and Transmit Rings
Obtaining and Interpreting Ring Information
Displaying Ring Allocation Capabilities of a Datalink
How to Configure a CPU Pool for a Datalink
How to Allocate CPUs to a Link
4. Monitoring Network Traffic and Resource Usage in Oracle Solaris
On NICs, receive (Rx) rings and transmit (Tx) rings are hardware resources through which the system receives and sends network packets, respectively. The following sections provide an overview of rings, followed by procedures that are used to allocate rings for networking processes. Examples are also provided to show how the mechanism works when you issue commands to allocate rings.
MAC clients such as VNICs and other datalinks are configured over a NIC to enable communication between a system and other network nodes. After a client is configured, it uses both Rx and Tx rings to receive or transmit network packets respectively. A MAC client can be either hardware-based or software-based. A hardware-based client fulfills any one of the following conditions:
It has dedicated use of one or more Rx rings.
It has dedicated use of one or more Tx rings.
It has dedicated use of one or more Rx rings and one or more Tx rings.
Clients that do not fulfill any of these conditions are software-based MAC clients.
Hardware-based clients can be assigned rings for exclusive use depending on the NIC. NICs such as nxge support dynamic ring allocation. On such NICs, you can configure not only hardware-based clients. You also have the flexibility to determine the number of rings to allocate to such clients, assuming that rings remain available for allocation. The use of rings is always optimized for the primary interface, for example, net0. The primary interface is also known as the primary client. Any available rings that have not been assigned for exclusive use by other clients are automatically assigned to the primary interface.
Other NICs such as ixge only support static ring allocation. On these NICs, you can only create hardware-based clients. The clients are automatically configured with a fixed set of rings per client. The fixed set is determined during the NIC driver's initial configuration. For more information about a driver's initial configuration for static ring allocation, refer to the Oracle Solaris 11.1 Tunable Parameters Reference Manual.
Software-based clients do not have exclusive use of rings. Rather, they share rings with other existing software-based clients or with the primary client. The rings that software-based clients use depends on the number of hardware-based clients that have priority in ring allocation.
It is important to understand the distinction between the primary client and other secondary clients. The primary client is the physical datalink of the NIC. Based on the generic names provided by Oracle Solaris during installation, the primary client would be named netN, where N is an instance number. For an explanation of generic names for datalinks, see Network Devices and Datalink Names in Introduction to Oracle Solaris 11 Networking. VNICs are secondary clients that are created over the physical datalink. If these clients are hardware-based clients, then they can have exclusive use of rings. Otherwise, the clients are software-based.
With VLANs, the assignment of rings proceeds differently depending on how the VLAN is created. VLANs are created in one of two ways:
By using the dladm create-vlan subcommand:
# dladm create-vlan -l link -v vid vlan
By using the dladm create-vnic subcommand:
# dladm create-vnic -l link -v vid vnic
A VLAN that is created by the dladm create-vlan subcommand shares the same MAC address as the underlying interface. Consequently, that VLAN also shares the Rx and Tx rings of the underlying interface. A VLAN that is created as a VNIC by using the dladm create-vnic command has a different MAC address from its underlying interface. The allocation of rings for such a VLAN is independent of the allocation for the underlying link. Thus, that VLAN can be assigned its own dedicated rings, assuming that the NIC supports hardware-based clients.
To administer rings, two ring properties can be set by using the dladm command:
rxrings refers to the number of assigned Rx rings to a specified link.
txrings refers to the number of assigned Tx rings to a specified link.
You can set each property to one of three possible values:
sw indicates that you are configuring a software-based client. The client does not have exclusive use of rings. Rather, the client shares rings with any other existing clients that are similarly configured.
n > 0 (number greater than zero) applies to the configuration of a hardware-based client only. The number refers to the quantity of rings that you allocate to the client for its exclusive use. You can specify a number only if the underlying NIC supports dynamic ring allocation.
hw also applies to the configuration of a hardware-based client. However, for such a client, you cannot specify the actual number of dedicated rings. Rather, the fixed number of rings per client is already set according to the NIC driver's initial configuration. You set the *rings properties to hw only if the underlying NIC supports static ring allocation.
To provide information about current ring assignments and use, the following additional read-only ring properties are available:
rxrings-available and txrings-available indicate the number of Rx and Tx rings that are available for allocation.
rxhwclnt-available and txhwclnt-available indicate the number of Rx and Tx hardware-based clients that can be configured over a NIC.
To manage the use of receive and transmit rings of datalinks, you use the following principal dladm subcommands:
dladm show-linkprop – Displays the current values of link properties, including Rx and Tx rings. The output provides the following information about a datalink's ring support capabilities. You need the information to determine what type of client you can configure to use Rx and Tx rings.
The available clients that you can create
The available rings that you can allocate to available clients
The capability to support dynamic or static ring allocation
If only static ring allocation is supported, the current distribution of rings to existing clients
Datalink Properties for Ring Allocation further describes how to interpret the output of this command.
dladm show-phys -H datalink – Displays how the rings of a physical datalink are currently being used by existing clients.
dladm create-vnic -p ring-properties vnic – Creates a client with a specific number of Tx or Rx rings to use to service traffic.
dladm set-linkprop -p ring-properties datalink – Allocates rings to a specific client, provided that rings are available and ring allocation is supported.
This section describes the dladm show-linkprop output that displays ring-related properties of a datalink.
This section provides examples of command output about ring-related properties and explains the type of information you can obtain. The following NICs are used in the examples:
net0 (over nxge)
net1 (over ixgbe)
net2 (over e1000g)
Example 3-1 nxge Ring Information
The following example shows ring information for nxge:
# dladm show-linkprop net0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net0 rxrings rw -- -- sw,<1-7> ... net0 txrings rw -- -- sw,<1-7> ... net0 rxrings-available r- 5 -- -- net0 txrings-available r- 5 -- -- net0 rxhwclnt-available r- 2 -- -- net0 txhwclnt-available r- 2 -- -- ...
With net0, the values of the POSSIBLE field are sw and <1-7> for rxrings and txrings. These values indicate that nxge supports hardware-based clients as well as software-based clients. The range <1-7> indicates the limits of the number of Rx rings or Tx rings you can set for clients. The range also indicates that the NIC supports dynamic ring allocation for both the receive and transmit sides.
In addition, the *rings-available properties indicate that five Rx rings and five Tx rings are available to allocate to hardware-based clients.
However, the *clnt-available properties show that you can configure only two clients that can have exclusive use of available Rx rings. Likewise, you can configure only two clients that can have exclusive use of available Tx rings.
Example 3-2 ixgbe Ring Information
The following example shows ring information for ixgbe:
# dladm show-linkprop net1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net1 rxrings rw -- -- sw,hw ... net1 txrings rw -- -- sw,hw,<1-7> ... net1 rxrings-available r- 0 -- -- net1 txrings-available r- 5 -- -- net1 rxhwclnt-available r- 0 -- -- net1 txhwclnt-available r- 7 -- -- ...
With net1, the POSSIBLE field values sw and hw for both rxrings and txrings indicate that ixgbe supports both hardware-based clients and software-based clients. For Rx rings, only static ring allocation is supported, where the hardware assigns a fixed set of Rx rings to each hardware-based client. However, for Tx rings, the range <1–7> indicates that dynamic allocation is supported. You can determine the number of Tx rings to assign to a hardware-based client, in this example, up to seven rings.
In addition, the *rings-available properties indicate that five Tx rings are available to allocate to hardware-based clients, but no Rx rings can be assigned.
Finally, based on the *hwclnt-available properties, you can configure seven hardware-based Tx clients to use Tx rings exclusively. However, you cannot create hardware-based clients with exclusive use of Rx rings because dynamic Rx ring allocation is not supported.
A zero (0) under the VALUE field for either of the *rings-available properties can mean one of the following:
No more rings are available to allocate to clients.
Dynamic ring allocation is not supported.
You can verify the meaning of the zero by comparing the POSSIBLE field for rxrings and txrings with the VALUE field for rxrings-available and txrings-available.
For example, suppose that txrings-available is 0, as follows:
# dladm show-linkprop net1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net1 rxrings rw -- -- sw,hw net1 txrings rw -- -- sw,hw,<1-7> net1 rxrings-available r- 0 -- -- net1 txrings-available r- 0 -- -- ...
In this output, the VALUE field for rxrings-available is 0 while the POSSIBLE field for rxrings is sw,hw. The combined information means that no Rx rings are available because the NIC does not support dynamic ring allocation. On the transmit side, the VALUE field for txrings-available is 0 while the POSSIBLE field for txrings is sw,hw,<1-7>. The combined information indicates that no Tx rings are available because all the Tx rings have already been allocated. However, as the POSSIBLE field for txrings indicates, dynamic ring allocation is supported. Thus, you can allocate Tx rings as these rings become available.
Example 3-3 e1000g Ring Information
The following example shows ring information for e1000g:
# dladm show-linkprop net2 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net2 rxrings rw -- -- -- ... net2 txrings rw -- -- -- ... net2 rxrings-available r- 0 -- -- net2 txrings-available r- 0 -- -- net2 rxhwclnt-available r- 0 -- -- net2 txhwclnt-available r- 0 -- -- ...
The output indicates that neither rings nor hardware-based clients can be configured because ring allocation is not supported in e1000g.
Two read-only datalink properties provide information about how rings are currently used by existing clients on the datalink:
rxrings-effective
txrings-effective
To obtain information about ring use and which rings are distributed to clients, use both the dladm show-linkprop and dladm show-phys -H subcommands.
The following examples show different types of output generated by the two commands with respect to the use of Rx and Tx rings and how these rings are distributed among clients.
Example 3-4 Ring Use of the Primary Client
The primary client is the interface that is configured over the physical datalink of the NIC. For this example, the NIC is an ixgbe card. By default its datalink is net0. The IP interface over net0 is the primary client.
# dladm show-linkprop net0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net0 rxrings rw -- -- sw,hw net0 rxrings-effective r 2 -- -- net0 txrings rw -- -- sw,hw,<1-7> net0 txrings-effective r 8 -- -- net0 txrings-available r- 7 -- -- net0 rxrings-available r- 0 -- -- net0 rxhwclnt-available r- 3 -- -- net0 txhwclnt-available r- 7 -- -- ... # dladm show-phys -H net0 LINK RINGTYPE RINGS CLIENTS net0 RX 0-1 <default,mcast> net0 TX 0-7 <default>net0 net0 RX 2-3 net0 net0 RX 4-5 -- net0 RX 6-7 --
The output provides the following information about the use and distribution of rings for the primary client net0:
rxrings-effective shows that net0 automatically receives two Rx rings. txrings-effective shows that net0 has use of eight Tx rings. By default, all unused rings are automatically assigned to the primary client.
Based on the output of the dladm show-phys -H command, the two Rx rings allocated to net0 are rings 2 and 3. For Tx rings, net0 uses rings 0 through 7.
Example 3-5 Ring Use of a Secondary Client
This example assumes the configuration of a VNIC client vnic1 over net0, the physical datalink of the ixgbe card.
# dladm show-linkprop vnic1 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... vnic1 rxrings rw hw -- sw,hw vnic1 rxrings-effective r- 2 -- -- vnic1 txrings rw hw -- sw,hw,<1-7> vnic1 txrings-effective r- 1 -- -- ... # dladm show-linkprop net0 LINK PROPERTY PERM VALUE DEFAULT POSSIBLE ... net0 rxrings rw -- -- sw,hw net0 rxrings-effective r- 2 -- -- net0 txrings rw -- -- sw,hw,<1-7> net0 txrings-effective r- -- -- -- net0 txrings-available r- 6 -- -- net0 rxrings-available r- 0 -- -- net0 rxhwclnt-available r- 3 -- -- net0 txhwclnt-available r- 6 -- -- ... # dladm show-phys -H net0 LINK RINGTYPE RINGS CLIENTS net0 RX 0-1 <default,mcast> net0 TX 0,2-7 <default>net0 net0 RX 2-3 net0 net0 RX 4-5 vnic1 net0 RX 6-7 -- net0 TX 1 vnic1
The combined output from the three commands provides the following information:
rxrings-effective for vnic1 shows that this VNIC automatically receives two Rx rings. txrings-effective shows that vnic1 has use of one Tx ring. These rings are statically allocated, as indicated by the hw value that is set for the *ring properties.
Based on the output of the dladm show-phys -H command, the two Rx rings allocated to net0 are rings 2 and 3. For Tx rings, net0 uses ring 0 and rings 2 through 7. vnic1 uses ring 1 for a Tx ring and rings 4 and 5 for Rx rings.
Note that vnic1 is configured as a hardware-based client with static ring allocation. Consequently, the number of available Tx hardware clients (txhwclnt-available) that can be created over net0 is reduced to six.
This procedure explains how to configure clients on a datalink based on the type of support for ring allocation. Make sure that you can interpret the output of the dladm commands that display datalink ring properties, as explained in Displaying Ring Allocation Capabilities of a Datalink and Displaying Ring Use and Ring Assignments on a Datalink. The information guides you in the way you configure the clients.
# dladm show-linkprop datalink
From the output, determine the following:
Whether the NIC supports hardware-based clients
The type of ring allocation that the NIC supports
The availability of rings to allocate to hardware-based clients
The availability of hardware-based clients that you can configure on the link
If the NIC supports dynamic ring allocation, create the hardware-based client with the following syntax:
# dladm create-vnic -p rxrings=number[,txrings=number] -l link vnic
If the client was previously created, use the following syntax:
# dladm set-linkprop -p rxrings=number[,txrings=number] vnic
Note - Some NICs support dynamic ring allocation on either Rx rings or Tx rings, but not on both types. You specify number on the ring type for which dynamic ring allocation is supported.
If the NIC supports static ring allocation, create the hardware-based client with the following syntax:
# dladm create-vnic -p rxrings=hw[,txrings=hw] -l link vnic
If the client was previously created, use the following syntax:
# dladm set-linkprop -p rxrings=hw[,txrings=hw] vnic
Note - Some NICs support static ring allocation on either Rx rings or Tx rings, but not on both types. You specify hw on the ring type for which static ring allocation is supported.
If the NIC supports only software-based clients, create the client with the following syntax:
# dladm create-vnic -p rxrings=sw[,txrings=sw] -l link vnic
If the client was previously created, use the following syntax:
# dladm set-linkprop -p rxrings=sw[,txrings=sw] vnic
# dladm show-linkprop vnic
# dladm show-phys -H datalink
See Also
For an example that shows how to use flows and how to allocate system resources, including Rx and Tx rings, to process network traffic in a virtual network, see Example 3-8.
|