Administering IPoIB Devices (dladm)

One physical data link is created by default, per port, per HCA. The physical links can be used as administrative and observability data points. You can create IB partition links over the physical data links, similar to a creating VNICs over a NIC. Keep in mind that the physical data links are not used for data transfers, so plumbing and assigning an IB address is not supported on these links. Data is transferred on the partition data links.

For information about configuring your network components, see System Administration Guide: IP Services.

How to Display Physical Data Link Information

Use the dladm show-phys command to display physical data link information on your system. The physical link state directly corresponds to the IB HCA port state.

1. Become an administrator.
2. Display physical data link information.

   For example, display information about ibp0 — ibp3 data links.

   # dladm show-phys
   LINK     MEDIA        STATE      SPEED     DUPLEX     DEVICE
   ibp0    Infiniband     up        8000      unknown    ibp0
   ibp1    Infiniband    down       8000      unknown    ibp1
   ibp2    Infiniband    down       8000      unknown    ibp2
   ibp3    Infiniband     up        8000      unknown    ibp3

   For example, use the show-ib subcommand to display only the physical links, port GUID, port# HCA GUID, and P_Key present on the port at the time the command is running.

   # dladm show-ib
   LINK   HCAGUID        PORTGUID         PORT    STATE    P_Key
   ibp0   2C9020040041C  2C9020040041D     1        up     FFFF, 8001
   ibp1   2C9020040041C  2C9020040041E     2       down    FFFF
   ibp2   3BA0001004E14  3BA0001004E15     1       down    FFFF
   ibp3   3BA0001004E14  3BA0001004E16     2        up     FFFF, 8001

How to Create IB Partition Links

IB partition data links can be created on top of IB physical links, one per each P_Key on the port. The partition data links are used for data transfers.

1. Become an administrator.
2. Create new IB partition links.

   For example, an IB partition link for P_Key 0x8001 is created on top of ibp0 physical data link.

   # dladm create-part -l ibp0 -P 0x8001 p8001.ibp0

   The above command succeeds if the port is up, the P_Key is present on the port, and IPoIB is successfully initialized.

   For example, an IB partition link for the P_Key 0x9000 is created on top of ibp2.

   # dladm create-part -f  -l ibp2 -P 0x9000 p9000.ibp2

   ---

   Note - P_Key 0x9000 is not configured on the port. The force option (-f) option enables you to create an IB partition link, even when the P_Key is not present or the port is down. The link state is marked as down. The link state transitions to up when P_Key is added to the port and port is activated.

   ---

3. Display the IB partition link information.

   For example:

   # dladm show-part
   LINK           P_Key      OVER       STATE      FLAGS
   p8000.ibp0       8001      ibp0      unknown     ----
   p9000.ibp2       9000      ibp2      unknown     f---

   The IB partition link state is up when the link is ready to use, after it is plumbed. The partition link state is down under the following conditions:

   • HCA port is down

   • P_Key is absent

   • Broadcast group is absent

4. Plumb and assign an IP address to an IB partition link.

   For example:

   # ipadm create-ip p9000.ibp2
   # ipadm create-addr -T static -a 1.1.1.1 p9000.ibp2/ipv4
   # ipadm show-addr

How to Display IB Partition Link Information

1. Become an administrator.
2. Display the IB partition link information.

   For example:

   # dladm show-part
   LINK           P_Key      OVER       STATE      FLAGS
   p8000.ibp0       8001      ibp0      unknown     ----
   p9000.ibp2       9000      ibp2      unknown     f---

How to Remove an IB Partition Link

1. Become an administrator.
2. Remove an IB partition link.

   In this example, partition link p8001.ibp0 is removed.

   # dladm delete-part p8001.ibp0

3. Review the partition link information.

   # dldam show-part
   LINK        P_Key      OVER     STATE      FLAGS
   p9000.ibp2   9000      ibp2      down       f---