Bridging Overview

Bridges are used to connect separate network segments, which are paths between two nodes. When connected by a bridge, the attached network segments communicate as if they were a single network segment. Bridging is implemented at the datalink layer (L2) of the networking stack. Bridges use a packet-forwarding mechanism to connect subnetworks together.

Although both bridging and routing can be used to distribute information about the locations of resources on the network, they differ in several ways. Routing is implemented at the IP layer (L3) and uses routing protocols. No routing protocols are used on the datalink layer. Instead, the destinations of forwarded packets are determined by examining the network traffic that is received on the links that are attached to the bridge.

When a packet is received, its source address is examined. The packet's source address associates the node from which the packet was sent to the link on which it is received. Thereafter, when a received packet uses that same address as the destination address, the bridge forwards the packet over the link to that address.

The link associated with a source address might be an intermediate link that is connected to another bridge in the bridged network. Over time, all of the bridges within the bridged network “learn” which of the links sends a packet toward a given node. Thus, the packet's destination address is used to direct the packet to its final destination by means of hop-by-hop bridging.

A local “link-down” notification indicates that all nodes on a given link are no longer reachable. In this situation, packet forwarding to the link is halted, and all forwarding entries over the link are flushed. Older forwarding entries are also flushed over time. When a link is restored, packets that are received over the link are treated as new. The “learning” process based on a packet's source address begins again. This process enables the bridge to properly forward packets over that link when the address is used as the destination address.

To forward packets to their destinations, bridges must listen in promiscuous mode on every link that is attached to the bridge. Listening in promiscuous mode causes bridges to become vulnerable to the occurrences of forwarding loops, in which packets circle forever at full-line rate. Thus, bridging uses the Spanning Tree Protocol (STP) mechanism to prevent network loops that would render the subnetworks unusable.

In addition to using STP and the Rapid Spanning Tree Protocol (RSTP) for bridges, Oracle Solaris supports the Transparent Interconnection of Lots of Links (TRILL) protection enhancement. STP is used by default, but you can use TRILL by specifying the -P trill option for the bridging commands.

Using a bridge configuration simplifies the administration of the various nodes in the network by connecting them into a single network. By connecting these segments through a bridge, all the nodes share a single broadcast network. Thus, each node can reach the other nodes by using network protocols such as IP rather than by using routers to forward traffic across network segments. If you do not use a bridge, you must configure IP routing to permit the forwarding of IP traffic between nodes.

Figure 4-1 Simple Bridged Network

The previous figure shows a simple bridged network configuration. The bridge, goldengate, is an Oracle Solaris system that has bridging configured. The systems sanfrancisco and sausalito are physically connected to the bridge. Network A uses a hub that is physically connected to the bridge on one side and to computer systems on the other side. The bridge ports are links, such as bge0, bge1, and bge2.

Bridged networks can be formed into rings that physically connect several bridges together. Such configurations are common in networks. This type of configuration could cause problems with old packets saturating the network links by endlessly looping around the ring. To protect against such looping conditions, Oracle Solaris bridges implement both the STP and TRILL protocols. Note that most hardware bridges also implement STP loop protection.

The following figure shows a bridged network that is configured in a ring. The configuration shows three bridges. Two systems are physically connected to the westminster bridge. One system is physically connected to the waterloo bridge. And one system is physically connected to the tower bridge. The bridges are physically connected to each other through the bridge ports.

Figure 4-2 Bridged Network Ring

When STP or RSTP is used for loop protection, the physical loop is mitigated by preventing one of the connections in the loop from forwarding packets. Figure 4-2 shows that the physical link between the westminster and tower bridges is not used to forward packets.

Note that by shutting down usable physical links to perform loop protection, STP and RSTP cost you bandwidth.

Unlike STP and RSTP, TRILL does not shut down physical links to prevent loops. Instead, TRILL computes the shortest-path information for each TRILL node in the network and uses that information to forward packets to individual destinations.

As a result, TRILL enables the system to keep all links in use at all times. Loops are not a problem as they are handled similarly to the way that IP handles loops. Namely, TRILL creates routes as needed and uses forwarding hop limits to avoid problems that are caused by momentary loop states.

---

Caution - Do not set local-mac-address?=false on SPARC platforms. If you do, the systems will errantly use the same MAC address on multiple ports and on the same network.

---

---

Caution - Do not configure a link into a bridge when the highest possible levels of performance are required. Bridging requires that the underlying interfaces are in promiscuous mode, which disables a number of important optimizations that are in the hardware, driver, and other layers of the system. The disabling of these performance enhancements is an unavoidable consequence of the bridging mechanism. You can use a bridge on a system where some of the system's links are not bridged and are thus not subject to those constraints. These performance issues only affect links that are configured to be part of a bridge.

---

For information about STP, see IEEE 802.1D-1998. For information about RSTP, see IEEE 820.1Q-2004. For information about TRILL, see the Internet Engineering Task Force (IETF) TRILL draft documents.

Link Properties

The following link properties can be viewed by the dladm show-linkprop command, and modified by the dladm set-linkprop and reset-linkprop commands:

default_tag

Defines the default virtual local area network (VLAN) ID for untagged packets that are sent to and from a link. Valid values are from 0 to 4094. The default value is 1. Only non-VLAN and non-virtual network interface card (VNIC) type links have this property. Setting this value to 0 disables the forwarding of untagged packets to and from the port. (This is a MAC property.)

---

Note - This property is also used outside the scope of bridging to specify the IEEE port VLAN identifier (PVID) for the link. When default_tag is non-zero, you cannot create a VLAN that has that same ID on the link because the base link itself automatically represents the PVID.

For example, if PVID is set to 5 on net0, you cannot create a VLAN with ID 5 on net0. To specify VLAN 5 in this situation, use net1.

You cannot set default_tag to be equal to the ID of any existing VLAN that is created on that link. For example, the following command creates VLAN 22 on net0:

# dladm create-vlan -l net0 -v 22 myvlan0

In this situation, you cannot set default_tag to 22, which would make both net0 and myvlan0 represent the same VLAN.

By setting default_tag to 0, you enable untagged packets on net0 to be unassociated with any VLAN. This situation prevents such packets from being forwarded by a configured bridge.

---

forward

Enables and disables traffic forwarding through the bridge. This property exists on all links except for VNIC links. Valid values are 1 (true) and 0 (false). The default value is 1. When traffic forwarding is disabled, a VLAN that is associated with a link instance will not forward traffic through the bridge. Disabling forwarding is equivalent to removing the VLAN from the “allowed set” for a traditional bridge. This means that VLAN-based I/O to the underlying link from local clients continues, but no bridge-based forwarding is performed.

stp

Enables and disables STP and RSTP. Valid values are 1 (true) and 0 (false). The default value is 1, which enables STP and RSTP. When this property is set to 0, the link does not use STP or RSTP and is placed into forwarding mode at all times. The forwarding mode uses bridge protocol data unit (BPDU) guarding. Disable STP and RSTP when you want to configure point-to-point links that are connected to end nodes. Only non-VLAN and non-VNIC type links have this property.

stp_cost

Represents STP and RSTP cost values for using the link. Valid values are from 1 to 65535. The default value is 0, which is used to signal that cost is automatically computed by link type. The following values represent the cost for several link types: 100 for 10 Mbit/sec, 19 for 100 Mbit/sec, 4 for 1 Gbit/sec, and 2 for 10 Gbit/sec.

stp_edge

Specifies whether the port is connected to other bridges. Valid values are 1 (true) and 0 (false). The default value is 1. If this property is set to 0, the daemon assumes that the port is connected to other bridges even if no BPDUs of any type are detected.

stp_p2p

Specifies the connection mode type. Valid values are true, false, and auto. The default value is auto, which automatically discovers point-to-point connections. Specify true to force to point-to-point mode. Specify false to force normal multipoint mode.

stp_priority

Sets the STP and RSTP port priority value. Valid values are from 0 to 255. The default value is 128. The STP and RSTP port priority value is used to determine the preferred root port of a bridge by prepending the value to the PVID. The lower the numerical value is, the higher the priority.

STP Daemon

Each bridge that you create by using the dladm create-bridge command is represented as an identically named Service Management Facility (SMF) instance of svc:/network/bridge. Each instance runs a copy of the /usr/lib/bridged daemon, which implements the STP.

For example, the following command creates a bridge called pontevecchio:

# dladm create-bridge pontevecchio

The system creates an SMF service instance called svc:/network/bridge:pontevecchio and an observability node called /dev/net/pontevecchio0.

For safety purposes, all ports run standard STP by default. A bridge that does not run some form of bridging protocol, such as STP, can form long-lasting forwarding loops in the network. Because Ethernet has no hop-count or transistor-to-transistor logic (TTL) on packets, any such loops are fatal to the network.

When a particular port is not connected to another bridge (for example, because the port has a direct point-to-point connection to a host system), you can administratively disable STP for that port. Even if all ports on a bridge have STP disabled, the STP daemon still runs. The daemon continues to run for the following reasons:

• To handle any new ports that are added

• To implement BPDU guarding

• To enable or disable forwarding on the ports, as necessary

When a port has STP disabled, the bridged daemon continues to listen for BPDUs (BPDU guarding). The daemon uses syslog to flag any errors and disables forwarding on the port to indicate a serious network misconfiguration. The link is re-enabled when the link goes down and comes up again, or when you manually remove the link and re-add it.

If you disable the SMF service instance for a bridge, bridge forwarding stops on those ports as the STP daemon is stopped. If the instance is restarted, STP starts from its initial state.

TRILL Daemon

Each bridge that you create by using the dladm create-bridge -P trill command is represented as an identically named SMF instance of svc:/network/bridge and svc:/network/routing/trill. Each instance of svc:/network/routing/trill runs a copy of the /usr/lib/trilld daemon, which implements the TRILL protocol.

For example, the following command creates a bridge called bridgeofsighs:

# dladm create-bridge -P trill bridgeofsighs

The system creates two SMF services called svc:/network/bridge:bridgeofsighs and svc:/network/routing/trill:bridgeofsighs. In addition, the system creates an observability node called /dev/net/bridgeofsighs0.

Debugging Bridges

Each bridge instance is assigned an observability node, which appears in the /dev/net/ directory and is named with the bridge name plus a trailing 0.

The observability node is intended for use with the snoop and wireshark utilities. This node operates like a standard Ethernet interface, except for the transmission of packets, which are silently dropped. You cannot plumb IP on top of an observability node, and you cannot perform bind requests (DL_BIND_REQ) unless you use the passive option.

When used, the observability node makes a single unmodified copy of every packet handled by the bridge available to the user to monitor and debug. This behavior is similar to a monitoring port on a traditional bridge and is subject to the usual data link provider interface (DLPI) promiscuous mode rules. You can use the pfmod command or features in the snoop and wireshark utilities to filter packets based on VLAN ID.

The delivered packets represent the data that is received by the bridge.

To see the packets that are transmitted and received on a particular link (after the bridging process is complete), run snoop on the individual links rather than on the bridge observability node.

You can also obtain statistics on how network packets use network resources on links by using the dlstat command. For information, see Chapter 4, Monitoring Network Traffic and Resource Usage in Oracle Solaris, in Using Virtual Networks in Oracle Solaris 11.1.

How Link Behavior Changes When Bridges Are Used

The following sections describe how link behavior changes when bridges are used in a network configuration.

For information about standard link behavior, see Deploying VLANs: An Overview.

DLPI Behavior

The following describes the differences in link behavior when a bridge is enabled:

• Link up (DL_NOTE_LINK_UP) and link down (DL_NOTE_LINK_DOWN) notifications are delivered in the aggregate. This means that when all external links are showing the link-down status, the upper-level clients that are using the MAC layers also see link-down events. When any external link on the bridge shows the link-up status, all upper-level clients see the link-up events.

  This aggregate link-up and link-down reporting is performed for the following reasons:

  • When the link-down status is seen, nodes on the link are no longer reachable. This is not true when the bridging code can still send and receive packets through another link. Administrative applications that require the actual status of links can use the existing MAC-layer kernel statistics to reveal the status. These applications are unlike ordinary clients, such as IP, in that they report hardware status information and do not get involved in forwarding packets.

  • When all external links are down, the status appears as though the bridge itself were shut down. In this special case, the system recognizes that no nodes could possibly be reachable. The trade-off is that bridges cannot be used to enable local-only communication in the case where all interfaces are “real” (not virtual), and all links are disconnected.

• All link-specific features are made generic. Links that support special hardware acceleration features are unable to use those features because actual output link determination is not made entirely by the client. The bridge forwarding function must choose an output link based on the destination MAC address, and this output link can be any link on the bridge.

Administering VLANs on Bridged Networks

By default, VLANs that are configured on the system forward packets among all the ports on a bridge instance. When you invoke the dladm create-vlan or dladm create-vnic -v command, and the underlying link is part of a bridge, the command also enables packet forwarding of the specified VLAN on that bridge link.

To configure a VLAN on a link and disable packet forwarding to or from other links on the bridge, you must disable forwarding by setting the forward property for the VLAN with the dladm set-linkprop command.

Use the dladm create-vlan command to automatically enable the VLAN for bridging when the underlying link is configured as part of a bridge.

VLANs are ignored by the standards-compliant STP. The bridging protocol computes just one loop-free topology by using tag-free BPDU messages, and uses this tree topology to enable and disable links. You must configure any duplicate links that are provisioned in your networks such that when those links are automatically disabled by STP, the configured VLANs are not disconnected. This means that you must either run all VLANs everywhere on your bridged backbone or carefully examine all redundant links.

The TRILL protocol does not follow the complex STP rules. Instead, TRILL automatically encapsulates packets that have the VLAN tag intact and passes them through the network. This means that TRILL binds isolated VLANs where the same VLAN ID has been reused within a single bridged network.

This important difference from STP means that you can reuse VLAN tags in isolated sections of the network to manage sets of VLANs that are larger than the 4094 limit. Although you cannot use TRILL to manage networks in this way, you might be able to implement other solutions, such as provider-based VLANs.

In an STP network with VLANs, it might be difficult to configure the failover characteristics to prevent VLAN partitioning when STP disables the “wrong” link. The relatively small loss of functionality in isolated VLANs is minor compared to the robustness of the TRILL model.

VLAN Behavior

The bridge performs packet forwarding by examining the allowed set of VLANs and the default_tag property for each link. The general process is as follows:

1. Input VLAN determination. This process begins when an incoming packet is received by the system on a link. When a packet is received, it is checked for a VLAN tag. If that tag is not present or if the tag is priority only (set to zero), the default_tag property value configured on that link (if not set to zero) is taken as the internal VLAN tag. If the tag is not present or zero and default_tag is zero, the packet is ignored. No untagged forwarding is performed. If the tag is present and is the same as the default_tag value, the packet is also ignored. Otherwise, the tag is taken to be the incoming VLAN packet.

2. Link membership check. If the input VLAN is not configured as an allowed VLAN on this link, the packet is ignored. Forwarding is then computed, and the same check is made for the output link.

3. Tag update. If the VLAN tag (nonzero at this point) is the same as the default_tag value on the output link, the tag on the packet (if any) is removed, regardless of priority. If the VLAN tag is not the same as the default_tag value on the output link, a tag is added if not currently present, and the tag is set for the outgoing packet with the current priority copied into the packet.

Viewing Bridge Configurations

The following examples show how to view information about bridge configurations and bridging services:

• You can view information about bridges by running the following command:

  # dladm show-bridge
  BRIDGE      PROTECT ADDRESS            PRIORITY DESROOT
  tonowhere   trill   32768/66:ca:b0:39:31:5d 32768 32768/66:ca:b0:39:31:5d
  sanluisrey  stp     32768/ee:2:63:ed:41:94 32768 32768/ee:2:63:ed:41:94
  pontoon     trill   32768/56:db:46:be:b9:62 32768 32768/56:db:46:be:b9:62

• You can view TRILL nickname information for a bridge by running the following command:

  # dladm show-bridge -t tonowhere
  NICK FLAGS LINK        NEXTHOP
  38628 --   simblue2    56:db:46:be:b9:62
  58753 L    --          --