STREAMS Multiplexers

STREAMS-related system calls are used to set up the “plumbing,” or stream interconnections, for multiplexing drivers. The subset of these calls that allows a user to connect (and disconnect) streams below a driver is referred to as multiplexing. This type of connection is referred to as a one-to-M, or lower, multiplexer configuration. This configuration must always contain a multiplexing driver, which is recognized by STREAMS as having special characteristics.

Multiple streams can be connected above a driver by open(2) calls. This accommodates the loop-around driver and the driver that handled multiple minor devices in Chapter 9, STREAMS Drivers. There is no difference between the connections to these drivers. Only the functions performed by the driver are different. In the multiplexing case, the driver routes data between multiple streams. In the device driver case, the driver routes data between user processes and associated physical ports. Multiplexing with streams connected above is referred to as an N-to-1, or upper, multiplexer. STREAMS does not provide any facilities beyond open and close to connect or disconnect upper streams for multiplexing.

From the driver's perspective, upper and lower configurations differ only in the way they are initially connected to the driver. The implementation requirements are the same: route the data and handle flow control. All multiplexer drivers require special developer-provided software to perform the multiplexing data routing and to handle flow control. STREAMS does not directly support flow control among multiplexed streams. M-to-N multiplexing configurations are implemented by using both of these mechanisms in a driver.

As discussed in Chapter 9, STREAMS Drivers, the multiple streams that represent minor devices are actually distinct streams in which the driver keeps track of each stream attached to it. The STREAMS subsystem does not recognize any relationship between the streams. The same is true for STREAMS multiplexers of any configuration. The multiplexed streams are distinct and the driver must be implemented to do most of the work.

In addition to upper and lower multiplexers, more complex configurations can be created by connecting streams containing multiplexers to other multiplexer drivers. With such a diversity of needs for multiplexers, providing general-purpose multiplexer drivers is not possible. Rather, STREAMS provides a general purpose multiplexing facility. The facility enables you to set up the intermodule or driver plumbing to create multiplexer configurations of generally unlimited interconnection.

Building a Multiplexer

The example in this section builds a protocol multiplexer with the multiplexing configuration shown in Figure 13-1. To free users from the need to know about the underlying protocol structure, a user-level daemon process is built to maintain the multiplexing configuration. Users can then access the transport protocol directly by opening the transport protocol (TP) driver device node.

An internetworking protocol driver (IP) routes data from a single upper stream to one of two lower streams. This driver supports two STREAMS connections beneath it. These connections are to two distinct networks; one for the IEEE 802.3 standard through the 802.3 driver, and another to the IEEE 802.4 standard through the 802.4 driver. The TP driver multiplexes upper streams over a single stream to the IP driver.

Figure 13-1 Protocol Multiplexer

Example 13-1 shows how this daemon process sets up the protocol multiplexer. The necessary declarations and initialization for the daemon program follow.

Example 13-1 Protocol Daemon

#include <fcntl.h>
#include <stropts.h>
void
main()
{
        int    fd_802_4,
                fd_802_3,
                fd_ip,
                fd_tp;
        /* daemon-ize this process */

        switch (fork()) {
            case 0:
                break;
            case -1:
                perror("fork failed");
                exit(2);
            default:
                exit(0);
    }
    (void)setsid();

This multilevel multiplexed stream configuration is built from the bottom up. The example begins by first constructing the IP multiplexer. This multiplexing device driver is treated like any other software driver. It owns a node in the file system and is opened just like any other STREAMS device driver.

The first step is to open the multiplexing driver and the 802.4 driver, thus creating separate streams above each driver as shown in Figure 13-2. The stream to the 802.4 driver may now be connected below the multiplexing IP driver using the I_LINK ioctl(2).

Figure 13-2 Streams Before Link

The sequence of instructions to this point is:

    if ((fd_802_4 = open("/dev/802_4", O_RDWR)) < 0) {
            perror("open of /dev/802_4 failed");
            exit(1);
        }
        if ((fd_ip = open("/dev/ip", O_RDWR)) < 0) {
            perror("open of /dev/ip failed");
            exit(2);
        }
        /* now link 802.4 to underside of IP */
        if (ioctl(fd_ip, I_LINK, fd_802_4) < 0) {
            perror("I_LINK ioctl failed");
            exit(3);
        }

I_LINK takes two file descriptors as arguments. The first file descriptor, fd_ip, is the stream connected to the multiplexing driver, and the second file descriptor, fd_802_4, is the stream to be connected below the multiplexer. The complete stream to the 802.4 driver is connected below the IP driver. The stream head's queues of the 802.4 driver are used by the IP driver to manage the lower half of the multiplexer.

I_LINK returns an integer value, muxid, which is used by the multiplexing driver to identify the stream just connected below it. muxid is ignored in the example, but it is useful for dismantling a multiplexer or routing data through the multiplexer. Its significance is discussed in Dismantling a Multiplexer.

The following sequence of system calls continues building the Internetworking Protocol multiplexer (IP):

    if ((fd_802_3 = open("/dev/802_3", O_RDWR)) < 0) {
            perror("open of /dev/802_3 failed");
            exit(4);
        }
        if (ioctl(fd_ip, I_LINK, fd_802_3) < 0) {
            perror("I_LINK ioctl failed");
            exit(5);
        }

The stream above the multiplexing driver used to establish the lower connections is the controlling stream and has special significance when dismantling the multiplexing configuration. This is illustrated in Dismantling a Multiplexer. The stream referenced by fd_ip is the controlling stream for the IP multiplexer.

The order in which the streams in the multiplexing configuration are opened is unimportant. If intermediate modules in the stream are necessary between the IP driver and media drivers, these modules must be added to the streams associated with the media drivers (using I_PUSH) before the media drivers are attached below the multiplexer.

The number of streams that can be linked to a multiplexer is restricted by the design of the particular multiplexer. The manual page describing each driver describes such restrictions (see SunOS Reference Manual, Intro(7)). However, only one I_LINK operation is allowed for each lower stream; a single stream cannot be linked below two multiplexers simultaneously.

Continuing with the example, the IP driver is now linked below the transport protocol (TP) multiplexing driver. As seen in Figure 13-1, only one link is supported below the transport driver. This link is formed by the following sequence of system calls:

    if ((fd_tp = open("/dev/tp", O_RDWR)) < 0) {
            perror("open of /dev/tp failed");
            exit(6);
        }
        if (ioctl(fd_tp, I_LINK, fd_ip) < 0) {
            perror("I_LINK ioctl failed");
            exit(7);
        }

Because the controlling stream of the IP multiplexer has been linked below the TP multiplexer, the controlling stream for the new multilevel multiplexer configuration is the stream above the TP multiplexer.

At this point, the file descriptors associated with the lower drivers can be closed without affecting the operation of the multiplexer. If these file descriptors are not closed, all subsequent read(2), write(2), ioctl(2), poll(2), getmsg(2), and putmsg(2) calls issued to them fail. That is because I_LINK associates the stream head of each linked stream with the multiplexer, so the user may not access that stream directly for the duration of the link.

The following sequence of system calls completes the daemon example:

    close(fd_802_4);
        close(fd_802_3);
        close(fd_ip);
        /* Hold multiplexer open forever or at least til this process
      is terminated by an external UNIX signal */
        pause();
    }

The transport driver supports several simultaneous streams. These streams are multiplexed over the single stream connected to the IP multiplexer. The mechanism for establishing multiple streams above the transport multiplexer is actually a by-product of the way in which streams are created between a user process and a driver. By opening different minor devices of a STREAMS driver, separate streams will be connected to that driver. The driver must be designed with the intelligence to route data from the single lower stream to the appropriate upper stream.

The daemon process maintains the multiplexed stream configuration through an open stream (the controlling stream) to the transport driver. Meanwhile, other users can access the services of the transport protocol by opening new streams to the transport driver; they are freed from the need for any unnecessary knowledge of the underlying protocol configurations and subnetworks that support the transport service.

Multilevel multiplexing configurations should be assembled from the bottom up. That is because the passing of ioctl(2) through the multiplexer is determined by the nature of the multiplexing driver and cannot generally be relied on.

Dismantling a Multiplexer

Streams connected to a multiplexing driver from above with open(2), can be dismantled by closing each stream with close(2). The mechanism for dismantling streams that have been linked below a multiplexing driver is less obvious, and is described in Disconnecting Lower Streams.

I_UNLINK ioctl(2) disconnects each multiplexer link below a multiplexing driver individually. This command has the form:

ioctl(fd, I_UNLINK, muxid);

where fd is a file descriptor associated with a stream connected to the multiplexing driver from above, and muxid is the identifier that was returned by I_LINK when a driver was linked below the multiplexer. Each lower driver may be disconnected individually in this way, or a special muxid value of MUXID_ALL can be used to disconnect all drivers from the multiplexer simultaneously.

In the multiplexing daemon program, the multiplexer is never explicitly dismantled. That is because all links associated with a multiplexing driver are automatically dismantled when the controlling stream associated with that multiplexer is closed. Because the controlling stream is open to a driver, only the final call of close for that stream will close it. In this case, the daemon is the only process that has opened the controlling stream, so the multiplexing configuration will be dismantled when the daemon exits.

For the automatic dismantling mechanism to work in the multilevel, multiplexed stream configuration, the controlling stream for each multiplexer at each level must be linked under the next higher-level multiplexer. In the example, the controlling stream for the IP driver was linked under the TP driver. This resulted in a single controlling stream for the full, multilevel configuration. Because the multiplexing program relied on closing the controlling stream to dismantle the multiplexed stream configuration instead of using explicit I_UNLINK calls, the muxid values returned by I_LINK could be ignored.

An important side effect of automatic dismantling on the close is that a process cannot build a multiplexing configuration with I_LINK and then exit. exit(2) closes all files associated with the process, including the controlling stream. To keep the configuration intact, the process must exist for the life of that multiplexer. That is the motivation for implementing the multiplexer as a daemon processs, see Multiplexing Driver Example.

If the process uses persistent links through I_PLINK ioctl(2), the multiplexer configuration remains intact after the process exits. These links are described in Persistent Links.

Routing Data Through a Multiplexer

As demonstrated, STREAMS provides a mechanism for building multiplexed stream configurations. However, the criteria by which a multiplexer routes data are driver dependent. For example, the protocol multiplexer might use address information found in a protocol header to determine the subnetwork over which data should be routed. You must define its routing criteria.

One routing option available to the multiplexer is to use the muxid value to determine the stream to which data is routed (remember that each multiplexer link has a muxid). I_LINK passes the muxid value to the driver and returns this value to the user. The driver can therefore specify that the muxid value accompany data routed through it. For example, if a multiplexer routed data from a single upper stream to one of several lower streams (as did the IP driver), the multiplexer can require the user to insert the muxid of the desired lower stream into the first four bytes of each message passed to it. The driver can then match the muxid in each message with the muxid of each lower stream, and route the data accordingly.