Runtime Linking Programming Interface

Dependencies specified during the link-edit of an application are processed by the runtime linker during process initialization. In addition to this mechanism, the application can extend its address space during its execution by binding to additional objects. The application effectively uses the same services of the runtime linker that are used to process the applications standard dependencies.

Delayed object binding has several advantages.

• By processing an object when the object is required rather than during the initialization of an application, startup time can be greatly reduced. If the services provided by an object are not needed during a particular run of the application, the object is not required. This scenario can occur for objects that provide help or debugging information.

• The application can choose between several different objects, depending on the exact services required, such as for a networking protocol.

• Any objects added to the process address space during execution can be freed after use.

An application can use the following typical scenario to access an additional shared object.

• A shared object is located and added to the address space of a running application using dlopen(3C). Any dependencies of this shared object are located and added at this time.

• The added shared object and its dependencies are relocated. Any initialization sections within these objects are called.

• The application locates symbols within the added objects using dlsym(3C). The application can then reference the data or call the functions defined by these new symbols.

• After the application has finished with the objects, the address space can be freed using dlclose(3C). Any termination sections that exist within the objects that are being freed are called at this time.

• Any error conditions that occur as a result of using the runtime linker interface routines can be displayed using dlerror(3C).

The services of the runtime linker are defined in the header file dlfcn.h and are made available to an application by the shared object libc.so.1. In the following example, the file main.c can make reference to any of the dlopen(3C) family of routines, and the application prog can bind to these routines at runtime.

$ cc -o prog main.c

Loading Additional Objects

Additional objects can be added to a running process's address space by using dlopen(3C). This function takes a path name and a binding mode as arguments, and returns a handle to the application. This handle can be used to locate symbols for use by the application using dlsym(3C).

If the path name is specified as a simple file name, one with no `/' in the name, then the runtime linker uses a set of rules to generate an appropriate path name. Path names that contain a `/' are used as provided.

These search path rules are exactly the same as are used to locate any initial dependencies. See Directories Searched by the Runtime Linker. For example, the file main.c contains the following code fragment.

#include        <stdio.h>
#include        <dlfcn.h>
 
int main(int argc, char **argv)
{
        void *handle;
        .....
 
        if ((handle = dlopen("foo.so.1", RTLD_LAZY)) == NULL) {
                (void) printf("dlopen: %s\n", dlerror());
                return (1);
        }
        .....

To locate the shared object foo.so.1, the runtime linker uses any LD_LIBRARY_PATH definition that is present at process initialization. Next, the runtime linker uses any runpath specified during the link-edit of prog. Finally, the runtime linker uses the default locations /lib and /usr/lib for 32–bit objects, or /lib/64 and /usr/lib/64 for 64–bit objects.

If the path name is specified as:

        if ((handle = dlopen("./foo.so.1", RTLD_LAZY)) == NULL) {

then the runtime linker searches for the file only in the current working directory of the process.

If the required object cannot be located, dlopen(3C) returns a NULL handle. In this case dlerror(3C) can be used to display the true reason for the failure. For example.

$ cc -o prog main.c
$ prog
dlopen: ld.so.1: prog: fatal: foo.so.1: open failed: No such \
    file or directory

If the object being added by dlopen(3C) has dependencies on other objects, they too are brought into the process's address space. This process continues until all the dependencies of the specified object are loaded. This dependency tree is referred to as a group.

If the object specified by dlopen(3C), or any of its dependencies, are already part of the process image, then the objects are not processed any further. A valid handle is returned to the application. This mechanism prevents the same object from being loaded more than once, and enables an application to obtain a handle to itself. For example, from the previous example, main.c can contain the following dlopen() call.

        if ((handle = dlopen(0, RTLD_LAZY)) == NULL) {

The handle returned from this dlopen(3C) can be used to locate symbols within the application itself, within any of the dependencies loaded as part of the process's initialization, or within any objects added to the process's address space, using a dlopen(3C) that specified the RTLD_GLOBAL flag.

Relocation Processing

After locating and loading any objects, the runtime linker must process each object and perform any necessary relocations. Any objects that are brought into the process's address space with dlopen(3C) must also be relocated in the same manner.

For simple applications this process is straightforward. However, for users who have more complex applications with many dlopen(3C) calls involving many objects, possibly with common dependencies, this process can be quite important.

Relocations can be categorized according to when they occur. The default behavior of the runtime linker is to process all immediate reference relocations at initialization and all lazy references during process execution, a mechanism commonly referred to as lazy binding.

This same mechanism is applied to any objects added with dlopen(3C) when the mode is defined as RTLD_LAZY. An alternative is to require all relocations of an object to be performed immediately when the object is added. You can use a mode of RTLD_NOW, or record this requirement in the object when it is built using the link-editor's -z now option. This relocation requirement is propagated to any dependencies of the object being opened.

Relocations can also be categorized into non-symbolic and symbolic. The remainder of this section covers issues regarding symbolic relocations, regardless of when these relocations occur, with a focus on some of the subtleties of symbol lookup.

Symbol Lookup

If an object acquired by dlopen(3C) refers to a global symbol, the runtime linker must locate this symbol from the pool of objects that make up the process. In the absence of direct binding, a default symbol search model is applied to objects obtained by dlopen(). However, the mode of a dlopen() together with the attributes of the objects that make up the process, provide for alternative symbol search models.

Objects that required direct binding, although maintaining all the attributes described later, search for symbols directly in the associated dependency. See Chapter 6, Direct Bindings.

By default, objects obtained with dlopen(3C) are assigned world symbol search scope, and local symbol visibility. The section, Default Symbol Lookup Model, uses this default model to illustrate typical object group interactions. The sections Defining a Global Object, Isolating a Group, and Object Hierarchies show examples of using dlopen(3C) modes and file attributes to extend the default symbol lookup model.

Default Symbol Lookup Model

For each object added by a basic dlopen(3C), the runtime linker first looks for the symbol in the dynamic executable. The runtime linker then looks in each of the objects provided during the initialization of the process. If the symbol is still not found, the runtime linker continues the search. The runtime linker next looks in the object acquired through the dlopen(3C) and in any of its dependencies.

The default symbol lookup model provides for transitioning into a lazy loading environment. If a symbol can not be found in the presently loaded objects, any pending lazy loaded objects are processed in an attempt to locate the symbol. This loading compensates for objects that have not fully defined their dependencies. However, this compensation can undermine the advantages of a lazy loading.

In the following example, the dynamic executable prog and the shared object B.so.1 have the following dependencies.

$ ldd prog
        A.so.1 =>        ./A.so.1
$ ldd B.so.1
        C.so.1 =>        ./C.so.1

If prog acquires the shared object B.so.1 by dlopen(3C), then any symbol required to relocate the shared objects B.so.1 and C.so.1 will first be looked for in prog, followed by A.so.1, followed by B.so.1, and finally in C.so.1. In this simple case, think of the shared objects acquired through the dlopen(3C) as if they had been added to the end of the original link-edit of the application. For example, the objects referenced in the previous listing can be expressed diagrammatically as shown in the following figure.

Figure 3-1 A Single dlopen() Request

Any symbol lookup required by the objects acquired from the dlopen(3C), that is shown as shaded blocks, proceeds from the dynamic executable prog through to the final shared object C.so.1.

This symbol lookup is established by the attributes assigned to the objects as they were loaded. Recall that the dynamic executable and all the dependencies loaded with the executable are assigned global symbol visibility, and that the new objects are assigned world symbol search scope. Therefore, the new objects are able to look for symbols in the original objects. The new objects also form a unique group in which each object has local symbol visibility. Therefore, each object within the group can look for symbols within the other group members.

These new objects do not affect the normal symbol lookup required by either the application or the applications initial dependencies. For example, if A.so.1 requires a function relocation after the previous dlopen(3C) has occurred, the runtime linker's normal search for the relocation symbol is to look in prog and then A.so.1. The runtime linker does not follow through and look in B.so.1 or C.so.1.

This symbol lookup is again a result of the attributes assigned to the objects as they were loaded. The world symbol search scope is assigned to the dynamic executable and all the dependencies loaded with it. This scope does not allow them to look for symbols in the new objects that only offer local symbol visibility.

These symbol search and symbol visibility attributes maintain associations between objects. These associations are based on their introduction into the process address space, and on any dependency relationship between the objects. Assigning the objects associated with a given dlopen(3C) to a unique group ensures that only objects associated with the same dlopen(3C) are allowed to look up symbols within themselves and their related dependencies.

This concept of defining associations between objects becomes more clear in applications that carry out more than one dlopen(3C). For example, suppose the shared object D.so.1 has the following dependency.

$ ldd D.so.1
        E.so.1 =>         ./E.so.1

and the prog application used dlopen(3C) to load this shared object in addition to the shared object B.so.1. The following figure illustrates the symbol lookup releationship between the objects.

Figure 3-2 Multiple dlopen() Requests

Suppose that both B.so.1 and D.so.1 contain a definition for the symbol foo, and both C.so.1 and E.so.1 contain a relocation that requires this symbol. Because of the association of objects to a unique group, C.so.1 is bound to the definition in B.so.1, and E.so.1 is bound to the definition in D.so.1. This mechanism is intended to provide the most intuitive binding of objects that are obtained from multiple calls to dlopen(3C).

When objects are used in the scenarios that have so far been described, the order in which each dlopen(3C) occurs has no effect on the resulting symbol binding. However, when objects have common dependencies, the resultant bindings can be affected by the order in which the dlopen(3C) calls are made.

In the following example, the shared objects O.so.1 and P.so.1 have the same common dependency.

$ ldd O.so.1
        Z.so.1 =>        ./Z.so.1
$ ldd P.so.1
        Z.so.1 =>        ./Z.so.1

In this example, the prog application will dlopen(3C) each of these shared objects. Because the shared object Z.so.1 is a common dependency of both O.so.1 and P.so.1, Z.so.1 is assigned to both of the groups that are associated with the two dlopen(3C) calls. This relationship is shown in the following figure.

Figure 3-3 Multiple dlopen() Requests With A Common Dependency

Z.so.1 is available for both O.so.1 and P.so.1 to look up symbols. More importantly, as far as dlopen(3C) ordering is concerned, Z.so.1 is also be able to look up symbols in both O.so.1 and P.so.1.

Therefore, if both O.so.1 and P.so.1 contain a definition for the symbol foo, which is required for a Z.so.1 relocation, the actual binding that occurs is unpredictable because it is affected by the order of the dlopen(3C) calls. If the functionality of symbol foo differs between the two shared objects in which it is defined, the overall outcome of executing code within Z.so.1 might vary depending on the application's dlopen(3C) ordering.

Defining a Global Object

The default assignment of local symbol visibility to the objects obtained by a dlopen(3C) can be promoted to global by augmenting the mode argument with the RTLD_GLOBAL flag. Under this mode, any objects obtained through a dlopen(3C) can be used by any other objects with world symbol search scope to locate symbols.

In addition, any object obtained by dlopen(3C) with the RTLD_GLOBAL flag is available for symbol lookup using dlopen() with a path name whose value is 0.

Isolating a Group

The default assignment of world symbol search scope to the objects obtained by a dlopen(3C) can be reduced to group by augmenting the mode argument with the RTLD_GROUP flag. Under this mode, any objects obtained through a dlopen(3C) will only be allowed to look for symbols within their own group.

Using the link-editor's -B group option, you can assign the group symbol search scope to objects when they are built.

Object Hierarchies

If an initial object is obtained from a dlopen(3C), and uses dlopen() to open a secondary object, both objects are assigned to a unique group. This situation can prevent either object from locating symbols from the other.

In some implementations the initial object has to export symbols for the relocation of the secondary object. This requirement can be satisfied by one of two mechanisms.

• Making the initial object an explicit dependency of the second object.

• Use the RTLD_PARENT mode flag to dlopen(3C) the secondary object.

If the initial object is an explicit dependency of the secondary object, the initial object is assigned to the secondary objects' group. The initial object is therefore able to provide symbols for the secondary objects' relocation.

If many objects can use dlopen(3C) to open the secondary object, and each of these initial objects must export the same symbols to satisfy the secondary objects' relocation, then the secondary object cannot be assigned an explicit dependency. In this case, the dlopen(3C) mode of the secondary object can be augmented with the RTLD_PARENT flag. This flag causes the propagation of the secondary objects' group to the initial object in the same manner as an explicit dependency would do.

There is one small difference between these two techniques. If you specify an explicit dependency, the dependency itself becomes part of the secondary objects' dlopen(3C) dependency tree, and thus becomes available for symbol lookup with dlsym(3C). If you obtain the secondary object with RTLD_PARENT, the initial object does not become available for symbol lookup with dlsym(3C).

When a secondary object is obtained by dlopen(3C) from an initial object with global symbol visibility, the RTLD_PARENT mode is both redundant and harmless. This case commonly occurs when dlopen(3C) is called from an application or from one of the dependencies of the application.

Obtaining New Symbols

A process can obtain the address of a specific symbol using dlsym(3C). This function takes a handle and a symbol name, and returns the address of the symbol to the caller. The handle directs the search for the symbol in the following manner.

• A handle can be returned from a dlopen(3C) of a named object. The handle enables symbols to be obtained from the named object and the objects that define its dependency tree. A handle returned using the mode RTLD_FIRST, enables symbols to be obtained only from the named object.

• A handle can be returned from a dlopen(3C) of a path name whose value is 0. The handle enables symbols to be obtained from the initiating object of the associated link-map and the objects that define its dependency tree. Typically, the initiating object is the dynamic executable. This handle also enables symbols to be obtained from any object obtained by a dlopen(3C) with the RTLD_GLOBAL mode, on the associated link-map. A handle returned using the mode RTLD_FIRST, enables symbols to be obtained only from the initiating object of the associated link-map.

• The special handle RTLD_DEFAULT, and RTLD_PROBE enable symbols to be obtained from the initiating object of the associated link-map and objects that define its dependency tree. This handle also enables symbols to be obtained from any object obtained by a dlopen(3C) that belongs to the same group as the caller. Use of RTLD_DEFAULT, or RTLD_PROBE follows the same model as used to resolve a symbolic relocation from the calling object.

• The special handle RTLD_NEXT enables symbols to be obtained from the next associated object on the callers link-map list.

In the following example, which is probably the most common, an application adds additional objects to its address space. The application then uses dlsym(3C) to locate function or data symbols. The application then uses these symbols to call upon services that are provided in these new objects. The file main.c contains the following code.

#include    <stdio.h>
#include    <dlfcn.h>
 
int main()
{
        void *handle;
        int  *dptr, (*fptr)();
 
        if ((handle = dlopen("foo.so.1", RTLD_LAZY)) == NULL) {
                (void) printf("dlopen: %s\n", dlerror());
                return (1);
        }
 
        if (((fptr = (int (*)())dlsym(handle, "foo")) == NULL) ||
            ((dptr = (int *)dlsym(handle, "bar")) == NULL)) {
                (void) printf("dlsym: %s\n", dlerror());
                return (1);
        }
 
        return ((*fptr)(*dptr));
}

The symbols foo and bar are searched for in the file foo.so.1, followed by any dependencies that are associated with this file. The function foo is then called with the single argument bar as part of the return() statement.

The application prog, built using the previous file main.c, contains the following dependencies.

$ ldd prog
        libc.so.1 =>     /lib/libc.so.1

If the file name specified in the dlopen(3C) had the value 0, the symbols foo and bar are searched for in prog, followed by /lib/libc.so.1.

The handle indicates the root at which to start a symbol search. From this root, the search mechanism follows the same model as described in Relocation Symbol Lookup.

If the required symbol cannot be located, dlsym(3C) returns a NULL value. In this case, dlerror(3C) can be used to indicate the true reason for the failure. In the following example, the application prog is unable to locate the symbol bar.

$ prog
dlsym: ld.so.1: main: fatal: bar: can't find symbol

Testing for Functionality

The special handles RTLD_DEFAULT, and RTLD_PROBE enable an application to test for the existence of a symbol.

The RTLD_DEFAULT handle employes the same rules used by the runtime linker to resolve any symbol reference from the calling object. See Default Symbol Lookup Model. Two aspects of this model should be noted.

• A symbol reference that matches the same symbol reference from the dynamic executable is bound to the procedure linkage table entry associated with the reference from the executable. See Procedure Linkage Table (Processor-Specific). This artifact of dynamic linking ensures that all components within a process see a single address for a function.

• If a symbol definition can not be found to satisfy a non-weak symbol reference within the objects that are presently loaded in the process, a lazy loading fall back is initiated. This fall back iterates through each loaded dynamic object, and loads any pending lazy loadable objects in an attempt to resolve the symbol. This model compensates for objects that have not fully defined their dependencies. However, this compensation can undermine the advantages of lazy loading. Unnecessary objects can be loaded, or an exhaustive loading of all lazy loadable objects can occur should the relocation symbol not be found.

RTLD_PROBE follows a similar model to RTLD_DEFAULT, but differs in the two aspects noted with RTLD_DEFAULT. RTLD_PROBE only binds to explicit symbol definitions, and is not bound to any procedure linkage table entry within the executable. In addition, RTLD_PROBE does not initiate an exhaustive lazy loading fall back. RTLD_PROBE is the most appropriate flag to use to detect the presence of a symbol within an existing process.

RTLD_DEFAULT and RTLD_PROBE can both initiate an explicit lazy load. An object can make reference to a function, and that reference can be established through a lazy loadable dependency. Prior to calling this function, RTLD_DEFAULT or RTLD_PROBE can be used to test for the existence of the function. Because the object makes reference to the function, an attempt is first made to load the associated lazy dependency. The rules for RTLD_DEFAULT and RTLD_PROBE are then followed to bind to the function. In the following example, an RTLD_PROBE call is used both to trigger a lazy load, and to bind to the loaded dependency if the dependency exists.

void foo()
{
    if (dlsym(RTLD_PROBE, "foo1")) {
        foo1(arg1);
        foo2(arg2);
        ....
}

To provide a robust and flexible model for testing for functionally, the associated lazy dependencies should be explicitly tagged as deferred. See Providing an Alternative to dlopen(). This tagging also provides a means of changing the deferred dependency at runtime.

The use of RTLD_DEFAULT or RTLD_PROBE provide a more robust alternative to the use of undefined weak references, as discussed in Weak Symbols.

Using Interposition

The special handle RTLD_NEXT enables an application to locate the next symbol in a symbol scope. For example, the application prog can contain the following code fragment.

        if ((fptr = (int (*)())dlsym(RTLD_NEXT, "foo")) == NULL) {
                (void) printf("dlsym: %s\n", dlerror());
                return (1);
        }
 
        return ((*fptr)());

In this case, foo is searched for in the shared objects associated with prog, which in this case is /lib/libc.so.1. If this code fragment was contained in the file B.so.1 from the example that is shown in Figure 3-1, then foo is searched for in C.so.1 only.

Use of RTLD_NEXT provides a means to exploit symbol interposition. For example, a function within an object can be interposed upon by a preceding object, which can then augment the processing of the original function. For example, the following code fragment can be placed in the shared object malloc.so.1.

#include    <sys/types.h>
#include    <dlfcn.h>
#include    <stdio.h>
 
void *
malloc(size_t size)
{
        static void *(*fptr)() = 0;
        char        buffer[50];
 
        if (fptr == 0) {
                fptr = (void *(*)())dlsym(RTLD_NEXT, "malloc");
                if (fptr == NULL) {
                        (void) printf("dlopen: %s\n", dlerror());
                        return (NULL);
                }
        }
 
        (void) sprintf(buffer, "malloc: %#x bytes\n", size);
        (void) write(1, buffer, strlen(buffer));
        return ((*fptr)(size));
}

malloc.so.1 can be interposed before the system library /lib/libc.so.1 where malloc(3C) usually resides. Any calls to malloc() are now interposed upon before the original function is called to complete the allocation.

$ cc -o malloc.so.1 -G -K pic malloc.c
$ cc -o prog file1.o file2.o ..... -R. malloc.so.1
$ prog
malloc: 0x32 bytes
malloc: 0x14 bytes
..........

Alternatively, the same interposition can be achieved using the following commands.

$ cc -o malloc.so.1 -G -K pic malloc.c
$ cc -o prog main.c
$ LD_PRELOAD=./malloc.so.1 prog
malloc: 0x32 bytes
malloc: 0x14 bytes
..........

The use of RTLD_NEXT within a dynamic executable or preloaded object, provides a predictable interposition technique. Be careful when using this technique in a generic object dependency, as the actual load order of objects is not always predictable.