Forking Issues in Process Creation

The default handling of fork() in the Solaris 9 product and earlier Oracle Solaris releases is somewhat different from the way fork() is handled in POSIX threads. For Oracle Solaris releases after Solaris 9, fork() behaves as specified for POSIX threads in all cases.

Table 5-1 compares the differences and similarities of fork() handling in Oracle Solaris threads and pthreads. When the comparable interface is not available either in POSIX threads or in Oracle Solaris threads, the `—' character appears in the table column.

Table 5-1 Comparing POSIX and Oracle Solaris fork() Handling

Fork-One Model

As shown in Table 5-1, the behavior of the pthreads fork(2) function is the same as the behavior of the Oracle Solaris fork1(2) function. Both the pthreads fork(2) function and the Oracle Solaris fork1(2) function create a new process, duplicating the complete address space in the child. However, both functions duplicate only the calling thread in the child process.

Duplication of the calling thread in the child process is useful when the child process immediately calls exec(), which is what happens after most calls to fork(). In this case, the child process does not need a duplicate of any thread other than the thread that called fork().

In the child, do not call any library functions after calling fork() and before calling exec(). One of the library functions might use a lock that was held in the parent at the time of the fork(). The child process may execute only Async-Signal-Safe operations until one of the exec() handlers is called. See Signal Handlers and Async-Signal Safety for more information about Async-Signal-Safe functions.

Fork-One Safety Problem and Solution

Besides the usual concerns such as locking shared data, a library should be well behaved with respect to forking a child process when only the thread that called fork() is running. The problem is that the sole thread in the child process might try to grab a lock held by a thread not duplicated in the child.

Most programs are not likely to encounter this problem. Most programs call exec() in the child right after the return from fork(). However, if the program has to carry out actions in the child before calling exec(), or never calls exec(), then the child could encounter deadlocks. Each library writer should provide a safe solution, although not providing a fork-safe library is not a large concern because this condition is rare.

For example, assume that T1 is in the middle of printing something and holds a lock for printf(), when T2 forks a new process. In the child process, if the sole thread (T2) calls printf(), T2 promptly deadlocks.

The POSIX fork() or the Oracle Solaris fork1() function duplicates only the thread that calls fork() or fork1() . If you call Oracle Solaris forkall() to duplicate all threads, this issue is not a concern.

However, forkall() can cause other problems and should be used with care. For instance, if a thread calls forkall(), the parent thread performing I/O to a file is replicated in the child process. Both copies of the thread will continue performing I/O to the same file, one in the parent and one in the child, leading to malfunctions or file corruption.

To prevent deadlock when calling fork1(), ensure that no locks are being held at the time of forking. The most obvious way to prevent deadlock is to have the forking thread acquire all the locks that could possibly be used by the child. Because you cannot acquire all locks for printf() because printf() is owned by libc, you must ensure that printf() is not being used at fork() time.

To manage the locks in your library, you should perform the following actions:

• Identify all the locks used by the library.

• Identify the locking order for the locks used by the library. If a strict locking order is not used, then lock acquisition must be managed carefully.

• Arrange to acquire all locks at fork time.

In the following example, the list of locks used by the library is { L1,...Ln}. The locking order for these locks is also L1...Ln.

mutex_lock(L1);
mutex_lock(L2);
fork1(...);
mutex_unlock(L1);
mutex_unlock(L2);

When using either Oracle Solaris threads or POSIX threads, you can add a call to pthread_atfork(f1, f2, f3) in your library's .init() section. The f1(), f2(), f3() are defined as follows:

f1() /* This is executed just before the process forks. */
{
 mutex_lock(L1); |
 mutex_lock(...); | -- ordered in lock order
 mutex_lock(Ln); |
 } V

f2() /* This is executed in the child after the process forks. */
 {
 mutex_unlock(L1);
 mutex_unlock(...);
 mutex_unlock(Ln);
 }

f3() /* This is executed in the parent after the process forks. */
 {
 mutex_unlock(L1);
 mutex_unlock(...);
 mutex_unlock(Ln);
 } 

Virtual Forks–vfork

The standard vfork(2) function is unsafe in multithreaded programs. vfork(2) , like fork1(2), copies only the calling thread in the child process. As in nonthreaded implementations, vfork() does not copy the address space for the child process.

Be careful that the thread in the child process does not change memory before the thread calls exec(2). vfork() gives the parent address space to the child. The parent gets its address space back after the child calls exec() or exits. The child must not change the state of the parent.

For example, disastrous problems occur if you create new threads between the call to vfork() and the call to exec().

Solution: pthread_atfork

Use pthread_atfork() to prevent deadlocks whenever you use the fork-one model.

#include <pthread.h>

int pthread_atfork(void (*prepare) (void), void (*
parent) (void),
    void (*child) (void) );

The pthread_atfork() function declares fork() handlers that are called before and after fork() in the context of the thread that called fork().

• The prepare handler is called before fork() starts.

• The parent handler is called after fork() returns in the parent.

• The child handler is called after fork() returns in the child.

Any handler argument can be set to NULL. The order in which successive calls to pthread_atfork() are made is significant.

For example, a prepare handler could acquire all the mutexes needed. Then the parent and child handlers could release the mutexes. The prepare handler acquiring all required mutexes ensures that all relevant locks are held by the thread calling the fork function before the process is forked. This technique prevents a deadlock in the child.

See the pthread_atfork(3C) man page for more information.

Fork-All Model

The Oracle Solaris forkall(2) function duplicates the address space and all the threads in the child. Address space duplication is useful, for example, when the child process never calls exec(2) but does use its copy of the parent address space.

When one thread in a process calls Oracle Solaris forkall(2), threads that are blocked in an interruptible system call will return EINTR.

Be careful not to create locks that are held by both the parent and child processes. Locks held in both parent and child processes occur when locks are allocated in shared memory by calling mmap() with the MAP_SHARED flag. This problem does not occur if the fork-one model is used.

Choosing the Right Fork

Starting with the Oracle Solaris 10 release, a call to fork() is identical to a call to fork1(). Specifically, only the calling thread is replicated in the child process. The behavior is the same as the POSIX fork().

In previous releases of the Oracle Solaris software, the behavior of fork() was dependent on whether the application was linked with the POSIX threads library. When linked with -lthread ( Oracle Solaris threads) but not linked with -lpthread (POSIX threads), fork() was the same as forkall(). When linked with -lpthread, regardless of whether fork() was also linked with -lthread , fork() was the same as fork1().

Starting with the Oracle Solaris 10 release, neither -lthread nor -lpthread is required for multithreaded applications. The -mt option is used to indicate that you are compiling a multithreaded application. The standard C library provides all threading support for both sets of application program interfaces. Applications that require replicate all fork semantics must call forkall().