pid Provider

The pid provider enables you to trace any instruction in a process. Unlike most other providers, pid probes are created on demand based on the probe descriptions found in your D programs. As a result, no pid probes are listed in the output of dtrace -l until you have enabled them yourself.

User Function Boundary Tracing

The simplest mode of operation for the pid provider is as the user space analogue to the fbt provider. The following example program traces all function entries and returns that are made from a single function. The $1 macro variable (the first operand on the command line) is the process ID for the process to trace. The $2 macro variable (the second operand on the command line) is the name of the function from which to trace all function calls.

Example 12-1 userfunc.d: Trace User Function Entry and Return

#!/usr/sbin/dtrace -s

#pragma D option flowindent

pid$1::$2:entry
{
        self->trace = 1;
}

pid$1::$2:return
/self->trace/
{
        self->trace = 0;
}

pid$1:::entry,
pid$1:::return
/self->trace/
{
}

Type in the above example script and save it in a file named userfunc.d, and then chmod it to be executable. This script produces output similar to the following example:

# ./userfunc.d 15032 execute
dtrace: script './userfunc.d' matched 11594 probes
  0  -> execute                               
  0    -> execute                             
  0      -> Dfix                              
  0      <- Dfix                              
  0      -> s_strsave                         
  0        -> malloc                          
  0        <- malloc                          
  0      <- s_strsave                         
  0      -> set                               
  0        -> malloc                          
  0        <- malloc                          
  0      <- set                               
  0      -> set1                              
  0        -> tglob                           
  0        <- tglob                           
  0      <- set1                              
  0      -> setq                              
  0        -> s_strcmp                        
  0        <- s_strcmp                        
...

The pid provider can only be used on processes that are already running. You can use the $target macro variable (see Chapter 9, Scripting) and the dtrace -c and -p options to create and grab processes of interest and instrument them using DTrace. For example, the following D script can be used to determine the distribution of function calls made to libc by a particular subject process:

pid$target:libc.so::entry
{
        @[probefunc] = count();
}

To determine the distribution of such calls made by the date(1) command, save the script in a file named libc.d and execute the following command:

# dtrace -s libc.d -c date
dtrace: script 'libc.d' matched 2476 probes
Fri Jul 30 14:08:54 PDT 2004
dtrace: pid 109196 has exited

  pthread_rwlock_unlock                                             1
  _fflush_u                                                         1
  rwlock_lock                                                       1
  rw_write_held                                                     1
  strftime                                                          1
  _close                                                            1
  _read                                                             1
  __open                                                            1
  _open                                                             1
  strstr                                                            1
  load_zoneinfo                                                     1

...
  _ti_bind_guard                                                   47
  _ti_bind_clear                                                   94

Tracing Arbitrary Instructions

You can use the pid provider to trace any instruction in any user function. Upon demand, the pid provider will create a probe for every instruction in a function. The name of each probe is the offset of its corresponding instruction in the function expressed as a hexadecimal integer. For example, to enable a probe associated with the instruction at offset 0x1c in function foo of module bar.so in the process with PID 123, you can use the following command:

# dtrace -n pid123:bar.so:foo:1c

To enable all of the probes in the function foo, including the probe for each instruction, you can use the command:

# dtrace -n pid123:bar.so:foo:

This command demonstrates an extremely powerful technique for debugging and analyzing user applications. Infrequent errors can be difficult to debug because they can be difficult to reproduce. Often, you can identify a problem after the failure has occurred, too late to reconstruct the code path. The following example demonstrates how to combine the pid provider with speculative tracing (see Chapter 7, Speculative Tracing) to solve this problem by tracing every instruction in a function.

Example 12-2 errorpath.d: Trace User Function Call Error Path

pid$1::$2:entry
{
        self->spec = speculation();
        speculate(self->spec);
        printf("%x %x %x %x %x", arg0, arg1, arg2, arg3, arg4);
}

pid$1::$2:
/self->spec/
{
        speculate(self->spec);
}

pid$1::$2:return
/self->spec && arg1 == 0/
{
        discard(self->spec);
        self->spec = 0;
}

pid$1::$2:return
/self->spec && arg1 != 0/
{
        commit(self->spec);
        self->spec = 0;
}

Executing errorpath.d results in output similar to the following example:

# ./errorpath.d 100461 _chdir
dtrace: script './errorpath.d' matched 19 probes
CPU     ID                    FUNCTION:NAME
  0  25253                     _chdir:entry 81e08 6d140 ffbfcb20 656c73 0
  0  25253                     _chdir:entry
  0  25269                         _chdir:0
  0  25270                         _chdir:4
  0  25271                         _chdir:8
  0  25272                         _chdir:c
  0  25273                        _chdir:10
  0  25274                        _chdir:14
  0  25275                        _chdir:18
  0  25276                        _chdir:1c
  0  25277                        _chdir:20
  0  25278                        _chdir:24
  0  25279                        _chdir:28
  0  25280                        _chdir:2c
  0  25268                    _chdir:return