ddi_cb_register

, ddi_cb_unregister

- register and unregister a device driver callback handler

Synopsis

#include <sys/sunddi.h> 

int ddi_cb_register(dev_info_t *dip, ddi_cb_flags_t flags,
      ddi_cb_func_t cbfunc, void *arg1, void *arg2,
      ddi_cb_handle_t * ret_hdlp);

int ddi_cb_unregister(ddi_cb_handle_t hdl);

Interface Level

Solaris DDI specific (Solaris DDI).

Parameters

ddi_cb_register()

dip

Pointer to the dev_info structure.

flags

Flags to determine which callback events can be handled.

cbfunc

Callback handler function.

arg1

First argument to the callback handler.

arg2

Second (optional) argument to the callback handler.

ret_hdlp

Pointer to return a handle to the registered callback.

ddi_cb_unregister()

hdl

Handle to the registered callback handler that is to be unregistered.

Description

The ddi_cb_register() function installs a callback handler which processes various actions that require the driver's attention while it is attached. The driver specifies which callback actions it can handle through the flags parameter. With each relevant action, the specified callback function passes the arg1 and arg2 arguments along with the description of each callback event to the driver.

The ddi_cb_unregister() function removes a previously installed callback handler and prevents future processing of actions.

The flags parameter consists of the following:

DDI_CB_FLAG_INTR

The device driver participates in interrupt resource management. The device driver may receive additional interrupt resources from the system, but only because it can accept callback notices informing it when it has more or less resources available. Callback notices can occur at anytime after the driver is attached. Interrupt availability varies based on the overall needs of the system.

DDI_CB_FLAG_SRIOV

Indicates to the DDI framework that the device driver is IOV capable. Normally IOV device drivers are expected to call pciv_vf_config(9F)) to configure the VFs during attach and register callbacks with this flag set so that they can be informed through callback notices when VFs are unconfigured by the DDI framework.

If the driver does not explicitly configure the VFs using pciv_vf_config() during attach then the PCIE framework will configure the VFs as part of the post-attach processing if this flag is set.

The callback notices while configuring/unconfiguring is performed with two separate callbacks to the driver PRE and POST. This helps the drivers to prepare for the event during PRE and do the necessary initializations/cleanup during POST notices.

DDI_CB_FLAG_COMM

The device driver uses pciv_send(9F) interfaces to communicate across different domains or within the same domain. The driver may receive callback notices that incoming data has been received.

DDI_CB_FLAG_LSR

The device driver supports Live Suspend and Resume (LSR) operations. When LSR operation (Live Suspend or Live Resume) happens, the device driver will be asked to suspend/resume various I/O activities of the device. The LSR operations may also impose various impacts (such as device reset, losing power, etc.) to the device, the driver needs to handle these impacts too. For more information about the activities and impacts, refer to the data type ddi_cb_lsr_t described below. Callback can occur at anytime after the driver is attached. Currently, LSR is only supported for PCI and PCIe devices.

The cdfunc is a callback handler with the following prototype:

typedef int (*ddi_cb_func_t)(dev_info_t *dip, 
              ddi_cb_action_t action, void *cbarg, 
              void *arg1, void *arg2);

The cbfunc routine with the arguments dip, action, cbarg, arg1 and arg2 is called upon receipt of any callbacks for which the driver is registered. The callback handler returns DDI_SUCCESS if the callback was handled successfully, DDI_ENOTSUP if it received a callback action that it did not know how to process, or DDI_FAILURE if it has an internal failure while processing an action.

For interrupt resource management, the action parameter can be one of the following:

DDI_CB_INTR_ADD

For interrupt resource management, the driver has more available interrupts. The driver can allocate more interrupt vectors and then set up more interrupt handling functions by using ddi_intr_alloc(9F).

DDI_CB_INTR_REMOVE

For interrupt resource management, the driver has fewer available interrupts. The driver must release any previously allocated interrupts in excess of what is now available by using ddi_intr_free(9F).

The cbarg parameter points to an action-specific argument. Each class of registered actions specifies its own data structure that a callback handler should dereference when it receives those actions.

The cbarg parameter is defined as an integer in the case of DDI_CB_INTR_ADD and DDI_CB_INTR_REMOVE actions. The callback handler should cast the cbarg parameter to an integer. The integer represents how many interrupts have been added or removed from the total number available to the device driver.

If a driver participates in interrupt resource management, it must register a callback with the DDI_CB_FLAG_INTR flag. The driver then receives the actions DDI_CB_INTR_ADD and DDI_CB_INTR_REMOVE whenever its interrupt availability has changed. The callback handler should use the interrupt functions ddi_intr_alloc(9F) and ddi_intr_free(9F) functions to respond accordingly. A driver is not required to allocate all interrupts that are available to it, but it is required to manage its allocations so that it never uses more interrupts than are currently available.

DDI_CB_PCIV_CONFIG_VF

The PF driver is being notified of its VF configuration request. The pciv_config_vf_t structure is being passed as cbarg to describe the configuration. The cmd field in pciv_config_vf_t indicates if the VFs are about to or have just been enabled or disabled.

cbarg (when action is set to DDI_CB_PCIV_CONFIG_VF)
   pciv_config_vf_t

typedef enum {
   PCIV_VFCFG_PARAM,       /* Retrieve VF configuration parameters */
   PCIV_VF_ENABLE,         /* Request to enable VFs synchronously */
   PCIV_VF_DISABLE,        /* Request to disable VFs synchronously */
   PCIV_EVT_VFENABLE_PRE,  /* VFs are just about to be enabled */
   PCIV_EVT_VFENABLE_POST, /* VFs have just been enabled */
   PCIV_EVT_VFDISABLE_PRE, /* VFs are just about to be disabled */
   PCIV_EVT_VFDISABLE_POST /* VFs have just been disabled */
} pciv_vf_config_cmd_t;

typedef struct pciv_config_vf {
   int                     version;
   pciv_vf_config_cmd_t    cmd; /* pre/post VF enable/disable */
   uint16_t                num_vf; /* number of VFs to be used */
   uint16_t                first_vf_offset; /* offset between 1st VF & PF */
   uint16_t                vf_stride; /* distance between VFs */
   boolean_t               ari_cap; /* ARI capable hierarchy */
   uint32_t                page_size; /* system page size */
} pciv_config_vf_t;

The cmd field in the pciv_config_vf_t informs the driver the reason for the callback execution. The driver can return one of the following codes back to the caller.

DDI_SUCCESS

The request was accepted and resources are properly configured.

DDI_NOTAPPLICABLE

The requested configuration is not applicable.

DDI_REQRESET

The requested configuration cannot be applied until device is reset (for (example, the PF hardware cannot dynamically adjust its internal resources to satisfy the request.)

DDI_REQREATTACH

The requested configuration cannot be applied unless the driver itself is reattached.

DDI_CB_COMM_RECV

Drivers are notified of incoming data described by pciv_recv_event_t, which is passed as the cbarg argument to the callback handler.

cbarg (when action is set to DDI_CB_COMM_RECV)
   pciv_recv_event_t

typedef enum {
   PCIV_EVT_READY = 0x1,   /* peer side has registered the recv cb */
   PCIV_EVT_NOT_READY,     /* peer side has unregistered the recv cb */
   PCIV_EVT_DRV_DATA,      /* private driver data event */
   PCIV_EVT_FABRIC         /* PCIv framework and fabric admin event */
} pciv_event_type_t;

typedef struct pciv_recv_event {
   pciv_event_type_t event;          /* event type */
   caddr_t           buf;            /* buffer address */
   size_t            nbyte;          /* size of buffer */
   uint32_t          src_func;       /* source function */
   dom_id_t          src_domain;     /* source domain */
} pciv_recv_event_t;

PCIV_EVT_READY

Both local and remote end has registered its event handler. The buf and nbyte fields should be ignored

PCIV_EVT_NOT_READY

Remote end has not registered its event handler. The buf and nbyte fields should be ignored

PCIV_EVT_DRV_DATA

Data in buf with nbyte size has been received or sent. Data can only be interpreted by the transmitter and receiver.

PCIV_EVT_FABRIC

Framework VF administration event, used between framework and PF drivers for VF administration purposes. Administration data is stored in buf with nbyte size.

buf, nbyte

Incoming data is stored in a buffer of address buf with length of nbyte. Not used for certain event types.

src_func

The source function number of the transmission. It is used by the PF driver to identify the transmission source virtual function number. Besides source VF number, it can also be:

PCIV_PF

Transmitter is the PF of the receiver.

PCIV_FRM

Transmitter is the PCI Express framework rather than a driver.

src_domain

The source domain of the transmission. It is used by PF driver to identify the domain the transmission is from.

For Live Suspend & Resume, the action parameter can be one of the following:

DDI_CB_LSR_SUSPEND

Live suspend the device. The detailed I/O activities that will be suspended and impacts that the operation will impose to the device are specified by the argument pointed to by cbarg. It is of type ddi_cb_lsr_t.

DDI_CB_LSR_RESUME

Live resume the device. The detailed activities and impacts are specified in the argument pointed to by cbarg. It will be the same as the previous suspend operation.

DDI_CB_LSR_QUERY_CAPABILITY

Query the device's LSR capabilities. The detailed activities and impacts that the device driver supports are returned in the argument pointed to by cbarg. The type of the argument is ddi_cb_lsr_t.

For all the LSR actions, the cbarg parameter points to the argument of the same type, ddi_cb_lsr_t, which is defined as:

typedef struct ddi_cb_lsr {
    uint64_t    activities;
    uint64_t    impacts;
    char        *reason;
} ddi_cb_lsr_t;

The activities member specifies what I/O activities of the device will be live suspended/resumed (DDI_CB_LSR_SUSPEND and DDI_CB_LSR_RESUME), or the I/O activities that the device driver supports (DDI_CB_LSR_QUERY_CAPABILITY).

For PCI/PCIe devices, the activities member in ddi_cb_lsr_t can be a value bit-ORed from the following:

DDI_CB_LSR_ACT_DMA

DMA activities

DDI_CB_LSR_ACT_PIO

Programmed I/O activities

DDI_CB_LSR_ACT_INTR

Interrupt activities

The impacts member specifies the side effects the LSR operation will impose on the device. For PCI/PCIe devices, it can be bit-ORed value of the following:

DDI_CB_LSR_IMP_DMA_ADDR_CHANGE

During the LSR operation, the device will undergo DMA address changes. The device driver should unbind and rebind the DMA buffers.

DDI_CB_LSR_IMP_DMA_PROP_CHANGE

During the LSR operation, the platform hardware changes may cause the driver's DMA transaction properties to be invalid, such as the lowest and highest DMA address range, DMA address alignment, DMA segment boundary, etc. The driver needs to destroy and reallocate DMA handles.

DDI_CB_LSR_IMP_DEVICE_RESET

During the LSR operation, the device will undergo device reset. The device driver must restore the hardware context during live resume.

DDI_CB_LSR_IMP_DEVICE_REPLACE

During the LSR operation, the device will be replaced. The device driver needs to check the device identity.

DDI_CB_LSR_IMP_LOSE_POWER

During the LSR operation, the device will lose power. The device driver needs to restore device hardware context during live resume. If necessary, the driver should reload firmware.

The reason member is a predefined string to specify the cause of the LSR operation. It is used for printing purposes. There is no predefined string and it may be NULL for DDI_CB_LSR_SUSPEND and DDI_CB_LSR_RESUME. For DDI_CB_LSR_QUERY_CAPABILITY, the driver must set it to NULL.

The driver's implementation should make the LSR operation transparent to user applications, such that when the driver is suspended, it can block user I/O requests (read/write/ioctl) but should not return failure.

The driver may have grouped activities, such as when one is suspended, the other has to be suspended as well. When the driver is asked to suspend one of the grouped activities, it can choose to suspend all the activities in the group.

The driver's callback function will only be asked to suspend after the former suspension has already been resumed.

The callback function should return as soon as possible. In normal cases, returning in milliseconds should be acceptable while returning in seconds is not.

Return Values

The ddi_cb_register() and ddi_cb_unregister() functions return:

DDI_SUCCESS

on success

DDI_EINVAL

An invalid parameter was given when registering a callback handler, or an invalid handle was given when unregistering.

DDI_EALREADY

An attempt was made to register a callback handler while a previous registration still exists.

The cbfunc routine must return:

DDI_SUCCESS

on success

DDI_ENOTSUP

The device does not support the operation

DDI_FAILURE

Implementation specific failure

Context

These functions can be called from kernel, non-interrupt context.

Examples

Example 1 An example using ddi_cb_register().

/*
    * attach(9F) routine.
    *
    * Creates soft state, registers callback handler, initializes
    * hardware, and sets up interrupt handling for the driver.
    */
    xx_attach(dev_info_t *dip, ddi_attach_cmd_t cmd)
    {
        xx_state_t              *statep = NULL;
        xx_intr_t               *intrs = NULL;
        ddi_intr_handle_t       *hdls;
        ddi_cb_handle_t         cb_hdl;
        int                     instance;
        int                     type;
        int                     types;
        int                     nintrs;
        int                     nactual;
        int                     inum;

        /* Get device instance */
        instance = ddi_get_instance(dip);

        switch (cmd) {
        case DDI_ATTACH:

             /* Get soft state */
             if (ddi_soft_state_zalloc(state_list, instance) != 0)
                     return (DDI_FAILURE);
             statep = ddi_get_soft_state(state_list, instance);
             ddi_set_driver_private(dip, (caddr_t)statep);
             statep->dip = dip;

             /* Initialize hardware */
             xx_initialize(statep);

             /* Allocate/bind dma handle, etc. */
             xx_dma_attach(statep);

             /* Initialize Programmed I/O, map register memory, etc. */
             xx_pio_attach(statep);

             /* Initialize LSR state */
             statep->lsr_state = 0;

             /* Register callback handler */
             if (ddi_cb_register(dip, DDI_CB_FLAG_INTR | DDI_CB_FLAG_LSR,
                 xx_cbfunc, statep, NULL, &cb_hdl) != 0) {
                     ddi_soft_state_free(state_list, instance);
                     return (DDI_FAILURE);
             }
             statep->cb_hdl = cb_hdl;

             /* Select interrupt type */
             ddi_intr_get_supported_types(dip, &types);
             if (types & DDI_INTR_TYPE_MSIX) {
                     type = DDI_INTR_TYPE_MSIX;
             } else if (types & DDI_INTR_TYPE_MSI) {
                     type = DDI_INTR_TYPE_MSI;
             } else {
                     type = DDI_INTR_TYPE_FIXED;
             }
             statep->type = type;

             /* Get number of supported interrupts */

             ddi_intr_get_nintrs(dip, type, &nintrs);

             /* Allocate interrupt handle array */
             statep->hdls_size = nintrs * sizeof (ddi_intr_handle_t);
             hdls = kmem_zalloc(statep->hdls_size, KMEM_SLEEP);

             /* Allocate interrupt setup array */
             statep->intrs_size = nintrs * sizeof (xx_intr_t);
             statep->intrs = kmem_zalloc(statep->intrs_size, KMEM_SLEEP);

             /* Allocate interrupt vectors */
             ddi_intr_alloc(dip, hdls, type, 0, nintrs, &nactual, 0);
             statep->nactual = nactual;

             /* Configure interrupt handling */
             xx_setup_interrupts(statep, nactual, statep->intrs);

             /* Install and enable interrupt handlers */
             for (inum = 0; inum < nactual; inum++) {
                     ddi_intr_add_handler(&statep->hdls[inum],
                         statep->intrs[inum].inthandler,
                         statep->intrs[inum].arg1,
                         statep->intrs[inum].arg2);
                     ddi_intr_enable(statep->hdls[inum]);
             }

             break;

        case DDI_RESUME:

                /* Get soft state */
                statep = ddi_get_soft_state(state_list, instance);
                if (statep == NULL)
                        return (DDI_FAILURE);

                /* Resume hardware */
                xx_resume(statep);

                break;
        }

        return (DDI_SUCESS);
    }

    /*
     * detach(9F) routine.
     *
     * Stops the hardware, disables interrupt handling, unregisters
     * a callback handler, and destroys the soft state for the driver.
     */
    xx_detach(dev_info_t *dip, ddi_detach_cmd_t cmd)
    {
        xx_state_t      *statep = NULL;
        int             instance;
        int             inum;


        /* Get device instance */
        instance = ddi_get_instance(dip);

        switch (cmd) {
        case DDI_DETACH:

                /* Get soft state */
                statep = ddi_get_soft_state(state_list, instance);
                if (statep == NULL)
                        return (DDI_FAILURE);

                /* Stop device */
                xx_uninitialize(statep);

                /* Disable and free interrupts */
                for (inum = 0; inum < statep->nactual; inum++) {
                        ddi_intr_disable(statep->hdls[inum]);
                        ddi_intr_remove_handler(statep->hdls[inum]);
                        ddi_intr_free(statep->hdls[inum]);
                }

                /* Clean up DMA, unbind/free dma handle etc. */
                xx_teardown_dma(statep);

                /* Clean up PIO, unmap register memory etc. */
                xx_teardown_pio(statep);

                /* Unregister callback handler */
                ddi_cb_unregister(statep->cb_hdl);

                /* Free interrupt handle array */
                kmem_free(statep->hdls, statep->hdls_size);

                /* Free interrupt setup array */
                kmem_free(statep->intrs, statep->intrs_size);

                /* Free soft state */
                ddi_soft_state_free(state_list, instance);

                break;

        case DDI_SUSPEND:

                /* Get soft state */
                statep = ddi_get_soft_state(state_list, instance);
                if (statep == NULL)
                        return (DDI_FAILURE);

                /* Suspend hardware */
                xx_quiesce(statep);

                break;
        }

        return (DDI_SUCCESS);
    }

    /*
     * (*ddi_cbfunc)() routine.
     *
     * Adapt interrupt usage when availability changes.
     */
    int
    xx_cbfunc(dev_info_t *dip, ddi_cb_action_t cbaction, void *cbarg,
        void *arg1, void *arg2)
    {
        xx_state_t      *statep = (xx_state_t *)arg1;
        int             count;
        int             inum;
        int             nactual;

        switch (cbaction) {
        case DDI_CB_INTR_ADD:
        case DDI_CB_INTR_REMOVE:

             /* Get change in availability */
             count = (int)(uintptr_t)cbarg;

             /* Suspend hardware */
             xx_quiesce(statep);

             /* Tear down previous interrupt handling */
             for (inum = 0; inum < statep->nactual; inum++) {
                     ddi_intr_disable(statep->hdls[inum]);
                     ddi_intr_remove_handler(statep->hdls[inum]);
             }

             /* Adjust interrupt vector allocations */
             if (cbaction == DDI_CB_INTR_ADD) {

                     /* Allocate additional interrupt vectors */
                     ddi_intr_alloc(dip, statep->hdls, statep->type,
                         statep->nactual, count, &nactual, 0);

                     /* Update actual count of available interrupts */
                     statep->nactual += nactual;

             } else {

                     /* Free removed interrupt vectors */
                     for (inum = statep->nactual - count;
                         inum < statep->nactual; inum++) {
                             ddi_intr_free(statep->hdls[inum]);
                     }

                     /* Update actual count of available interrupts */
                     statep->nactual -= count;
             }

             /* Configure interrupt handling */
             xx_setup_interrupts(statep, statep->nactual, statep->intrs);

             /* Install and enable interrupt handlers */
             for (inum = 0; inum < statep->nactual; inum++) {
                     ddi_intr_add_handler(&statep->hdls[inum],
                         statep->intrs[inum].inthandler,
                         statep->intrs[inum].arg1,
                         statep->intrs[inum].arg2);
                     ddi_intr_enable(statep->hdls[inum]);
             }

             /* Resume hardware */
             xx_resume(statep);

             break;

        case DDI_CB_LSR_SUSPEND:

                plsr = (ddi_cb_lsr_t *) cbarg;
                if (plsr->impacts &
                    (DDI_CB_LSR_IMP_LOSE_POWER
                    | DDI_CB_LSR_IMP_DEVICE_REPLACE))
                        return DDI_NOTSUP;

                if (plsr->activities & DDI_CB_LSR_ACT_DMA) {
                        xx_stop_dma_engine(statep);
                        if (plsr->impacts
                            & DDI_CB_LSR_IMP_DMA_ADDR_CHANGE) {
                                xx_unbind_dma_buffer(statep);
                        }
                        if (plsr->impacts
                            & DDI_CB_LSR_IMP_DMA_PROP_CHANGE) {
                                xx_destroy_dma_handle(statep);
                        }
                }

                if (plsr->activities & DDI_CB_LSR_ACT_INTR) {
                        xx_disable_interrupts(statep);
                }

                if (plsr->activities & DDI_CB_LSR_ACT_PIO) {
                        xx_stop_pio(statep);
                }

                if (plsr->impacts & DDI_CB_LSR_IMP_DEVICE_RESET) {
                        /*
                         * may save the context to be restored for re-loaded
                         * firmware on resume
                         */
                        xx_save_hardware_context(statep);
                }

                statep->saved_lsr = *plsr;
                statep->lsr_state = XX_LSR_SUSPENDED;

                break;

        case DDI_CB_LSR_RESUME:

                if (statep->lsr_state != XX_LSR_SUSPENDED)
                        return (DDI_EINVAL);

                plsr = (ddi_cb_lsr_t *) cbarg;

                if (plsr->activities != statep->saved_lsr.activities
                    || plsr->impacts != statep->saved_lsr.impacts)
                        return (DDI_EINVAL);

                if (plsr->impacts & DDI_CB_LSR_IMP_DEVICE_RESET) {
                        xx_restore_hardware_context(statep);
                }

                if (plsr->activities & DDI_CB_LSR_ACT_DMA) {
                        if (plsr->impacts
                            & DDI_CB_LSR_IMP_DMA_PROP_CHANGE) {
                                xx_alloc_dma_handle(statep);
                        }
                        if (plsr->impacts
                            & DDI_CB_LSR_IMP_DMA_ADDR_CHANGE) {
                                xx_bind_dma_buffer(statep);
                        }
                        xx_start_dma_engine(statep);
                }

                if (plsr->activities & DDI_CB_LSR_ACT_INTR) {
                        xx_enable_interrupts(statep);
                }

                if (plsr->activities & DDI_CB_LSR_ACT_PIO) {
                        xx_enable_pio(statep);
                }

                statep->lsr_state = 0;

                break;

        case DDI_CB_LSR_QUERY_CAPABILITY:
                /*
                 * return the supported activities and impacts
                 */
                plsr = (ddi_cb_lsr_t *) cbarg;

                plsr->activities = DDI_CB_LSR_ACT_DMA | DDI_CB_LSR_ACT_PIO
                    | DDI_CB_LSR_ACT_INTR;
                plsr->impacts = DDI_CB_LSR_IMP_DEVICE_RESET
                    | DDI_CB_LSR_IMP_DMA_ADDR_CHANGE
                    | DDI_CB_LSR_IMP_DMA_PROP_CHANGE;

                    break;

     default:
             return (DDI_ENOTSUP);
     }

     return (DDI_SUCCESS);
 }

Attributes

See attributes(5) for descriptions of the following attributes:

See Also

attributes(5), ddi_intr_alloc(9F), ddi_intr_free(9F), ddi_intr_set_nreq(9F), pciv_send(9F), pciv_vf_config(9F)

Notes

Users of these interfaces that register for DDI_CB_FLAG_INTR become participants in interrupt resource management. With that participation comes a responsibility to properly adjust interrupt usage. In the case of a DDI_CB_INTR_ADD action, the system guarantees that a driver can allocate a total number of interrupt resources up to its new number of available interrupts. The total number of interrupt resources is the sum of all resources allocated by the function ddi_intr_alloc(9F), minus all previously released by the function ddi_intr_free(9F). In the case of a DDI_CB_INTR_REMOVE action, the driver might have more interrupts allocated than are now currently available. It is necessary for the driver to release the excess interrupts, or it will have a negative impact on the interrupt availability for other drivers in the system.

A failure to release interrupts in response to a DDI_CB_INTR_REMOVE callback generates the following warning on the system console:

WARNING: <driver><instance>: failed to release interrupts for
        IRM (nintrs = ##, navail=##).

Participation in interrupt resource management ends when a driver uses the ddi_cb_unregister() function to unregister its callback function. The callback function must still operate properly until after the call to the ddi_cb_unregister() function completes. If addinterrupts were given to the driver because of its participation, then a final use of the callback function occurs to release the additional interrupts. The call to the ddi_cb_unregister() function blocks until the final use of the registered callback function is finished.