PCI Power Management — The Linux Kernel documentation (2024)

1.1. Native and Platform-Based Power Management¶

In general, power management is a feature allowing one to save energy by puttingdevices into states in which they draw less power (low-power states) at theprice of reduced functionality or performance.

Usually, a device is put into a low-power state when it is underutilized orcompletely inactive. However, when it is necessary to use the device onceagain, it has to be put back into the “fully functional” state (full-powerstate). This may happen when there are some data for the device to handle oras a result of an external event requiring the device to be active, which maybe signaled by the device itself.

PCI devices may be put into low-power states in two ways, by using the devicecapabilities introduced by the PCI Bus Power Management Interface Specification,or with the help of platform firmware, such as an ACPI BIOS. In the firstapproach, that is referred to as the native PCI power management (native PCI PM)in what follows, the device power state is changed as a result of writing aspecific value into one of its standard configuration registers. The secondapproach requires the platform firmware to provide special methods that may beused by the kernel to change the device’s power state.

Devices supporting the native PCI PM usually can generate wakeup signals calledPower Management Events (PMEs) to let the kernel know about external eventsrequiring the device to be active. After receiving a PME the kernel is supposedto put the device that sent it into the full-power state. However, the PCI BusPower Management Interface Specification doesn’t define any standard method ofdelivering the PME from the device to the CPU and the operating system kernel.It is assumed that the platform firmware will perform this task and therefore,even though a PCI device is set up to generate PMEs, it also may be necessary toprepare the platform firmware for notifying the CPU of the PMEs coming from thedevice (e.g. by generating interrupts).

In turn, if the methods provided by the platform firmware are used for changingthe power state of a device, usually the platform also provides a method forpreparing the device to generate wakeup signals. In that case, however, itoften also is necessary to prepare the device for generating PMEs using thenative PCI PM mechanism, because the method provided by the platform depends onthat.

Thus in many situations both the native and the platform-based power managementmechanisms have to be used simultaneously to obtain the desired result.

1.2. Native PCI Power Management¶

The PCI Bus Power Management Interface Specification (PCI PM Spec) wasintroduced between the PCI 2.1 and PCI 2.2 Specifications. It defined astandard interface for performing various operations related to powermanagement.

The implementation of the PCI PM Spec is optional for conventional PCI devices,but it is mandatory for PCI Express devices. If a device supports the PCI PMSpec, it has an 8 byte power management capability field in its PCIconfiguration space. This field is used to describe and control the standardfeatures related to the native PCI power management.

The PCI PM Spec defines 4 operating states for devices (D0-D3) and for buses(B0-B3). The higher the number, the less power is drawn by the device or busin that state. However, the higher the number, the longer the latency forthe device or bus to return to the full-power state (D0 or B0, respectively).

There are two variants of the D3 state defined by the specification. The firstone is D3hot, referred to as the software accessible D3, because devices can beprogrammed to go into it. The second one, D3cold, is the state that PCI devicesare in when the supply voltage (Vcc) is removed from them. It is not possibleto program a PCI device to go into D3cold, although there may be a programmableinterface for putting the bus the device is on into a state in which Vcc isremoved from all devices on the bus.

PCI bus power management, however, is not supported by the Linux kernel at thetime of this writing and therefore it is not covered by this document.

Note that every PCI device can be in the full-power state (D0) or in D3cold,regardless of whether or not it implements the PCI PM Spec. In addition tothat, if the PCI PM Spec is implemented by the device, it must support D3hotas well as D0. The support for the D1 and D2 power states is optional.

PCI devices supporting the PCI PM Spec can be programmed to go to any of thesupported low-power states (except for D3cold). While in D1-D3hot thestandard configuration registers of the device must be accessible to software(i.e. the device is required to respond to PCI configuration accesses), althoughits I/O and memory spaces are then disabled. This allows the device to beprogrammatically put into D0. Thus the kernel can switch the device back andforth between D0 and the supported low-power states (except for D3cold) and thepossible power state transitions the device can undergo are the following:

The transition from D3cold to D0 occurs when the supply voltage is provided tothe device (i.e. power is restored). In that case the device returns to D0 witha full power-on reset sequence and the power-on defaults are restored to thedevice by hardware just as at initial power up.

PCI devices supporting the PCI PM Spec can be programmed to generate PMEswhile in any power state (D0-D3), but they are not required to be capableof generating PMEs from all supported power states. In particular, thecapability of generating PMEs from D3cold is optional and depends on thepresence of additional voltage (3.3Vaux) allowing the device to remainsufficiently active to generate a wakeup signal.

1.3. ACPI Device Power Management¶

The platform firmware support for the power management of PCI devices issystem-specific. However, if the system in question is compliant with theAdvanced Configuration and Power Interface (ACPI) Specification, like themajority of x86-based systems, it is supposed to implement device powermanagement interfaces defined by the ACPI standard.

For this purpose the ACPI BIOS provides special functions called “controlmethods” that may be executed by the kernel to perform specific tasks, such asputting a device into a low-power state. These control methods are encodedusing special byte-code language called the ACPI Machine Language (AML) andstored in the machine’s BIOS. The kernel loads them from the BIOS and executesthem as needed using an AML interpreter that translates the AML byte code intocomputations and memory or I/O space accesses. This way, in theory, a BIOSwriter can provide the kernel with a means to perform actions dependingon the system design in a system-specific fashion.

ACPI control methods may be divided into global control methods, that are notassociated with any particular devices, and device control methods, that haveto be defined separately for each device supposed to be handled with the help ofthe platform. This means, in particular, that ACPI device control methods canonly be used to handle devices that the BIOS writer knew about in advance. TheACPI methods used for device power management fall into that category.

The ACPI specification assumes that devices can be in one of four power stateslabeled as D0, D1, D2, and D3 that roughly correspond to the native PCI PMD0-D3 states (although the difference between D3hot and D3cold is not takeninto account by ACPI). Moreover, for each power state of a device there is aset of power resources that have to be enabled for the device to be put intothat state. These power resources are controlled (i.e. enabled or disabled)with the help of their own control methods, _ON and _OFF, that have to bedefined individually for each of them.

To put a device into the ACPI power state Dx (where x is a number between 0 and3 inclusive) the kernel is supposed to (1) enable the power resources requiredby the device in this state using their _ON control methods and (2) execute the_PSx control method defined for the device. In addition to that, if the deviceis going to be put into a low-power state (D1-D3) and is supposed to generatewakeup signals from that state, the _DSW (or _PSW, replaced with _DSW by ACPI3.0) control method defined for it has to be executed before _PSx. Powerresources that are not required by the device in the target power state and arenot required any more by any other device should be disabled (by executing their_OFF control methods). If the current power state of the device is D3, it canonly be put into D0 this way.

However, quite often the power states of devices are changed during asystem-wide transition into a sleep state or back into the working state. ACPIdefines four system sleep states, S1, S2, S3, and S4, and denotes the systemworking state as S0. In general, the target system sleep (or working) statedetermines the highest power (lowest number) state the device can be putinto and the kernel is supposed to obtain this information by executing thedevice’s _SxD control method (where x is a number between 0 and 4 inclusive).If the device is required to wake up the system from the target sleep state, thelowest power (highest number) state it can be put into is also determined by thetarget state of the system. The kernel is then supposed to use the device’s_SxW control method to obtain the number of that state. It also is supposed touse the device’s _PRW control method to learn which power resources need to beenabled for the device to be able to generate wakeup signals.

1.4. Wakeup Signaling¶

Wakeup signals generated by PCI devices, either as native PCI PMEs, or asa result of the execution of the _DSW (or _PSW) ACPI control method beforeputting the device into a low-power state, have to be caught and handled asappropriate. If they are sent while the system is in the working state(ACPI S0), they should be translated into interrupts so that the kernel canput the devices generating them into the full-power state and take care of theevents that triggered them. In turn, if they are sent while the system issleeping, they should cause the system’s core logic to trigger wakeup.

On ACPI-based systems wakeup signals sent by conventional PCI devices areconverted into ACPI General-Purpose Events (GPEs) which are hardware signalsfrom the system core logic generated in response to various events that need tobe acted upon. Every GPE is associated with one or more sources of potentiallyinteresting events. In particular, a GPE may be associated with a PCI devicecapable of signaling wakeup. The information on the connections between GPEsand event sources is recorded in the system’s ACPI BIOS from where it can beread by the kernel.

If a PCI device known to the system’s ACPI BIOS signals wakeup, the GPEassociated with it (if there is one) is triggered. The GPEs associated with PCIbridges may also be triggered in response to a wakeup signal from one of thedevices below the bridge (this also is the case for root bridges) and, forexample, native PCI PMEs from devices unknown to the system’s ACPI BIOS may behandled this way.

A GPE may be triggered when the system is sleeping (i.e. when it is in one ofthe ACPI S1-S4 states), in which case system wakeup is started by its core logic(the device that was the source of the signal causing the system wakeup to occurmay be identified later). The GPEs used in such situations are referred to aswakeup GPEs.

Usually, however, GPEs are also triggered when the system is in the workingstate (ACPI S0) and in that case the system’s core logic generates a SystemControl Interrupt (SCI) to notify the kernel of the event. Then, the SCIhandler identifies the GPE that caused the interrupt to be generated which,in turn, allows the kernel to identify the source of the event (that may bea PCI device signaling wakeup). The GPEs used for notifying the kernel ofevents occurring while the system is in the working state are referred to asruntime GPEs.

Unfortunately, there is no standard way of handling wakeup signals sent byconventional PCI devices on systems that are not ACPI-based, but there is onefor PCI Express devices. Namely, the PCI Express Base Specification introduceda native mechanism for converting native PCI PMEs into interrupts generated byroot ports. For conventional PCI devices native PMEs are out-of-band, so theyare routed separately and they need not pass through bridges (in principle theymay be routed directly to the system’s core logic), but for PCI Express devicesthey are in-band messages that have to pass through the PCI Express hierarchy,including the root port on the path from the device to the Root Complex. Thusit was possible to introduce a mechanism by which a root port generates aninterrupt whenever it receives a PME message from one of the devices below it.The PCI Express Requester ID of the device that sent the PME message is thenrecorded in one of the root port’s configuration registers from where it may beread by the interrupt handler allowing the device to be identified. [PMEmessages sent by PCI Express endpoints integrated with the Root Complex don’tpass through root ports, but instead they cause a Root Complex Event Collector(if there is one) to generate interrupts.]

In principle the native PCI Express PME signaling may also be used on ACPI-basedsystems along with the GPEs, but to use it the kernel has to ask the system’sACPI BIOS to release control of root port configuration registers. The ACPIBIOS, however, is not required to allow the kernel to control these registersand if it doesn’t do that, the kernel must not modify their contents. Of coursethe native PCI Express PME signaling cannot be used by the kernel in that case.

2.1. Device Power Management Callbacks¶

The PCI Subsystem participates in the power management of PCI devices in anumber of ways. First of all, it provides an intermediate code layer betweenthe device power management core (PM core) and PCI device drivers.Specifically, the pm field of the PCI subsystem’s struct bus_type object,pci_bus_type, points to a struct dev_pm_ops object, pci_dev_pm_ops, containingpointers to several device power management callbacks:

const struct dev_pm_ops pci_dev_pm_ops = { .prepare = pci_pm_prepare, .complete = pci_pm_complete, .suspend = pci_pm_suspend, .resume = pci_pm_resume, .freeze = pci_pm_freeze, .thaw = pci_pm_thaw, .poweroff = pci_pm_poweroff, .restore = pci_pm_restore, .suspend_noirq = pci_pm_suspend_noirq, .resume_noirq = pci_pm_resume_noirq, .freeze_noirq = pci_pm_freeze_noirq, .thaw_noirq = pci_pm_thaw_noirq, .poweroff_noirq = pci_pm_poweroff_noirq, .restore_noirq = pci_pm_restore_noirq, .runtime_suspend = pci_pm_runtime_suspend, .runtime_resume = pci_pm_runtime_resume, .runtime_idle = pci_pm_runtime_idle,};

These callbacks are executed by the PM core in various situations related todevice power management and they, in turn, execute power management callbacksprovided by PCI device drivers. They also perform power management operationsinvolving some standard configuration registers of PCI devices that devicedrivers need not know or care about.

The structure representing a PCI device, struct pci_dev, contains several fieldsthat these callbacks operate on:

struct pci_dev { ... pci_power_t current_state; /* Current operating state. */ int pm_cap; /* PM capability offset in the configuration space */ unsigned int pme_support:5; /* Bitmask of states from which PME# can be generated */ unsigned int pme_poll:1; /* Poll device's PME status bit */ unsigned int d1_support:1; /* Low power state D1 is supported */ unsigned int d2_support:1; /* Low power state D2 is supported */ unsigned int no_d1d2:1; /* D1 and D2 are forbidden */ unsigned int wakeup_prepared:1; /* Device prepared for wake up */ unsigned int d3hot_delay; /* D3hot->D0 transition time in ms */ ...};

They also indirectly use some fields of the struct device that is embedded instruct pci_dev.

2.2. Device Initialization¶

The PCI subsystem’s first task related to device power management is toprepare the device for power management and initialize the fields of structpci_dev used for this purpose. This happens in two functions defined indrivers/pci/, pci_pm_init() and pci_acpi_setup().

The first of these functions checks if the device supports native PCI PMand if that’s the case the offset of its power management capability structurein the configuration space is stored in the pm_cap field of the device’s structpci_dev object. Next, the function checks which PCI low-power states aresupported by the device and from which low-power states the device can generatenative PCI PMEs. The power management fields of the device’s struct pci_dev andthe struct device embedded in it are updated accordingly and the generation ofPMEs by the device is disabled.

The second function checks if the device can be prepared to signal wakeup withthe help of the platform firmware, such as the ACPI BIOS. If that is the case,the function updates the wakeup fields in struct device embedded in thedevice’s struct pci_dev and uses the firmware-provided method to prevent thedevice from signaling wakeup.

At this point the device is ready for power management. For driverless devices,however, this functionality is limited to a few basic operations carried outduring system-wide transitions to a sleep state and back to the working state.

2.3. Runtime Device Power Management¶

The PCI subsystem plays a vital role in the runtime power management of PCIdevices. For this purpose it uses the general runtime power management(runtime PM) framework described in Runtime Power Management Framework for I/O Devices.Namely, it provides subsystem-level callbacks:

pci_pm_runtime_suspend()pci_pm_runtime_resume()pci_pm_runtime_idle()

that are executed by the core runtime PM routines. It also implements theentire mechanics necessary for handling runtime wakeup signals from PCI devicesin low-power states, which at the time of this writing works for both the nativePCI Express PME signaling and the ACPI GPE-based wakeup signaling described inSection 1.

First, a PCI device is put into a low-power state, or suspended, with the helpof pm_schedule_suspend() or pm_runtime_suspend() which for PCI devices callpci_pm_runtime_suspend() to do the actual job. For this to work, the device’sdriver has to provide a pm->runtime_suspend() callback (see below), which isrun by pci_pm_runtime_suspend() as the first action. If the driver’s callbackreturns successfully, the device’s standard configuration registers are saved,the device is prepared to generate wakeup signals and, finally, it is put intothe target low-power state.

The low-power state to put the device into is the lowest-power (highest number)state from which it can signal wakeup. The exact method of signaling wakeup issystem-dependent and is determined by the PCI subsystem on the basis of thereported capabilities of the device and the platform firmware. To prepare thedevice for signaling wakeup and put it into the selected low-power state, thePCI subsystem can use the platform firmware as well as the device’s native PCIPM capabilities, if supported.

It is expected that the device driver’s pm->runtime_suspend() callback willnot attempt to prepare the device for signaling wakeup or to put it into alow-power state. The driver ought to leave these tasks to the PCI subsystemthat has all of the information necessary to perform them.

A suspended device is brought back into the “active” state, or resumed,with the help of pm_request_resume() or pm_runtime_resume() which both callpci_pm_runtime_resume() for PCI devices. Again, this only works if the device’sdriver provides a pm->runtime_resume() callback (see below). However, beforethe driver’s callback is executed, pci_pm_runtime_resume() brings the deviceback into the full-power state, prevents it from signaling wakeup while in thatstate and restores its standard configuration registers. Thus the driver’scallback need not worry about the PCI-specific aspects of the device resume.

Note that generally pci_pm_runtime_resume() may be called in two differentsituations. First, it may be called at the request of the device’s driver, forexample if there are some data for it to process. Second, it may be calledas a result of a wakeup signal from the device itself (this sometimes isreferred to as “remote wakeup”). Of course, for this purpose the wakeup signalis handled in one of the ways described in Section 1 and finally converted intoa notification for the PCI subsystem after the source device has beenidentified.

The pci_pm_runtime_idle() function, called for PCI devices by pm_runtime_idle()and pm_request_idle(), executes the device driver’s pm->runtime_idle()callback, if defined, and if that callback doesn’t return error code (or is notpresent at all), suspends the device with the help of pm_runtime_suspend().Sometimes pci_pm_runtime_idle() is called automatically by the PM core (forexample, it is called right after the device has just been resumed), in whichcases it is expected to suspend the device if that makes sense. Usually,however, the PCI subsystem doesn’t really know if the device really can besuspended, so it lets the device’s driver decide by running itspm->runtime_idle() callback.

2.4. System-Wide Power Transitions¶

There are a few different types of system-wide power transitions, described inDevice Power Management Basics. Each of them requires devices to behandled in a specific way and the PM core executes subsystem-level powermanagement callbacks for this purpose. They are executed in phases such thateach phase involves executing the same subsystem-level callback for every devicebelonging to the given subsystem before the next phase begins. These phasesalways run after tasks have been frozen.

pci_pm_prepare()pci_pm_suspend()pci_pm_suspend_noirq()

pci_pm_resume_noirq()pci_pm_resume()pci_pm_complete()

pci_pm_prepare()pci_pm_freeze()pci_pm_freeze_noirq()

pci_pm_thaw_noirq()pci_pm_thaw()pci_pm_complete()

pci_pm_poweroff()pci_pm_poweroff_noirq()

pci_pm_restore_noirq()pci_pm_restore()

3.1. Power Management Callbacks¶

PCI device drivers participate in power management by providing callbacks to beexecuted by the PCI subsystem’s power management routines described above and bycontrolling the runtime power management of their devices.

At the time of this writing there are two ways to define power managementcallbacks for a PCI device driver, the recommended one, based on using adev_pm_ops structure described in Device Power Management Basics, andthe “legacy” one, in which the .suspend() and .resume() callbacks from structpci_driver are used. The legacy approach, however, doesn’t allow one to defineruntime power management callbacks and is not really suitable for any newdrivers. Therefore it is not covered by this document (refer to the source codeto learn more about it).

It is recommended that all PCI device drivers define a struct dev_pm_ops objectcontaining pointers to power management (PM) callbacks that will be executed bythe PCI subsystem’s PM routines in various circ*mstances. A pointer to thedriver’s struct dev_pm_ops object has to be assigned to the driver.pm field inits struct pci_driver object. Once that has happened, the “legacy” PM callbacksin struct pci_driver are ignored (even if they are not NULL).

The PM callbacks in struct dev_pm_ops are not mandatory and if they are notdefined (i.e. the respective fields of struct dev_pm_ops are unset) the PCIsubsystem will handle the device in a simplified default manner. If they aredefined, though, they are expected to behave as described in the followingsubsections.

3.2. Device Runtime Power Management¶

In addition to providing device power management callbacks PCI device driversare responsible for controlling the runtime power management (runtime PM) oftheir devices.

The PCI device runtime PM is optional, but it is recommended that PCI devicedrivers implement it at least in the cases where there is a reliable way ofverifying that the device is not used (like when the network cable is detachedfrom an Ethernet adapter or there are no devices attached to a USB controller).

To support the PCI runtime PM the driver first needs to implement theruntime_suspend() and runtime_resume() callbacks. It also may need to implementthe runtime_idle() callback to prevent the device from being suspended againevery time right after the runtime_resume() callback has returned(alternatively, the runtime_suspend() callback will have to check if thedevice should really be suspended and return -EAGAIN if that is not the case).

The runtime PM of PCI devices is enabled by default by the PCI core. PCIdevice drivers do not need to enable it and should not attempt to do so.However, it is blocked by pci_pm_init() that runs the pm_runtime_forbid()helper function. In addition to that, the runtime PM usage counter ofeach PCI device is incremented by local_pci_probe() before executing theprobe callback provided by the device’s driver.

If a PCI driver implements the runtime PM callbacks and intends to use theruntime PM framework provided by the PM core and the PCI subsystem, it needsto decrement the device’s runtime PM usage counter in its probe callbackfunction. If it doesn’t do that, the counter will always be different fromzero for the device and it will never be runtime-suspended. The simplestway to do that is by calling pm_runtime_put_noidle(), but if the driverwants to schedule an autosuspend right away, for example, it may callpm_runtime_put_autosuspend() instead for this purpose. Generally, itjust needs to call a function that decrements the devices usage counterfrom its probe routine to make runtime PM work for the device.

It is important to remember that the driver’s runtime_suspend() callbackmay be executed right after the usage counter has been decremented, becauseuser space may already have caused the pm_runtime_allow() helper functionunblocking the runtime PM of the device to run via sysfs, so the driver mustbe prepared to cope with that.

The driver itself should not call pm_runtime_allow(), though. Instead, itshould let user space or some platform-specific code do that (user space cando it via sysfs as stated above), but it must be prepared to handle theruntime PM of the device correctly as soon as pm_runtime_allow() is called(which may happen at any time, even before the driver is loaded).

When the driver’s remove callback runs, it has to balance the decrementationof the device’s runtime PM usage counter at the probe time. For this reason,if it has decremented the counter in its probe callback, it must runpm_runtime_get_noresume() in its remove callback. [Since the core carriesout a runtime resume of the device and bumps up the device’s usage counterbefore running the driver’s remove callback, the runtime PM of the deviceis effectively disabled for the duration of the remove execution and allruntime PM helper functions incrementing the device’s usage counter arethen effectively equivalent to pm_runtime_get_noresume().]

The runtime PM framework works by processing requests to suspend or resumedevices, or to check if they are idle (in which cases it is reasonable tosubsequently request that they be suspended). These requests are representedby work items put into the power management workqueue, pm_wq. Although thereare a few situations in which power management requests are automaticallyqueued by the PM core (for example, after processing a request to resume adevice the PM core automatically queues a request to check if the device isidle), device drivers are generally responsible for queuing power managementrequests for their devices. For this purpose they should use the runtime PMhelper functions provided by the PM core, discussed inRuntime Power Management Framework for I/O Devices.

Devices can also be suspended and resumed synchronously, without placing arequest into pm_wq. In the majority of cases this also is done by theirdrivers that use helper functions provided by the PM core for this purpose.

For more information on the runtime PM of devices refer toRuntime Power Management Framework for I/O Devices.