DMA up to 64 bits (DDI only)

Beginning with DDI 8, drivers can support devices that access memory above 32 bits and up to 64 bits for DMA operations. Three requirements must be met:

• The device hardware must support 64-bit memory access.

• The driver must be written for DDI version 8 or later.

• The driver must be written to support DMA above 32 bits. Specifically, the driver must set the phys_dmasize member of the physreq(D4) structure to the appropriate size for the device, up to 64. All DMA above 32 bits is implemented using 64-bit format. For example, if the device supports 36-bit DMA, set phys_dmasize to 36. Transfers will use the 64-bit format, with bits 0-35 containing valid data and the upper (36-63) bits set to 0.

The driver must not set phys_dmasize to a value above 32 unless the hardware can support larger memory access; this would result in truncated physical addresses which will corrupt system memory. Setting phys_dmasize to 32 when the device can support 64-bit memory does not corrupt the kernel although it does underutilize the hardware.

Two examples help to clarify this implementation. Assume a disk that is connected to a controller that supports up to 32-bit DMA. The DDI 8 HBA driver for this controller must set the phys_dmasize to 32 and set the BA_SCGTH flag in the bcb_addrtypes member of the bcb(D4) structure. When an application reads data from this device, the process is handled differently depending on the processor and its capabilities:

Scenario 1: Running on Pentium processor with 256 MB of RAM

The application allocates a buffer. This buffer can be anywhere in the memory. In this case, the physical addresses of all the buffers will automatically fall within the 28-bit address space. This is less than 32-bit and, assuming that the alignments and such are correct, the same buffer is passed to the HBA driver, the buffer is filled with the data, and returned to the application.

Scenario 2: Running on Xeon processor with 16 GB of RAM

The application allocates a buffer. If, for example, that buffer is at the 15 GB level, the physical addresses of the buffer are represented by 34 bits. This is greater than the 32-bit addresses that are supported by the HBA, so even if the alignments and such are correct, the same buffer cannot be passed down to the HBA driver. Instead, the kernel allocates a new buffer from the lower memory space (or waits until one is available) The HBA driver fills in the new buffer with the data and returns the buffer back to the kernel. The kernel then copies the contents of this buffer into the application's buffer, frees the buffer in lower memory, and passes the original buffer back to the kernel.

If the HBA controller had supported 36-bit DMA, an allocation, a copy, and a free operation would have been avoided, which would result in better performance. and passes that buffer to the HBA driver.