Examining a kernel stack trace

The trace command prints a symbolic stack traceback for any process or interrupt stack. When used on a postmortem memory image without options, trace prints the kernel stack that was present in the u-area of the user process that was executing at the time of the panic. trace also displays the contents of the processor registers.

You cannot use trace to examine the kernel stack of the current process on an live system. However, you can use it to examine the kernel stack of processes which are sleeping (for example, while waiting for I/O to complete).

Following an interrupt or exception, the kernel creates an additional stack frame on the system stack for the interrupt handler to use. If there was an interrupt outstanding on the crashed system being examined, crash also displays this interrupt stack and its register values.

Note the following when reading trace output:

1. The top two lines give the information from crash proc for this process.

2. The three bottom lines show the contents of the application program registers in the exception frame when the call was entered.

3. The lines above the register contents show the associated frame pointers, the associated function, and the pointer to the function argument(s). Note that trace does not differentiate between argument 0 and argument 1; both are reported as ARG #1.

4. If there is a second set of registers listed in the middle of the output, they are the registers from a second exception frame called during execution. Frequently this happens for an error exception, in which case cmn_err will appear in the NAME column to indicate a kernel error. The list of associated fram pointers are listed above the register contents.

Exception frames look very much like standard C stacks. trace searches for a pattern that could be an exception frame. Occasionally, trace will make a mistake. To verify that this is a real exception frame, verify that the frame pointers (FP) have numbers beginning with 4000, indicating that they are in the user area.

trace run on a postmortem dump displays the interrupt stack only if there were interrupts outstanding when the system crashed. If the interrupt stack was corrupted, using trace -i address enables you to get beyond the bad spot.