KDB
is a kernel debugger that works like a Reverse
Polish Notation (RPN) calculator.
KDB can set breakpoints, display kernel
stack traces and various kernel structures, and modify
the contents of memory, I/O, and registers.
The debugger supports basic arithmetic operations, conditional
execution, variables, and macros.
KDB does conversions from a kernel symbol name to its
virtual address, from a virtual address to the value
at that address, and
from a virtual address to the name of the nearest kernel symbol.
You have a choice of different numeric bases,
address spaces, and operand sizes.
This is an advanced tool, only for those who are thoroughly
familiar with the UNIX kernel.
Because UNIX systems differ, you could damage your
system by following some of the examples in this discussion.
You can invoke the debugger by using the kdb command
or the sysi86(SI86TODEMON) system call
on all systems,
<Ctrl><Alt>D (from the
console only) on an AT-bus system, or the interrupt
character (from the console only) on a Multibus system.
In addition,
KDB is entered automatically under various conditions,
such as panics and breakpoint traps.
Any time the kdb>> prompt
appears, you are in the debugger.
I/O is done via the console (kd), or a serial terminal.
To exit the debugger, type q or <Ctrl>D.
When you exit and re-enter the debugger, its
state is preserved, including the contents of the value stack.
Using kdb as a calculator
KDB operates as an RPN calculator, similar to
dc(C).
This calculator has a 32-level value stack for storing results and
intermediate values.
Commands and values you enter operate on the value stack,
which is an internal data structure in KDB.
It has no connection with the kernel stack or any
other stack in the system.
To use KDB, at the kdb>>
prompt type one or more items (values or commands) on a line.
Separate items with spaces or tabs.
Press <Enter> to end a line and send its contents to KDB
for processing.
Each item is processed separately,
from left to right.
The values can be:
Numbers
Use positive or negative integers.
Numbers must begin with a digit, or a minus sign for
negative numbers.
Begin octal numbers with ``0o'' and hex numbers with ``0x''.
Otherwise, numbers are assumed to be in the default
base -- the default is hex, unless you change it.
(See
``Resetting the numeric base''
for instructions.)
Character constants
You can have KDB convert characters to a number by
entering one to four characters inside single quotes.
C-style escapes are supported in character constants.
Strings
Use C-style strings, enclosed in double quotes.
Kernel symbol names
When you type a kernel symbol name, its address is pushed
onto the value stack.
When you enter a number or a string, it is pushed
onto the value stack, becoming the new Top Of Stack (TOS).
Values remain on the value stack until they are popped
off as a result of a command.
In the descriptions below, [TOS] means the value on the
top of the stack and
[TOS-1] means the value immediately below it (pushed previously).
Stack operations
KDB provides these commands for examining or changing the value stack:
stk
Print all values on the stack
p
Print [TOS]
dup
Push [TOS]
pop
Pop 1 value
clrstk
Pop all values
Stack operation examples
stk
Displays the entire stack.
For example, starting with an empty
value stack, the input:
5 "xyzzy" 7 stk
produces the output:
5
"xyzzy"
7
p
Displays the top value on the stack.
In the example, this is:
7
The next example uses the p command to display
the address of a kernel symbol.
The input:
lbolt p
produces an address, for example:
D01821BC
dup
Uses a value twice
in a calculation.
For example:
5 3 * dup 2 + * p
would produce the output:
FF
which is the value of (((5 * 3) + 2) * (5 * 3)).
pop
Removes the top
value from the value stack.
For example, if this is the stack:
5
"xyzzy"
7
the input:
pop stk
removes the top value from the stack and
displays the resulting stack:
5
"xyzzy"
clrstk
Clears the value stack.
Remember that the contents of the stack are saved when
you exit and re-enter KDB.
Arithmetic operations
You can perform arithmetic operations on the top two values on the stack:
+
compute [TOS-1] + [TOS]; pop 2; push result
-
compute [TOS-1] - [TOS]; pop 2; push result
*
compute [TOS-1] * [TOS]; pop 2; push result
/
compute [TOS-1] / [TOS]; pop 2; push result
%
compute [TOS-1] % [TOS]; pop 2; push result
>>
compute [TOS-1] >> [TOS]; pop 2; push result
<<
compute [TOS-1] << [TOS]; pop 2; push result
<
compute [TOS-1] < [TOS]; pop 2; push result
>
compute [TOS-1] > [TOS]; pop 2; push result
==
compute [TOS-1] == [TOS]; pop 2; push result
!=
compute [TOS-1] != [TOS]; pop 2; push result
&
compute [TOS-1] & [TOS]; pop 2; push result
|
compute [TOS-1] | [TOS]; pop 2; push result
^
compute [TOS-1] ^ [TOS]; pop 2; push result
&&
compute [TOS-1] && [TOS]; pop 2; push result
||
compute [TOS-1] || [TOS]; pop 2; push result
!
replace [TOS] with ![TOS]
++
replace [TOS] with [TOS] + 1
--
replace [TOS] with [TOS] - 1
For example, this input (subtracting 5 from 7):
7 5 - p
would produce this output:
2
The power of KDB's calculator feature lies in its ability
to evaluate expressions like this:
callout 16
This pushes the address of the callout table on
the stack and adds 16 to it.
If the size of a callout table entry is 16 bytes,
the result of the calculation is the address of the second entry
in the callout table.
(Use the size command of
crash(ADM)
to find
the sizes of common system tables.)
CAUTION:
Make sure the divide operator (slash character)
is both preceded and followed by spaces.
If any other character
appears next to the slash, it indicates a suffix instead of division.
Reading and writing to memory
These commands operate like an RPN calculator, but they perform
specific debugging operations instead of calculations.
To examine and set the contents of memory (and I/O) use the commands:
r
Replace [TOS] with the value at virtual address [TOS].
w
Write [TOS-1] into virtual address [TOS]; pop 2.
dump
Show [TOS] bytes starting at virtual address [TOS-1]; pop 2.
fdump
Show [TOS-1] formatted items at [TOS-2] with format [TOS]; pop 3.
Examples
r
You can find, for example, the value
of the (long) kernel variable, lbolt, by typing:
lbolt r p
This puts the virtual address of lbolt on the
stack, replaces it with the value found
at that address, and prints the result.
w
To change the value of lbolt to 2000, type:
2000 lbolt w
This writes 2000 at lbolt's virtual address.
You could increment lbolt by typing:
lbolt r ++ lbolt w
This puts the virtual address of lbolt on
the stack, replaces it with the value found
at that address, adds 1 to the value, and writes the
result at lbolt's virtual address.
dump
This command displays a range
of memory, both in hex and ASCII.
For example, if you typed:
putbuf 10 dump
This shows 10 bytes, starting at the virtual
address of putbuf,
you would see something like:
........ ........ ........ 61746F74 D0108C50 ............tota
6572206C 6D206C61 726F6D65 ........ D0108C60 l real memor....
In each line, the block of four values on the left shows the values of
16 bytes, displayed as four 4-byte longwords in hex.
The dots represent values outside of the requested range.
(dump may also display question marks here:
that means the address is invalid.)
The next column is the address of the first of the 16 bytes.
The last column is the same 16 bytes displayed in
ASCII.
Dots represent values outside the requested range, or
unprintable characters.
fdump
This command displays an arbitrary memory structure in
a formatted fashion according to a format string.
The syntax for fdump is:
addresscountformat_stringfdump
This displays count items starting at address
address.
fdump keeps track of a memory address
called dot while it is
processing the format string; some format commands
change the value of
dot; others use dot to access memory;
dot is initially
set to address.
format_string is a (double-quoted) string
which is a sequence of
any of the command characters listed below.
Any digits (0-9) in the format string are interpreted as the
``current number'' (num), which can modify
the following command;
most commands reset num after they have finished.
Any other character in the format string is printed literally.
Number commands
x
Print value at dot in hex (base 16), unsigned.
o
Print value at dot in octal (base 8), unsigned.
d
Print value at dot in decimal (base 10), signed.
u
Print value at dot in decimal (base 10), unsigned.
Size modifiers (prefix to number commands)
B
byte (char)
H
half (short)
I
int
L
long
Other print commands
c
Print value at dot as a character.
s
Print value at dot as a (null-terminated) string.
i
Print value at dot as a machine instruction.
p
Print value at dot symbolically.
a
Print dot symbolically (hex if num non-zero).
n
Print a newline character.
t
Print a tab character.
Register modification (256 registers; first 200 are general purpose)
m
Move dot to register num.
M
Move register num to dot.
r
Move num2 to register num.
R
Move register num to num.
.
Move num to dot.
/
Move num to num2.
Indirection
*
Push dot; move value at dot to dot.
^
Pop dot from indirection stack.
Quoting
\
Print the next character literally.
`` ''
Print the characters between the quotes literally.
Other commands
+
Increment dot (by num if non-zero).
-
Decrement dot (by num if non-zero).
;
Repeat next command num times.
( )
Group commands together.
All print commands advance dot by the size of the object
printed.
Print commands are modified by non-zero num
and/or num2 as follows.
For number and instruction commands, num specifies the
output field width.
For character and string commands, num specifies the
style of
character printing: default is that all non-printable
characters show as a dot (.);
1 means to use ``control-character'' style which
prints control
characters as a caret (^) followed by
the control-character letter.
For string commands, num2 specifies the maximum number
of characters to be printed.
Note that the stack used for pushes and pops in the indirection commands
is not the KDB value stack; it is a special stack just for those commands.
As an example of using fdump, consider the following
kernel data structure:
struct cdevsw {
int (*d_open)();
int (*d_close)();
int (*d_read)();
int (*d_write)();
int (*d_ioctl)();
int (*d_mmap)();
int (*d_segmap)();
int (*d_poll)();
int (*d_msgio)();
struct streamtab *d_str;
char *d_name;
int *d_flag;
int d_cpu;
} cdevsw[];
The following fdump command could be used to
print out three elements from this cdevsw array:
Suffixes can be appended to many KDB commands.
They always begin with the slash character (/).
CAUTION:
Do not leave spaces before or after
the ``/''.
When the
``/'' is preceded and followed by a space, it
indicates division instead of a suffix.
Operand-size suffixes
The r, w and dump commands can also work
with units of bytes and words, as well as the default longs.
To do this, append one of these suffixes to the command:
/b
byte
/w
word (2 bytes)
/l
long (4 bytes) -- this is the default
/L
long long (8 bytes)
For example, to display the value of a short (2-byte) variable at
address 0xD0008120, type:
0xD0008120 r/w p
Entering the dump command with /b displays
16 1-byte values per line, with /w displays
eight 2-byte values per line, and with /l (or nothing)
displays four 4-byte values per line.
Address-space suffixes
The r, w and dump commands, by default, work
with kernel virtual addresses.
You can change to physical addresses, I/O addresses, or user
process virtual addresses by appending one of these suffixes
to the command:
/k
kernel virtual -- this is the default
/p
physical
/io
I/O port
/un
user process number n virtual (n is a process slot number in hex)
/cpun
CPU number n (n is in hex)
/cn
same as /cpun
Examples of using suffixes
/p
For example, to dump 40 (hex) bytes
in longword format from physical address 2000,
type:
2000 40 dump/p
The default address is kernel virtual, so the /p suffix
is required for the physical address.
Note that an operand-size suffix is not required, because
long is the default.
/io
For example, to read from port
300 (in bytes) and display the result, type:
300 r/io/b p
/un
For example, to dump 20 longwords from
process 16's u-area at an offset of 1000, type:
1000 u + 20 dump/u16
/cpun
For example, to print a stack-trace of CPU 5's stack, type:
stack/cpu5
or
stack/c5
Suffix formats
Address-space suffixes can be combined with operand-size suffixes;
only the first slash is required.
For example, to do the read from I/O port 300
shown above, any of these command lines is acceptable:
300 r/io/b 300 r/b/io 300 r/iob 300 r/bio
Suffixes can also be attached directly to an address
as shorthand for ``read and print''.
Thus, 2000 r/p p can be shortened to 2000/p.
Since the default address-space is kernel virtual,
the common operation of ``read and print from kernel virtual''
can be even further shortened.
Type lbolt/ to read and print the value of the
(long) kernel variable, lbolt.
Displaying and writing to registers
You can examine the CPU's general registers
(and some pseudo-registers) with these commands:
%register
This pushes the contents of the 32-, 16- or 8-bit register.
register can be any of the following:
You can modify the values of general-purpose
registers with these commands:
w%register
write [TOS] into register (with the exception of stack
pointer w%esp and w%sp); pop 1
w%trap
write [TOS] into the trap number pseudo-register; pop 1
Register sets
The commands listed above can also be used
to access specific register sets.
Multiple sets of general registers may have been saved
on the kernel stack (one for each interrupt, trap, and so on).
For more information see
``Printing kernel stack traces''.
Register sets are numbered from 0 to 19, with 0 being the current
(most recent) set.
By default, the general-register commands use register set 0,
but you can override this with a ``register-set suffix'':
/rsn
register set number n
Note that by combining suffixes, you can access
any register of any process.
For example, you can get the eax register from process 5's
register set 1 by typing:
%eax/u5rs1
This command will
push the contents of that register (%eax) in register set 1
(/rs1) of user process 5 (/u5).
CPU control registers
In addition to the general registers,
you can examine the values of CPU control registers with these commands:
cr0
push the contents of register cr0
cr2
push the contents of register cr2
cr3
push the contents of register cr3
cr4
push the contents of register cr4
Creating debugger variables
KDB allows you to create named variables that are stored in the
debugger and hold debugger values (numbers or strings).
Two KDB commands
apply to variables:
=variable
store [TOS] in [variable]; pop 1
vars
show values of debugger variables
Examples
=variable
This command assigns a value to a debugger variable.
For example:
5 = abc
creates the variable abc if it does not exist,
and sets the variable equal to 5.
Now whenever you use the variable name, its
value is pushed onto the stack.
For example:
abc abc + 2 - p
will yield 8 (5 + 5 - 2).
Note that variable names share the same namespace
as debugger macros and kernel global symbols.
vars
Look at all the existing variables.
Variables are shown in the following format:
name=value
The vars command also lists macros, in this format:
name::value
Setting breakpoints
Set and modify breakpoints with these commands:
B
Either of the following:
set breakpoint number [TOS] at address [TOS-1]; pop 2
set breakpoint number [TOS] at address [TOS-2] with command string [TOS-1]; pop 3
b
Either of the following:
set first free breakpoint address [TOS]; pop 1
set first free breakpoint at address [TOS-1] with command string [TOS]; pop 2
bn
set breakpoint (like b) and push breakpoint number
brkoff
disable breakpoint number [TOS]; pop 1
brkon
re-enable breakpoint number [TOS]; pop 1
brksoff
disable all breakpoints
brkson
re-enable all (disabled) breakpoints
trace
set breakpoint number [TOS] trace count to [TOS-1]; pop 2
clrbrk
clear breakpoint number [TOS]; pop 1
clrbrks
clear all breakpoints
clraddrbrks
clear all breakpoints for address [TOS]; pop 1
curbrk
push the current breakpoint number, or -1 if
not entered from a breakpoint
?brk
show current breakpoint settings
You can have up to 20 breakpoints, numbered 0 through 19,
set at one time.
B, b, and bn
The B command lets
you set specific breakpoints,
while the b command automatically picks the first un-set
breakpoint.
This example sets breakpoint 3 at a specific address:
0xD0125098 3 B
Normally, you will just set a breakpoint at a certain address.
For example:
read b
This sets an instruction breakpoint at the beginning of the
kernel read routine, using the next available breakpoint number.
When the specified address is executed (after
exiting from the debugger),
you enter the debugger again, with a message indicating which breakpoint
was triggered.
Debugger command strings can be added to the breakpoint commands.
Enter a quoted string of commands after the address:
read "stack" b
which is used as a series of debugger commands that
are executed when the breakpoint is triggered.
If there are several items in the string, separate them with spaces:
ie6unitdata_req "300 r/bio p" b
After these commands are executed, you are prompted for debugger
commands, as usual, unless the q (quit) command is executed
in the command string.
The bn command works like b except that it leaves the
breakpoint number on the value stack.
This is useful for macros.
You can append breakpoint-type suffixes to the
breakpoint commands (B, b, and bn).
By default, breakpoints are ``instruction'' breakpoints,
which trigger when
the specified address is executed.
The suffixes cause breakpoints to trigger on data accesses instead.
The breakpoint-type suffixes are:
/a
data access breakpoint
/m
data modify breakpoint
/i
instruction execution breakpoint -- this is the default
/io
I/O port access
With access and modify breakpoints, you can also
use operand-size suffixes to control the size of the
address range that will trigger the breakpoint.
The default is /l (4 bytes); you can also
use /w (word) and /b (byte).
(See the discussion of suffixes
in
``Reading and writing to memory''
for more information.)
brkoff and brkon
These commands let you temporarily
disable and re-enable a breakpoint, instead of clearing
it with clrbrk and then re-entering it later.
This is especially handy for breakpoints with command strings.
trace
This command sets a trace count for a breakpoint.
This causes the debugger to just print a message
and decrement the count
when the breakpoint is triggered,
instead of entering the debugger, until the count reaches zero.
Commands attached to the breakpoint are not executed.
?brk
Use this command to determine the current breakpoint settings.
Each set breakpoint is displayed with: the breakpoint number,
the address in hex, the symbolic address,
the current state, and the type:
0: 0xD003907C(read) ON /i
The possible states are:
ON
set and enabled
DISABLED
set, but currently disabled
OFF
un-set (these breakpoints are not displayed by ?brk)
The possible types (in this example /i) are the same as
the breakpoint-type suffixes described earlier.
If a breakpoint has a non-zero trace count, that is displayed
after the breakpoint state.
If a breakpoint has a command string, it is displayed
at the end of the line.
For example, with a count of 5 and a stack command,
the above breakpoint would display as:
0: 0xD003907C(read) ON 0x5 /i "stack"
Single-stepping through instructions
You can use these commands for single-stepping:
s
single step 1 instruction
ss
single step [TOS] instructions; pop 1
S
single step 1 instruction (passing calls)
SS
single step [TOS] instructions (passing calls); pop 1
s and ss single-step all instructions.
S and SS single-step all instructions except
call instructions.
They do not step down into the called routine,
but instead skip ahead to the return from the call,
treating the whole subroutine sequence as a single instruction.
Branch-stepping through instructions
Four commands are available for branch-stepping:
bs
branch step: execute until a branch is taken
bss
branch step through [TOS] branches; pop 1
lbr
show from-and to- address for last branch taken
lcall
as call but using long long (8 bytes)
lint
show from- and to- address for last interrupt or exception
Some advanced CPUs support a branch-stepping feature that
allows instruction execution to proceed until the next jump or call
instruction which causes a branch in program flow.
These commands allow use of this feature when available.
On CPUs that do not support branch-stepping,
these commands will not be active.
Examining kernel data structures
The ps command shows information about each
active process in the system.
This information includes process IDs, flags,
and command names, on one line,
with an additional line per LightWeight Process (LWP) within the process
with LWP IDs, flags,
states, and scheduling information.
The CPU field of an LWP line is blank for non-running LWPs;
otherwise it is the CPU number on which that
LWP is currently running.
Printing kernel stack traces
KDB provides the following commands to look at kernel stack traces:
stack
kernel stack trace for the current process
lstack
kernel stack trace for LWP [TOS]; pop 1
pstack
kernel stack trace for process; pop 1
tstack
``try'' kernel stack trace from [TOS]; pop 1
stackargs
set maximum number of arguments in stack trace to [TOS]; pop 1
NOTE:
The argument to lstack can be specified
either as the address of the LWP structure
or -1 for the current LWP for the given CPU.
(-1 lstack is equivalent to the stack command.)
tstack is sometimes useful if lstack or stack
fail to give
a full stack trace for some reason.
(For example, another CPU might be hung and fails to
respond to the stack-trace
request.)
It takes a single argument, which is a stack pointer
value, and attempts
to find a potentially valid trace starting from some
stack address greater
than or equal to that value.
The output of stack, lstack and tstack
have the same format.
A typical stack trace
(for the current process, entered using <Ctrl><Alt>D)
looks like this:
(current) idle stack:
DEBUGGER ENTERED FROM USER REQUEST
kdcksysrq(D101FD40 D00DE624 81 20)........esp:FFFE9C94 ret:D008F592
*kdintr+0x186(1)...........................esp:FFFE9CD8 ret:D0011A3A
INTERRUPT TO devint0+0x78 from 8:D001218A (r0ptr:FFFE9CEC)
eax: 8 ebx: ------- ecx:FFFFFFFF edx: 8 efl: 246
esi: ------- edi: ------- esp:FFFE9D04 ebp: ------- regset: 0
idle(0 D00EDDD0 D106BC00).................esp:FFFE9D0C ret:D006F11F
*swtch(0 0 0)..............................esp:FFFE9D40 ret:D002464C
>use_private+0xAB()
The stack trace shows a history of which routine called which other
routine, up until the point the debugger was entered
(or in the case of a non-current process, until the process was
context-switched out).
The most-recently-entered routine is shown on the first line.
In the example, the debugger was entered from kdcksysrq,
which, in turn, was called by kdintr;
idle was called from swtch, and so on.
The stack trace ends at the point the kernel was entered from user mode.
In the case of a system process or an idle stack (as shown here)
where there is no user mode, the stack trace ends at the top-level
routine (use_private in this case).
(An idle stack is a per-CPU private stack which is used when the CPU is idle
or is otherwise not running an LWP.)
Routine trace format
The trace for each routine has four parts:
its address, the arguments passed to it,
the value of the esp register on entry to the routine,
and its return address.
For example:
The address that was called usually appears in symbolic form.
A routine name may also include:
An asterisk (``*''): *kdintr+0x186
This means the routine was called indirectly.
There is insufficient information in the stack format
to be 100% sure of the correctness of indirect call traces.
Whenever you see an asterisk in a stack trace,
there is a small chance that some part of the stack
trace from that point on is incorrect.
An offset (a plus sign (``+'') and a hex number): *kdintr+0x186
The offset may mean that the actual address called was
somewhere past the start of the indicated routine.
This will most likely happen if a subroutine was declared static.
Since the debugger only has access to global symbols,
it finds the nearest preceding global symbol.
The offset may also mean that the exact address
called cannot be determined.
The address displayed in this case is the return address into this
routine from the routine it called.
This will most likely happen if this routine was called indirectly
via a function pointer.
Arguments
The arguments passed to the routine appear as a list of hex numbers,
enclosed in parentheses.
Since the actual number of arguments passed cannot be
determined, KDB assumes that each routine has no more than
a certain maximum number of arguments. The default
is three, but you can change it with the stackargs command.
If a routine actually has:
Fewer arguments than displayed
Only the first ones are real.
In rare cases when the debugger can deduce that a
routine could not have been called with
the maximum number of arguments (because there is not
enough room on the stack),
it displays only the maximum possible number of arguments.
In the above stack trace, the call to kdintr is
shown with only one argument ``(1)''.
More arguments than displayed
Increase the number with stackargs and
then display the stack trace again,
or dump out a portion of the stack directly
in order to see all the arguments (continue to
``Trap frames''
for details).
esp register
The value of the esp register on entry to the routine
is shown as a hex number following esp:.
This value can be used as a ``frame pointer'' to access
arguments and local variables for the routine.
The following diagram illustrates the stack layout:
For example, if you want to see all the arguments to a routine that
takes five arguments, find its esp value from the
stack trace -- say
0xD2473CD4 -- and enter these commands:
0xD2473CD4 4 + 5 4 * dump
or, more succinctly:
0xD2473CD8 14 dump
Return address
This is the address within its caller to which the routine returns.
It is shown as a hex number following ret:.
Trap frames
In addition to lines for each routine, stack traces
will often include
``trap frames'' created when an event causes
suspension of current processing,
saving all register values on the stack.
Typical events are interrupts, hardware exceptions,
and system calls.
Trap frames are three lines each, starting with
an uppercase, non-indented keyword (like INTERRUPT
in the following example). The next two lines contain
the values of the registers at the time the event occurred.
The first line of a trap frame is in one of these formats:
INTERRUPT TO devint0+0x78 from 8:D001218A (r0ptr:E0000D84)
TRAP 0x1 from 8:D001218A (r0ptr:E0000D94)
TRAP EVENT from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
SYSTEM CALL from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
SIGNAL <Return> from 17:830676D (r0ptr: E0000D94, ss:esp: 1F:80468E8)
These represent interrupts, hardware exception traps, trap event processing,
system calls, and returns from old-style signal handlers, respectively.
The number after TRAP is the hardware exception number;
the most common are 0x1 for breakpoint traps
and 0xE for page faults.
See /usr/include/sys/trap.h for a full list of trap numbers.
The colon-separated numbers after the word from are the segment
and offset (cs and eip) at the time the event occurred.
The values in parentheses show the r0ptr value for the beginning of
the trap frame (for example, the address of the saved eax register;
see /usr/include/sys/reg.h for the layout of saved registers),
and the user stack pointer segment and offset at the time
the event occurred.
The user stack information is only displayed if the trap frame is
for an entry into the kernel from user mode.
Resetting the numeric base
If you do not start numbers with ``0o'' (for octal)
or ``0x'' (for hex), KDB
assumes they are in the default numeric base.
Initially, the defaults for both input and output
are set to 16 (hex), but you can use these
commands to change them:
ibase
set default input base to [TOS]; pop 1
ibinary
set default input base to 2
ioctal
set default input base to 8
idecimal
set default input base to 10
ihex
set default input base to 16
obase
set output base to [TOS]; pop 1
ooctal
set output base to 8
odecimal
set output base to 10
ohex
set output base to 16
Converting address spaces
Use these commands to convert a virtual address to a physical address:
kvtop
convert kernel virtual address [TOS] to physical
uvtop
convert user proc number [TOS] address [TOS-1] to physical; pop 1
Performing conditional execution
KDB provides two commands for conditional execution:
then
if [TOS] = 0, skip to endif; pop 1
endif
end scope of then command
In other words, a sequence like:
conditionthencommandsendif
executes commands if and only if the condition
is true (non-zero).
These are mostly useful for macros and breakpoint command strings.
For example, imagine you wish to set a
breakpoint for when the function
inb is called with 2E as its first argument.
Use the following command:
inb "%esp 4 + r 2E != then q" b
This sets a breakpoint at inb,
but enters the debugger only if the content of %esp+4
is equal to 2E.
This works because esp points to the
return address on the stack,
and the longword after that is the first argument.
For the second argument, you would add 8 instead of 4
(see
``Printing kernel stack traces''
for details of the stack layout).
If you do a ?brk command, the display for
that breakpoint includes the string of debugger
commands:
0: 0xD003907C(inb) ON /i "%esp 4 + r 2E != then q"
Calling a kernel function
Use these commands to call an arbitrary kernel function:
call
call the function at address [TOS-1] with [TOS] arguments,
given by:
[TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
vcall
call the function at address [TOS-1] with [TOS] arguments,
given by:
[TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
lcall
as call, but using long long arguments
Use the call command to call a function which returns a value;
this value will be the [TOS] value on the value stack
after the call command.
Use the vcall (``void call'') command to
call a function with no return value.
To call psignal with two arguments,
the current process and 9, type:
curproc r 9 psignal 2 call
curproc r gives the value of the current process,
the first argument, and 9 is the second
argument.
psignal is converted into the address at which
that function can be called, and 2 specifies
the number of arguments to pass to psignal.
Performing a system dump
Some systems support dumping all of memory to disk with a
sysdump kernel function.
For these systems, you can take a dump by calling
this function with the command:
sysdump 0 vcall
All of memory and the current state is dumped to the
dump partition on the disk, so
you can use
crash(ADM)
to do a
postmortem.
Miscellaneous commands
Some miscellaneous KDB commands are:
findsym
Print kernel symbol with address closest to [TOS]; pop 1
dis
Disassemble [TOS] instructions starting at address [TOS-1]; pop 2
nonverbose
Turn verbose mode off
verbose
Turn verbose mode on
newdebug
Switch to another debugger on next debugger entry
help
Print a help message
?
Print a help message (same as help)
cmds
Print a list of all debugger commands
Writing macros
KDB provides the ability to assign a string of
commands to a single new command name, called a macro.
When a debugging task involves repeating the
same set of commands many times (possibly doing other
things in between),
it is easier to define a macro and use it in place
of the whole set of commands.
These commands are used for macros:
::macro
define macro as command string [TOS]; pop 1
P
print [TOS] in raw form; pop 1
PP
print [TOS] values in raw form, from [TOS-[TOS]],...[TOS-1]; pop [TOS]+1
vars
show values of debugger macros and variables
Macro Examples
::macro
Use this command to define macros.
For example:
"curproc r 16 - p" :: newaddr
Note that macro names share the same namespace
as debugger variables and kernel global symbols.
P and PP
These commands are provided to aid in writing macros.
P and PP print values in raw form,
without the embellishments provided by the
p command, such as quotes around strings
and automatic newlines after each value.
This allows complete control over formatting.
For example, the input:
"The value of curproc is " curproc r ".\n" 3 PP
might produce the output:
The value of curproc is 0xD1011E80
To put something like this into a macro means putting
strings inside strings,
so you will have to escape the inner quotes:
"\"The value of curproc is \" curproc r \".\n\" 3 PP" :: pcurproc
vars
Use this command to show the macro definitions.
Macros are shown in this format:
name :: value
Note that the vars command also shows
the values of variables, in this format:
name = value
Output control
Some KDB commands send a great deal of output to the screen.
There are various ways you can slow the output down if necessary:
XON/XOFF flow control is supported,
using the <Ctrl>Q/<Ctrl>S keys.
Output may be interrupted (and command execution aborted)
with either the <Del> or <Ctrl>C key.
Output is automatically paginated.
That is, after each screenful of output,
a [MORE]--- prompt is
output, and KDB waits for an input key before continuing.
The number of lines per screen is controlled by the db_nlines
global variable.
You can change the number of lines to, for example, 60, with:
60 db_nlines w
Executing debugger commands at boot time
KDB allows you to specify an arbitrary command sequence to be executed
at boot time, when the system is coming up.
You can do this by writing the commands into the file
/stand/kdb.rc,
then rebuilding the kernel
with
idbuild(ADM).
The following line needs to be added to the /stand/boot file:
At boot time, after the (possibly blank) string is executed,
the system enters KDB at the kdb>> prompt,
unless a q command was executed as part of the string -- just
like conditional breakpoints.
(A non-existent or zero-length kdb.rc file acts as a single
q command, so KDB is not entered.)
Using a serial terminal
KDB can be used from a serial terminal as well as the console.
This is particularly useful if you are trying to debug a scenario that
involves graphics or multiple virtual terminals on the console.
Before you attempt to use the debugger from a serial terminal,
make sure there is a
getty(ADM)
or
ttymon(ADM)
running on it.
It may be either logged in or waiting at the login prompt.
This ensures that the baud rate and other parameters are properly set.
You can switch from the console to a terminal,
and vice-versa, with the newterm command.
This immediately switches you to the new terminal.
The debugger continues to use this terminal until you give it
the newterm command again, even if you exit and re-enter KDB.
The newterm command takes two arguments.
The first argument is a string which is the device
name of a console-capable
device driver (for example, iasy).
The second argument is the unit number of the actual device
(usually the minor device number) you wish to use.
For example the following command switches control of KDB to a terminal on the
first serial port:
"iasy" 0 newterm
Likewise, the following command returns control of KDB to the
console (where kd is device name of the console driver):
"kd" 0 newterm
Once you exit from KDB, you can invoke it again from either
the console or a serial terminal.
Use the kdb command to invoke the debugger
from a terminal; <Ctrl><Alt>D only works from the
console.
Regardless of where you invoke KDB, its I/O appears
where you directed it during the last KDB session.
Entering the debugger from a driver
If you are debugging a DDI8 device driver (or another part
of the kernel), you can directly invoke the kernel debugger using
call_demon(D3).
Pre-DDI8 drivers have in the past invoked the kernel debugger using
code similar to the code shown below.
Note that this is discouraged as this code introduces
non-conforming symbols into the driver's name space.
#include <sys/xdebug.h>
(*cdebugger) (DR_OTHER, NO_FRAME);
DR_OTHER
tells the debugger that the reason for entering is other.
See sys/xdebug.h for a list of other reason codes.
Note that this mechanism cannot be used for debugging early kernel
startup code or driver init routines, since the debugger cannot
be used until its init routine (kdb_init) has been
called.
Configuring KDB
The Kernel Debugger (KDB) can be configured two ways: it can be
enabled at all times, or it can be enabled only on demand. If
demand mode is chosen, KDB must be manually enabled before it can
be invoked.
Enabling KDB
KDB can be invoked by a console key sequence, by a system crash,
or by the kdb command. However, it can only be invoked if it
is enabled. KDB is enabled when it is loaded into the kernel.
If you choose to enable it always, KDB will be loaded into the
kernel at all times and can be invoked at any time.
If you choose to enable it on demand, KDB must first be loaded
with the modadmin -l kdb command before it can be invoked.
KDB can be unloaded with the modadmin -U kdb command.
The advantage to having KDB always enabled is that it will
be ready even for unanticipated use. The disadvantage is that
whenever it is enabled, KDB consumes about 250K of main memory.
Most of this memory is used for the kernel symbol table, which
is locked in memory whenever KDB is loaded, unless the PAGESYMTAB
tunable is set to 2.
This will affect system performance on systems with small memories.
Security checks
KDB also provides optional security checks which prohibit an
unauthorized user from invoking the kernel debugger. If you do not
turn on these security checks, the kernel debugger can be entered
from a key sequence on the console, presenting a potential security
breach if your console is not physically secure.
Disabling the <Ctrl><Alt>D sequence
If the security check is not enabled, a user can type a key
sequence at the console without having to log into the computer
and enter the kernel debugger. Unless the console is in a room
that is locked or accessible to only a controlled group of people,
this security check should be enabled to prevent a security breach.
KDB can only be called from the
console using <Ctrl><Alt>D
if the kdb_security flag was set to 0 when
the kernel was built.
To disable the <Ctrl><Alt>D key sequence, reset
the kdb_security flag by using
/etc/conf/bin/idtune to change
the KDBSECURITY tunable to 1.
Note that the flag setting does not affect the kdb command.
In order to enter the debugger after disabling this
key sequence, the user must log into the
computer using a privileged account and then type the kdb command.
Command summary
+
compute [TOS-1] + [TOS]; pop 2; push result
-
compute [TOS-1] - [TOS]; pop 2; push result
*
compute [TOS-1] * [TOS]; pop 2; push result
/
compute [TOS-1] / [TOS]; pop 2; push result
%
compute [TOS-1] % [TOS]; pop 2; push result
>>
compute [TOS-1] >> [TOS]; pop 2; push result
<<
compute [TOS-1] << [TOS]; pop 2; push result
<
compute [TOS-1] < [TOS]; pop 2; push result
>
compute [TOS-1] > [TOS]; pop 2; push result
==
compute [TOS-1] == [TOS]; pop 2; push result
!=
compute [TOS-1] != [TOS]; pop 2; push result
&
compute [TOS-1] & [TOS]; pop 2; push result
|
compute [TOS-1] | [TOS]; pop 2; push result
^
compute [TOS-1] ^ [TOS]; pop 2; push result
&&
compute [TOS-1] && [TOS]; pop 2; push result
||
compute [TOS-1] || [TOS]; pop 2; push result
!
replace [TOS] with ![TOS]
++
replace [TOS] with [TOS] + 1
--
replace [TOS] with [TOS] - 1
%register
push the contents of the 32-, 16- or 8-bit register.
register can be any of the following:
set breakpoint number [TOS] at address [TOS-1]; pop 2
set breakpoint number [TOS] at address [TOS-2] with command string [TOS-1]; pop 3
b
Either of the following:
set first free breakpoint address [TOS]; pop 1
set first free breakpoint at address [TOS-1] with command string [TOS]; pop 2
bn
set breakpoint (like b) and push breakpoint number
brkoff
disable breakpoint number [TOS]; pop 1
brkon
re-enable breakpoint number [TOS]; pop 1
brksoff
disable all breakpoints
brkson
re-enable all (disabled) breakpoints
bs
branch step: execute until a branch is taken
bss
branch step through [TOS] branches; pop 1
call
call the function at address [TOS-1] with [TOS] arguments,
given by [TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2;
push function return value
clraddrbrks
clear all breakpoints for address [TOS]; pop 1
clrbrk
clear breakpoint number [TOS]; pop 1
clrbrks
clear all breakpoints
clrstk
pop all values
cmds
print a list of all debugger commands
cr0
push the contents of register cr0
cr2
push the contents of register cr2
cr3
push the contents of register cr3
cr4
push the contents of register cr4
curbrk
push the current breakpoint number, or -1
if not entered from a breakpoint
dis
disassemble [TOS] instructions starting at address [TOS-1]; pop 2
dump
show [TOS] bytes starting at virtual address [TOS-1]; pop 2
dup
push [TOS]
endif
end scope of then command
findsym
print kernel symbol with address closest to [TOS]; pop 1
fdump
show [TOS-1] formatted items at [TOS-2] with format [TOS]; pop 3
help
print a help message
ibase
set default input base to [TOS]; pop 1
ibinary
set default input base to 2
idecimal
set default input base to 10
ihex
set default input base to 16
ioctal
set default input base to 8
kvtop
convert kernel virtual addr [TOS] to physical
lbr
show from- and to- address for last branch taken
lint
show from- and to- address for last interrupt or exception
lstack
kernel stack trace for LWP [TOS]; pop 1
newterm
switch KDB console I/O to device [TOS-1] unit number [TOS]; pop 2
newdebug
switch to another debugger on next debugger entry
nonverbose
turn verbose mode off
obase
set output base to [TOS]; pop 1
odecimal
set output base to 10
ohex
set output base to 16
ooctal
set output base to 8
P
print [TOS] in raw form; pop 1
p
print [TOS]
PP
print [TOS] values in raw form,
from [TOS-[TOS]],...[TOS-1]; pop [TOS]+1
pop
pop 1 value
ps
show process information
pstack
kernel stack trace for process; pop 1
q
exit from the debugger
r
replace [TOS] with the value at virtual address [TOS]
S
single step 1 instruction (passing calls)
s
single step 1 instruction
SS
single step [TOS] instructions (passing calls); pop 1
ss
single step [TOS] instructions; pop 1
stack
kernel stack trace for the current process
stackargs
set the maximum number of arguments in the stack trace to [TOS]; pop 1
stk
print all values on the stack
then
if [TOS] = 0, skip to endif; pop 1
trace
set breakpoint number [TOS] trace count to [TOS-1]; pop 2
tstack
``try'' kernel stack trace from [TOS]; pop 1
uvtop
convert user process number [TOS] address [TOS-1] to physical; pop 1
vars
show values of debugger variables
vcall
call the function at address [TOS-1] with [TOS] arguments,
given by [TOS-([TOS]+1)],...[TOS-2]; pop [TOS]+2
verbose
turn verbose mode on
w
write [TOS-1] into virtual address [TOS]; pop 2
w%register
write [TOS] into register; pop 1
w%trap
write [TOS] into the trap number pseudo-register; pop 1
Command suffixes
Operand size
/b
byte
/w
word (2 bytes)
/l
long (4 bytes) -- this is the default
/L
long long (8 bytes)
Address space
/k
kernel virtual -- this is the default
/p
physical
/io
I/O port
/un
user process number n virtual
/cpun
CPU number n -- valid only for multiprocessors
/cn
CPU number n (same as cpun) -- valid
only for multiprocessors
Register set
/rsn
register set number n
Breakpoint type
/a
data access breakpoint
/m
data modify breakpoint
/i
instruction execution breakpoint -- this is the default
Command Recall and Line Editing
kdb supports command recall and line editing with a set of
emacs-style control characters and escape sequences.
The emacson and emacsoff commands control whether
command recall and line editing are turned on or off, respectively.
In emacsoff mode, only the backspace
key (delete character to the left of the cursor) is active.
The default is emacson.
You can change the default behaviour using the kdb_emacs variable
in the file kdb_util/space.c.
If kdb_emacs is ``0'', emacsoff is the
default; if it is non-zero, emacson is the default.
The active keys in emacson mode are:
cursor motion
ctrl-b
move cursor left one character
(non-destructive backspace)
To reboot from the
kdb(ADM)
prompt, enter the following:
kdb>> 1 0 mdboot 2 call
Loading
kdb(ADM)
dynamically does not enable the Debug Extension bit of cr4
on Pentium processors and higher. Use statically linked
kdb on these processors.
On some multi-processor systems, the keyboard may become irrecoverably
corrupted while
kdb(ADM)
is running. The only solution to this is to press the restart button
on the computer.