docs: debugging-via-ohci1394.txt: add it to the core-api book
There is an special chapter inside the core-api book about some debug infrastructure like tracepoints and debug objects. It sounded to me that this is the best place to add a chapter explaining how to use a FireWire controller to do remote kernel debugging, as explained on this document. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/9b489d36d08ad89d3ad5aefef1f52a0715b29716.1588345503.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This commit is contained in:
Mauro Carvalho Chehab
committed by
Jonathan Corbet
Jonathan Corbet
parent
283b69bf79
commit
a74e2a2264
185
Documentation/core-api/debugging-via-ohci1394.rst
Normal file
185
Documentation/core-api/debugging-via-ohci1394.rst
Normal file
@@ -0,0 +1,185 @@
|
||||
===========================================================================
|
||||
Using physical DMA provided by OHCI-1394 FireWire controllers for debugging
|
||||
===========================================================================
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
Basically all FireWire controllers which are in use today are compliant
|
||||
to the OHCI-1394 specification which defines the controller to be a PCI
|
||||
bus master which uses DMA to offload data transfers from the CPU and has
|
||||
a "Physical Response Unit" which executes specific requests by employing
|
||||
PCI-Bus master DMA after applying filters defined by the OHCI-1394 driver.
|
||||
|
||||
Once properly configured, remote machines can send these requests to
|
||||
ask the OHCI-1394 controller to perform read and write requests on
|
||||
physical system memory and, for read requests, send the result of
|
||||
the physical memory read back to the requester.
|
||||
|
||||
With that, it is possible to debug issues by reading interesting memory
|
||||
locations such as buffers like the printk buffer or the process table.
|
||||
|
||||
Retrieving a full system memory dump is also possible over the FireWire,
|
||||
using data transfer rates in the order of 10MB/s or more.
|
||||
|
||||
With most FireWire controllers, memory access is limited to the low 4 GB
|
||||
of physical address space. This can be a problem on IA64 machines where
|
||||
memory is located mostly above that limit, but it is rarely a problem on
|
||||
more common hardware such as x86, x86-64 and PowerPC.
|
||||
|
||||
At least LSI FW643e and FW643e2 controllers are known to support access to
|
||||
physical addresses above 4 GB, but this feature is currently not enabled by
|
||||
Linux.
|
||||
|
||||
Together with a early initialization of the OHCI-1394 controller for debugging,
|
||||
this facility proved most useful for examining long debugs logs in the printk
|
||||
buffer on to debug early boot problems in areas like ACPI where the system
|
||||
fails to boot and other means for debugging (serial port) are either not
|
||||
available (notebooks) or too slow for extensive debug information (like ACPI).
|
||||
|
||||
Drivers
|
||||
-------
|
||||
|
||||
The firewire-ohci driver in drivers/firewire uses filtered physical
|
||||
DMA by default, which is more secure but not suitable for remote debugging.
|
||||
Pass the remote_dma=1 parameter to the driver to get unfiltered physical DMA.
|
||||
|
||||
Because the firewire-ohci driver depends on the PCI enumeration to be
|
||||
completed, an initialization routine which runs pretty early has been
|
||||
implemented for x86. This routine runs long before console_init() can be
|
||||
called, i.e. before the printk buffer appears on the console.
|
||||
|
||||
To activate it, enable CONFIG_PROVIDE_OHCI1394_DMA_INIT (Kernel hacking menu:
|
||||
Remote debugging over FireWire early on boot) and pass the parameter
|
||||
"ohci1394_dma=early" to the recompiled kernel on boot.
|
||||
|
||||
Tools
|
||||
-----
|
||||
|
||||
firescope - Originally developed by Benjamin Herrenschmidt, Andi Kleen ported
|
||||
it from PowerPC to x86 and x86_64 and added functionality, firescope can now
|
||||
be used to view the printk buffer of a remote machine, even with live update.
|
||||
|
||||
Bernhard Kaindl enhanced firescope to support accessing 64-bit machines
|
||||
from 32-bit firescope and vice versa:
|
||||
- http://v3.sk/~lkundrak/firescope/
|
||||
|
||||
and he implemented fast system dump (alpha version - read README.txt):
|
||||
- http://halobates.de/firewire/firedump-0.1.tar.bz2
|
||||
|
||||
There is also a gdb proxy for firewire which allows to use gdb to access
|
||||
data which can be referenced from symbols found by gdb in vmlinux:
|
||||
- http://halobates.de/firewire/fireproxy-0.33.tar.bz2
|
||||
|
||||
The latest version of this gdb proxy (fireproxy-0.34) can communicate (not
|
||||
yet stable) with kgdb over an memory-based communication module (kgdbom).
|
||||
|
||||
Getting Started
|
||||
---------------
|
||||
|
||||
The OHCI-1394 specification regulates that the OHCI-1394 controller must
|
||||
disable all physical DMA on each bus reset.
|
||||
|
||||
This means that if you want to debug an issue in a system state where
|
||||
interrupts are disabled and where no polling of the OHCI-1394 controller
|
||||
for bus resets takes place, you have to establish any FireWire cable
|
||||
connections and fully initialize all FireWire hardware __before__ the
|
||||
system enters such state.
|
||||
|
||||
Step-by-step instructions for using firescope with early OHCI initialization:
|
||||
|
||||
1) Verify that your hardware is supported:
|
||||
|
||||
Load the firewire-ohci module and check your kernel logs.
|
||||
You should see a line similar to::
|
||||
|
||||
firewire_ohci 0000:15:00.1: added OHCI v1.0 device as card 2, 4 IR + 4 IT
|
||||
... contexts, quirks 0x11
|
||||
|
||||
when loading the driver. If you have no supported controller, many PCI,
|
||||
CardBus and even some Express cards which are fully compliant to OHCI-1394
|
||||
specification are available. If it requires no driver for Windows operating
|
||||
systems, it most likely is. Only specialized shops have cards which are not
|
||||
compliant, they are based on TI PCILynx chips and require drivers for Windows
|
||||
operating systems.
|
||||
|
||||
The mentioned kernel log message contains the string "physUB" if the
|
||||
controller implements a writable Physical Upper Bound register. This is
|
||||
required for physical DMA above 4 GB (but not utilized by Linux yet).
|
||||
|
||||
2) Establish a working FireWire cable connection:
|
||||
|
||||
Any FireWire cable, as long at it provides electrically and mechanically
|
||||
stable connection and has matching connectors (there are small 4-pin and
|
||||
large 6-pin FireWire ports) will do.
|
||||
|
||||
If an driver is running on both machines you should see a line like::
|
||||
|
||||
firewire_core 0000:15:00.1: created device fw1: GUID 00061b0020105917, S400
|
||||
|
||||
on both machines in the kernel log when the cable is plugged in
|
||||
and connects the two machines.
|
||||
|
||||
3) Test physical DMA using firescope:
|
||||
|
||||
On the debug host, make sure that /dev/fw* is accessible,
|
||||
then start firescope::
|
||||
|
||||
$ firescope
|
||||
Port 0 (/dev/fw1) opened, 2 nodes detected
|
||||
|
||||
FireScope
|
||||
---------
|
||||
Target : <unspecified>
|
||||
Gen : 1
|
||||
[Ctrl-T] choose target
|
||||
[Ctrl-H] this menu
|
||||
[Ctrl-Q] quit
|
||||
|
||||
------> Press Ctrl-T now, the output should be similar to:
|
||||
|
||||
2 nodes available, local node is: 0
|
||||
0: ffc0, uuid: 00000000 00000000 [LOCAL]
|
||||
1: ffc1, uuid: 00279000 ba4bb801
|
||||
|
||||
Besides the [LOCAL] node, it must show another node without error message.
|
||||
|
||||
4) Prepare for debugging with early OHCI-1394 initialization:
|
||||
|
||||
4.1) Kernel compilation and installation on debug target
|
||||
|
||||
Compile the kernel to be debugged with CONFIG_PROVIDE_OHCI1394_DMA_INIT
|
||||
(Kernel hacking: Provide code for enabling DMA over FireWire early on boot)
|
||||
enabled and install it on the machine to be debugged (debug target).
|
||||
|
||||
4.2) Transfer the System.map of the debugged kernel to the debug host
|
||||
|
||||
Copy the System.map of the kernel be debugged to the debug host (the host
|
||||
which is connected to the debugged machine over the FireWire cable).
|
||||
|
||||
5) Retrieving the printk buffer contents:
|
||||
|
||||
With the FireWire cable connected, the OHCI-1394 driver on the debugging
|
||||
host loaded, reboot the debugged machine, booting the kernel which has
|
||||
CONFIG_PROVIDE_OHCI1394_DMA_INIT enabled, with the option ohci1394_dma=early.
|
||||
|
||||
Then, on the debugging host, run firescope, for example by using -A::
|
||||
|
||||
firescope -A System.map-of-debug-target-kernel
|
||||
|
||||
Note: -A automatically attaches to the first non-local node. It only works
|
||||
reliably if only connected two machines are connected using FireWire.
|
||||
|
||||
After having attached to the debug target, press Ctrl-D to view the
|
||||
complete printk buffer or Ctrl-U to enter auto update mode and get an
|
||||
updated live view of recent kernel messages logged on the debug target.
|
||||
|
||||
Call "firescope -h" to get more information on firescope's options.
|
||||
|
||||
Notes
|
||||
-----
|
||||
|
||||
Documentation and specifications: http://halobates.de/firewire/
|
||||
|
||||
FireWire is a trademark of Apple Inc. - for more information please refer to:
|
||||
https://en.wikipedia.org/wiki/FireWire
|
||||
@@ -98,6 +98,7 @@ Interfaces for kernel debugging
|
||||
|
||||
debug-objects
|
||||
tracepoint
|
||||
debugging-via-ohci1394
|
||||
|
||||
Everything else
|
||||
===============
|
||||
|
||||
Reference in New Issue
Block a user