perf report has an option (--max-stack) to set the maximum stack depth
when processing callchains. The option defaults to the hard-coded
maximum definition PERF_MAX_STACK_DEPTH which is 127. The intention of
the option is to allow the user to reduce the processing time by
reducing the amount of the callchain that is processed.
It is also possible, when processing instruction traces, to synthesize
callchains. Synthesized callchains do not have the kernel size
limitation and are whatever size the user requests, although validation
presently prevents the user requested a value greater that 1024. The
default value is 16.
To allow for synthesized callchains, make the max_stack value at least
the same size as the synthesized callchain size.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-16-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support for generating branch stack context for PT samples. The
decoder reports a configurable number of branches as branch context for
each sample. Internally it keeps track of them by using a simple sliding
window. We also flush the last branch buffer on each sample to avoid
overlapping intervals.
This is useful for:
- Reporting accurate basic block edge frequencies through the perf
report branch view
- Using with --branch-history to get the wider context of samples
- Other users of LBRs
Also the Documentation is updated.
Examples:
Record with Intel PT:
perf record -e intel_pt//u ls
Branch stacks are used by default if synthesized so:
perf report --itrace=ile
is the same as:
perf report --itrace=ile -b
Branch history can be requested also:
perf report --itrace=igle --branch-history
Based-on-patch-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-15-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A non-synthesized event might not have a branch stack if branch stacks
have been synthesized (using itrace options).
An example of that is when Intel PT records sched_switch events for
decoding purposes. Those sched_switch events do not have branch stacks
even though the Intel PT decoder may be synthesizing other events that
do due to the itrace options.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-12-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
perf report looks at event sample types to determine if branch stacks
have been sampled. Adjust the validation to know about instruction
tracing options.
This change allows the use of the -b option which otherwise would
complain with an error like:
Error:
Selected -b but no branch data. Did you call perf record without -b?
# To display the perf.data header info,
# please use --header/--header-only options.
#
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-10-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
TSC packets contain only 7 bytes of TSC. The 8th byte is assumed to
change so infrequently that its value can be inferred. However the
logic must cater for a 7 byte wraparound, which it does by adding 1 to
the top byte.
The existing code was doing that with a while loop even though the
addition should only need to be done once. That logic won't work (will
loop forever) if TSC wraps around at the 8th byte. Theoretically that
would take at least 10 years, unless something else went wrong.
And what else could go wrong. Well, if the chunks of trace data are
processed out of order, it will make it look like the 7-byte TSC has
gone backwards (i.e. wrapped). If that happens 256 times then stuck in
the while loop it will be.
Fix that by getting rid of the unnecessary while loop.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Processing instruction tracing data (e.g. Intel PT) can synthesize
callchains e.g.
$ perf record -e intel_pt//u uname
$ perf report --stdio --itrace=ige
However perf report's callgraph option gets extra validation, so:
$ perf report --stdio --itrace=ige -gflat
Error:
Selected -g or --branch-history but no callchain data. Did
you call 'perf record' without -g?
# To display the perf.data header info,
# please use --header/--header-only options.
#
Fix the validation to know about instruction tracing options so
above command works.
A side-effect of the change is that the default option to
accumulate the callchain of child functions comes into force.
To get the previous behaviour the --no-children option can be
used e.g.
$ perf report --stdio --itrace=ige -gflat --no-children
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-3-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instruction tracing options (i.e. --itrace) include an option for
sampling instructions at an arbitrary period. e.g.
--itrace=i10us
means make an 'instructions' sample for every 10us of trace.
Currently the logic does not distinguish between a period of
zero and no period being specified at all, so it gets treated
as the default period which is 100000. That doesn't really
make sense.
Fix it so that zero period is accepted and treated as meaning
"as often as possible".
In the case of Intel PT that is the same as a period of 1 and
a unit of 'instructions' (i.e. --itrace=i1i).
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-2-git-send-email-adrian.hunter@intel.com
[ Add a few lines describing this in the Documentation/intel-pt.txt file ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A copy of /proc/kcore containing the kernel text can be made to the
buildid cache. e.g.
perf buildid-cache -v -k /proc/kcore
To workaround objdump limitations, a copy is also made when annotating
against /proc/kcore.
The copying process stops working from libelf about v1.62 onwards (the
problem was found with v1.63).
The cause is that a call to gelf_getphdr() in kcore__add_phdr() fails
because additional validation has been added to gelf_getphdr().
The use of gelf_getphdr() is a misguided attempt to get default
initialization of the Gelf_Phdr structure. That should not be
necessary because every member of the Gelf_Phdr structure is
subsequently assigned. So just remove the call to gelf_getphdr().
Similarly, a call to gelf_getehdr() in gelf_kcore__init() can be
removed also.
Committer notes:
Note to stable@kernel.org, from Adrian in the cover letter for this
patchkit:
The "Fix copying of /proc/kcore" problem goes back to v3.13 if you think
it is important enough for stable.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@kernel.org
Link: http://lkml.kernel.org/r/1443089122-19082-3-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We have map_groups__find_by_name() to look at the list of modules that
are in place for a given machine, so use it instead of traversing the
machine dso list, which also includes DSOs for userspace.
When merging the user and kernel DSO lists a bug was introduced where
'perf probe' stopped being able to add probes to modules using its short
name:
# perf probe -m usbnet --add usbnet_start_xmit
usbnet_start_xmit is out of .text, skip it.
Error: Failed to add events.
#
With this fix it works again:
# perf probe -m usbnet --add usbnet_start_xmit
Added new event:
probe:usbnet_start_xmit (on usbnet_start_xmit in usbnet)
You can now use it in all perf tools, such as:
perf record -e probe:usbnet_start_xmit -aR sleep 1
#
Reported-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Stephane Eranian <eranian@google.com>
Fixes: 3d39ac5386 ("perf machine: No need to have two DSOs lists")
Link: http://lkml.kernel.org/r/20150924015008.GE1897@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
User visible changes:
- Fix a segfault in 'perf probe' when removing uprobe events. (Masami Hiramatsu)
- Synthesize COMM event for workloads started from the command line in 'perf
record' so that we can have the pid->comm mapping before we get the real
PERF_RECORD_COMM switching from perf to the workload. (Namhyung Kim)
- Fix build tools/vm/ due to removal of tools/lib/api/fs/debugfs.h.
(Arnaldo Carvalho de Melo)
Infrastructure changes:
- Fix the make tarball targets by including the recently added err.h header in
the perf MANIFEST file. (Jiri Olsa)
- Don't assume that the event parser returns a non empty evlist. (Wang Nan)
- Add way to disambiguate feature detection state files, needed to use
tools/build feature detection for multiple components in a single O= output
dir, which will be the case with tools/perf/ and tools/lib/bpf/.
(Arnaldo Carvalho de Melo)
- Fixup FEATURE_{TESTS,DISPLAY} inversion in tools/lib/bpf/. (Arnaldo Carvalho de Melo)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
When perf creates a new child to profile, the events are enabled on
exec(). And in this case, it doesn't synthesize any event for the
child since they'll be generated during exec(). But there's an window
between the enabling and the event generation.
It used to be overcome since samples are only in kernel (so we always
have the map) and the comm is overridden by a later COMM event.
However it won't work if events are processed and displayed before the
COMM event overrides like in 'perf script'. This leads to those early
samples (like native_write_msr_safe) not having a comm but pid (like
':15328').
So it needs to synthesize COMM event for the child explicitly before
enabling so that it can have a correct comm. But at this time, the
comm will be "perf" since it's not exec-ed yet.
Committer note:
Before this patch:
# perf record usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.017 MB perf.data (7 samples) ]
# perf script --show-task-events
:4429 4429 27909.079372: 1 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
:4429 4429 27909.079375: 1 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
:4429 4429 27909.079376: 10 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
:4429 4429 27909.079377: 223 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
:4429 4429 27909.079378: 6571 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
usleep 4429 27909.079380: PERF_RECORD_COMM exec: usleep:4429/4429
usleep 4429 27909.079381: 185403 cycles: ffffffff810a72d3 flush_signal_handlers (/lib/modules/4.
usleep 4429 27909.079444: 2241110 cycles: 7fc575355be3 _dl_start (/usr/lib64/ld-2.20.so)
usleep 4429 27909.079875: PERF_RECORD_EXIT(4429:4429):(4429:4429)
After:
# perf record usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.017 MB perf.data (7 samples) ]
# perf script --show-task
perf 0 0.000000: PERF_RECORD_COMM: perf:8446/8446
perf 8446 30154.038944: 1 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
perf 8446 30154.038948: 1 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
perf 8446 30154.038949: 9 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
perf 8446 30154.038950: 230 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
perf 8446 30154.038951: 6772 cycles: ffffffff8105f45a native_write_msr_safe (/lib/modules/4.
usleep 8446 30154.038952: PERF_RECORD_COMM exec: usleep:8446/8446
usleep 8446 30154.038954: 196923 cycles: ffffffff81766440 _raw_spin_lock (/lib/modules/4.3.0-rc1
usleep 8446 30154.039021: 2292130 cycles: 7f609a173dc4 memcpy (/usr/lib64/ld-2.20.so)
usleep 8446 30154.039349: PERF_RECORD_EXIT(8446:8446):(8446:8446)
#
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1442881495-2928-1-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix a segfault bug and a small mistake in perf probe -d.
Since the "ulist" in perf_del_probe_events is never initialized,
strlist__add(ulist, *) always causes a segfault when removing
uprobe events by perf probe -d.
Also, the "str" local variable is never released if fail to
allocate the "klist". This fixes it too.
This has been introduced by the commit e607f1426b ("perf probe:
Print deleted events in cmd_probe()").
Reported-by: Milian Wolff <milian.wolff@kdab.com>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20150916125241.4446.44805.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If a session contains no events, we can get stuck in an infinite loop in
__perf_session__process_events, with a non-zero file_size and data_offset, but
a zero data_size.
In this case, we can mmap the entirety of the file (consisting of the file and
attribute headers), and fetch_mmaped_event will correctly refuse to read any
(unmapped and non-existent) event headers. This causes
__perf_session__process_events to unmap the file and retry with the exact same
parameters, getting stuck in an infinite loop.
This has been observed to result in an exit-time hang when counting
rare/unschedulable events with perf record, and can be triggered artificially
with the script below:
----
#!/bin/sh
printf "REPRO: launching perf\n";
./perf record -e software/config=9/ sleep 1 &
PERF_PID=$!;
sleep 0.002;
kill -2 $PERF_PID;
printf "REPRO: waiting for perf (%d) to exit...\n" "$PERF_PID";
wait $PERF_PID;
printf "REPRO: perf exited\n";
----
To avoid this, have __perf_session__process_events bail out early when
the file has no data (i.e. it has no events).
Commiter note:
I only managed to reproduce this when setting
/proc/sys/kernel/kptr_restrict to '1' and changing the code to
purposefully not process any samples and no synthesized samples, i.e.
kptr_restrict prevents 'record' from synthesizing the kernel mmaps for
vmlinux + modules and since it is a workload started from perf, we don't
synthesize mmap/comm records for existing threads.
Adrian Hunter managed to reproduce it in his environment tho.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Tested-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1442423929-12253-1-git-send-email-mark.rutland@arm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The existing numa test checks only if numa.h and numa_available() are
present, but that can be satisfied with an old libnuma that is not
enough for the 'perf bench numa' entry, so add a test to check for that:
[acme@rhel5 linux]$ make NO_AUXTRACE=1 NO_LIBPERL=1 -C tools/perf O=/tmp/build/perf install-bin
make: Entering directory `/home/acme/git/linux/tools/perf'
BUILD: Doing 'make -j2' parallel build
Auto-detecting system features:
... libelf: [ on ]
... libnuma: [ on ]
... numa_num_possible_cpus: [ OFF ]
... libperl: [ on ]
<SNIP>
config/Makefile:577: Old numa library found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev >= 2.0.8
INSTALL binaries
<SNIP>
This fixes the build on old systems such as RHEL/CentOS 5.11.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Victor Kamensky <victor.kamensky@linaro.org>
Cc: Vinson Lee <vlee@twopensource.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-zqriqkezppi2de2iyjin1tnc@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Per-pkg events need to be captured once per processor socket. The code
in check_per_pkg() ensures only one value per processor package is used.
However there is a problem with this function in case the first CPU of
the package does not measure anything for the per-pkg event, but other
CPUs do.
Consider the following:
$ create cgroup FOO; echo $$ >FOO/tasks; taskset -c 1 noploop &
$ perf stat -a -I 1000 -e intel_cqm/llc_occupancy/ -G FOO sleep 100
1.00000 <not counted> Bytes intel_cqm/llc_occupancy/ FOO
The reason for this is that CPU0 in the cgroup has nothing running on it.
Yet check_per_plg() will mark socket0 as processed and no other event
value will be considered for the socket.
This patch fixes the problem by having check_per_pkg() only consider
events which actually ran.
Signed-off-by: Stephane Eranian <eranian@google.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kan Liang <kan.liang@intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1441286620-10117-1-git-send-email-eranian@google.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
