Currently if a process creates a bunch of threads using pthread_create
and then perf is run in system_wide mode, the mmaps for those threads
are not captured with a synthesized mmap event.
The reason is those threads are not visible when walking the /proc/
directory looking for /proc/<pid>/maps files. Instead they are
discovered using the /proc/<pid>/tasks file (which the synthesized comm
event uses).
This causes problems when a program is trying to map a data address to a
tid. Because the tid has no maps, the event is dropped. Changing the
program to look up using the pid instead of the tid, finds the correct
maps but creates ugly hacks in the program to carry the correct tid
around.
Fix this by moving the walking of the /proc/<pid>/tasks up a level (out
of the comm function) based on Arnaldo's suggestion.
Tweaked things a bit to special case the 'full' bit and 'guest' check.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1393429527-167840-2-git-send-email-dzickus@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When trying to map a bunch of instruction addresses to their respective
threads, I kept getting a lot of bogus entries [I forget the exact
reason as I patched my code months ago].
Looking through ip__resolve_ams, I noticed the check for
if (al.sym)
and realized, most times I have an al.map definition but sometimes an
al.sym is undefined. In the cases where al.sym is undefined, the loop
keeps going even though a valid al.map exists.
Modify this check to use the more reliable al.map. This fixed my bogus
entries.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1393386227-149412-2-git-send-email-dzickus@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When compiling perf tool code with gcc 4.4.7 I'm getting
following error:
CC util/session.o
cc1: warnings being treated as errors
util/session.c: In function ‘perf_session_deliver_event’:
tools/perf/util/include/linux/bitops.h:109: error: dereferencing pointer ‘p’ does break strict-aliasing rules
tools/perf/util/include/linux/bitops.h:101: error: dereferencing pointer ‘p’ does break strict-aliasing rules
util/session.c:697: note: initialized from here
tools/perf/util/include/linux/bitops.h:101: note: initialized from here
make[1]: *** [util/session.o] Error 1
make: *** [util/session.o] Error 2
The aliased types here are u64 and unsigned long pointers, which is safe
for the find_first_bit processing.
This error shows up for me only for gcc 4.4 on 32bit x86, even for
-Wstrict-aliasing=3, while newer gcc are quiet and scream here for
-Wstrict-aliasing={2,1}. Looks like newer gcc changed the rules for
strict alias warnings.
The gcc documentation offers workaround for valid aliasing by using
__may_alias__ attribute:
http://gcc.gnu.org/onlinedocs/gcc-4.4.0/gcc/Type-Attributes.html
Using this workaround for the find_first_bit function.
Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1393434867-20271-1-git-send-email-jolsa@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Stephane reported that perf report and annotate failed to process data
using lots of (> 500) shared libraries. It was because of the limit on
number of open files (ulimit -n).
Currently when perf loads a DSO, it'll look for normal and dynamic
symbol tables. And if it fails to find out both tables, it'll iterate
all of possible symtab types. But many of them are useless since they
have no additional information and the problem is that it's not closing
those files even though they're not used. Fix it.
Reported-by: Stephane Eranian <eranian@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Cody P Schafer <cody@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung.kim@lge.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1392859976-32760-2-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The TUI of perf report and top support annotation, but stdio and GTK
don't. So it should be checked before calling hist_entry__inc_addr_
samples() to avoid wasting resources that will never be used.
perf annotate need it regardless of UI and sort keys, so the check
of whether to allocate resources should be on the tools that have
annotate as an option in the TUI, 'report' and 'top', not on the
function called by all of them.
It caused perf annotate on ppc64 to produce zero output, since the
buckets were not being allocated.
Reported-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Anton Blanchard <anton@samba.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung.kim@lge.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1392859976-32760-1-git-send-email-namhyung@kernel.org
[ Renamed (report,top)__needs_annotate() to ui__has_annotation() ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allow to add events on the local functions without debuginfo.
(With the debuginfo, we can add events even on inlined functions)
Currently, probing on local functions requires debuginfo to
locate actual address. It is also possible without debuginfo since
we have symbol maps.
Without this change;
----
# ./perf probe -a t_show
Added new event:
probe:t_show (on t_show)
You can now use it in all perf tools, such as:
perf record -e probe:t_show -aR sleep 1
# ./perf probe -x perf -a identity__map_ip
no symbols found in /kbuild/ksrc/linux-3/tools/perf/perf, maybe install a debug package?
Failed to load map.
Error: Failed to add events. (-22)
----
As the above results, perf probe just put one event
on the first found symbol for kprobe event. Moreover,
for uprobe event, perf probe failed to find local
functions.
With this change;
----
# ./perf probe -a t_show
Added new events:
probe:t_show (on t_show)
probe:t_show_1 (on t_show)
probe:t_show_2 (on t_show)
probe:t_show_3 (on t_show)
You can now use it in all perf tools, such as:
perf record -e probe:t_show_3 -aR sleep 1
# ./perf probe -x perf -a identity__map_ip
Added new events:
probe_perf:identity__map_ip (on identity__map_ip in /kbuild/ksrc/linux-3/tools/perf/perf)
probe_perf:identity__map_ip_1 (on identity__map_ip in /kbuild/ksrc/linux-3/tools/perf/perf)
probe_perf:identity__map_ip_2 (on identity__map_ip in /kbuild/ksrc/linux-3/tools/perf/perf)
probe_perf:identity__map_ip_3 (on identity__map_ip in /kbuild/ksrc/linux-3/tools/perf/perf)
You can now use it in all perf tools, such as:
perf record -e probe_perf:identity__map_ip_3 -aR sleep 1
----
Now we succeed to put events on every given local functions
for both kprobes and uprobes. :)
Note that this also introduces some symbol rbtree
iteration macros; symbols__for_each, dso__for_each_symbol,
and map__for_each_symbol. These are for walking through
the symbol list in a map.
Changes from v2:
- Fix add_exec_to_probe_trace_events() not to convert address
to tp->symbol any more.
- Fix to set kernel probes based on ref_reloc_sym.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: "David A. Long" <dave.long@linaro.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: yrl.pp-manager.tt@hitachi.com
Link: http://lkml.kernel.org/r/20140206053225.29635.15026.stgit@kbuild-fedora.yrl.intra.hitachi.co.jp
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Find the given address from offline dwarfs instead of online kernel
dwarfs.
On the KASLR enabled kernel, the kernel text section is loaded with
random offset, and the debuginfo__new_online_kernel can't handle it. So
let's move to the offline dwarf loader instead of using the online dwarf
loader.
As a result, since we don't need debuginfo__new_online_kernel any more,
this also removes the functions related to that.
Without this change;
# ./perf probe -l
probe:t_show (on _stext+901288 with m v)
probe:t_show_1 (on _stext+939624 with m v t)
probe:t_show_2 (on _stext+980296 with m v fmt)
probe:t_show_3 (on _stext+1014392 with m v file)
With this change;
# ./perf probe -l
probe:t_show (on t_show@linux-3/kernel/trace/ftrace.c with m v)
probe:t_show_1 (on t_show@linux-3/kernel/trace/trace.c with m v t)
probe:t_show_2 (on t_show@kernel/trace/trace_printk.c with m v fmt)
probe:t_show_3 (on t_show@kernel/trace/trace_events.c with m v file)
Changes from v2:
- Instead of retrying, directly opens offline dwarf.
- Remove debuginfo__new_online_kernel and related functions.
- Refer map->reloc to get the correct address of a symbol.
- Add a special case for handling ref_reloc_sym based address.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: "David A. Long" <dave.long@linaro.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: yrl.pp-manager.tt@hitachi.com
Link: http://lkml.kernel.org/r/20140206053218.29635.74821.stgit@kbuild-fedora.yrl.intra.hitachi.co.jp
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since several local symbols can have same name (e.g. t_show), we need to
use the relative address from the symbol referred by kmap->ref_reloc_sym
instead of the target symbol name itself.
Because the kernel address space layout randomize (kASLR) changes the
absolute address of kernel symbols, we can't rely on the absolute
address.
Note that this works only with debuginfo.
E.g. without this change;
----
# ./perf probe -a "t_show \$vars"
Added new events:
probe:t_show (on t_show with $vars)
probe:t_show_1 (on t_show with $vars)
probe:t_show_2 (on t_show with $vars)
probe:t_show_3 (on t_show with $vars)
You can now use it in all perf tools, such as:
perf record -e probe:t_show_3 -aR sleep 1
----
OK, we have 4 different t_show()s. All functions have
different arguments as below;
----
# cat /sys/kernel/debug/tracing/kprobe_events
p:probe/t_show t_show m=%di:u64 v=%si:u64
p:probe/t_show_1 t_show m=%di:u64 v=%si:u64 t=%si:u64
p:probe/t_show_2 t_show m=%di:u64 v=%si:u64 fmt=%si:u64
p:probe/t_show_3 t_show m=%di:u64 v=%si:u64 file=%si:u64
----
However, all of them have been put on the *same* address.
----
# cat /sys/kernel/debug/kprobes/list
ffffffff810d9720 k t_show+0x0 [DISABLED]
ffffffff810d9720 k t_show+0x0 [DISABLED]
ffffffff810d9720 k t_show+0x0 [DISABLED]
ffffffff810d9720 k t_show+0x0 [DISABLED]
----
With this change;
----
# ./perf probe -a "t_show \$vars"
Added new events:
probe:t_show (on t_show with $vars)
probe:t_show_1 (on t_show with $vars)
probe:t_show_2 (on t_show with $vars)
probe:t_show_3 (on t_show with $vars)
You can now use it in all perf tools, such as:
perf record -e probe:t_show_3 -aR sleep 1
# cat /sys/kernel/debug/tracing/kprobe_events
p:probe/t_show _stext+889880 m=%di:u64 v=%si:u64
p:probe/t_show_1 _stext+928568 m=%di:u64 v=%si:u64 t=%si:u64
p:probe/t_show_2 _stext+969512 m=%di:u64 v=%si:u64 fmt=%si:u64
p:probe/t_show_3 _stext+1001416 m=%di:u64 v=%si:u64 file=%si:u64
# cat /sys/kernel/debug/kprobes/list
ffffffffb50d95e0 k t_show+0x0 [DISABLED]
ffffffffb50e2d00 k t_show+0x0 [DISABLED]
ffffffffb50f4990 k t_show+0x0 [DISABLED]
ffffffffb50eccf0 k t_show+0x0 [DISABLED]
----
This time, each event is put in different address
correctly.
Note that currently this doesn't support address-based
probe on modules (thus the probes on modules are symbol
based), since it requires relative address probe syntax
for kprobe-tracer, and it isn't implemented yet.
One more note, this allows us to put events on correct
address, but --list option should be updated to show
correct corresponding source code.
Changes from v2:
- Refer kmap->ref_reloc_sym instead of "_stext".
- Refer map->reloc to catch up the kASLR perf fix.
Changes from v1:
- Use _stext relative address instead of actual
absolute address recorded in debuginfo.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: "David A. Long" <dave.long@linaro.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: yrl.pp-manager.tt@hitachi.com
Link: http://lkml.kernel.org/r/20140206053216.29635.22584.stgit@kbuild-fedora.yrl.intra.hitachi.co.jp
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adding call-graph option support into .perfconfig file, so it's now
possible use call-graph option like:
[top]
call-graph = fp
[record]
call-graph = dwarf,8192
Above options ONLY setup the unwind method. To enable perf record/top to
actually use it the command line option -g/-G must be specified.
The --call-graph option overloads .perfconfig setup.
Assuming above configuration:
$ perf record -g ls
- enables dwarf unwind with user stack size dump 8192 bytes
$ perf top -G
- enables frame pointer unwind
$ perf record --call-graph=fp ls
- enables frame pointer unwind
$ perf top --call-graph=dwarf,4096 ls
- enables dwarf unwind with user stack size dump 4096 bytes
Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1391427883-13443-2-git-send-email-jolsa@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
"perf list" listing of hardware events doesn't work on older ARM devices.
The change enabling event detection:
commit b41f1cec91
Author: Namhyung Kim <namhyung.kim@lge.com>
Date: Tue Aug 27 11:41:53 2013 +0900
perf list: Skip unsupported events
uses the following code in tools/perf/util/parse-events.c:
struct perf_event_attr attr = {
.type = type,
.config = config,
.disabled = 1,
.exclude_kernel = 1,
};
On ARM machines pre-dating the Cortex-A15 this doesn't work, as these
machines don't support .exclude_kernel. So starting with 3.12 "perf
list" does not report any hardware events at all on older machines (seen
on Rasp-Pi, Pandaboard, Beagleboard, etc).
This version of the patch makes changes suggested by Namhyung Kim to
check for EACCESS and retry (instead of just dropping the
exclude_kernel) so we can properly handle machines where
/proc/sys/kernel/perf_event_paranoid is set to 2.
Reported-by: Chad Paradis <chad.paradis@umit.maine.edu>
Signed-off-by: Vince Weaver <vincent.weaver@maine.edu>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Chad Paradis <chad.paradis@umit.maine.edu>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/alpine.DEB.2.10.1312301536150.28814@vincent-weaver-1.um.maine.edu
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix perf-probe not to add offset value twice to uprobe probe address
when post processing.
The tevs[i].point.address struct member is the address of symbol+offset,
but current perf-probe adjusts the point.address by adding the offset.
As a result, the probe address becomes symbol+offset+offset. This may
cause unexpected code corruption. Urgent fix is needed.
Without this fix:
---
# ./perf probe -x ./perf dso__load_vmlinux+4
# ./perf probe -l
probe_perf:dso__load_vmlinux (on 0x000000000006d2b8)
# nm ./perf.orig | grep dso__load_vmlinux\$
000000000046d0a0 T dso__load_vmlinux
---
You can see the given offset is 3 but the actual probed address is
dso__load_vmlinux+8.
With this fix:
---
# ./perf probe -x ./perf dso__load_vmlinux+4
# ./perf probe -l
probe_perf:dso__load_vmlinux (on 0x000000000006d2b4)
---
Now the problem is fixed.
Note: This bug is introduced by
commit fb7345bbf7
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: "David A. Long" <dave.long@linaro.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: yrl.pp-manager.tt@hitachi.com
Link: http://lkml.kernel.org/r/20140205051858.6519.27314.stgit@kbuild-fedora.yrl.intra.hitachi.co.jp
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
