I get such truncated annotation results in 'perf top':
: Disassembly of section .text: ▒
: ▒
: ffffffff810966a8 <nr_iowait_cpu>: ▒
4.94 : ffffffff810966a8: movslq %edi,%rdi ▒
3.70 : ffffffff810966ab: mov $0x13700,%rax ▒
0.00 : ffffffff810966b2: add -0x7e32cb00(,%rdi,8),%rax ▒
8.64 : ffffffff810966ba: mov 0x7e0(%rax),%eax ▒
82.72 : ffffffff810966c0: cltq ▒
Note the missing 'retq' which is there in the original function:
ffffffff810966a8 <nr_iowait_cpu>:
ffffffff810966a8: 48 63 ff movslq %edi,%rdi
ffffffff810966ab: 48 c7 c0 00 37 01 00 mov $0x13700,%rax
ffffffff810966b2: 48 03 04 fd 00 35 cd add -0x7e32cb00(,%rdi,8),%rax
ffffffff810966b9: 81
ffffffff810966ba: 8b 80 e0 07 00 00 mov 0x7e0(%rax),%eax
ffffffff810966c0: 48 98 cltq
ffffffff810966c2: c3 retq
ffffffff810966c3 <this_cpu_load>:
I'm using a fairly recent binutils:
GNU objdump version 2.21.51.0.6-2.fc16 20110118
AFAICS the bug is simply that sym->end points to the last byte
of the symbol in question - while objdump's --stop-address
expects the last byte plus 1 to disassemble the full range.
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20111223130804.GA24305@elte.hu
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
perf does not properly handle monitoring of processes with named threads.
For example:
$ ps -C myapp -L
PID LWP TTY TIME CMD
25118 25118 ? 00:00:00 myapp
25118 25119 ? 00:00:00 myapp:worker
perf record -e cs -c 1 -fo /tmp/perf.data -p 25118 -- sleep 10
perf report --stdio -i /tmp/perf.data
100.00% myapp:worker [kernel.kallsyms] [k] perf_event_task_sched_out
The process name is set to the name of the last thread it finds for the
process.
The Problem:
perf-top and perf-record both create a thread_map of threads to be
monitored. That map is used in perf_event__synthesize_thread_map which
loops over the entries in thread_map and calls __event__synthesize_thread
to generate COMM and MMAP events.
__event__synthesize_thread calls perf_event__synthesize_comm which opens
/proc/pid/status, reads the name of the task and its thread group id.
That's all fine. The problem is that it then reads /proc/pid/task and
generates COMM events for each task it finds - but using the name found
in /proc/pid/status where pid is the thread of interest.
The end result (looping over thread_map + synthesizing comm events for
each thread each time) means the name of the last thread processed sets
the name for all threads in the process - which is not good for
multithreaded processes with named threads.
The Fix:
perf_event__synthesize_comm has an input argument (full) that decides
whether to process task entries for each pid it is passed. It currently
never set to 0 (perf_event__synthesize_comm has a single caller and it
always passes the value 1). Let's fix that.
Add the full input argument to __event__synthesize_thread which passes
it to perf_event__synthesize_comm. For thread/process monitoring set full
to 0 which means COMM and MMAP events are only generated for the pid
passed to it. For system wide monitoring set full to 1 so that COMM events
are generated for all threads in a process.
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1324578603-12762-2-git-send-email-dsahern@gmail.com
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Implement a simple test for the self-monitoring data from the
perf mmap data area control page:
6: x86 rdpmc test:
0: 6053
1: 60053
2: 600059
3: 6000059
4: 60000075
5: 600000247
Ok
The counts are expected to increase monotonically - these
are recovered via RDPMC, without calling into the kernel.
It might be nice to add logic to automagically turn these numbers into OK/FAIL.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Arun Sharma <asharma@fb.com>
Link: http://lkml.kernel.org/n/tip-evf5yii88ljdgmaihccbxxw1@git.kernel.org
[ various small improvements ]
Signed-off-by: Ingo Molnar <mingo@elte.hu>
The problem is that when SAMPLE_PERIOD is not set, the kernel generates
a number of samples in proportion to an event's period. Number of these
samples may be too big and the kernel throttles all samples above a
defined limit.
E.g.: I want to trace when a process sleeps. I created a process which
sleeps for 1ms and for 4ms. perf got 100 events in both cases.
swapper 0 [000] 1141.371830: sched_stat_sleep: comm=foo pid=1801 delay=1386750 [ns]
swapper 0 [000] 1141.369444: sched_stat_sleep: comm=foo pid=1801 delay=4499585 [ns]
In the first case a kernel want to send 4499585 events and in the second
case it wants to send 1386750 events. perf-reports shows that process
sleeps in both places equal time.
Instead of this we can get only one sample with an attribute period. As
result we have less data transferring between kernel and user-space and
we avoid throttling of samples.
The patch "events: Don't divide events if it has field period" added a
kernel part of this functionality.
Acked-by: Arun Sharma <asharma@fb.com>
Cc: Arun Sharma <asharma@fb.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: devel@openvz.org
Link: http://lkml.kernel.org/r/1324391565-1369947-1-git-send-email-avagin@openvz.org
Signed-off-by: Andrew Vagin <avagin@openvz.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The option is documented in man perf-script but was not yet implemented:
-a
Force system-wide collection. Scripts run without a
<command> normally use -a by default, while scripts run
with a <command> normally don't - this option allows the
latter to be run in system-wide mode.
As with perf record you now can profile in system-wide mode for the
runtime of a given command, e.g.:
# perf script -a syscall-counts sleep 2
Cc: Ingo Molnar <mingo@elte.hu>
Link: http://lkml.kernel.org/r/1322229925-10075-1-git-send-email-robert.richter@amd.com
Signed-off-by: Robert Richter <robert.richter@amd.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A update is made to the sched:sched_switch event that adds some
logic to the first parameter of the __print_flags() that shows the
state of tasks. This change cause perf to fail parsing the flags.
A simple fix is needed to have the parser be able to process ops
within the argument.
Cc: stable@vger.kernel.org
Reported-by: Andrew Vagin <avagin@openvz.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
perf stat is failing on PowerPC:
Error: open_counter returned with 95 (Operation not supported). /bin/dmesg may provide additional information.
Fatal: Not all events could be opened.
commit 370faf1dd0 (perf stat: Fail softly on unsupported events)
added a check for failure returning ENOENT, but the POWER backend
returns EOPNOTSUPP. It looks like alpha, blackfin and mips do the
same.
With the patch applied, things work as expected:
Performance counter stats for '/bin/true':
0.362176 task-clock # 0.623 CPUs utilized
0 context-switches # 0.000 M/sec
0 CPU-migrations # 0.000 M/sec
28 page-faults # 0.077 M/sec
1,677,020 cycles # 4.630 GHz
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
431,220 instructions # 0.26 insns per cycle
101,889 branches # 281.325 M/sec
4,145 branch-misses # 4.07% of all branches
0.000581361 seconds time elapsed
Cc: <stable@kernel.org> # 3.0+
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20111202093833.5fef7226@kryten
Signed-off-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This new test will validate these new routines extracted from 'perf
record':
- perf_evlist__config_attrs
- perf_evlist__prepare_workload
- perf_evlist__start_workload
In addition to several other perf_evlist methods.
It consists of starting a simple workload, setting up just one event to
monitor ("cycles") requesting that several PERF_SAMPLE_ fields be
present in all events.
It then will check that the expected PERF_RECORD_ events are produced
and will sanity check all its fields.
Some checks performed:
. PERF_SAMPLE_TIME monotonically increases.
. PERF_SAMPLE_CPU is the one requested with sched_setaffinity
. PERF_SAMPLE_TID and PERF_SAMPLE_PID matches the one we forked
in perf_evlist__prepare_workload and that is stored in
evlist->workload.pid
. For the events where these fields are also present in its
pre-sample_id_all fields (e.g. event->mmap.pid), that they are what
is expected too.
. That we get a bunch of mmaps:
PATH/libcSUFFIX
PATH/ldSUFFIX
[vdso]
PATH/sleep
Example:
[root@emilia ~]# taskset -c 3,4 perf test -v1 perf_sample
6: Validate PERF_RECORD_* events & perf_sample fields:
--- start ---
7159480799825 3 PERF_RECORD_SAMPLE
7159480805584 3 PERF_RECORD_SAMPLE
7159480807814 3 PERF_RECORD_SAMPLE
7159480810430 3 PERF_RECORD_SAMPLE
7159480861511 3 PERF_RECORD_MMAP 8086/8086: [0x7fffffffd000(0x2000) @ 0x7fffffffd000]: //anon
7159481052516 3 PERF_RECORD_COMM: sleep:8086
7159481070188 3 PERF_RECORD_MMAP 8086/8086: [0x400000(0x6000) @ 0]: /bin/sleep
7159481077104 3 PERF_RECORD_MMAP 8086/8086: [0x3d06400000(0x221000) @ 0]: /lib64/ld-2.12.so
7159481092912 3 PERF_RECORD_MMAP 8086/8086: [0x7fff1adff000(0x1000) @ 0x7fff1adff000]: [vdso]
7159481196779 3 PERF_RECORD_MMAP 8086/8086: [0x3d06800000(0x37f000) @ 0]: /lib64/libc-2.12.so
7160481558435 3 PERF_RECORD_EXIT(8086:8086):(8086:8086)
---- end ----
Validate PERF_RECORD_* events & perf_sample fields: Ok
[root@emilia ~]#
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/n/tip-svag18v2z4idas0dyz3umjpq@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To obtain a list of available tests:
[root@emilia linux]# perf test list
1: vmlinux symtab matches kallsyms
2: detect open syscall event
3: detect open syscall event on all cpus
4: read samples using the mmap interface
5: parse events tests
[root@emilia linux]#
To list just a subset:
[root@emilia linux]# perf test list syscall
2: detect open syscall event
3: detect open syscall event on all cpus
[root@emilia linux]#
To run a subset:
[root@emilia linux]# perf test detect
2: detect open syscall event: Ok
3: detect open syscall event on all cpus: Ok
[root@emilia linux]#
Specific tests can be chosen by number:
[root@emilia linux]# perf test 1 3 parse
1: vmlinux symtab matches kallsyms: Ok
3: detect open syscall event on all cpus: Ok
5: parse events tests: Ok
[root@emilia linux]#
Now to write more tests!
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/n/tip-nqec2145qfxdgimux28aw7v8@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently the meaning of -C varies by perf command: for perf-top,
perf-stat, perf-record it means cpu list. For perf-report it means comm
list. Then perf-annotate, perf-report and perf-script use -c for cpu
list.
Fix annotate, report and script to use -C for cpu list to be consistent
with top, stat and record. This means report needs to use -c for comm
list which does introduce a backward compatibility change.
v1 -> v2
- update perf-script.txt too
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1321209008-7004-1-git-send-email-dsahern@gmail.com
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
