Pull perf tooling updates from Thomas Gleixner:
"A set of perf improvements and fixes:
perf db-export:
- Improvements in how COMM details are exported to databases for post
processing and use in the sql-viewer.py UI.
- Export switch events to the database.
BPF:
- Bump rlimit(MEMLOCK) for 'perf test bpf' and 'perf trace', just
like selftests/bpf/bpf_rlimit.h do, which makes errors due to
exhaustion of this limit, which are kinda cryptic (EPERM sometimes)
less frequent.
perf version:
- Fix segfault due to missing OPT_END(), noticed on PowerPC.
perf vendor events:
- Add JSON files for IBM s/390 machine type 8561.
perf cs-etm (ARM):
- Fix two cases of error returns not bing done properly: Invalid
ERR_PTR() use and loss of propagation error codes"
* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (28 commits)
perf version: Fix segfault due to missing OPT_END()
perf vendor events s390: Add JSON files for machine type 8561
perf cs-etm: Return errcode in cs_etm__process_auxtrace_info()
perf cs-etm: Remove errnoeous ERR_PTR() usage in cs_etm__process_auxtrace_info
perf scripts python: export-to-postgresql.py: Export switch events
perf scripts python: export-to-sqlite.py: Export switch events
perf db-export: Export switch events
perf db-export: Factor out db_export__threads()
perf script: Add scripting operation process_switch()
perf scripts python: exported-sql-viewer.py: Use new 'has_calls' column
perf scripts python: exported-sql-viewer.py: Remove redundant semi-colons
perf scripts python: export-to-postgresql.py: Add has_calls column to comms table
perf scripts python: export-to-sqlite.py: Add has_calls column to comms table
perf db-export: Also export thread's current comm
perf db-export: Factor out db_export__comm()
perf scripts python: export-to-postgresql.py: Export comm details
perf scripts python: export-to-sqlite.py: Export comm details
perf db-export: Export comm details
perf db-export: Fix a white space issue in db_export__sample()
perf db-export: Move export__comm_thread into db_export__sample()
...
Pull tracing updates from Steven Rostedt:
"The main changes in this release include:
- Add user space specific memory reading for kprobes
- Allow kprobes to be executed earlier in boot
The rest are mostly just various clean ups and small fixes"
* tag 'trace-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (33 commits)
tracing: Make trace_get_fields() global
tracing: Let filter_assign_type() detect FILTER_PTR_STRING
tracing: Pass type into tracing_generic_entry_update()
ftrace/selftest: Test if set_event/ftrace_pid exists before writing
ftrace/selftests: Return the skip code when tracing directory not configured in kernel
tracing/kprobe: Check registered state using kprobe
tracing/probe: Add trace_event_call accesses APIs
tracing/probe: Add probe event name and group name accesses APIs
tracing/probe: Add trace flag access APIs for trace_probe
tracing/probe: Add trace_event_file access APIs for trace_probe
tracing/probe: Add trace_event_call register API for trace_probe
tracing/probe: Add trace_probe init and free functions
tracing/uprobe: Set print format when parsing command
tracing/kprobe: Set print format right after parsed command
kprobes: Fix to init kprobes in subsys_initcall
tracepoint: Use struct_size() in kmalloc()
ring-buffer: Remove HAVE_64BIT_ALIGNED_ACCESS
ftrace: Enable trampoline when rec count returns back to one
tracing/kprobe: Do not run kprobe boot tests if kprobe_event is on cmdline
tracing: Make a separate config for trace event self tests
...
Export switch events to a new table 'context_switches' and create a view
'context_switches_view'. The table and view will show automatically in
the exported-sql-viewer.py script.
If the table ends up empty, then it and the view are dropped.
Committer testing:
Use the exported-sql-viewer.py and look at "Tables" ->
"context_switches":
id machine_id time cpu thread_out_id comm_out_id thread_in_id comm_in_id flags
1 1 187836111885918 7 1 1 2 2 3
2 1 187836111889369 7 1 1 2 2 0
3 1 187836112464618 7 2 3 1 1 1
4 1 187836112465511 7 2 3 1 1 0
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20190710085810.1650-21-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that a thread's current comm is exported, it shows up in the call graph
and call tree even if it has no calls. That can happen because the calls
are recorded against the main thread's initial comm.
Add a table column to make it easy for the exported-sql-viewer.py script to
select only comms with calls.
Committer testing:
$ rm -f simple-retpoline.db
$ sudo ~acme/bin/perf script -i simple-retpoline.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls
2019-07-10 12:25:33.200529 Creating database ...
2019-07-10 12:25:33.211548 Writing records...
2019-07-10 12:25:33.549630 Adding indexes
2019-07-10 12:25:33.560715 Dropping unused tables
2019-07-10 12:25:33.580201 Done
$ sha256sum tools/perf/scripts/python/export-to-sqlite.py ~/libexec/perf-core/scripts/python/export-to-sqlite.py
2922b642c392004dffa1d8789296478c85904623f5895bcb9b6cbf33e3ca999f tools/perf/scripts/python/export-to-sqlite.py
2922b642c392004dffa1d8789296478c85904623f5895bcb9b6cbf33e3ca999f /home/acme/libexec/perf-core/scripts/python/export-to-sqlite.py
$
$ sqlite3 simple-retpoline.db
SQLite version 3.26.0 2018-12-01 12:34:55
Enter ".help" for usage hints.
sqlite> .schema comms
CREATE TABLE comms (id integer NOT NULL PRIMARY KEY,comm varchar(16),c_thread_id bigint,c_time bigint,exec_flag boolean, has_calls boolean);
sqlite> select id,has_calls from comms;
0|1
1|1
sqlite> select distinct comm_id from calls;
0
1
sqlite>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20190710085810.1650-15-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that a thread's current comm is exported, it shows up in the call
graph and call tree even if it has no calls. That can happen because the
calls are recorded against the main thread's initial comm.
Add a table column to make it easy for the exported-sql-viewer.py script
to select only comms with calls.
Committer notes:
Running the export-to-sqlite.py worked without warnings and using the
exported-sql-viewer.py worked as before.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20190710085810.1650-14-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Calls to db_export__thread() already have main_thread so there is no
reason to get it again, instead pass it as a parameter. Note that one
difference in this approach is that the main thread is not created if it
does not exist. It is better if it is not created because:
- If main_thread is being traced it will have been created already.
- If it is not being traced, there will be no other information about
it, and it will never get deleted because there will be no EXIT event.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20190710085810.1650-4-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Circa v5.2 this started to fail:
# perf trace -e /wb/augmented_raw_syscalls.o
event syntax error: '/wb/augmented_raw_syscalls.o'
\___ Operation not permitted
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [<options>] [<command>]
or: perf trace [<options>] -- <command> [<options>]
or: perf trace record [<options>] [<command>]
or: perf trace record [<options>] -- <command> [<options>]
-e, --event <event> event/syscall selector. use 'perf list' to list available events
#
In verbose mode we some -EPERM when creating a BPF map:
# perf trace -v -e /wb/augmented_raw_syscalls.o
<SNIP>
libbpf: failed to create map (name: '__augmented_syscalls__'): Operation not permitted
libbpf: failed to load object '/wb/augmented_raw_syscalls.o'
bpf: load objects failed: err=-1: (Operation not permitted)
event syntax error: '/wb/augmented_raw_syscalls.o'
\___ Operation not permitted
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [<options>] [<command>]
or: perf trace [<options>] -- <command> [<options>]
or: perf trace record [<options>] [<command>]
or: perf trace record [<options>] -- <command> [<options>]
-e, --event <event> event/syscall selector. use 'perf list' to list available events
#
If we bumped 'ulimit -l 128' to get it from the 64k default to double that, it
worked, so use the recently added rlimit__bump_memlock() helper:
# perf trace -e /wb/augmented_raw_syscalls.o -e open*,*sleep sleep 1
0.000 ( 0.007 ms): sleep/28042 openat(dfd: CWD, filename: "/etc/ld.so.cache", flags: RDONLY|CLOEXEC) = 3
0.022 ( 0.004 ms): sleep/28042 openat(dfd: CWD, filename: "/lib64/libc.so.6", flags: RDONLY|CLOEXEC) = 3
0.201 ( 0.007 ms): sleep/28042 openat(dfd: CWD, filename: "", flags: RDONLY|CLOEXEC) = 3
0.241 (1000.421 ms): sleep/28042 nanosleep(rqtp: 0x7ffd6c3e6ed0) = 0
#
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-j6f2ioa6hj9dinzpjvlhcjoc@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I noticed that the 'perf test bpf' was failing:
# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Skip
41.2: BPF pinning : Skip
41.3: BPF prologue generation : Skip
41.4: BPF relocation checker : Skip
# ulimit -l
64
#
Using verbose mode we get just a line bout -EPERF being returned from
libbpf's bpf_load_program_xattr(), that ends up being used in 'perf
test bpf' initial program loading capability query:
Missing basic BPF support, skip this test: Operation not permitted
Not that informative, but on a separate problem when creating BPF maps
bumping rlimit(MEMLOCK) helped, so I tried it here as well, works:
# ulimit -l 128
# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Ok
41.2: BPF pinning : Ok
41.3: BPF prologue generation : Ok
41.4: BPF relocation checker : Ok
#
So use the recently added rlimit__bump_memlock() helper:
# ulimit -l 64
# perf test bpf
41: BPF filter :
41.1: Basic BPF filtering : Ok
41.2: BPF pinning : Ok
41.3: BPF prologue generation : Ok
41.4: BPF relocation checker : Ok
# ulimit -l
64
#
I.e. the bumping of memlock is restricted to the 'perf test' instance,
not changing the global value.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-b9fubkhr4jm192lu7y8hgjvo@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Just like the BPF guys did when faced with failures with map creation,
etc, i.e. their solution is:
tools/testing/selftests/bpf/bpf_rlimit.h
For perf use this function in 'perf test' and in 'perf trace'.
Make it bump to 4 times the current value, if it fails twice the current
value and if it still fails, warn that things like BPF map creation may
fail, to help in diagnosing the problem.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-muvqef2i7n6pzqbmu7tn2d2y@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Based on the following report from Smatch, fix the potential NULL
pointer dereference check.
tools/perf/util/intel-pt.c:3200
intel_pt_process_auxtrace_info() error: we previously assumed
'session->itrace_synth_opts' could be null (see line 3196)
tools/perf/util/intel-pt.c:3206
intel_pt_process_auxtrace_info() warn: variable dereferenced before
check 'session->itrace_synth_opts' (see line 3200)
tools/perf/util/intel-pt.c
3196 if (session->itrace_synth_opts && session->itrace_synth_opts->set) {
3197 pt->synth_opts = *session->itrace_synth_opts;
3198 } else {
3199 itrace_synth_opts__set_default(&pt->synth_opts,
3200 session->itrace_synth_opts->default_no_sample);
^^^^^^^^^^^^^^^^^^^^^^^^^^
3201 if (!session->itrace_synth_opts->default_no_sample &&
3202 !session->itrace_synth_opts->inject) {
3203 pt->synth_opts.branches = false;
3204 pt->synth_opts.callchain = true;
3205 }
3206 if (session->itrace_synth_opts)
^^^^^^^^^^^^^^^^^^^^^^^^^^
3207 pt->synth_opts.thread_stack =
3208 session->itrace_synth_opts->thread_stack;
3209 }
'session->itrace_synth_opts' is impossible to be a NULL pointer in
intel_pt_process_auxtrace_info(), thus this patch removes the NULL test
for 'session->itrace_synth_opts'.
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Link: http://lkml.kernel.org/r/20190708143937.7722-4-leo.yan@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Based on the following report from Smatch, fix the potential NULL
pointer dereference check.
tools/perf/util/intel-bts.c:898
intel_bts_process_auxtrace_info() error: we previously assumed
'session->itrace_synth_opts' could be null (see line 894)
tools/perf/util/intel-bts.c:899
intel_bts_process_auxtrace_info() warn: variable dereferenced before
check 'session->itrace_synth_opts' (see line 898)
tools/perf/util/intel-bts.c
894 if (session->itrace_synth_opts && session->itrace_synth_opts->set) {
895 bts->synth_opts = *session->itrace_synth_opts;
896 } else {
897 itrace_synth_opts__set_default(&bts->synth_opts,
898 session->itrace_synth_opts->default_no_sample);
^^^^^^^^^^^^^^^^^^^^^^^^^^
899 if (session->itrace_synth_opts)
^^^^^^^^^^^^^^^^^^^^^^^^^^
900 bts->synth_opts.thread_stack =
901 session->itrace_synth_opts->thread_stack;
902 }
'session->itrace_synth_opts' is impossible to be a NULL pointer in
intel_bts_process_auxtrace_info(), thus this patch removes the NULL test
for 'session->itrace_synth_opts'.
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Link: http://lkml.kernel.org/r/20190708143937.7722-3-leo.yan@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In pipe mode, session->header.env.arch is not populated until the events
are processed. Therefore, the following command crashes:
perf record -o - | perf script
(gdb) bt
It fails when we try to compare env.arch against uts.machine:
if (!strcmp(uts.machine, session->header.env.arch) ||
(!strcmp(uts.machine, "x86_64") &&
!strcmp(session->header.env.arch, "i386")))
native_arch = true;
In pipe mode, it is tricky to find env.arch at this stage. To keep it
simple, let's just assume native_arch is always true for pipe mode.
Reported-by: David Carrillo Cisneros <davidca@fb.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: kernel-team@fb.com
Cc: stable@vger.kernel.org #v5.1+
Fixes: 3ab481a1cf ("perf script: Support insn output for normal samples")
Link: http://lkml.kernel.org/r/20190621014438.810342-1-songliubraving@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Drop power_events_view before its dependent tables.
SQLite does not seem to mind but the fix was needed for PostgreSQL
(export-to-postgresql.py script), so do the same fix for the SQLite. It is
more logical and keeps the 2 scripts following the same approach.
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Fixes: 5130c6e555 ("perf scripts python: export-to-sqlite.py: Export Intel PT power and ptwrite events")
Link: http://lkml.kernel.org/r/20190708055232.5032-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
PostgreSQL can error if power_events_view is not dropped before its
dependent tables e.g.
Exception: Query failed: ERROR: cannot drop table mwait because other
objects depend on it
DETAIL: view power_events_view depends on table mwait
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Fixes: aba44287a2 ("perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events")
Link: http://lkml.kernel.org/r/20190708055232.5032-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
tool
Based on the following report from Smatch, fix the potential NULL
pointer dereference check.
tools/perf/util/cs-etm.c:2545
cs_etm__process_auxtrace_info() error: we previously assumed
'session->itrace_synth_opts' could be null (see line 2541)
tools/perf/util/cs-etm.c
2541 if (session->itrace_synth_opts && session->itrace_synth_opts->set) {
2542 etm->synth_opts = *session->itrace_synth_opts;
2543 } else {
2544 itrace_synth_opts__set_default(&etm->synth_opts,
2545 session->itrace_synth_opts->default_no_sample);
^^^^^^^^^^^^^^^^^^^^^^^^^^
2546 etm->synth_opts.callchain = false;
2547 }
'session->itrace_synth_opts' is impossible to be a NULL pointer in
cs_etm__process_auxtrace_info(), thus this patch removes the NULL
test for 'session->itrace_synth_opts'.
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Link: http://lkml.kernel.org/r/20190708143937.7722-5-leo.yan@linaro.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In places where the equivalent was already being done, i.e.:
free(a);
a = NULL;
And in placs where struct members are being freed so that if we have
some erroneous reference to its struct, then accesses to freed members
will result in segfaults, which we can detect faster than use after free
to areas that may still have something seemingly valid.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-jatyoofo5boc1bsvoig6bb6i@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
