Every event in the perf-kvm has a 'stats' structure, which contains
max/min/average/etc times of handling this event.
The problem is that the 'perf-kvm stat report' command always shows
that 'min time' is 0us for every event. Example:
# perf kvm stat report
Analyze events for all VCPUs:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
[..]
0xB2 MSCH 12 0.07% 0.00% 0us 8us 7.31us ( +- 2.11% )
0xB2 CHSC 12 0.07% 0.00% 0us 18us 9.39us ( +- 9.49% )
0xB2 STPX 8 0.05% 0.00% 0us 2us 1.88us ( +- 7.18% )
0xB2 STSI 7 0.04% 0.00% 0us 44us 16.49us ( +- 38.20% )
[..]
This happens because the 'stats' structure is not initialized and
stats->min equals to 0. Lets initialize the structure for every
event after its allocation using init_stats() function. This initializes
stats->min to -1 and makes 'Min time' statistics counting work:
# perf kvm stat report
Analyze events for all VCPUs:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
[..]
0xB2 MSCH 12 0.07% 0.00% 6us 8us 7.31us ( +- 2.11% )
0xB2 CHSC 12 0.07% 0.00% 7us 18us 9.39us ( +- 9.49% )
0xB2 STPX 8 0.05% 0.00% 1us 2us 1.88us ( +- 7.18% )
0xB2 STSI 7 0.04% 0.00% 1us 44us 16.49us ( +- 38.20% )
[..]
Signed-off-by: Alexander Yarygin <yarygin@linux.vnet.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: David Ahern <dsahern@gmail.com>
Link: http://lkml.kernel.org/r/1397053319-2130-3-git-send-email-borntraeger@de.ibm.com
[ Fixing the perf examples changelog output ]
Signed-off-by: Jiri Olsa <jolsa@redhat.com>
When we use perf kvm record-report, there is a bug in report subcommand.
Example:
# perf kvm stat record -a sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.678 MB perf.data.guest (~29641 samples) ]
# perf kvm stat report
failed to open perf.data: No such file or directory (try 'perf record' first)
Initializing perf session failed
This bug was introduced by f5fc14124.
+ struct perf_data_file file = {
+ .path = input_name,
+ .mode = PERF_DATA_MODE_READ,
+ };
kvm->tool = eops;
- kvm->session = perf_session__new(kvm->file_name, O_RDONLY, 0, false,
- &kvm->tool);
+ kvm->session = perf_session__new(&file, false, &kvm->tool);
It changed the path from kvm->file_name to input_name, this patch change the path back to
'kvm->file_name', then it works well.
Verification:
# perf kvm stat record -a sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.807 MB perf.data.guest (~35264 samples) ]
# perf kvm stat report
Analyze events for all VCPUs:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
EPT_VIOLATION 200 32.79% 1.25% 0us 12064us 62.35us ( +- 96.74% )
EPT_MISCONFIG 134 21.97% 0.21% 0us 35us 15.25us ( +- 4.14% )
EXCEPTION_NMI 96 15.74% 0.02% 0us 11us 1.95us ( +- 9.81% )
APIC_ACCESS 79 12.95% 0.02% 0us 13us 2.94us ( +- 11.20% )
HLT 65 10.66% 98.47% 0us 16706us 15084.86us ( +- 1.89% )
IO_INSTRUCTION 27 4.43% 0.02% 0us 29us 6.42us ( +- 15.53% )
EXTERNAL_INTERRUPT 5 0.82% 0.01% 0us 77us 23.65us ( +- 57.90% )
TPR_BELOW_THRESHOLD 4 0.66% 0.00% 0us 1us 1.22us ( +- 4.36% )
Total Samples:610, Total events handled time:995745.54us.
Signed-off-by: Dongsheng Yang <yangds.fnst@cn.fujitsu.com>
Link: http://lkml.kernel.org/r/1386632823-17539-1-git-send-email-yangds.fnst@cn.fujitsu.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As there is no -v option for perf kvm, the all debug message for perf
kvm will nerver be printed out to user.
Example:
# perf kvm --guestmount /tmp/guestmount/ record -a
Not enough memory for reading perf file header
It is confusing message for newbies such as me. With this patch applied,
we can use -v option to get the detail.
Example:
# perf kvm --guestmount /tmp/guestmount/ record -a -v
Can't access file /tmp/guestmount//15069/proc/kallsyms
Not enough memory for reading perf file header
Signed-off-by: Dongsheng Yang <yangds.fnst@cn.fujitsu.com>
Cc: David Ahern <dsahern@gmail.com>
Link: http://lkml.kernel.org/r/1386609311-23889-1-git-send-email-yangds.fnst@cn.fujitsu.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
perf kvm stat currently requires back to back record and report commands
to see stats. e.g,.
perf kvm stat record -p $pid -- sleep 1
perf kvm stat report
This is inconvenvient for on box monitoring of a VM. This patch
introduces a 'live' mode that in effect combines the record plus report
into one command. e.g., to monitor a single VM:
perf kvm stat live -p $pid
or all VMs:
perf kvm stat live
Same stats options for the record+report path work with the live mode.
Display rate defaults to 1 second and can be changed using the -d
option.
v4:
- address comments from Xiao -- verify_vcpu check should not look at
processors on line for the host, prune configurable options.
- set attr->{mmap,comm,task} to 0 - don't need task events so trim events
we have to deal with
- better control of time for queue event flushing to reduce frequency of
"Timestamp below last timeslice flush" failures.
v3:
updated to use existing tracepoint parsing code
v2:
removed ABSTIME arg from timerfd_settime as mentioned by Namhyung
only call perf_kvm__handle_stdin when poll returns activity.
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Runzhen Wang <runzhen@linux.vnet.ibm.com>
Cc: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1375753297-69645-3-git-send-email-dsahern@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This introduces the tools/lib/lk library, that will gradually have the
routines that now are used in tools/perf/ and other tools and that can
be shared.
Start by carving out debugfs routines for general use.
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/1361374353-30385-5-git-send-email-bp@alien8.de
[ committer note: Add tools/lib/lk/ to perf's MANIFEST so that its tarballs continue to build ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Make perf build for x86 once the UAPI disintegration patches for that arch
have been applied by adding the appropriate -I flags - in the right order -
and then converting some #includes that use ../.. notation to find main kernel
headerfiles to use <asm/foo.h> and <linux/foo.h> instead.
Note that -Iarch/foo/include/uapi is present _before_ -Iarch/foo/include.
This makes sure we get the userspace version of the pt_regs struct. Ideally,
we wouldn't have the latter -I flag at all, but unfortunately we want
asm/svm.h and asm/vmx.h in builtin-kvm.c and these aren't part of the UAPI -
at least not for x86. I wonder if the bits outside of the __KERNEL__ guards
*should* be transferred there.
I note also that perf seems to do its dependency handling manually by listing
all the header files it might want to use in LIB_H in the Makefile. Can this
be changed to use -MD?
Note that to do make this work, we need to export and UAPI disintegrate
linux/hw_breakpoint.h, which I think should've been exported previously so that
perf can access the bits. We have to do this in the same patch to maintain
bisectability.
Signed-off-by: David Howells <dhowells@redhat.com>
Add 'perf kvm stat' support to analyze kvm vmexit/mmio/ioport smartly
Usage:
- kvm stat
run a command and gather performance counter statistics, it is the alias of
perf stat
- trace kvm events:
perf kvm stat record, or, if other tracepoints are interesting as well, we
can append the events like this:
perf kvm stat record -e timer:* -a
If many guests are running, we can track the specified guest by using -p or
--pid, -a is used to track events generated by all guests.
- show the result:
perf kvm stat report
The output example is following:
13005
13059
total 2 guests are running on the host
Then, track the guest whose pid is 13059:
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.253 MB perf.data.guest (~11065 samples) ]
See the vmexit events:
Analyze events for all VCPUs:
VM-EXIT Samples Samples% Time% Avg time
APIC_ACCESS 460 70.55% 0.01% 22.44us ( +- 1.75% )
HLT 93 14.26% 99.98% 832077.26us ( +- 10.42% )
EXTERNAL_INTERRUPT 64 9.82% 0.00% 35.35us ( +- 14.21% )
PENDING_INTERRUPT 24 3.68% 0.00% 9.29us ( +- 31.39% )
CR_ACCESS 7 1.07% 0.00% 8.12us ( +- 5.76% )
IO_INSTRUCTION 3 0.46% 0.00% 18.00us ( +- 11.79% )
EXCEPTION_NMI 1 0.15% 0.00% 5.83us ( +- -nan% )
Total Samples:652, Total events handled time:77396109.80us.
See the mmio events:
Analyze events for all VCPUs:
MMIO Access Samples Samples% Time% Avg time
0xfee00380:W 387 84.31% 79.28% 8.29us ( +- 3.32% )
0xfee00300:W 24 5.23% 9.96% 16.79us ( +- 1.97% )
0xfee00300:R 24 5.23% 7.83% 13.20us ( +- 3.00% )
0xfee00310:W 24 5.23% 2.93% 4.94us ( +- 3.84% )
Total Samples:459, Total events handled time:4044.59us.
See the ioport event:
Analyze events for all VCPUs:
IO Port Access Samples Samples% Time% Avg time
0xc050:POUT 3 100.00% 100.00% 13.75us ( +- 10.83% )
Total Samples:3, Total events handled time:41.26us.
And, --vcpu is used to track the specified vcpu and --key is used to sort the
result:
Analyze events for VCPU 0:
VM-EXIT Samples Samples% Time% Avg time
HLT 27 13.85% 99.97% 405790.24us ( +- 12.70% )
EXTERNAL_INTERRUPT 13 6.67% 0.00% 27.94us ( +- 22.26% )
APIC_ACCESS 146 74.87% 0.03% 21.69us ( +- 2.91% )
IO_INSTRUCTION 2 1.03% 0.00% 17.77us ( +- 20.56% )
CR_ACCESS 2 1.03% 0.00% 8.55us ( +- 6.47% )
PENDING_INTERRUPT 5 2.56% 0.00% 6.27us ( +- 3.94% )
Total Samples:195, Total events handled time:10959950.90us.
Signed-off-by: Dong Hao <haodong@linux.vnet.ibm.com>
Signed-off-by: Runzhen Wang <runzhen@linux.vnet.ibm.com>
[ Dong Hao <haodong@linux.vnet.ibm.com>
Runzhen Wang <runzhen@linux.vnet.ibm.com>:
- rebase it on current acme's tree
- fix the compiling-error on i386 ]
Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
Acked-by: David Ahern <dsahern@gmail.com>
Cc: Avi Kivity <avi@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: kvm@vger.kernel.org
Cc: Runzhen Wang <runzhen@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1347870675-31495-4-git-send-email-haodong@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
perf defines both __used and __unused variables to use for marking
unused variables. The variable __used is defined to
__attribute__((__unused__)), which contradicts the kernel definition to
__attribute__((__used__)) for new gcc versions. On Android, __used is
also defined in system headers and this leads to warnings like: warning:
'__used__' attribute ignored
__unused is not defined in the kernel and is not a standard definition.
If __unused is included everywhere instead of __used, this leads to
conflicts with glibc headers, since glibc has a variables with this name
in its headers.
The best approach is to use __maybe_unused, the definition used in the
kernel for __attribute__((unused)). In this way there is only one
definition in perf sources (instead of 2 definitions that point to the
same thing: __used and __unused) and it works on both Linux and Android.
This patch simply replaces all instances of __used and __unused with
__maybe_unused.
Signed-off-by: Irina Tirdea <irina.tirdea@intel.com>
Acked-by: Pekka Enberg <penberg@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung.kim@lge.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/1347315303-29906-7-git-send-email-irina.tirdea@intel.com
[ committer note: fixed up conflict with a116e05 in builtin-sched.c ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Make use of exclude_guest and exlude_host in perf-kvm to do only
guest-only counting by default.
Cc: Gleb Natapov <gleb@redhat.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Gleb Natapov <gleb@redhat.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
[ committer note: Moved perf_{guest,host} & event_attr_init to util.c ]
[ so as not to drag more stuff to the python binding]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
