Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
perf metrics:
Andi Kleen:
- Fixes for SkylakeX and CascadeLakeX Intel vendor events.
- Avoid extra ':' for --raw metrics.
- Don't include duration_time in group.
perf script:
Arnaldo Carvalho de Melo/Jiri Olsa:
- Fix processing guest samples.
perf diff:
Jin Yao:
- Do diffs by basic blocks.
objtool:
Jiri Olsa:
- Fix build by linking against tools/lib/ctype.o sources.
perf pmu:
John Garry:
- Support more complex PMU event aliasing.
- Add support for Hisi hip08 DDRC, HHA and L3C PMU aliasing.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
perf annotate:
Mao Han:
- Add support for the csky processor architecture.
perf stat:
Andi Kleen:
- Fix metrics with --no-merge.
- Don't merge events in the same PMU.
- Fix group lookup for metric group.
Intel PT:
Adrian Hunter:
- Improve CBR (Core to Bus Ratio) packets support.
- Fix thread stack return from kernel for kernel only case.
- Export power and ptwrite events to sqlite and postgresql.
core libraries:
Arnaldo Carvalho de Melo:
- Find routines in tools/perf/util/ that have implementations in the kernel
libraries (lib/*.c), such as strreplace(), strim(), skip_spaces() and reuse
them after making a copy into tools/lib and tools/include/.
This continues the effort of having tools/ code looking as much as possible
like kernel source code, to help encourage people to work on both the kernel
and in tools hosted in the kernel sources.
That in turn will help moving stuff that uses those routines to
tools/lib/perf/ where they will be made available for use in other tools.
In the process ditch old cruft, remove unused variables and add missing
include directives for headers providing things used in places that were
building by sheer luck.
Kyle Meyer:
- Bump MAX_NR_CPUS and MAX_CACHES to get these tools to work on more machines.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Fix a regression introduced when removing bi_phys_segments for Write Zeroes
requests, which need to have a segment count of zero, as they don't have a
payload.
Fixes: 14ccb66b3f ("block: remove the bi_phys_segments field in struct bio")
Reported-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Generic kprobe_page_fault() calls into kprobe_fault_handler() which must be
available with and without CONFIG_KPROBES. There is one stub implementation
for !CONFIG_KPROBES. For CONFIG_KPROBES all subscribing archs must provide
a kprobe_fault_handler() definition. Currently mips has an implementation
which is defined as 'static inline'. Make it available for generic kprobes
to comply with the above new requirement.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: James Hogan <jhogan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mips@vger.kernel.org
Cc: linux-mm@kvack.org
Reported-by: kbuild test robot <lkp@intel.com>
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Paul Burton <paul.burton@mips.com>
Fixes: 773734b44557 ("mm, kprobes: generalize and rename notify_page_fault() as kprobe_page_fault()")
Cc: linux-kernel@vger.kernel.org
Device that bound to XDP socket will not have zero refcount until the
userspace application will not close it. This leads to hang inside
'netdev_wait_allrefs()' if device unregistering requested:
# ip link del p1
< hang on recvmsg on netlink socket >
# ps -x | grep ip
5126 pts/0 D+ 0:00 ip link del p1
# journalctl -b
Jun 05 07:19:16 kernel:
unregister_netdevice: waiting for p1 to become free. Usage count = 1
Jun 05 07:19:27 kernel:
unregister_netdevice: waiting for p1 to become free. Usage count = 1
...
Fix that by implementing NETDEV_UNREGISTER event notification handler
to properly clean up all the resources and unref device.
This should also allow socket killing via ss(8) utility.
Fixes: 965a990984 ("xsk: add support for bind for Rx")
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Jonathan Lemon <jonathan.lemon@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Device pointer stored in umem regardless of zero-copy mode,
so we heed to hold the device in all cases.
Fixes: c9b47cc1fa ("xsk: fix bug when trying to use both copy and zero-copy on one queue id")
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Jonathan Lemon <jonathan.lemon@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Selftests are reporting this failure in test_lwt_seg6local.sh:
+ ip netns exec ns2 ip -6 route add fb00::6 encap bpf in obj test_lwt_seg6local.o sec encap_srh dev veth2
Error fetching program/map!
Failed to parse eBPF program: Operation not permitted
The problem is __attribute__((always_inline)) alone is not enough to prevent
clang from inserting those functions in .text. In that case, .text is not
marked as relocateable.
See the output of objdump -h test_lwt_seg6local.o:
Idx Name Size VMA LMA File off Algn
0 .text 00003530 0000000000000000 0000000000000000 00000040 2**3
CONTENTS, ALLOC, LOAD, READONLY, CODE
This causes the iproute bpf loader to fail in bpf_fetch_prog_sec:
bpf_has_call_data returns true but bpf_fetch_prog_relo fails as there's no
relocateable .text section in the file.
To fix this, convert to 'static __always_inline'.
v2: Use 'static __always_inline' instead of 'static inline
__attribute__((always_inline))'
Fixes: c99a84eac0 ("selftests/bpf: test for seg6local End.BPF action")
Signed-off-by: Jiri Benc <jbenc@redhat.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
The progs for bpf selftests use several different notations to force
function inlining. Standardize to what most of them use,
static __always_inline.
Suggested-by: Song Liu <liu.song.a23@gmail.com>
Signed-off-by: Jiri Benc <jbenc@redhat.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Adds support for fq's Earliest Departure Time to HBM (Host Bandwidth
Manager). Includes a new BPF program supporting EDT, and also updates
corresponding programs.
It will drop packets with an EDT of more than 500us in the future
unless the packet belongs to a flow with less than 2 packets in flight.
This is done so each flow has at least 2 packets in flight, so they
will not starve, and also to help prevent delayed ACK timeouts.
It will also work with ECN enabled traffic, where the packets will be
CE marked if their EDT is more than 50us in the future.
The table below shows some performance numbers. The flows are back to
back RPCS. One server sending to another, either 2 or 4 flows.
One flow is a 10KB RPC, the rest are 1MB RPCs. When there are more
than one flow of a given RPC size, the numbers represent averages.
The rate limit applies to all flows (they are in the same cgroup).
Tests ending with "-edt" ran with the new BPF program supporting EDT.
Tests ending with "-hbt" ran on top HBT qdisc with the specified rate
(i.e. no HBM). The other tests ran with the HBM BPF program included
in the HBM patch-set.
EDT has limited value when using DCTCP, but it helps in many cases when
using Cubic. It usually achieves larger link utilization and lower
99% latencies for the 1MB RPCs.
HBM ends up queueing a lot of packets with its default parameter values,
reducing the goodput of the 10KB RPCs and increasing their latency. Also,
the RTTs seen by the flows are quite large.
Aggr 10K 10K 10K 1MB 1MB 1MB
Limit rate drops RTT rate P90 P99 rate P90 P99
Test rate Flows Mbps % us Mbps us us Mbps ms ms
-------- ---- ----- ---- ----- --- ---- ---- ---- ---- ---- ----
cubic 1G 2 904 0.02 108 257 511 539 647 13.4 24.5
cubic-edt 1G 2 982 0.01 156 239 656 967 743 14.0 17.2
dctcp 1G 2 977 0.00 105 324 408 744 653 14.5 15.9
dctcp-edt 1G 2 981 0.01 142 321 417 811 660 15.7 17.0
cubic-htb 1G 2 919 0.00 1825 40 2822 4140 879 9.7 9.9
cubic 200M 2 155 0.30 220 81 532 655 74 283 450
cubic-edt 200M 2 188 0.02 222 87 1035 1095 101 84 85
dctcp 200M 2 188 0.03 111 77 912 939 111 76 325
dctcp-edt 200M 2 188 0.03 217 74 1416 1738 114 76 79
cubic-htb 200M 2 188 0.00 5015 8 14ms 15ms 180 48 50
cubic 1G 4 952 0.03 110 165 516 546 262 38 154
cubic-edt 1G 4 973 0.01 190 111 1034 1314 287 65 79
dctcp 1G 4 951 0.00 103 180 617 905 257 37 38
dctcp-edt 1G 4 967 0.00 163 151 732 1126 272 43 55
cubic-htb 1G 4 914 0.00 3249 13 7ms 8ms 300 29 34
cubic 5G 4 4236 0.00 134 305 490 624 1310 10 17
cubic-edt 5G 4 4865 0.00 156 306 425 759 1520 10 16
dctcp 5G 4 4936 0.00 128 485 221 409 1484 7 9
dctcp-edt 5G 4 4924 0.00 148 390 392 623 1508 11 26
v1 -> v2: Incorporated Andrii's suggestions
v2 -> v3: Incorporated Yonghong's suggestions
v3 -> v4: Removed credit update that is not needed
Signed-off-by: Lawrence Brakmo <brakmo@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Rewrite gfs2_allocate_page_backing to call gfs2_iomap_get_alloc and operate on
struct iomap directly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
No need to indirect through get_blocks and buffer_heads when we can just use
the iomap version.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
There is no need to keep these two functions separate.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
The only difference between the two is that gfs2_ordered_aops sets the
set_page_dirty method to __set_page_dirty_buffers, but given that
__set_page_dirty_buffers is the default, if no method is set, there is no need
to to do that. Merge the two sets of operations into one.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
Refer to the Intel SDM Vol.4, the package C-state residency counters
of modern IA micro-architecture are all ticking in TSC frequency,
hence we can apply simple math to transform the ticks into microseconds.
i.e.,
residency (ms) = count / tsc_khz
residency (us) = count / tsc_khz * 1000
This also aligns to other sysfs debug entries of residency counter in
the same metric in microseconds, benefits reading and scripting.
v2: restore the accidentally deleted newline, no function change.
v3: apply kernel do_div() macro to calculate division
Signed-off-by: Harry Pan <harry.pan@intel.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Add documentation for the new attributes for exposing reset causes for
the next type of resets: caused by different watchdog, BIOS reload,
ASIC reset request.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Add more attributes for reset cause indication for the cases when
system reset has been caused by watchdog, BIOS reload and COMEX
thermal shutdown.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Modify DMI matching order: perform matching based on DMI_BOARD_NAME
before matching based on DMI_BOARD_VENDOR and DMI_PRODUCT_NAME in order
to reduce the number of ‘dmi_table’ entries necessary for new systems
support and keep matching order in logical way.
For example, the existing check for DMI_PRODUCT_NAME with prefixes
“MSN27", “MSN24”, "MSB” matches systems MSN2700-BXXXX, MSN2700-XXXX,
MSN2410-BXXXX, MSB7800-XXXX, where ‘XXXX’ specifies some systems
hardware flavors.
At the same time these systems also matched by DMI_BOARD_NAME
“VMOD0001”, because they all have the same platform configuration (LED,
interrupt control, mux etcetera).
New systems with different platform configuration, but with similar
DMI_PRODUCT_NAME MSN2700-2XXXX, MSN2700-2XXXX, MSB7800-2XXXX are about
to be added. These system have similar DMI_PRODUCT_NAME, since they
have same ports configuration as their predecessors. All new systems
will be matched by DMI_BOARD_NAME “VMOD0008”.
With the change provided in the patch it is enough just to add
“VMOD0008” match following natural after “VMOD0007”, otherwise
“VMOD0008” or all “MSN2700-2XXXX”, “MSN2700-2XXXX”, “MSB7800-2XXXX”
should be added on top of ‘mlxplat_dmi_table” in order to be matched
before “MSN27", “MSN24”, "MSB”.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Use separated regamp structures for old and next generation systems.
Next generation systems don’t require write protection removing.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Activate 'i2c-mlxcpld' driver with 'platform_device_register_resndata'
instead off 'platform_device_register_simple' in order to pass platform
specific info.
Add platform i2c data for the next generation systems.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
The Intel(R) Speed select technologies contains four features.
Performance profile:An non architectural mechanism that allows multiple
optimized performance profiles per system via static and/or dynamic
adjustment of core count, workload, Tjmax, and TDP, etc. aka ISS
in the documentation.
Base Frequency: Enables users to increase guaranteed base frequency on
certain cores (high priority cores) in exchange for lower base frequency
on remaining cores (low priority cores). aka PBF in the documenation.
Turbo frequency: Enables the ability to set different turbo ratio limits
to cores based on priority. aka FACT in the documentation.
Core power: An Interface that allows user to define per core/tile
priority.
There is a multi level help for commands and options. This can be used
to check required arguments for each feature and commands for the
feature.
To start navigating the features start with
$sudo intel-speed-select --help
For help on a specific feature for example
$sudo intel-speed-select perf-profile --help
To get help for a command for a feature for example
$sudo intel-speed-select perf-profile get-lock-status --help
Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
Acked-by: Len Brown <len.brown@intel.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Provide a keyctl() operation to grant/remove permissions. The grant
operation, wrapped by libkeyutils, looks like:
int ret = keyctl_grant_permission(key_serial_t key,
enum key_ace_subject_type type,
unsigned int subject,
unsigned int perm);
Where key is the key to be modified, type and subject represent the subject
to which permission is to be granted (or removed) and perm is the set of
permissions to be granted. 0 is returned on success. SET_SECURITY
permission is required for this.
The subject type currently must be KEY_ACE_SUBJ_STANDARD for the moment
(other subject types will come along later).
For subject type KEY_ACE_SUBJ_STANDARD, the following subject values are
available:
KEY_ACE_POSSESSOR The possessor of the key
KEY_ACE_OWNER The owner of the key
KEY_ACE_GROUP The key's group
KEY_ACE_EVERYONE Everyone
perm lists the permissions to be granted:
KEY_ACE_VIEW Can view the key metadata
KEY_ACE_READ Can read the key content
KEY_ACE_WRITE Can update/modify the key content
KEY_ACE_SEARCH Can find the key by searching/requesting
KEY_ACE_LINK Can make a link to the key
KEY_ACE_SET_SECURITY Can set security
KEY_ACE_INVAL Can invalidate
KEY_ACE_REVOKE Can revoke
KEY_ACE_JOIN Can join this keyring
KEY_ACE_CLEAR Can clear this keyring
If an ACE already exists for the subject, then the permissions mask will be
overwritten; if perm is 0, it will be deleted.
Currently, the internal ACL is limited to a maximum of 16 entries.
For example:
int ret = keyctl_grant_permission(key,
KEY_ACE_SUBJ_STANDARD,
KEY_ACE_OWNER,
KEY_ACE_VIEW | KEY_ACE_READ);
Signed-off-by: David Howells <dhowells@redhat.com>
Felipe writes:
USB: more changes for v5.3 merge window
Turns out a few more important changes came about. We have the new
Cadence DRD Driver being added here and that's the biggest, most
important part.
Together with that we have suport for new imx7ulp phy. Support for
TigerLake Devices on dwc3. Also a couple important fixes which weren't
completed in time for the -rc cycle.
Signed-off-by: Felipe Balbi <felipe.balbi@linux.intel.com>
* tag 'usb-for-v5.3-part2' of git://git.kernel.org/pub/scm/linux/kernel/git/balbi/usb:
usb: renesas_usbhs: add a workaround for a race condition of workqueue
usb: gadget: udc: renesas_usb3: remove redundant assignment to ret
usb: dwc2: use a longer AHB idle timeout in dwc2_core_reset()
USB: gadget: function: fix issue Unneeded variable: "value"
usb: phy: phy-mxs-usb: add imx7ulp support
doc: dt-binding: mxs-usb-phy: add compatible for 7ulp
usb:cdns3 Fix for stuck packets in on-chip OUT buffer.
usb:cdns3 Add Cadence USB3 DRD Driver
usb:gadget Simplify usb_decode_get_set_descriptor function.
usb:gadget Patch simplify usb_decode_set_clear_feature function.
usb:gadget Separated decoding functions from dwc3 driver.
dt-bindings: add binding for USBSS-DRD controller.
usb: dwc3: pci: add support for TigerLake Devices
The pointer clk is being initialized with a value that is never
read and it is being updated later with a new value. The
initialization is redundant and can be removed.
Addresses-Coverity: ("Unused value")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
The intel-int3496.txt file is a documentation for an ACPI driver.
There's no reason to keep it on a separate directory.
So, instead of keeping it on some random location, move it
to a sub-directory inside the ACPI documentation dir,
renaming it to .rst.
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Acked-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
With CONFIG_OPTIMIZE_INLINING enabled, Laura Abbott reported error
with gcc 9.1.1:
arch/powerpc/mm/book3s64/radix_tlb.c: In function '_tlbiel_pid':
arch/powerpc/mm/book3s64/radix_tlb.c:104:2: warning: asm operand 3 probably doesn't match constraints
104 | asm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1)
| ^~~
arch/powerpc/mm/book3s64/radix_tlb.c:104:2: error: impossible constraint in 'asm'
Fixing _tlbiel_pid() is enough to address the warning above, but I
inlined more functions to fix all potential issues.
To meet the "i" (immediate) constraint for the asm operands, functions
propagating "ric" must be always inlined.
Fixes: 9012d01166 ("compiler: allow all arches to enable CONFIG_OPTIMIZE_INLINING")
Reported-by: Laura Abbott <labbott@redhat.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Based on the following report from Smatch, fix the potential NULL
pointer dereference check:
tools/lib/bpf/libbpf.c:3493
bpf_prog_load_xattr() warn: variable dereferenced before check 'attr'
(see line 3483)
3479 int bpf_prog_load_xattr(const struct bpf_prog_load_attr *attr,
3480 struct bpf_object **pobj, int *prog_fd)
3481 {
3482 struct bpf_object_open_attr open_attr = {
3483 .file = attr->file,
3484 .prog_type = attr->prog_type,
^^^^^^
3485 };
At the head of function, it directly access 'attr' without checking
if it's NULL pointer. This patch moves the values assignment after
validating 'attr' and 'attr->file'.
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
GCC8 started emitting warning about using strncpy with number of bytes
exactly equal destination size, which is generally unsafe, as can lead
to non-zero terminated string being copied. Use IFNAMSIZ - 1 as number
of bytes to ensure name is always zero-terminated.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Magnus Karlsson <magnus.karlsson@intel.com>
Acked-by: Yonghong Song <yhs@fb.com>
Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Pull irqchip updates for Linux 5.3 from Marc Zyngier:
- ACPI support for the exiu and mb86s7x drivers
- New Renesas RZ/A1, Amazon al-fic drivers
- Add quirk for Amazon Graviton GICv2m widget
- Large Renesas driver cleanup
- CSky mpintc trigger type fixes
- Meson G12A driver support
- Various minor cleanups
There are currently no tests for ALU64 shift operations when the shift
amount is 0. This adds 6 new tests to make sure they are equivalent
to a no-op. The x32 JIT had such bugs that could have been caught by
these tests.
Cc: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
The current x32 BPF JIT does not correctly compile shift operations when
the immediate shift amount is 0. The expected behavior is for this to
be a no-op.
The following program demonstrates the bug. The expexceted result is 1,
but the current JITed code returns 2.
r0 = 1
r1 = 1
r1 <<= 0
if r1 == 1 goto end
r0 = 2
end:
exit
This patch simplifies the code and fixes the bug.
Fixes: 03f5781be2 ("bpf, x86_32: add eBPF JIT compiler for ia32")
Co-developed-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
The current x32 BPF JIT for shift operations is not correct when the
shift amount in a register is 0. The expected behavior is a no-op, whereas
the current implementation changes bits in the destination register.
The following example demonstrates the bug. The expected result of this
program is 1, but the current JITed code returns 2.
r0 = 1
r1 = 1
r2 = 0
r1 <<= r2
if r1 == 1 goto end
r0 = 2
end:
exit
The bug is caused by an incorrect assumption by the JIT that a shift by
32 clear the register. On x32 however, shifts use the lower 5 bits of
the source, making a shift by 32 equivalent to a shift by 0.
This patch fixes the bug using double-precision shifts, which also
simplifies the code.
Fixes: 03f5781be2 ("bpf, x86_32: add eBPF JIT compiler for ia32")
Co-developed-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
When equivalent state is found the current state needs to propagate precision marks.
Otherwise the verifier will prune the search incorrectly.
There is a price for correctness:
before before broken fixed
cnst spill precise precise
bpf_lb-DLB_L3.o 1923 8128 1863 1898
bpf_lb-DLB_L4.o 3077 6707 2468 2666
bpf_lb-DUNKNOWN.o 1062 1062 544 544
bpf_lxc-DDROP_ALL.o 166729 380712 22629 36823
bpf_lxc-DUNKNOWN.o 174607 440652 28805 45325
bpf_netdev.o 8407 31904 6801 7002
bpf_overlay.o 5420 23569 4754 4858
bpf_lxc_jit.o 39389 359445 50925 69631
Overall precision tracking is still very effective.
Fixes: b5dc0163d8 ("bpf: precise scalar_value tracking")
Reported-by: Lawrence Brakmo <brakmo@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Tested-by: Lawrence Brakmo <brakmo@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
syzbot reported following spat:
BUG: KASAN: use-after-free in __write_once_size include/linux/compiler.h:221
BUG: KASAN: use-after-free in hlist_del_rcu include/linux/rculist.h:455
BUG: KASAN: use-after-free in xfrm_hash_rebuild+0xa0d/0x1000 net/xfrm/xfrm_policy.c:1318
Write of size 8 at addr ffff888095e79c00 by task kworker/1:3/8066
Workqueue: events xfrm_hash_rebuild
Call Trace:
__write_once_size include/linux/compiler.h:221 [inline]
hlist_del_rcu include/linux/rculist.h:455 [inline]
xfrm_hash_rebuild+0xa0d/0x1000 net/xfrm/xfrm_policy.c:1318
process_one_work+0x814/0x1130 kernel/workqueue.c:2269
Allocated by task 8064:
__kmalloc+0x23c/0x310 mm/slab.c:3669
kzalloc include/linux/slab.h:742 [inline]
xfrm_hash_alloc+0x38/0xe0 net/xfrm/xfrm_hash.c:21
xfrm_policy_init net/xfrm/xfrm_policy.c:4036 [inline]
xfrm_net_init+0x269/0xd60 net/xfrm/xfrm_policy.c:4120
ops_init+0x336/0x420 net/core/net_namespace.c:130
setup_net+0x212/0x690 net/core/net_namespace.c:316
The faulting address is the address of the old chain head,
free'd by xfrm_hash_resize().
In xfrm_hash_rehash(), chain heads get re-initialized without
any hlist_del_rcu:
for (i = hmask; i >= 0; i--)
INIT_HLIST_HEAD(odst + i);
Then, hlist_del_rcu() gets called on the about to-be-reinserted policy
when iterating the per-net list of policies.
hlist_del_rcu() will then make chain->first be nonzero again:
static inline void __hlist_del(struct hlist_node *n)
{
struct hlist_node *next = n->next; // address of next element in list
struct hlist_node **pprev = n->pprev;// location of previous elem, this
// can point at chain->first
WRITE_ONCE(*pprev, next); // chain->first points to next elem
if (next)
next->pprev = pprev;
Then, when we walk chainlist to find insertion point, we may find a
non-empty list even though we're supposedly reinserting the first
policy to an empty chain.
To fix this first unlink all exact and inexact policies instead of
zeroing the list heads.
Add the commands equivalent to the syzbot reproducer to xfrm_policy.sh,
without fix KASAN catches the corruption as it happens, SLUB poisoning
detects it a bit later.
Reported-by: syzbot+0165480d4ef07360eeda@syzkaller.appspotmail.com
Fixes: 1548bc4e05 ("xfrm: policy: delete inexact policies from inexact list on hash rebuild")
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
