Mathieu Desnoyers
d341e5a754
selftests/rseq: Change type of rseq_offset to ptrdiff_t
...
commit 889c5d60fbcf332c8b6ab7054d45f2768914a375 upstream.
Just before the 2.35 release of glibc, the __rseq_offset userspace ABI
was changed from int to ptrdiff_t.
Adapt to this change in the kernel selftests.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://sourceware.org/pipermail/libc-alpha/2022-February/136024.html
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:22 +02:00
Mathieu Desnoyers
7e617278bf
selftests/rseq: x86-32: use %gs segment selector for accessing rseq thread area
...
commit 127b6429d235ab7c358223bbfd8a8b8d8cc799b6 upstream.
Rather than use rseq_get_abi() and pass its result through a register to
the inline assembler, directly access the per-thread rseq area through a
memory reference combining the %gs segment selector, the constant offset
of the field in struct rseq, and the rseq_offset value (in a register).
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-16-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:22 +02:00
Mathieu Desnoyers
27f6361cb4
selftests/rseq: x86-64: use %fs segment selector for accessing rseq thread area
...
commit 4e15bb766b6c6e963a4d33629034d0ec3b7637df upstream.
Rather than use rseq_get_abi() and pass its result through a register to
the inline assembler, directly access the per-thread rseq area through a
memory reference combining the %fs segment selector, the constant offset
of the field in struct rseq, and the rseq_offset value (in a register).
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-15-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:22 +02:00
Mathieu Desnoyers
a4312e2d81
selftests/rseq: Fix: work-around asm goto compiler bugs
...
commit b53823fb2ef854222853be164f3b1e815f315144 upstream.
gcc and clang each have their own compiler bugs with respect to asm
goto. Implement a work-around for compiler versions known to have those
bugs.
gcc prior to 4.8.2 miscompiles asm goto.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670
gcc prior to 8.1.0 miscompiles asm goto at O1.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103908
clang prior to version 13.0.1 miscompiles asm goto at O2.
https://github.com/llvm/llvm-project/issues/52735
Work around these issues by adding a volatile inline asm with
memory clobber in the fallthrough after the asm goto and at each
label target. Emit this for all compilers in case other similar
issues are found in the future.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-14-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:22 +02:00
Mathieu Desnoyers
ba4d79af71
selftests/rseq: Fix warnings about #if checks of undefined tokens
...
commit d7ed99ade3e62b755584eea07b4e499e79240527 upstream.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-12-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:21 +02:00
Mathieu Desnoyers
35c6f5047f
selftests/rseq: Fix ppc32 offsets by using long rather than off_t
...
commit 26dc8a6d8e11552f3b797b5aafe01071ca32d692 upstream.
The semantic of off_t is for file offsets. We mean to use it as an
offset from a pointer. We really expect it to fit in a single register,
and not use a 64-bit type on 32-bit architectures.
Fix runtime issues on ppc32 where the offset is always 0 due to
inconsistency between the argument type (off_t -> 64-bit) and type
expected by the inline assembler (32-bit).
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-11-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:21 +02:00
Mathieu Desnoyers
4a78bf83e2
selftests/rseq: Introduce rseq_get_abi() helper
...
commit e546cd48ccc456074ddb8920732aef4af65d7ca7 upstream.
This is done in preparation for the selftest uplift to become compatible
with glibc-2.35.
glibc-2.35 exposes the rseq per-thread data in the TCB, accessible
at an offset from the thread pointer, rather than through an actual
Thread-Local Storage (TLS) variable, as the kernel selftests initially
expected.
Introduce a rseq_get_abi() helper, initially using the __rseq_abi
TLS variable, in preparation for changing this userspace ABI for one
which is compatible with glibc-2.35.
Note that the __rseq_abi TLS and glibc-2.35's ABI for per-thread data
cannot actively coexist in a process, because the kernel supports only
a single rseq registration per thread.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Link: https://lkml.kernel.org/r/20220124171253.22072-6-mathieu.desnoyers@efficios.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org >
2022-07-07 17:52:21 +02:00
Peter Oskolkov
ea366dd79c
rseq/selftests,x86_64: Add rseq_offset_deref_addv()
...
This patch adds rseq_offset_deref_addv() function to
tools/testing/selftests/rseq/rseq-x86.h, to be used in a selftest in
the next patch in the patchset.
Once an architecture adds support for this function they should define
"RSEQ_ARCH_HAS_OFFSET_DEREF_ADDV".
Signed-off-by: Peter Oskolkov <posk@google.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Link: https://lkml.kernel.org/r/20200923233618.2572849-2-posk@google.com
2020-09-25 14:23:27 +02:00
Mathieu Desnoyers
24fa5d1efe
rseq/selftests: x86: use ud1 instruction as RSEQ_SIG opcode
...
Use ud1 as the guard instruction for the restartable sequence abort
handler. Its benefit compared to nopl is to trap execution if the
program ends up trying to execute it by mistake, which makes debugging
easier.
The 4-byte signature per se is unchanged (it is the instruction
operand). Only the opcode is changed from nopl to ud1.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Suggested-by: Peter Zijlstra <peterz@infradead.org >
CC: Peter Zijlstra <peterz@infradead.org >
CC: Thomas Gleixner <tglx@linutronix.de >
CC: Joel Fernandes <joelaf@google.com >
CC: Catalin Marinas <catalin.marinas@arm.com >
CC: Dave Watson <davejwatson@fb.com >
CC: Will Deacon <will.deacon@arm.com >
CC: Shuah Khan <shuah@kernel.org >
CC: Andi Kleen <andi@firstfloor.org >
CC: linux-kselftest@vger.kernel.org
CC: "H . Peter Anvin" <hpa@zytor.com >
CC: Chris Lameter <cl@linux.com >
CC: Russell King <linux@arm.linux.org.uk >
CC: Michael Kerrisk <mtk.manpages@gmail.com >
CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com >
CC: Paul Turner <pjt@google.com >
CC: Boqun Feng <boqun.feng@gmail.com >
CC: Josh Triplett <josh@joshtriplett.org >
CC: Steven Rostedt <rostedt@goodmis.org >
CC: Ben Maurer <bmaurer@fb.com >
CC: linux-api@vger.kernel.org
CC: Andy Lutomirski <luto@amacapital.net >
CC: Andrew Morton <akpm@linux-foundation.org >
CC: Linus Torvalds <torvalds@linux-foundation.org >
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org >
2019-05-07 15:32:05 -06:00
Mathieu Desnoyers
a3e3131f94
rseq/selftests: Introduce __rseq_cs_ptr_array, rename __rseq_table to __rseq_cs
...
The entries within __rseq_table are aligned on 32 bytes due to
linux/rseq.h struct rseq_cs uapi requirements, but the start of the
__rseq_table section is not guaranteed to be 32-byte aligned. It can
cause padding to be added at the start of the section, which makes it
hard to use as an array of items by debuggers.
Considering that __rseq_table does not really consist of a table due to
the presence of padding, rename this section to __rseq_cs.
Create a new __rseq_cs_ptr_array section which contains 64-bit packed
pointers to entries within the __rseq_cs section.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
CC: Thomas Gleixner <tglx@linutronix.de >
CC: Joel Fernandes <joelaf@google.com >
CC: Peter Zijlstra <peterz@infradead.org >
CC: Catalin Marinas <catalin.marinas@arm.com >
CC: Dave Watson <davejwatson@fb.com >
CC: Will Deacon <will.deacon@arm.com >
CC: Shuah Khan <shuah@kernel.org >
CC: Andi Kleen <andi@firstfloor.org >
CC: linux-kselftest@vger.kernel.org
CC: "H . Peter Anvin" <hpa@zytor.com >
CC: Chris Lameter <cl@linux.com >
CC: Russell King <linux@arm.linux.org.uk >
CC: Michael Kerrisk <mtk.manpages@gmail.com >
CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com >
CC: Paul Turner <pjt@google.com >
CC: Boqun Feng <boqun.feng@gmail.com >
CC: Josh Triplett <josh@joshtriplett.org >
CC: Steven Rostedt <rostedt@goodmis.org >
CC: Ben Maurer <bmaurer@fb.com >
CC: linux-api@vger.kernel.org
CC: Andy Lutomirski <luto@amacapital.net >
CC: Andrew Morton <akpm@linux-foundation.org >
CC: Linus Torvalds <torvalds@linux-foundation.org >
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org >
2019-05-07 15:31:36 -06:00
Mathieu Desnoyers
4fe2088e16
rseq/selftests: Add __rseq_exit_point_array section for debuggers
...
Knowing all exit points is useful to assist debuggers stepping over the
rseq critical sections without requiring them to disassemble the content
of the critical section to figure out the exit points.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
CC: Thomas Gleixner <tglx@linutronix.de >
CC: Joel Fernandes <joelaf@google.com >
CC: Peter Zijlstra <peterz@infradead.org >
CC: Catalin Marinas <catalin.marinas@arm.com >
CC: Dave Watson <davejwatson@fb.com >
CC: Will Deacon <will.deacon@arm.com >
CC: Shuah Khan <shuah@kernel.org >
CC: Andi Kleen <andi@firstfloor.org >
CC: linux-kselftest@vger.kernel.org
CC: "H . Peter Anvin" <hpa@zytor.com >
CC: Chris Lameter <cl@linux.com >
CC: Russell King <linux@arm.linux.org.uk >
CC: Michael Kerrisk <mtk.manpages@gmail.com >
CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com >
CC: Paul Turner <pjt@google.com >
CC: Boqun Feng <boqun.feng@gmail.com >
CC: Josh Triplett <josh@joshtriplett.org >
CC: Steven Rostedt <rostedt@goodmis.org >
CC: Ben Maurer <bmaurer@fb.com >
CC: linux-api@vger.kernel.org
CC: Andy Lutomirski <luto@amacapital.net >
CC: Andrew Morton <akpm@linux-foundation.org >
CC: Linus Torvalds <torvalds@linux-foundation.org >
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org >
2019-05-07 15:31:13 -06:00
Mathieu Desnoyers
fe22983d92
rseq/selftests: x86: Work-around bogus gcc-8 optimisation
...
gcc-8 version 8.1.0, 8.2.0, and 8.3.0 generate broken assembler with asm
goto that have a thread-local storage "m" input operand on both x86-32
and x86-64. For instance:
__thread int var;
static int fct(void)
{
asm goto ( "jmp %l[testlabel]\n\t"
: : [var] "m" (var) : : testlabel);
return 0;
testlabel:
return 1;
}
int main()
{
return fct();
}
% gcc-8 -O2 -o test-asm-goto test-asm-goto.c
/tmp/ccAdHJbe.o: In function `main':
test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2'
collect2: error: ld returned 1 exit status
% gcc-8 -m32 -O2 -o test-asm-goto test-asm-goto.c
/tmp/ccREsVXA.o: In function `main':
test-asm-goto.c:(.text.startup+0x1): undefined reference to `.L2'
collect2: error: ld returned 1 exit status
Work-around this compiler bug in the rseq-x86.h header by passing the
address of the __rseq_abi TLS as a register operand rather than using
the "m" input operand.
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90193
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
CC: Ingo Molnar <mingo@redhat.com >
CC: Peter Zijlstra <peterz@infradead.org >
CC: Thomas Gleixner <tglx@linutronix.de >
CC: Joel Fernandes <joelaf@google.com >
CC: Catalin Marinas <catalin.marinas@arm.com >
CC: Dave Watson <davejwatson@fb.com >
CC: Will Deacon <will.deacon@arm.com >
CC: Shuah Khan <shuah@kernel.org >
CC: Andi Kleen <andi@firstfloor.org >
CC: linux-kselftest@vger.kernel.org
CC: "H . Peter Anvin" <hpa@zytor.com >
CC: Chris Lameter <cl@linux.com >
CC: Russell King <linux@arm.linux.org.uk >
CC: Michael Kerrisk <mtk.manpages@gmail.com >
CC: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com >
CC: Paul Turner <pjt@google.com >
CC: Boqun Feng <boqun.feng@gmail.com >
CC: Josh Triplett <josh@joshtriplett.org >
CC: Steven Rostedt <rostedt@goodmis.org >
CC: Ben Maurer <bmaurer@fb.com >
CC: linux-api@vger.kernel.org
CC: Andy Lutomirski <luto@amacapital.net >
CC: Andrew Morton <akpm@linux-foundation.org >
CC: Linus Torvalds <torvalds@linux-foundation.org >
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org >
2019-05-07 15:30:14 -06:00
Mathieu Desnoyers
2e155fb7d6
rseq/selftests: Provide rseq library
...
This rseq helper library provides a user-space API to the rseq()
system call.
The rseq fast-path exposes the instruction pointer addresses where the
rseq assembly blocks begin and end, as well as the associated abort
instruction pointer, in the __rseq_table section. This section allows
debuggers may know where to place breakpoints when single-stepping
through assembly blocks which may be aborted at any point by the kernel.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Thomas Gleixner <tglx@linutronix.de >
Cc: Joel Fernandes <joelaf@google.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Catalin Marinas <catalin.marinas@arm.com >
Cc: Dave Watson <davejwatson@fb.com >
Cc: Will Deacon <will.deacon@arm.com >
Cc: Shuah Khan <shuahkh@osg.samsung.com >
Cc: Andi Kleen <andi@firstfloor.org >
Cc: linux-kselftest@vger.kernel.org
Cc: "H . Peter Anvin" <hpa@zytor.com >
Cc: Chris Lameter <cl@linux.com >
Cc: Russell King <linux@arm.linux.org.uk >
Cc: Andrew Hunter <ahh@google.com >
Cc: Michael Kerrisk <mtk.manpages@gmail.com >
Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com >
Cc: Paul Turner <pjt@google.com >
Cc: Boqun Feng <boqun.feng@gmail.com >
Cc: Josh Triplett <josh@joshtriplett.org >
Cc: Steven Rostedt <rostedt@goodmis.org >
Cc: Ben Maurer <bmaurer@fb.com >
Cc: linux-api@vger.kernel.org
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Link: https://lkml.kernel.org/r/20180602124408.8430-13-mathieu.desnoyers@efficios.com
2018-06-06 11:58:34 +02:00