Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull RCU updates from Ingo Molnar:
 "The main RCU related changes in this cycle were:

   - Additional cleanups after RCU flavor consolidation

   - Grace-period forward-progress cleanups and improvements

   - Documentation updates

   - Miscellaneous fixes

   - spin_is_locked() conversions to lockdep

   - SPDX changes to RCU source and header files

   - SRCU updates

   - Torture-test updates, including nolibc updates and moving nolibc to
     tools/include"

* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (71 commits)
  locking/locktorture: Convert to SPDX license identifier
  linux/torture: Convert to SPDX license identifier
  torture: Convert to SPDX license identifier
  linux/srcu: Convert to SPDX license identifier
  linux/rcutree: Convert to SPDX license identifier
  linux/rcutiny: Convert to SPDX license identifier
  linux/rcu_sync: Convert to SPDX license identifier
  linux/rcu_segcblist: Convert to SPDX license identifier
  linux/rcupdate: Convert to SPDX license identifier
  linux/rcu_node_tree: Convert to SPDX license identifier
  rcu/update: Convert to SPDX license identifier
  rcu/tree: Convert to SPDX license identifier
  rcu/tiny: Convert to SPDX license identifier
  rcu/sync: Convert to SPDX license identifier
  rcu/srcu: Convert to SPDX license identifier
  rcu/rcutorture: Convert to SPDX license identifier
  rcu/rcu_segcblist: Convert to SPDX license identifier
  rcu/rcuperf: Convert to SPDX license identifier
  rcu/rcu.h: Convert to SPDX license identifier
  RCU/torture.txt: Remove section MODULE PARAMETERS
  ...
This commit is contained in:
Linus Torvalds
2019-03-05 14:49:11 -08:00
63 changed files with 1396 additions and 1269 deletions

View File

@@ -3,7 +3,85 @@
* Copyright (C) 2017-2018 Willy Tarreau <w@1wt.eu>
*/
/* some archs (at least aarch64) don't expose the regular syscalls anymore by
/*
* This file is designed to be used as a libc alternative for minimal programs
* with very limited requirements. It consists of a small number of syscall and
* type definitions, and the minimal startup code needed to call main().
* All syscalls are declared as static functions so that they can be optimized
* away by the compiler when not used.
*
* Syscalls are split into 3 levels:
* - The lower level is the arch-specific syscall() definition, consisting in
* assembly code in compound expressions. These are called my_syscall0() to
* my_syscall6() depending on the number of arguments. The MIPS
* implementation is limited to 5 arguments. All input arguments are cast
* to a long stored in a register. These expressions always return the
* syscall's return value as a signed long value which is often either a
* pointer or the negated errno value.
*
* - The second level is mostly architecture-independent. It is made of
* static functions called sys_<name>() which rely on my_syscallN()
* depending on the syscall definition. These functions are responsible
* for exposing the appropriate types for the syscall arguments (int,
* pointers, etc) and for setting the appropriate return type (often int).
* A few of them are architecture-specific because the syscalls are not all
* mapped exactly the same among architectures. For example, some archs do
* not implement select() and need pselect6() instead, so the sys_select()
* function will have to abstract this.
*
* - The third level is the libc call definition. It exposes the lower raw
* sys_<name>() calls in a way that looks like what a libc usually does,
* takes care of specific input values, and of setting errno upon error.
* There can be minor variations compared to standard libc calls. For
* example the open() call always takes 3 args here.
*
* The errno variable is declared static and unused. This way it can be
* optimized away if not used. However this means that a program made of
* multiple C files may observe different errno values (one per C file). For
* the type of programs this project targets it usually is not a problem. The
* resulting program may even be reduced by defining the NOLIBC_IGNORE_ERRNO
* macro, in which case the errno value will never be assigned.
*
* Some stdint-like integer types are defined. These are valid on all currently
* supported architectures, because signs are enforced, ints are assumed to be
* 32 bits, longs the size of a pointer and long long 64 bits. If more
* architectures have to be supported, this may need to be adapted.
*
* Some macro definitions like the O_* values passed to open(), and some
* structures like the sys_stat struct depend on the architecture.
*
* The definitions start with the architecture-specific parts, which are picked
* based on what the compiler knows about the target architecture, and are
* completed with the generic code. Since it is the compiler which sets the
* target architecture, cross-compiling normally works out of the box without
* having to specify anything.
*
* Finally some very common libc-level functions are provided. It is the case
* for a few functions usually found in string.h, ctype.h, or stdlib.h. Nothing
* is currently provided regarding stdio emulation.
*
* The macro NOLIBC is always defined, so that it is possible for a program to
* check this macro to know if it is being built against and decide to disable
* some features or simply not to include some standard libc files.
*
* Ideally this file should be split in multiple files for easier long term
* maintenance, but provided as a single file as it is now, it's quite
* convenient to use. Maybe some variations involving a set of includes at the
* top could work.
*
* A simple static executable may be built this way :
* $ gcc -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \
* -static -include nolibc.h -lgcc -o hello hello.c
*
* A very useful calling convention table may be found here :
* http://man7.org/linux/man-pages/man2/syscall.2.html
*
* This doc is quite convenient though not necessarily up to date :
* https://w3challs.com/syscalls/
*
*/
/* Some archs (at least aarch64) don't expose the regular syscalls anymore by
* default, either because they have an "_at" replacement, or because there are
* more modern alternatives. For now we'd rather still use them.
*/
@@ -19,18 +97,6 @@
#define NOLIBC
/* Build a static executable this way :
* $ gcc -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \
* -static -include nolibc.h -lgcc -o hello hello.c
*
* Useful calling convention table found here :
* http://man7.org/linux/man-pages/man2/syscall.2.html
*
* This doc is even better :
* https://w3challs.com/syscalls/
*/
/* this way it will be removed if unused */
static int errno;
@@ -81,9 +147,9 @@ typedef signed long time_t;
/* for poll() */
struct pollfd {
int fd;
short int events;
short int revents;
int fd;
short int events;
short int revents;
};
/* for select() */
@@ -239,7 +305,7 @@ struct stat {
"syscall\n" \
: "=a" (_ret) \
: "0"(_num) \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
); \
_ret; \
})
@@ -255,7 +321,7 @@ struct stat {
: "=a" (_ret) \
: "r"(_arg1), \
"0"(_num) \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
); \
_ret; \
})
@@ -272,7 +338,7 @@ struct stat {
: "=a" (_ret) \
: "r"(_arg1), "r"(_arg2), \
"0"(_num) \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
); \
_ret; \
})
@@ -290,7 +356,7 @@ struct stat {
: "=a" (_ret) \
: "r"(_arg1), "r"(_arg2), "r"(_arg3), \
"0"(_num) \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
: "rcx", "r8", "r9", "r10", "r11", "memory", "cc" \
); \
_ret; \
})
@@ -1006,7 +1072,7 @@ struct sys_stat_struct {
: "=r"(_num), "=r"(_arg4) \
: "r"(_num) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})
@@ -1025,7 +1091,7 @@ struct sys_stat_struct {
: "0"(_num), \
"r"(_arg1) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})
@@ -1045,7 +1111,7 @@ struct sys_stat_struct {
: "0"(_num), \
"r"(_arg1), "r"(_arg2) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})
@@ -1066,7 +1132,7 @@ struct sys_stat_struct {
: "0"(_num), \
"r"(_arg1), "r"(_arg2), "r"(_arg3) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})
@@ -1087,7 +1153,7 @@ struct sys_stat_struct {
: "0"(_num), \
"r"(_arg1), "r"(_arg2), "r"(_arg3), "r"(_arg4) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})
@@ -1110,7 +1176,7 @@ struct sys_stat_struct {
: "0"(_num), \
"r"(_arg1), "r"(_arg2), "r"(_arg3), "r"(_arg4), "r"(_arg5) \
: "memory", "cc", "at", "v1", "hi", "lo", \
\
"t0", "t1", "t2", "t3", "t4", "t5", "t6", "t7", "t8", "t9" \
); \
_arg4 ? -_num : _num; \
})

1
tools/memory-model/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
litmus

View File

@@ -156,6 +156,8 @@ lock.cat
README
This file.
scripts Various scripts, see scripts/README.
===========
LIMITATIONS

View File

@@ -29,7 +29,8 @@ enum Barriers = 'wmb (*smp_wmb*) ||
'sync-rcu (*synchronize_rcu*) ||
'before-atomic (*smp_mb__before_atomic*) ||
'after-atomic (*smp_mb__after_atomic*) ||
'after-spinlock (*smp_mb__after_spinlock*)
'after-spinlock (*smp_mb__after_spinlock*) ||
'after-unlock-lock (*smp_mb__after_unlock_lock*)
instructions F[Barriers]
(* Compute matching pairs of nested Rcu-lock and Rcu-unlock *)

View File

@@ -30,7 +30,9 @@ let wmb = [W] ; fencerel(Wmb) ; [W]
let mb = ([M] ; fencerel(Mb) ; [M]) |
([M] ; fencerel(Before-atomic) ; [RMW] ; po? ; [M]) |
([M] ; po? ; [RMW] ; fencerel(After-atomic) ; [M]) |
([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M])
([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M]) |
([M] ; po ; [UL] ; (co | po) ; [LKW] ;
fencerel(After-unlock-lock) ; [M])
let gp = po ; [Sync-rcu] ; po?
let strong-fence = mb | gp

View File

@@ -23,6 +23,7 @@ smp_wmb() { __fence{wmb}; }
smp_mb__before_atomic() { __fence{before-atomic}; }
smp_mb__after_atomic() { __fence{after-atomic}; }
smp_mb__after_spinlock() { __fence{after-spinlock}; }
smp_mb__after_unlock_lock() { __fence{after-unlock-lock}; }
// Exchange
xchg(X,V) __xchg{mb}(X,V)

View File

@@ -0,0 +1,70 @@
============
LKMM SCRIPTS
============
These scripts are run from the tools/memory-model directory.
checkalllitmus.sh
Run all litmus tests in the litmus-tests directory, checking
the results against the expected results recorded in the
"Result:" comment lines.
checkghlitmus.sh
Run all litmus tests in the https://github.com/paulmckrcu/litmus
archive that are C-language and that have "Result:" comment lines
documenting expected results, comparing the actual results to
those expected.
checklitmushist.sh
Run all litmus tests having .litmus.out files from previous
initlitmushist.sh or newlitmushist.sh runs, comparing the
herd output to that of the original runs.
checklitmus.sh
Check a single litmus test against its "Result:" expected result.
cmplitmushist.sh
Compare output from two different runs of the same litmus tests,
with the absolute pathnames of the tests to run provided one
name per line on standard input. Not normally run manually,
provided instead for use by other scripts.
initlitmushist.sh
Run all litmus tests having no more than the specified number
of processes given a specified timeout, recording the results
in .litmus.out files.
judgelitmus.sh
Given a .litmus file and its .litmus.out herd output, check the
.litmus.out file against the .litmus file's "Result:" comment to
judge whether the test ran correctly. Not normally run manually,
provided instead for use by other scripts.
newlitmushist.sh
For all new or updated litmus tests having no more than the
specified number of processes given a specified timeout, run
and record the results in .litmus.out files.
parseargs.sh
Parse command-line arguments. Not normally run manually,
provided instead for use by other scripts.
runlitmushist.sh
Run the litmus tests whose absolute pathnames are provided one
name per line on standard input. Not normally run manually,
provided instead for use by other scripts.
README
This file

View File

@@ -1,42 +1,27 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Run herd tests on all .litmus files in the specified directory (which
# defaults to litmus-tests) and check each file's result against a "Result:"
# comment within that litmus test. If the verification result does not
# match that specified in the litmus test, this script prints an error
# message prefixed with "^^^". It also outputs verification results to
# a file whose name is that of the specified litmus test, but with ".out"
# appended.
# Run herd tests on all .litmus files in the litmus-tests directory
# and check each file's result against a "Result:" comment within that
# litmus test. If the verification result does not match that specified
# in the litmus test, this script prints an error message prefixed with
# "^^^". It also outputs verification results to a file whose name is
# that of the specified litmus test, but with ".out" appended.
#
# Usage:
# checkalllitmus.sh [ directory ]
# checkalllitmus.sh
#
# The LINUX_HERD_OPTIONS environment variable may be used to specify
# arguments to herd, whose default is defined by the checklitmus.sh script.
# Thus, one would normally run this in the directory containing the memory
# model, specifying the pathname of the litmus test to check.
# Run this in the directory containing the memory model.
#
# This script makes no attempt to run the litmus tests concurrently.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, you can access it online at
# http://www.gnu.org/licenses/gpl-2.0.html.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
litmusdir=${1-litmus-tests}
. scripts/parseargs.sh
litmusdir=litmus-tests
if test -d "$litmusdir" -a -r "$litmusdir" -a -x "$litmusdir"
then
:
@@ -45,6 +30,14 @@ else
exit 255
fi
# Create any new directories that have appeared in the github litmus
# repo since the last run.
if test "$LKMM_DESTDIR" != "."
then
find $litmusdir -type d -print |
( cd "$LKMM_DESTDIR"; sed -e 's/^/mkdir -p /' | sh )
fi
# Find the checklitmus script. If it is not where we expect it, then
# assume that the caller has the PATH environment variable set
# appropriately.
@@ -57,7 +50,7 @@ fi
# Run the script on all the litmus tests in the specified directory
ret=0
for i in litmus-tests/*.litmus
for i in $litmusdir/*.litmus
do
if ! $clscript $i
then
@@ -66,8 +59,8 @@ do
done
if test "$ret" -ne 0
then
echo " ^^^ VERIFICATION MISMATCHES"
echo " ^^^ VERIFICATION MISMATCHES" 1>&2
else
echo All litmus tests verified as was expected.
echo All litmus tests verified as was expected. 1>&2
fi
exit $ret

View File

@@ -0,0 +1,65 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Runs the C-language litmus tests having a maximum number of processes
# to run, defaults to 6.
#
# sh checkghlitmus.sh
#
# Run from the Linux kernel tools/memory-model directory. See the
# parseargs.sh scripts for arguments.
. scripts/parseargs.sh
T=/tmp/checkghlitmus.sh.$$
trap 'rm -rf $T' 0
mkdir $T
# Clone the repository if it is not already present.
if test -d litmus
then
:
else
git clone https://github.com/paulmckrcu/litmus
( cd litmus; git checkout origin/master )
fi
# Create any new directories that have appeared in the github litmus
# repo since the last run.
if test "$LKMM_DESTDIR" != "."
then
find litmus -type d -print |
( cd "$LKMM_DESTDIR"; sed -e 's/^/mkdir -p /' | sh )
fi
# Create a list of the C-language litmus tests previously run.
( cd $LKMM_DESTDIR; find litmus -name '*.litmus.out' -print ) |
sed -e 's/\.out$//' |
xargs -r egrep -l '^ \* Result: (Never|Sometimes|Always|DEADLOCK)' |
xargs -r grep -L "^P${LKMM_PROCS}"> $T/list-C-already
# Create a list of C-language litmus tests with "Result:" commands and
# no more than the specified number of processes.
find litmus -name '*.litmus' -exec grep -l -m 1 "^C " {} \; > $T/list-C
xargs < $T/list-C -r egrep -l '^ \* Result: (Never|Sometimes|Always|DEADLOCK)' > $T/list-C-result
xargs < $T/list-C-result -r grep -L "^P${LKMM_PROCS}" > $T/list-C-result-short
# Form list of tests without corresponding .litmus.out files
sort $T/list-C-already $T/list-C-result-short | uniq -u > $T/list-C-needed
# Run any needed tests.
if scripts/runlitmushist.sh < $T/list-C-needed > $T/run.stdout 2> $T/run.stderr
then
errs=
else
errs=1
fi
sed < $T/list-C-result-short -e 's,^,scripts/judgelitmus.sh ,' |
sh > $T/judge.stdout 2> $T/judge.stderr
if test -n "$errs"
then
cat $T/run.stderr 1>&2
fi
grep '!!!' $T/judge.stdout

View File

@@ -1,40 +1,24 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Run a herd test and check the result against a "Result:" comment within
# the litmus test. If the verification result does not match that specified
# in the litmus test, this script prints an error message prefixed with
# "^^^" and exits with a non-zero status. It also outputs verification
# Run a herd test and invokes judgelitmus.sh to check the result against
# a "Result:" comment within the litmus test. It also outputs verification
# results to a file whose name is that of the specified litmus test, but
# with ".out" appended.
#
# Usage:
# checklitmus.sh file.litmus
#
# The LINUX_HERD_OPTIONS environment variable may be used to specify
# arguments to herd, which default to "-conf linux-kernel.cfg". Thus,
# one would normally run this in the directory containing the memory model,
# specifying the pathname of the litmus test to check.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, you can access it online at
# http://www.gnu.org/licenses/gpl-2.0.html.
# Run this in the directory containing the memory model, specifying the
# pathname of the litmus test to check. The caller is expected to have
# properly set up the LKMM environment variables.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
litmus=$1
herdoptions=${LINUX_HERD_OPTIONS--conf linux-kernel.cfg}
herdoptions=${LKMM_HERD_OPTIONS--conf linux-kernel.cfg}
if test -f "$litmus" -a -r "$litmus"
then
@@ -43,44 +27,8 @@ else
echo ' --- ' error: \"$litmus\" is not a readable file
exit 255
fi
if grep -q '^ \* Result: ' $litmus
then
outcome=`grep -m 1 '^ \* Result: ' $litmus | awk '{ print $3 }'`
else
outcome=specified
fi
echo Herd options: $herdoptions > $litmus.out
/usr/bin/time herd7 -o ~/tmp $herdoptions $litmus >> $litmus.out 2>&1
grep "Herd options:" $litmus.out
grep '^Observation' $litmus.out
if grep -q '^Observation' $litmus.out
then
:
else
cat $litmus.out
echo ' ^^^ Verification error'
echo ' ^^^ Verification error' >> $litmus.out 2>&1
exit 255
fi
if test "$outcome" = DEADLOCK
then
echo grep 3 and 4
if grep '^Observation' $litmus.out | grep -q 'Never 0 0$'
then
ret=0
else
echo " ^^^ Unexpected non-$outcome verification"
echo " ^^^ Unexpected non-$outcome verification" >> $litmus.out 2>&1
ret=1
fi
elif grep '^Observation' $litmus.out | grep -q $outcome || test "$outcome" = Maybe
then
ret=0
else
echo " ^^^ Unexpected non-$outcome verification"
echo " ^^^ Unexpected non-$outcome verification" >> $litmus.out 2>&1
ret=1
fi
tail -2 $litmus.out | head -1
exit $ret
echo Herd options: $herdoptions > $LKMM_DESTDIR/$litmus.out
/usr/bin/time $LKMM_TIMEOUT_CMD herd7 $herdoptions $litmus >> $LKMM_DESTDIR/$litmus.out 2>&1
scripts/judgelitmus.sh $litmus

View File

@@ -0,0 +1,60 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Reruns the C-language litmus tests previously run that match the
# specified criteria, and compares the result to that of the previous
# runs from initlitmushist.sh and/or newlitmushist.sh.
#
# sh checklitmushist.sh
#
# Run from the Linux kernel tools/memory-model directory.
# See scripts/parseargs.sh for list of arguments.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
. scripts/parseargs.sh
T=/tmp/checklitmushist.sh.$$
trap 'rm -rf $T' 0
mkdir $T
if test -d litmus
then
:
else
echo Run scripts/initlitmushist.sh first, need litmus repo.
exit 1
fi
# Create the results directory and populate it with subdirectories.
# The initial output is created here to avoid clobbering the output
# generated earlier.
mkdir $T/results
find litmus -type d -print | ( cd $T/results; sed -e 's/^/mkdir -p /' | sh )
# Create the list of litmus tests already run, then remove those that
# are excluded by this run's --procs argument.
( cd $LKMM_DESTDIR; find litmus -name '*.litmus.out' -print ) |
sed -e 's/\.out$//' |
xargs -r grep -L "^P${LKMM_PROCS}"> $T/list-C-already
xargs < $T/list-C-already -r grep -L "^P${LKMM_PROCS}" > $T/list-C-short
# Redirect output, run tests, then restore destination directory.
destdir="$LKMM_DESTDIR"
LKMM_DESTDIR=$T/results; export LKMM_DESTDIR
scripts/runlitmushist.sh < $T/list-C-short > $T/runlitmushist.sh.out 2>&1
LKMM_DESTDIR="$destdir"; export LKMM_DESTDIR
# Move the newly generated .litmus.out files to .litmus.out.new files
# in the destination directory.
cdir=`pwd`
ddir=`awk -v c="$cdir" -v d="$LKMM_DESTDIR" \
'END { if (d ~ /^\//) print d; else print c "/" d; }' < /dev/null`
( cd $T/results; find litmus -type f -name '*.litmus.out' -print |
sed -e 's,^.*$,cp & '"$ddir"'/&.new,' | sh )
sed < $T/list-C-short -e 's,^,'"$LKMM_DESTDIR/"',' |
sh scripts/cmplitmushist.sh
exit $?

View File

@@ -0,0 +1,87 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Compares .out and .out.new files for each name on standard input,
# one full pathname per line. Outputs comparison results followed by
# a summary.
#
# sh cmplitmushist.sh
T=/tmp/cmplitmushist.sh.$$
trap 'rm -rf $T' 0
mkdir $T
# comparetest oldpath newpath
perfect=0
obsline=0
noobsline=0
obsresult=0
badcompare=0
comparetest () {
grep -v 'maxresident)k\|minor)pagefaults\|^Time' $1 > $T/oldout
grep -v 'maxresident)k\|minor)pagefaults\|^Time' $2 > $T/newout
if cmp -s $T/oldout $T/newout && grep -q '^Observation' $1
then
echo Exact output match: $2
perfect=`expr "$perfect" + 1`
return 0
fi
grep '^Observation' $1 > $T/oldout
grep '^Observation' $2 > $T/newout
if test -s $T/oldout -o -s $T/newout
then
if cmp -s $T/oldout $T/newout
then
echo Matching Observation result and counts: $2
obsline=`expr "$obsline" + 1`
return 0
fi
else
echo Missing Observation line "(e.g., herd7 timeout)": $2
noobsline=`expr "$noobsline" + 1`
return 0
fi
grep '^Observation' $1 | awk '{ print $3 }' > $T/oldout
grep '^Observation' $2 | awk '{ print $3 }' > $T/newout
if cmp -s $T/oldout $T/newout
then
echo Matching Observation Always/Sometimes/Never result: $2
obsresult=`expr "$obsresult" + 1`
return 0
fi
echo ' !!!' Result changed: $2
badcompare=`expr "$badcompare" + 1`
return 1
}
sed -e 's/^.*$/comparetest &.out &.out.new/' > $T/cmpscript
. $T/cmpscript > $T/cmpscript.out
cat $T/cmpscript.out
echo ' ---' Summary: 1>&2
grep '!!!' $T/cmpscript.out 1>&2
if test "$perfect" -ne 0
then
echo Exact output matches: $perfect 1>&2
fi
if test "$obsline" -ne 0
then
echo Matching Observation result and counts: $obsline 1>&2
fi
if test "$noobsline" -ne 0
then
echo Missing Observation line "(e.g., herd7 timeout)": $noobsline 1>&2
fi
if test "$obsresult" -ne 0
then
echo Matching Observation Always/Sometimes/Never result: $obsresult 1>&2
fi
if test "$badcompare" -ne 0
then
echo "!!!" Result changed: $badcompare 1>&2
exit 1
fi
exit 0

View File

@@ -0,0 +1,68 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Runs the C-language litmus tests matching the specified criteria.
# Generates the output for each .litmus file into a corresponding
# .litmus.out file, and does not judge the result.
#
# sh initlitmushist.sh
#
# Run from the Linux kernel tools/memory-model directory.
# See scripts/parseargs.sh for list of arguments.
#
# This script can consume significant wallclock time and CPU, especially as
# the value of --procs rises. On a four-core (eight hardware threads)
# 2.5GHz x86 with a one-minute per-run timeout:
#
# --procs wallclock CPU timeouts tests
# 1 0m11.241s 0m1.086s 0 19
# 2 1m12.598s 2m8.459s 2 393
# 3 1m30.007s 6m2.479s 4 2291
# 4 3m26.042s 18m5.139s 9 3217
# 5 4m26.661s 23m54.128s 13 3784
# 6 4m41.900s 26m4.721s 13 4352
# 7 5m51.463s 35m50.868s 13 4626
# 8 10m5.235s 68m43.672s 34 5117
# 9 15m57.80s 105m58.101s 69 5156
# 10 16m14.13s 103m35.009s 69 5165
# 20 27m48.55s 198m3.286s 156 5269
#
# Increasing the timeout on the 20-process run to five minutes increases
# the runtime to about 90 minutes with the CPU time rising to about
# 10 hours. On the other hand, it decreases the number of timeouts to 101.
#
# Note that there are historical tests for which herd7 will fail
# completely, for example, litmus/manual/atomic/C-unlock-wait-00.litmus
# contains a call to spin_unlock_wait(), which no longer exists in either
# the kernel or LKMM.
. scripts/parseargs.sh
T=/tmp/initlitmushist.sh.$$
trap 'rm -rf $T' 0
mkdir $T
if test -d litmus
then
:
else
git clone https://github.com/paulmckrcu/litmus
( cd litmus; git checkout origin/master )
fi
# Create any new directories that have appeared in the github litmus
# repo since the last run.
if test "$LKMM_DESTDIR" != "."
then
find litmus -type d -print |
( cd "$LKMM_DESTDIR"; sed -e 's/^/mkdir -p /' | sh )
fi
# Create a list of the C-language litmus tests with no more than the
# specified number of processes (per the --procs argument).
find litmus -name '*.litmus' -exec grep -l -m 1 "^C " {} \; > $T/list-C
xargs < $T/list-C -r grep -L "^P${LKMM_PROCS}" > $T/list-C-short
scripts/runlitmushist.sh < $T/list-C-short
exit 0

View File

@@ -0,0 +1,78 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Given a .litmus test and the corresponding .litmus.out file, check
# the .litmus.out file against the "Result:" comment to judge whether
# the test ran correctly.
#
# Usage:
# judgelitmus.sh file.litmus
#
# Run this in the directory containing the memory model, specifying the
# pathname of the litmus test to check.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
litmus=$1
if test -f "$litmus" -a -r "$litmus"
then
:
else
echo ' --- ' error: \"$litmus\" is not a readable file
exit 255
fi
if test -f "$LKMM_DESTDIR/$litmus".out -a -r "$LKMM_DESTDIR/$litmus".out
then
:
else
echo ' --- ' error: \"$LKMM_DESTDIR/$litmus\".out is not a readable file
exit 255
fi
if grep -q '^ \* Result: ' $litmus
then
outcome=`grep -m 1 '^ \* Result: ' $litmus | awk '{ print $3 }'`
else
outcome=specified
fi
grep '^Observation' $LKMM_DESTDIR/$litmus.out
if grep -q '^Observation' $LKMM_DESTDIR/$litmus.out
then
:
else
echo ' !!! Verification error' $litmus
if ! grep -q '!!!' $LKMM_DESTDIR/$litmus.out
then
echo ' !!! Verification error' >> $LKMM_DESTDIR/$litmus.out 2>&1
fi
exit 255
fi
if test "$outcome" = DEADLOCK
then
if grep '^Observation' $LKMM_DESTDIR/$litmus.out | grep -q 'Never 0 0$'
then
ret=0
else
echo " !!! Unexpected non-$outcome verification" $litmus
if ! grep -q '!!!' $LKMM_DESTDIR/$litmus.out
then
echo " !!! Unexpected non-$outcome verification" >> $LKMM_DESTDIR/$litmus.out 2>&1
fi
ret=1
fi
elif grep '^Observation' $LKMM_DESTDIR/$litmus.out | grep -q $outcome || test "$outcome" = Maybe
then
ret=0
else
echo " !!! Unexpected non-$outcome verification" $litmus
if ! grep -q '!!!' $LKMM_DESTDIR/$litmus.out
then
echo " !!! Unexpected non-$outcome verification" >> $LKMM_DESTDIR/$litmus.out 2>&1
fi
ret=1
fi
tail -2 $LKMM_DESTDIR/$litmus.out | head -1
exit $ret

View File

@@ -0,0 +1,61 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# Runs the C-language litmus tests matching the specified criteria
# that do not already have a corresponding .litmus.out file, and does
# not judge the result.
#
# sh newlitmushist.sh
#
# Run from the Linux kernel tools/memory-model directory.
# See scripts/parseargs.sh for list of arguments.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
. scripts/parseargs.sh
T=/tmp/newlitmushist.sh.$$
trap 'rm -rf $T' 0
mkdir $T
if test -d litmus
then
:
else
echo Run scripts/initlitmushist.sh first, need litmus repo.
exit 1
fi
# Create any new directories that have appeared in the github litmus
# repo since the last run.
if test "$LKMM_DESTDIR" != "."
then
find litmus -type d -print |
( cd "$LKMM_DESTDIR"; sed -e 's/^/mkdir -p /' | sh )
fi
# Create a list of the C-language litmus tests previously run.
( cd $LKMM_DESTDIR; find litmus -name '*.litmus.out' -print ) |
sed -e 's/\.out$//' |
xargs -r grep -L "^P${LKMM_PROCS}"> $T/list-C-already
# Form full list of litmus tests with no more than the specified
# number of processes (per the --procs argument).
find litmus -name '*.litmus' -exec grep -l -m 1 "^C " {} \; > $T/list-C-all
xargs < $T/list-C-all -r grep -L "^P${LKMM_PROCS}" > $T/list-C-short
# Form list of new tests. Note: This does not handle litmus-test deletion!
sort $T/list-C-already $T/list-C-short | uniq -u > $T/list-C-new
# Form list of litmus tests that have changed since the last run.
sed < $T/list-C-short -e 's,^.*$,if test & -nt '"$LKMM_DESTDIR"'/&.out; then echo &; fi,' > $T/list-C-script
sh $T/list-C-script > $T/list-C-newer
# Merge the list of new and of updated litmus tests: These must be (re)run.
sort -u $T/list-C-new $T/list-C-newer > $T/list-C-needed
scripts/runlitmushist.sh < $T/list-C-needed
exit 0

View File

@@ -0,0 +1,136 @@
#!/bin/sh
# SPDX-License-Identifier: GPL-2.0+
#
# the corresponding .litmus.out file, and does not judge the result.
#
# . scripts/parseargs.sh
#
# Include into other Linux kernel tools/memory-model scripts.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
T=/tmp/parseargs.sh.$$
mkdir $T
# Initialize one parameter: initparam name default
initparam () {
echo if test -z '"$'$1'"' > $T/s
echo then >> $T/s
echo $1='"'$2'"' >> $T/s
echo export $1 >> $T/s
echo fi >> $T/s
echo $1_DEF='$'$1 >> $T/s
. $T/s
}
initparam LKMM_DESTDIR "."
initparam LKMM_HERD_OPTIONS "-conf linux-kernel.cfg"
initparam LKMM_JOBS `getconf _NPROCESSORS_ONLN`
initparam LKMM_PROCS "3"
initparam LKMM_TIMEOUT "1m"
scriptname=$0
usagehelp () {
echo "Usage $scriptname [ arguments ]"
echo " --destdir path (place for .litmus.out, default by .litmus)"
echo " --herdopts -conf linux-kernel.cfg ..."
echo " --jobs N (number of jobs, default one per CPU)"
echo " --procs N (litmus tests with at most this many processes)"
echo " --timeout N (herd7 timeout (e.g., 10s, 1m, 2hr, 1d, '')"
echo "Defaults: --destdir '$LKMM_DESTDIR_DEF' --herdopts '$LKMM_HERD_OPTIONS_DEF' --jobs '$LKMM_JOBS_DEF' --procs '$LKMM_PROCS_DEF' --timeout '$LKMM_TIMEOUT_DEF'"
exit 1
}
usage () {
usagehelp 1>&2
}
# checkarg --argname argtype $# arg mustmatch cannotmatch
checkarg () {
if test $3 -le 1
then
echo $1 needs argument $2 matching \"$5\"
usage
fi
if echo "$4" | grep -q -e "$5"
then
:
else
echo $1 $2 \"$4\" must match \"$5\"
usage
fi
if echo "$4" | grep -q -e "$6"
then
echo $1 $2 \"$4\" must not match \"$6\"
usage
fi
}
while test $# -gt 0
do
case "$1" in
--destdir)
checkarg --destdir "(path to directory)" "$#" "$2" '.\+' '^--'
LKMM_DESTDIR="$2"
mkdir $LKMM_DESTDIR > /dev/null 2>&1
if ! test -e "$LKMM_DESTDIR"
then
echo "Cannot create directory --destdir '$LKMM_DESTDIR'"
usage
fi
if test -d "$LKMM_DESTDIR" -a -w "$LKMM_DESTDIR" -a -x "$LKMM_DESTDIR"
then
:
else
echo "Directory --destdir '$LKMM_DESTDIR' insufficient permissions to create files"
usage
fi
shift
;;
--herdopts|--herdopt)
checkarg --destdir "(herd options)" "$#" "$2" '.*' '^--'
LKMM_HERD_OPTIONS="$2"
shift
;;
-j[1-9]*)
njobs="`echo $1 | sed -e 's/^-j//'`"
trailchars="`echo $njobs | sed -e 's/[0-9]\+\(.*\)$/\1/'`"
if test -n "$trailchars"
then
echo $1 trailing characters "'$trailchars'"
usagehelp
fi
LKMM_JOBS="`echo $njobs | sed -e 's/^\([0-9]\+\).*$/\1/'`"
;;
--jobs|--job|-j)
checkarg --jobs "(number)" "$#" "$2" '^[1-9][0-9]\+$' '^--'
LKMM_JOBS="$2"
shift
;;
--procs|--proc)
checkarg --procs "(number)" "$#" "$2" '^[0-9]\+$' '^--'
LKMM_PROCS="$2"
shift
;;
--timeout)
checkarg --timeout "(timeout spec)" "$#" "$2" '^\([0-9]\+[smhd]\?\|\)$' '^--'
LKMM_TIMEOUT="$2"
shift
;;
*)
echo Unknown argument $1
usage
;;
esac
shift
done
if test -z "$LKMM_TIMEOUT"
then
LKMM_TIMEOUT_CMD=""; export LKMM_TIMEOUT_CMD
else
LKMM_TIMEOUT_CMD="timeout $LKMM_TIMEOUT"; export LKMM_TIMEOUT_CMD
fi
rm -rf $T

View File

@@ -0,0 +1,87 @@
#!/bin/bash
# SPDX-License-Identifier: GPL-2.0+
#
# Runs the C-language litmus tests specified on standard input, using up
# to the specified number of CPUs (defaulting to all of them) and placing
# the results in the specified directory (defaulting to the same place
# the litmus test came from).
#
# sh runlitmushist.sh
#
# Run from the Linux kernel tools/memory-model directory.
# This script uses environment variables produced by parseargs.sh.
#
# Copyright IBM Corporation, 2018
#
# Author: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
T=/tmp/runlitmushist.sh.$$
trap 'rm -rf $T' 0
mkdir $T
if test -d litmus
then
:
else
echo Directory \"litmus\" missing, aborting run.
exit 1
fi
# Prefixes for per-CPU scripts
for ((i=0;i<$LKMM_JOBS;i++))
do
echo dir="$LKMM_DESTDIR" > $T/$i.sh
echo T=$T >> $T/$i.sh
echo herdoptions=\"$LKMM_HERD_OPTIONS\" >> $T/$i.sh
cat << '___EOF___' >> $T/$i.sh
runtest () {
echo ' ... ' /usr/bin/time $LKMM_TIMEOUT_CMD herd7 $herdoptions $1 '>' $dir/$1.out '2>&1'
if /usr/bin/time $LKMM_TIMEOUT_CMD herd7 $herdoptions $1 > $dir/$1.out 2>&1
then
if ! grep -q '^Observation ' $dir/$1.out
then
echo ' !!! Herd failed, no Observation:' $1
fi
else
exitcode=$?
if test "$exitcode" -eq 124
then
exitmsg="timed out"
else
exitmsg="failed, exit code $exitcode"
fi
echo ' !!! Herd' ${exitmsg}: $1
fi
}
___EOF___
done
awk -v q="'" -v b='\\' '
{
print "echo `grep " q "^P[0-9]" b "+(" q " " $0 " | tail -1 | sed -e " q "s/^P" b "([0-9]" b "+" b ")(.*$/" b "1/" q "` " $0
}' | bash |
sort -k1n |
awk -v ncpu=$LKMM_JOBS -v t=$T '
{
print "runtest " $2 >> t "/" NR % ncpu ".sh";
}
END {
for (i = 0; i < ncpu; i++) {
print "sh " t "/" i ".sh > " t "/" i ".sh.out 2>&1 &";
close(t "/" i ".sh");
}
print "wait";
}' | sh
cat $T/*.sh.out
if grep -q '!!!' $T/*.sh.out
then
echo ' ---' Summary: 1>&2
grep '!!!' $T/*.sh.out 1>&2
nfail="`grep '!!!' $T/*.sh.out | wc -l`"
echo 'Number of failed herd runs (e.g., timeout): ' $nfail 1>&2
exit 1
else
echo All runs completed successfully. 1>&2
exit 0
fi

View File

@@ -40,17 +40,24 @@ mkdir $T
cat > $T/init << '__EOF___'
#!/bin/sh
# Run in userspace a few milliseconds every second. This helps to
# exercise the NO_HZ_FULL portions of RCU.
# exercise the NO_HZ_FULL portions of RCU. The 192 instances of "a" was
# empirically shown to give a nice multi-millisecond burst of user-mode
# execution on a 2GHz CPU, as desired. Modern CPUs will vary from a
# couple of milliseconds up to perhaps 100 milliseconds, which is an
# acceptable range.
#
# Why not calibrate an exact delay? Because within this initrd, we
# are restricted to Bourne-shell builtins, which as far as I know do not
# provide any means of obtaining a fine-grained timestamp.
a4="a a a a"
a16="$a4 $a4 $a4 $a4"
a64="$a16 $a16 $a16 $a16"
a192="$a64 $a64 $a64"
while :
do
q=
for i in \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a \
a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a
for i in $a192
do
q="$q $i"
done
@@ -124,8 +131,8 @@ if echo -e "#if __x86_64__||__i386__||__i486__||__i586__||__i686__" \
| grep -q '^yes'; then
# architecture supported by nolibc
${CROSS_COMPILE}gcc -fno-asynchronous-unwind-tables -fno-ident \
-nostdlib -include ../bin/nolibc.h -lgcc -s -static -Os \
-o init init.c
-nostdlib -include ../../../../include/nolibc/nolibc.h \
-lgcc -s -static -Os -o init init.c
else
${CROSS_COMPILE}gcc -s -static -Os -o init init.c
fi