Peter Zijlstra 1cde2930e1 sched/preempt: Add static_key() to preempt_notifiers ...

Avoid touching the curr->preempt_notifier cacheline when not needed.

Provides a small improvement on pipe-bench:

  taskset 01 perf stat --repeat 10 -- perf bench sched pipe

before:

 Performance counter stats for 'perf bench sched pipe' (10 runs):

      12385.016204      task-clock (msec)         #    1.001 CPUs utilized            ( +-  0.34% )
         2,000,023      context-switches          #    0.161 M/sec                    ( +-  0.00% )
                 0      cpu-migrations            #    0.000 K/sec
               175      page-faults               #    0.014 K/sec                    ( +-  0.26% )
    41,376,162,250      cycles                    #    3.341 GHz                      ( +-  0.11% )
    17,389,139,321      stalled-cycles-frontend   #   42.03% frontend cycles idle     ( +-  0.25% )
   <not supported>      stalled-cycles-backend
    68,788,588,003      instructions              #    1.66  insns per cycle
                                                  #    0.25  stalled cycles per insn  ( +-  0.02% )
    13,449,387,620      branches                  # 1085.940 M/sec                    ( +-  0.02% )
        20,880,690      branch-misses             #    0.16% of all branches          ( +-  0.98% )

      12.372646094 seconds time elapsed                                          ( +-  0.34% )

after:

 Performance counter stats for 'perf bench sched pipe' (10 runs):

      12180.936528      task-clock (msec)         #    1.001 CPUs utilized            ( +-  0.33% )
         2,000,077      context-switches          #    0.164 M/sec                    ( +-  0.00% )
                 0      cpu-migrations            #    0.000 K/sec
               174      page-faults               #    0.014 K/sec                    ( +-  0.27% )
    40,691,545,577      cycles                    #    3.341 GHz                      ( +-  0.06% )
    16,446,333,371      stalled-cycles-frontend   #   40.42% frontend cycles idle     ( +-  0.18% )
   <not supported>      stalled-cycles-backend
    68,570,100,387      instructions              #    1.69  insns per cycle
                                                  #    0.24  stalled cycles per insn  ( +-  0.01% )
    13,389,740,014      branches                  # 1099.237 M/sec                    ( +-  0.01% )
        20,175,440      branch-misses             #    0.15% of all branches          ( +-  0.52% )

      12.169253010 seconds time elapsed                                          ( +-  0.33% )

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

2015-06-19 10:06:45 +02:00

..

bpf

bpf: fix 64-bit divide

2015-04-27 23:11:49 -04:00

configs

x86: Add "make tinyconfig" to configure the tiniest possible kernel

2014-08-08 16:30:24 -07:00

debug

debug: prevent entering debug mode on panic/exception.

2015-02-19 12:39:03 -06:00

events

perf: Annotate inherited event ctx->mutex recursion

2015-05-08 11:59:40 +02:00

gcov

gcov: fix softlockups

2015-04-17 09:04:08 -04:00

irq

genirq: Set IRQCHIP_SKIP_SET_WAKE flag for dummy_irq_chip

2015-04-24 20:57:06 +02:00

livepatch

Merge branch 'for-4.1/core-noarch' into for-linus

2015-04-13 23:57:20 +02:00

locking

sched/stop_machine: Fix deadlock between multiple stop_two_cpus()

2015-06-19 10:03:12 +02:00

power

Merge back earlier suspend/hibernate material for v4.1.

2015-04-10 12:01:59 +02:00

printk

Merge tag 'tty-4.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty

2015-04-21 09:33:10 -07:00

rcu

rcu: Control grace-period delays directly from value

2015-04-14 19:33:59 -07:00

sched

sched/preempt: Add static_key() to preempt_notifiers

2015-06-19 10:06:45 +02:00

time

Merge branch 'linus' into sched/core, to resolve conflict

2015-06-02 08:05:42 +02:00

trace

tracing: Make ftrace_print_array_seq compute buf_len

2015-05-06 23:03:23 -04:00

.gitignore

…

acct.c

acct: check FMODE_CAN_WRITE

2015-04-11 22:27:55 -04:00

async.c

kernel/async.c: switch to pr_foo()

2014-10-09 22:26:04 -04:00

audit_tree.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2015-04-26 17:22:07 -07:00

audit_watch.c

VFS: audit: d_backing_inode() annotations

2015-04-15 15:06:55 -04:00

audit.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2015-04-26 17:22:07 -07:00

audit.h

Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit

2015-04-22 14:49:23 -07:00

auditfilter.c

Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit

2015-02-11 20:07:47 -08:00

auditsc.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2015-04-26 17:22:07 -07:00

backtracetest.c

…

bounds.c

page-cgroup: get rid of NR_PCG_FLAGS

2014-08-08 15:57:18 -07:00

capability.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

cgroup_freezer.c

cgroup: rename cgroup_subsys->base_cftypes to ->legacy_cftypes

2014-07-15 11:05:09 -04:00

cgroup.c

cgroup: remove use of seq_printf return value

2015-04-15 16:35:25 -07:00

compat.c

all arches, signal: move restart_block to struct task_struct

2015-02-12 18:54:12 -08:00

configs.c

…

context_tracking.c

context_tracking: Export context_tracking_user_enter/exit

2015-03-09 15:43:00 +01:00

cpu_pm.c

…

cpu.c

Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-04-14 13:36:04 -07:00

cpuset.c

kernel, cpuset: remove exception for __GFP_THISNODE

2015-04-14 16:49:03 -07:00

crash_dump.c

crash_dump: Make is_kdump_kernel() accessible from modules

2014-08-25 15:42:19 -07:00

cred.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

delayacct.c

delayacct: Remove braindamaged type conversions

2014-07-23 10:18:06 -07:00

dma.c

…

elfcore.c

…

exec_domain.c

Remove rest of exec domains.

2015-04-12 21:03:31 +02:00

exit.c

Remove execution domain support

2015-04-12 20:58:24 +02:00

extable.c

ftrace/x86/extable: Add is_ftrace_trampoline() function

2014-11-19 15:25:26 -05:00

fork.c

sched/preempt, mm/fault: Count pagefault_disable() levels in pagefault_disabled

2015-05-19 08:39:13 +02:00

freezer.c

freezer: remove obsolete comments in __thaw_task()

2014-10-21 23:44:20 +02:00

futex_compat.c

…

futex.c

futex: Implement lockless wakeups

2015-05-08 12:21:40 +02:00

groups.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

hung_task.c

kernel/hung_task.c: change hung_task.c to use for_each_process_thread()

2015-04-15 16:35:22 -07:00

irq_work.c

percpu: Convert remaining __get_cpu_var uses in 3.18-rcX

2014-10-29 11:18:18 -04:00

jump_label.c

…

kallsyms.c

kernel/kallsyms.c: use __seq_open_private()

2014-10-14 02:18:16 +02:00

kcmp.c

kcmp: fix standard comparison bug

2014-09-10 15:42:12 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

locking/mcs: Better differentiate between MCS variants

2015-01-14 15:07:32 +01:00

Kconfig.preempt

…

kexec.c

kexec: allocate the kexec control page with KEXEC_CONTROL_MEMORY_GFP

2015-04-23 16:52:01 +02:00

kmod.c

usermodehelper: kill the kmod_thread_locker logic

2014-12-10 17:41:17 -08:00

kprobes.c

kprobes: makes kprobes/enabled works correctly for optimized kprobes.

2015-02-13 21:21:42 -08:00

ksysfs.c

…

kthread.c

kernel/kthread.c: partial revert of 81c98869fa ("kthread: ensure locality of task_struct allocations")

2014-10-09 22:25:51 -04:00

latencytop.c

…

Makefile

modsign: change default key details

2015-04-30 09:35:41 -07:00

module_signing.c

…

module-internal.h

…

module.c

module: Call module notifier on failure after complete_formation()

2015-05-09 03:29:24 +09:30

notifier.c

rcu: Make SRCU optional by using CONFIG_SRCU

2015-01-06 11:04:29 -08:00

nsproxy.c

bury struct proc_ns in fs/proc

2014-12-04 14:34:54 -05:00

padata.c

padata: use %*pb[l] to print bitmaps including cpumasks and nodemasks

2015-02-13 21:21:38 -08:00

panic.c

livepatch: kernel: add TAINT_LIVEPATCH

2014-12-22 15:40:48 +01:00

params.c

params: handle quotes properly for values not of form foo="bar".

2015-04-15 13:31:23 +09:30

pid_namespace.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2014-12-16 15:53:03 -08:00

pid.c

fork: report pid reservation failure properly

2015-04-17 09:04:06 -04:00

profile.c

profile: use %*pb[l] to print bitmaps including cpumasks and nodemasks

2015-02-13 21:21:38 -08:00

ptrace.c

ptrace: ptrace_detach() can no longer race with SIGKILL

2015-04-17 09:04:06 -04:00

range.c

kernel: avoid overflow in cmp_range

2015-01-17 10:02:23 +13:00

reboot.c

kernel/reboot.c: add orderly_reboot for graceful reboot

2015-04-15 16:35:23 -07:00

relay.c

VFS: kernel/: d_inode() annotations

2015-04-15 15:06:55 -04:00

resource.c

kernel/resource.c: remove deprecated __check_region() and friends

2015-04-15 16:35:22 -07:00

seccomp.c

seccomp: cap SECCOMP_RET_ERRNO data to MAX_ERRNO

2015-02-17 14:34:55 -08:00

signal.c

signals, sched: Change all uses of JOBCTL_* from 'int' to 'long'

2015-05-08 12:04:36 +02:00

smp.c

smp: Fix error case handling in smp_call_function_*()

2015-04-19 13:19:23 -07:00

smpboot.c

smpboot: Add common code for notification from dying CPU

2015-03-11 13:20:25 -07:00

smpboot.h

…

softirq.c

Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-02-09 15:24:03 -08:00

stacktrace.c

stacktrace: introduce snprint_stack_trace for buffer output

2014-12-13 12:42:48 -08:00

stop_machine.c

sched/stop_machine: Fix deadlock between multiple stop_two_cpus()

2015-06-19 10:03:12 +02:00

sys_ni.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

sys.c

prctl: avoid using mmap_sem for exe_file serialization

2015-04-17 09:04:07 -04:00

sysctl_binary.c

kernel: add panic_on_warn

2014-12-10 17:41:10 -08:00

sysctl.c

kernel/sysctl.c: detect overflows when converting to int

2015-04-17 09:04:08 -04:00

system_certificates.S

…

system_keyring.c

KEYS: validate certificate trust only with builtin keys

2014-07-17 09:35:17 -04:00

task_work.c

…

taskstats.c

netlink: make nlmsg_end() and genlmsg_end() void

2015-01-18 01:03:45 -05:00

test_kprobes.c

kernel/test_kprobes.c: use current logging functions

2014-08-08 15:57:18 -07:00

torture.c

torture: Address race in module cleanup

2014-09-16 13:41:06 -07:00

tracepoint.c

…

tsacct.c

sched: Make task->start_time nanoseconds based

2014-07-23 10:18:05 -07:00

uid16.c

groups: Consolidate the setgroups permission checks

2014-12-05 17:19:27 -06:00

up.c

…

user_namespace.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2014-12-17 12:31:40 -08:00

user-return-notifier.c

scheduler: Replace __get_cpu_var with this_cpu_ptr

2014-08-26 13:45:45 -04:00

user.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2014-12-17 12:31:40 -08:00

utsname_sysctl.c

…

utsname.c

copy address of proc_ns_ops into ns_common

2014-12-04 14:34:47 -05:00

watchdog.c

watchdog: fix double lock in watchdog_nmi_enable_all

2015-05-19 10:57:03 -07:00

workqueue_internal.h

…

workqueue.c

workqueue: Reorder sysfs code

2015-04-06 11:16:04 -04:00