Tim Chen
ddf1d169c0
locking/mcs: Allow architecture specific asm files to be used for contended case
...
This patch allows each architecture to add its specific assembly optimized
arch_mcs_spin_lock_contended and arch_mcs_spinlock_uncontended for
MCS lock and unlock functions.
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com >
Cc: Scott J Norton <scott.norton@hp.com >
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com >
Cc: AswinChandramouleeswaran <aswin@hp.com >
Cc: George Spelvin <linux@horizon.com >
Cc: Rik vanRiel <riel@redhat.com >
Cc: Andrea Arcangeli <aarcange@redhat.com >
Cc: MichelLespinasse <walken@google.com >
Cc: Peter Hurley <peter@hurleysoftware.com >
Cc: Andi Kleen <andi@firstfloor.org >
Cc: Alex Shi <alex.shi@linaro.org >
Cc: Dave Hansen <dave.hansen@intel.com >
Cc: Tim Chen <tim.c.chen@linux.intel.com >
Cc: Arnd Bergmann <arnd@arndb.de >
Cc: "Figo.zhang" <figo1802@gmail.com >
Cc: "Paul E.McKenney" <paulmck@linux.vnet.ibm.com >
Cc: "H. Peter Anvin" <hpa@zytor.com >
Cc: Davidlohr Bueso <davidlohr.bueso@hp.com >
Cc: Waiman Long <waiman.long@hp.com >
Cc: Ingo Molnar <mingo@elte.hu >
Cc: Will Deacon <will.deacon@arm.com >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Matthew R Wilcox <matthew.r.wilcox@intel.com >
Signed-off-by: Peter Zijlstra <peterz@infradead.org >
Link: http://lkml.kernel.org/r/1390347382.3138.67.camel@schen9-DESK
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2014-02-09 21:18:52 +01:00
Will Deacon
e207552e64
locking/mcs: Allow architectures to hook in to contended paths
...
When contended, architectures may be able to reduce the polling overhead
in ways which aren't expressible using a simple relax() primitive.
This patch allows architectures to hook into the mcs_{lock,unlock}
functions for the contended cases only.
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Signed-off-by: Will Deacon <will.deacon@arm.com >
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com >
Signed-off-by: Peter Zijlstra <peterz@infradead.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Link: http://lkml.kernel.org/r/1390347370.3138.65.camel@schen9-DESK
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2014-01-28 13:13:28 +01:00
Jason Low
5faeb8adb9
locking/mcs: Micro-optimize the MCS code, add extra comments
...
Remove unnecessary operation to assign locked status to 1 if lock is
acquired without contention. Lock status will not be checked by lock
holder again once it is acquired and any lock
contenders will not be looking at the lock holder's lock status.
Make the cmpxchg(lock, node, NULL) == node check in mcs_spin_unlock()
likely() as it is likely that a race did not occur most of the time.
Also add in more comments describing how the local node is used in MCS locks.
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com >
Signed-off-by: Jason Low <jason.low2@hp.com >
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com >
Signed-off-by: Peter Zijlstra <peterz@infradead.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Link: http://lkml.kernel.org/r/1390347365.3138.64.camel@schen9-DESK
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2014-01-28 13:13:28 +01:00
Tim Chen
e72246748f
locking/mutexes/mcs: Restructure the MCS lock defines and locking code into its own file
...
We will need the MCS lock code for doing optimistic spinning for rwsem
and queued rwlock. Extracting the MCS code from mutex.c and put into
its own file allow us to reuse this code easily.
We also inline mcs_spin_lock and mcs_spin_unlock functions
for better efficiency.
Note that using the smp_load_acquire/smp_store_release pair used in
mcs_lock and mcs_unlock is not sufficient to form a full memory barrier
across cpus for many architectures (except x86). For applications that
absolutely need a full barrier across multiple cpus with mcs_unlock and
mcs_lock pair, smp_mb__after_unlock_lock() should be used after mcs_lock.
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com >
Signed-off-by: Davidlohr Bueso <davidlohr@hp.com >
Signed-off-by: Peter Zijlstra <peterz@infradead.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Link: http://lkml.kernel.org/r/1390347360.3138.63.camel@schen9-DESK
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2014-01-28 13:13:27 +01:00
