Peter Zijlstra
cef4402d76
x86/paravirt: Mark unused patch_default label
...
A bugfix commit:
45dbea5f55arnd@arndb.de >
Reported-by: Piotr Gregor <piotrgregor@rsyncme.org >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andy Lutomirski <luto@kernel.org >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Brian Gerst <brgerst@gmail.com >
Cc: Denys Vlasenko <dvlasenk@redhat.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Josh Poimboeuf <jpoimboe@redhat.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Fixes: 45dbea5f55mingo@kernel.org >
2016-12-22 17:43:35 +01:00
Linus Torvalds
518bacf5a5
Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
...
Pull x86 FPU updates from Ingo Molnar:
"The main changes in this cycle were:
- do a large round of simplifications after all CPUs do 'eager' FPU
context switching in v4.9: remove CR0 twiddling, remove leftover
eager/lazy bts, etc (Andy Lutomirski)
- more FPU code simplifications: remove struct fpu::counter, clarify
nomenclature, remove unnecessary arguments/functions and better
structure the code (Rik van Riel)"
* 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/fpu: Remove clts()
x86/fpu: Remove stts()
x86/fpu: Handle #NM without FPU emulation as an error
x86/fpu, lguest: Remove CR0.TS support
x86/fpu, kvm: Remove host CR0.TS manipulation
x86/fpu: Remove irq_ts_save() and irq_ts_restore()
x86/fpu: Stop saving and restoring CR0.TS in fpu__init_check_bugs()
x86/fpu: Get rid of two redundant clts() calls
x86/fpu: Finish excising 'eagerfpu'
x86/fpu: Split old_fpu & new_fpu handling into separate functions
x86/fpu: Remove 'cpu' argument from __cpu_invalidate_fpregs_state()
x86/fpu: Split old & new FPU code paths
x86/fpu: Remove __fpregs_(de)activate()
x86/fpu: Rename lazy restore functions to "register state valid"
x86/fpu, kvm: Remove KVM vcpu->fpu_counter
x86/fpu: Remove struct fpu::counter
x86/fpu: Remove use_eager_fpu()
x86/fpu: Remove the XFEATURE_MASK_EAGER/LAZY distinction
x86/fpu: Hard-disable lazy FPU mode
x86/crypto, x86/fpu: Remove X86_FEATURE_EAGER_FPU #ifdef from the crc32c code
2016-12-12 14:27:49 -08:00
Peter Zijlstra
45dbea5f55
x86/paravirt: Fix native_patch()
...
While chasing a regression I noticed we potentially patch the wrong
code in native_patch().
If we do not select the native code sequence, we must use the default
patcher, not fall-through the switch case.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Alok Kataria <akataria@vmware.com >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Chris Wright <chrisw@sous-sol.org >
Cc: Jeremy Fitzhardinge <jeremy@goop.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Pan Xinhui <xinhui.pan@linux.vnet.ibm.com >
Cc: Paolo Bonzini <pbonzini@redhat.com >
Cc: Peter Anvin <hpa@zytor.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Rusty Russell <rusty@rustcorp.com.au >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: kernel test robot <xiaolong.ye@intel.com >
Fixes: 3cded41794http://lkml.kernel.org/r/20161208154349.270616999@infradead.org
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-12-11 13:09:19 +01:00
Peter Zijlstra
3cded41794
x86/paravirt: Optimize native pv_lock_ops.vcpu_is_preempted()
...
Avoid the pointless function call to pv_lock_ops.vcpu_is_preempted()
when a paravirt spinlock enabled kernel is ran on native hardware.
Do this by patching out the CALL instruction with "XOR %RAX,%RAX"
which has the same effect (0 return value).
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: David.Laight@ACULAB.COM
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Pan Xinhui <xinhui.pan@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: benh@kernel.crashing.org
Cc: boqun.feng@gmail.com
Cc: borntraeger@de.ibm.com
Cc: bsingharora@gmail.com
Cc: dave@stgolabs.net
Cc: jgross@suse.com
Cc: kernellwp@gmail.com
Cc: konrad.wilk@oracle.com
Cc: mpe@ellerman.id.au
Cc: paulmck@linux.vnet.ibm.com
Cc: paulus@samba.org
Cc: pbonzini@redhat.com
Cc: rkrcmar@redhat.com
Cc: will.deacon@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-11-22 12:48:11 +01:00
Andy Lutomirski
af25ed59b5
x86/fpu: Remove clts()
...
The kernel doesn't use clts() any more. Remove it and all of its
paravirt infrastructure.
A careful reader may notice that xen_clts() appears to have been
buggy -- it didn't update xen_cr0_value.
Signed-off-by: Andy Lutomirski <luto@kernel.org >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Brian Gerst <brgerst@gmail.com >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Denys Vlasenko <dvlasenk@redhat.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Josh Poimboeuf <jpoimboe@redhat.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Paolo Bonzini <pbonzini@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Rusty Russell <rusty@rustcorp.com.au >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: kvm list <kvm@vger.kernel.org >
Link: http://lkml.kernel.org/r/3d3c8ca62f17579b9849a013d71e59a4d5d1b079.1477951965.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-11-01 07:47:55 +01:00
Peter Zijlstra
cfd8983f03
x86, locking/spinlocks: Remove ticket (spin)lock implementation
...
We've unconditionally used the queued spinlock for many releases now.
Its time to remove the old ticket lock code.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: Waiman Long <waiman.long@hpe.com >
Cc: Waiman.Long@hpe.com
Cc: david.vrabel@citrix.com
Cc: dhowells@redhat.com
Cc: pbonzini@redhat.com
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/20160518184302.GO3193@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2016-09-30 10:56:00 +02:00
Boris Ostrovsky
88c15ec90f
x86/paravirt: Remove the unused irq_enable_sysexit pv op
...
As result of commit "x86/xen: Avoid fast syscall path for Xen PV
guests", the irq_enable_sysexit pv op is not called by Xen PV guests
anymore and since they were the only ones who used it we can
safely remove it.
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com >
Reviewed-by: Borislav Petkov <bp@suse.de >
Acked-by: Andy Lutomirski <luto@kernel.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Brian Gerst <brgerst@gmail.com >
Cc: Denys Vlasenko <dvlasenk@redhat.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: david.vrabel@citrix.com
Cc: konrad.wilk@oracle.com
Cc: virtualization@lists.linux-foundation.org
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/1447970147-1733-3-git-send-email-boris.ostrovsky@oracle.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-11-23 10:48:16 +01:00
Andy Lutomirski
9261e050b6
x86/asm/tsc, x86/paravirt: Remove read_tsc() and read_tscp() paravirt hooks
...
We've had ->read_tsc() and ->read_tscp() paravirt hooks since
the very beginning of paravirt, i.e.,
d3561b7fa010936321093632luto@kernel.org >
Signed-off-by: Borislav Petkov <bp@suse.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Brian Gerst <brgerst@gmail.com >
Cc: Denys Vlasenko <dvlasenk@redhat.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Huang Rui <ray.huang@amd.com >
Cc: John Stultz <john.stultz@linaro.org >
Cc: Len Brown <lenb@kernel.org >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Ralf Baechle <ralf@linux-mips.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: kvm ML <kvm@vger.kernel.org >
Cc: virtualization@lists.linux-foundation.org
Link: http://lkml.kernel.org/r/d08a2600fb298af163681e5efd8e599d889a5b97.1434501121.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-07-06 15:23:26 +02:00
Peter Zijlstra (Intel)
f233f7f158
locking/pvqspinlock, x86: Implement the paravirt qspinlock call patching
...
We use the regular paravirt call patching to switch between:
native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath()
native_queued_spin_unlock() __pv_queued_spin_unlock()
We use a callee saved call for the unlock function which reduces the
i-cache footprint and allows 'inlining' of SPIN_UNLOCK functions
again.
We further optimize the unlock path by patching the direct call with a
"movb $0,%arg1" if we are indeed using the native unlock code. This
makes the unlock code almost as fast as the !PARAVIRT case.
This significantly lowers the overhead of having
CONFIG_PARAVIRT_SPINLOCKS enabled, even for native code.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Signed-off-by: Waiman Long <Waiman.Long@hp.com >
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org >
Cc: Andrew Morton <akpm@linux-foundation.org >
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com >
Cc: Borislav Petkov <bp@alien8.de >
Cc: Daniel J Blueman <daniel@numascale.com >
Cc: David Vrabel <david.vrabel@citrix.com >
Cc: Douglas Hatch <doug.hatch@hp.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Paolo Bonzini <paolo.bonzini@gmail.com >
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Scott J Norton <scott.norton@hp.com >
Cc: Thomas Gleixner <tglx@linutronix.de >
Cc: virtualization@lists.linux-foundation.org
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/1429901803-29771-10-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-08 12:37:09 +02:00
Jeremy Fitzhardinge
41edafdb78
x86/pvops: add a paravirt_ident functions to allow special patching
...
Impact: Optimization
Several paravirt ops implementations simply return their arguments,
the most obvious being the make_pte/pte_val class of operations on
native.
On 32-bit, the identity function is literally a no-op, as the calling
convention uses the same registers for the first argument and return.
On 64-bit, it can be implemented with a single "mov".
This patch adds special identity functions for 32 and 64 bit argument,
and machinery to recognize them and replace them with either nops or a
mov as appropriate.
At the moment, the only users for the identity functions are the
pagetable entry conversion functions.
The result is a measureable improvement on pagetable-heavy benchmarks
(2-3%, reducing the pvops overhead from 5 to 2%).
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com >
Signed-off-by: H. Peter Anvin <hpa@zytor.com >
2009-01-30 14:51:44 -08:00
Paolo Ciarrocchi
d9336a9b47
x86: coding style fixes to arch/x86/kernel/paravirt_patch_32.c
...
Before:
total: 3 errors, 1 warnings, 49 lines checked
After:
total: 2 errors, 1 warnings, 49 lines checked
paolo@paolo-desktop:~/linux.trees.git$ md5sum /tmp/paravirt_patch_32.o.*
a78eea4264723e18c49dcfbe0ee0aae7 /tmp/paravirt_patch_32.o.after
a78eea4264723e18c49dcfbe0ee0aae7 /tmp/paravirt_patch_32.o.before
Signed-off-by: Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com >
Signed-off-by: Ingo Molnar <mingo@elte.hu >
2008-08-15 16:53:25 +02:00
Jeremy Fitzhardinge
d75cd22fdd
x86/paravirt: split sysret and sysexit
...
Don't conflate sysret and sysexit; they're different instructions with
different semantics, and may be in use at the same time (at least
within the same kernel, depending on whether its an Intel or AMD
system).
sysexit - just return to userspace, does no register restoration of
any kind; must explicitly atomically enable interrupts.
sysret - reloads flags from r11, so no need to explicitly enable
interrupts on 64-bit, responsible for restoring usermode %gs
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citirx.com >
Cc: xen-devel <xen-devel@lists.xensource.com >
Cc: Stephen Tweedie <sct@redhat.com >
Cc: Eduardo Habkost <ehabkost@redhat.com >
Cc: Mark McLoughlin <markmc@redhat.com >
Signed-off-by: Ingo Molnar <mingo@elte.hu >
2008-07-08 13:13:15 +02:00
Glauber de Oliveira Costa
2f485ef568
x86: move patching code to arch-specific file.
...
The core patching code for paravirt is sufficiently different
among i386 and x86_64, and we move them to specific files.
Signed-off-by: Glauber de Oliveira Costa <gcosta@redhat.com >
Signed-off-by: Ingo Molnar <mingo@elte.hu >
Signed-off-by: Thomas Gleixner <tglx@linutronix.de >
2008-01-30 13:32:10 +01:00
