Ingo Molnar
e102f30f4e
x86/fpu: Move fpu_copy() to fpu/core.c
...
Move fpu_copy() where its only user is.
Beyond readability this also speeds up compilation, as fpu-internal.h
is included in over a dozen .c files.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:21 +02:00
Ingo Molnar
6522d78377
x86/fpu: Remove __save_init_fpu()
...
__save_init_fpu() is just a trivial wrapper around fpu_save_init().
Remove the extra layer of obfuscation.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:21 +02:00
Ingo Molnar
085cc281a0
x86/fpu: Add kernel_fpu_disabled()
...
Instead of open-coded in_kernel_fpu access, Use kernel_fpu_disabled() in
interrupted_kernel_fpu_idle(), matching the other kernel_fpu_*() methods.
Also add some documentation for in_kernel_fpu.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:21 +02:00
Ingo Molnar
3103ae3a6d
x86/fpu: Add debug check to kernel_fpu_disable()
...
We are not supposed to call kernel_fpu_disable() if we have not
previously enabled it.
Also use kernel_fpu_disable()/enable() in the __kernel_fpu_begin/end()
primitives, instead of writing to in_kernel_fpu directly,
so that we get the debugging checks.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:20 +02:00
Ingo Molnar
416d49ac67
x86/fpu: Make kernel_fpu_disable/enable() static
...
This allows the compiler to inline them and to eliminate them:
arch/x86/kernel/fpu/core.o:
text data bss dec hex filename
6741 4 8 6753 1a61 core.o.before
6716 4 8 6728 1a48 core.o.after
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:20 +02:00
Ingo Molnar
f55f88e25e
x86/fpu: Make task_xstate_cachep static
...
It's now local to fpu/core.c, make it static.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:20 +02:00
Ingo Molnar
5a12bf6332
x86/fpu: Uninline fpstate_free() and move it next to the allocation function
...
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:20 +02:00
Ingo Molnar
a752b53d9d
x86/fpu: Factor out fpu__copy()
...
Introduce fpu__copy() and use it in arch_dup_task_struct(),
thus moving another chunk of FPU logic to fpu/core.c.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:19 +02:00
Ingo Molnar
8ffb53ab98
x86/fpu: Move task_xstate_cachep handling to core.c
...
This code was historically in process.c, now we have FPU core internals in
fpu/core.c instead - move it there.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:19 +02:00
Ingo Molnar
3e261c14e4
x86/fpu: Simplify the xsave_state*() methods
...
These functions (xsave_state() and xsave_state_booting()) have a 'mask'
argument that is always -1.
Propagate this into the functions instead and eliminate the extra argument.
Does not change the generated code, because these were inlined functions.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:18 +02:00
Ingo Molnar
4d1640927b
x86/fpu: Factor out the FPU bug detection code into fpu__init_check_bugs()
...
Move the boot-time FPU bug detection code to the other FPU boot time
init code in fpu/init.c.
No change in code size:
text data bss dec hex filename
13044568 1884440 1130496 16059504 f50c70 vmlinux.before
13044568 1884440 1130496 16059504 f50c70 vmlinux.after
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:18 +02:00
Ingo Molnar
3a0aee4801
x86/fpu: Rename math_state_restore() to fpu__restore()
...
Move to the new fpu__*() namespace.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:18 +02:00
Ingo Molnar
93b90712c6
x86/fpu: Move math_state_restore() to fpu/core.c
...
It's another piece of FPU internals that is better off close to
the other FPU internals.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:17 +02:00
Ingo Molnar
81683cc827
x86/fpu: Factor out fpu__flush_thread() from flush_thread()
...
flush_thread() open codes a lot of FPU internals - create a separate
function for it in fpu/core.c.
Turns out that this does not hurt performance:
text data bss dec hex filename
11843039 1884440 1130496 14857975 e2b6f7 vmlinux.before
11843039 1884440 1130496 14857975 e2b6f7 vmlinux.after
and since this is a slowpath clarity comes first anyway.
We can reconsider inlining decisions after the FPU code has been cleaned up.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:17 +02:00
Ingo Molnar
146ed598d1
x86/fpu: Move the no_387 handling and FPU detection code into init.c
...
Both no_387() and fpu__detect() run at boot time, so they belong
into init.c.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:17 +02:00
Ingo Molnar
4445e6e9a5
x86/fpu: Remove unnecessary includes from core.c
...
fpu/core.c includes a lot of files for mostly historic reasons.
It only needs fpu-internal.h, which already includes all
the required headers.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:16 +02:00
Ingo Molnar
0c86753790
x86/fpu: Split out the boot time FPU init code into fpu/init.c
...
Move boot time FPU initialization code into init.c, to better
isolate it into its own domain.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:16 +02:00
Ingo Molnar
ce4c4c2624
x86/fpu: Move i387.c and xsave.c to arch/x86/kernel/fpu/
...
Create a new subdirectory for the FPU support code in arch/x86/kernel/fpu/.
Rename 'i387.c' to 'core.c' - as this really collects the core FPU support
code, nothing i387 specific.
We'll better organize this directory in later patches.
Reviewed-by: Borislav Petkov <bp@alien8.de >
Cc: Andy Lutomirski <luto@amacapital.net >
Cc: Dave Hansen <dave.hansen@linux.intel.com >
Cc: Fenghua Yu <fenghua.yu@intel.com >
Cc: H. Peter Anvin <hpa@zytor.com >
Cc: Linus Torvalds <torvalds@linux-foundation.org >
Cc: Oleg Nesterov <oleg@redhat.com >
Cc: Peter Zijlstra <peterz@infradead.org >
Cc: Thomas Gleixner <tglx@linutronix.de >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
2015-05-19 15:47:15 +02:00