x86/mm: Split read_cr3() into read_cr3_pa() and __read_cr3()

The kernel has several code paths that read CR3.  Most of them assume that
CR3 contains the PGD's physical address, whereas some of them awkwardly
use PHYSICAL_PAGE_MASK to mask off low bits.

Add explicit mask macros for CR3 and convert all of the CR3 readers.
This will keep them from breaking when PCID is enabled.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: xen-devel <xen-devel@lists.xen.org>
Link: http://lkml.kernel.org/r/883f8fb121f4616c1c1427ad87350bb2f5ffeca1.1497288170.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
This commit is contained in:
Andy Lutomirski
2017-06-12 10:26:14 -07:00
committed by Ingo Molnar
parent 3f365cf304
commit 6c690ee103
20 changed files with 79 additions and 29 deletions

View File

@@ -156,7 +156,7 @@ static inline void __native_flush_tlb(void)
* back:
*/
preempt_disable();
native_write_cr3(native_read_cr3());
native_write_cr3(__native_read_cr3());
preempt_enable();
}
@@ -264,7 +264,7 @@ static inline void reset_lazy_tlbstate(void)
this_cpu_write(cpu_tlbstate.state, 0);
this_cpu_write(cpu_tlbstate.loaded_mm, &init_mm);
WARN_ON(read_cr3() != __pa_symbol(swapper_pg_dir));
WARN_ON(read_cr3_pa() != __pa_symbol(swapper_pg_dir));
}
static inline void arch_tlbbatch_add_mm(struct arch_tlbflush_unmap_batch *batch,