xiaomi-sm8450/android_kernel_xiaomi_sm8450
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ANDROID: fix mmu_notifier race caused by not taking mmap_lock during SPF

Browse Source 
When pagefaults are handled speculatively,the pair of
mmu_notifier_invalidate_range_start/mmu_notifier_invalidate_range_end
calls happen without mmap_lock being taken. This enables the following
race:

mmu_notifier_invalidate_range_start
                                       mmap_write_lock
                                       mmu_notifier_register
                                       mmap_write_unlock
mmu_notifier_invalidate_range_end

In this case mmu_notifier_invalidate_range_end will see a new
subscriber not seen at the time of mmu_notifier_invalidate_range_start
and will call ops->invalidate_range_end for that subscriber without
the matching ops->invalidate_range_start, creating imbalance.
Fix this by introducing a new mm->mmu_notifier_lock percpu_rw_semaphore
to synchronize mmu_notifier_invalidate_range_start/
mmu_notifier_invalidate_range_end with mmu_notifier_register when
handling pagefaults speculatively without holding mmap_lock.
percpu_rw_semaphore is used instead of rw_semaphore to prevent cache
line bouncing in the pagefault path.

Fixes: 86ee4a531e ("FROMLIST: x86/mm: add speculative pagefault handling")

Bug: 161210518
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Change-Id: I9c363b2348efcad19818f93b010abf956870ab55
This commit is contained in:
Suren Baghdasaryan
2021-11-04 13:42:56 -07:00
parent 2fc2c66b9c
commit 6971350406
5 changed files with 108 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
kernel/fork.c
View File

@@ -1072,7 +1072,8 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p,
mm_init_owner(mm, p);
mm_init_pasid(mm);
RCU_INIT_POINTER(mm->exe_file, NULL);
mmu_notifier_subscriptions_init(mm);
if (!mmu_notifier_subscriptions_init(mm))
goto fail_nopgd;
init_tlb_flush_pending(mm);
#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
mm->pmd_huge_pte = NULL;