Suren Baghdasaryan 6971350406 ANDROID: fix mmu_notifier race caused by not taking mmap_lock during SPF ...

When pagefaults are handled speculatively,the pair of
mmu_notifier_invalidate_range_start/mmu_notifier_invalidate_range_end
calls happen without mmap_lock being taken. This enables the following
race:

mmu_notifier_invalidate_range_start
                                       mmap_write_lock
                                       mmu_notifier_register
                                       mmap_write_unlock
mmu_notifier_invalidate_range_end

In this case mmu_notifier_invalidate_range_end will see a new
subscriber not seen at the time of mmu_notifier_invalidate_range_start
and will call ops->invalidate_range_end for that subscriber without
the matching ops->invalidate_range_start, creating imbalance.
Fix this by introducing a new mm->mmu_notifier_lock percpu_rw_semaphore
to synchronize mmu_notifier_invalidate_range_start/
mmu_notifier_invalidate_range_end with mmu_notifier_register when
handling pagefaults speculatively without holding mmap_lock.
percpu_rw_semaphore is used instead of rw_semaphore to prevent cache
line bouncing in the pagefault path.

Fixes: 86ee4a531e ("FROMLIST: x86/mm: add speculative pagefault handling")

Bug: 161210518
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Change-Id: I9c363b2348efcad19818f93b010abf956870ab55

2021-12-15 21:44:56 +00:00

76 KiB

Raw Blame History

View Raw