xiaomi-sm8450/android_kernel_xiaomi_sm8450
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'hmm-devmem-cleanup.4' into rdma.git hmm

Browse Source 
Christoph Hellwig says:

====================
Below is a series that cleans up the dev_pagemap interface so that it is
more easily usable, which removes the need to wrap it in hmm and thus
allowing to kill a lot of code

Changes since v3:
 - pull in "mm/swap: Fix release_pages() when releasing devmap pages" and
   rebase the other patches on top of that
 - fold the hmm_devmem_add_resource into the DEVICE_PUBLIC memory removal
   patch
 - remove _vm_normal_page as it isn't needed without DEVICE_PUBLIC memory
 - pick up various ACKs

Changes since v2:
 - fix nvdimm kunit build
 - add a new memory type for device dax
 - fix a few issues in intermediate patches that didn't show up in the end
   result
 - incorporate feedback from Michal Hocko, including killing of
   the DEVICE_PUBLIC memory type entirely

Changes since v1:
 - rebase
 - also switch p2pdma to the internal refcount
 - add type checking for pgmap->type
 - rename the migrate method to migrate_to_ram
 - cleanup the altmap_valid flag
 - various tidbits from the reviews
====================

Conflicts resolved by:
 - Keeping Ira's version of the code in swap.c
 - Using the delete for the section in hmm.rst
 - Using the delete for the devmap code in hmm.c and .h

* branch 'hmm-devmem-cleanup.4': (24 commits)
  mm: don't select MIGRATE_VMA_HELPER from HMM_MIRROR
  mm: remove the HMM config option
  mm: sort out the DEVICE_PRIVATE Kconfig mess
  mm: simplify ZONE_DEVICE page private data
  mm: remove hmm_devmem_add
  mm: remove hmm_vma_alloc_locked_page
  nouveau: use devm_memremap_pages directly
  nouveau: use alloc_page_vma directly
  PCI/P2PDMA: use the dev_pagemap internal refcount
  device-dax: use the dev_pagemap internal refcount
  memremap: provide an optional internal refcount in struct dev_pagemap
  memremap: replace the altmap_valid field with a PGMAP_ALTMAP_VALID flag
  memremap: remove the data field in struct dev_pagemap
  memremap: add a migrate_to_ram method to struct dev_pagemap_ops
  memremap: lift the devmap_enable manipulation into devm_memremap_pages
  memremap: pass a struct dev_pagemap to ->kill and ->cleanup
  memremap: move dev_pagemap callbacks into a separate structure
  memremap: validate the pagemap type passed to devm_memremap_pages
  mm: factor out a devm_request_free_mem_region helper
  mm: export alloc_pages_vma
  ...

Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
This commit is contained in:
Jason Gunthorpe
2019-07-02 15:07:52 -03:00
parent 9ec3f4cb35 b6b346a066
commit cc5dfd59e3
33 changed files with 370 additions and 1013 deletions
Download Patch File Download Diff File Expand all files Collapse all files

198
include/linux/hmm.h
View File

@@ -62,7 +62,7 @@
#include <linux/kconfig.h>
#include <asm/pgtable.h>
#if IS_ENABLED(CONFIG_HMM)
#ifdef CONFIG_HMM_MIRROR
#include <linux/device.h>
#include <linux/migrate.h>
@@ -324,9 +324,6 @@ static inline uint64_t hmm_pfn_from_pfn(const struct hmm_range *range,
return hmm_device_entry_from_pfn(range, pfn);
}
#if IS_ENABLED(CONFIG_HMM_MIRROR)
/*
* Mirroring: how to synchronize device page table with CPU page table.
*
@@ -550,197 +547,4 @@ static inline void hmm_mm_init(struct mm_struct *mm)
static inline void hmm_mm_init(struct mm_struct *mm) {}
#endif /* IS_ENABLED(CONFIG_HMM_MIRROR) */
#if IS_ENABLED(CONFIG_DEVICE_PRIVATE) || IS_ENABLED(CONFIG_DEVICE_PUBLIC)
struct hmm_devmem;
struct page *hmm_vma_alloc_locked_page(struct vm_area_struct *vma,
unsigned long addr);
/*
* struct hmm_devmem_ops - callback for ZONE_DEVICE memory events
*
* @free: call when refcount on page reach 1 and thus is no longer use
* @fault: call when there is a page fault to unaddressable memory
*
* Both callback happens from page_free() and page_fault() callback of struct
* dev_pagemap respectively. See include/linux/memremap.h for more details on
* those.
*
* The hmm_devmem_ops callback are just here to provide a coherent and
* uniq API to device driver and device driver should not register their
* own page_free() or page_fault() but rely on the hmm_devmem_ops call-
* back.
*/
struct hmm_devmem_ops {
/*
* free() - free a device page
* @devmem: device memory structure (see struct hmm_devmem)
* @page: pointer to struct page being freed
*
* Call back occurs whenever a device page refcount reach 1 which
* means that no one is holding any reference on the page anymore
* (ZONE_DEVICE page have an elevated refcount of 1 as default so
* that they are not release to the general page allocator).
*
* Note that callback has exclusive ownership of the page (as no
* one is holding any reference).
*/
void (*free)(struct hmm_devmem *devmem, struct page *page);
/*
* fault() - CPU page fault or get user page (GUP)
* @devmem: device memory structure (see struct hmm_devmem)
* @vma: virtual memory area containing the virtual address
* @addr: virtual address that faulted or for which there is a GUP
* @page: pointer to struct page backing virtual address (unreliable)
* @flags: FAULT_FLAG_* (see include/linux/mm.h)
* @pmdp: page middle directory
* Return: VM_FAULT_MINOR/MAJOR on success or one of VM_FAULT_ERROR
* on error
*
* The callback occurs whenever there is a CPU page fault or GUP on a
* virtual address. This means that the device driver must migrate the
* page back to regular memory (CPU accessible).
*
* The device driver is free to migrate more than one page from the
* fault() callback as an optimization. However if the device decides
* to migrate more than one page it must always priotirize the faulting
* address over the others.
*
* The struct page pointer is only given as a hint to allow quick
* lookup of internal device driver data. A concurrent migration
* might have already freed that page and the virtual address might
* no longer be backed by it. So it should not be modified by the
* callback.
*
* Note that mmap semaphore is held in read mode at least when this
* callback occurs, hence the vma is valid upon callback entry.
*/
vm_fault_t (*fault)(struct hmm_devmem *devmem,
struct vm_area_struct *vma,
unsigned long addr,
const struct page *page,
unsigned int flags,
pmd_t *pmdp);
};
/*
* struct hmm_devmem - track device memory
*
* @completion: completion object for device memory
* @pfn_first: first pfn for this resource (set by hmm_devmem_add())
* @pfn_last: last pfn for this resource (set by hmm_devmem_add())
* @resource: IO resource reserved for this chunk of memory
* @pagemap: device page map for that chunk
* @device: device to bind resource to
* @ops: memory operations callback
* @ref: per CPU refcount
* @page_fault: callback when CPU fault on an unaddressable device page
*
* This is a helper structure for device drivers that do not wish to implement
* the gory details related to hotplugging new memoy and allocating struct
* pages.
*
* Device drivers can directly use ZONE_DEVICE memory on their own if they
* wish to do so.
*
* The page_fault() callback must migrate page back, from device memory to
* system memory, so that the CPU can access it. This might fail for various
* reasons (device issues, device have been unplugged, ...). When such error
* conditions happen, the page_fault() callback must return VM_FAULT_SIGBUS and
* set the CPU page table entry to "poisoned".
*
* Note that because memory cgroup charges are transferred to the device memory,
* this should never fail due to memory restrictions. However, allocation
* of a regular system page might still fail because we are out of memory. If
* that happens, the page_fault() callback must return VM_FAULT_OOM.
*
* The page_fault() callback can also try to migrate back multiple pages in one
* chunk, as an optimization. It must, however, prioritize the faulting address
* over all the others.
*/
typedef vm_fault_t (*dev_page_fault_t)(struct vm_area_struct *vma,
unsigned long addr,
const struct page *page,
unsigned int flags,
pmd_t *pmdp);
struct hmm_devmem {
struct completion completion;
unsigned long pfn_first;
unsigned long pfn_last;
struct resource *resource;
struct device *device;
struct dev_pagemap pagemap;
const struct hmm_devmem_ops *ops;
struct percpu_ref ref;
dev_page_fault_t page_fault;
};
/*
* To add (hotplug) device memory, HMM assumes that there is no real resource
* that reserves a range in the physical address space (this is intended to be
* use by unaddressable device memory). It will reserve a physical range big
* enough and allocate struct page for it.
*
* The device driver can wrap the hmm_devmem struct inside a private device
* driver struct.
*/
struct hmm_devmem *hmm_devmem_add(const struct hmm_devmem_ops *ops,
struct device *device,
unsigned long size);
struct hmm_devmem *hmm_devmem_add_resource(const struct hmm_devmem_ops *ops,
struct device *device,
struct resource *res);
/*
* hmm_devmem_page_set_drvdata - set per-page driver data field
*
* @page: pointer to struct page
* @data: driver data value to set
*
* Because page can not be on lru we have an unsigned long that driver can use
* to store a per page field. This just a simple helper to do that.
*/
static inline void hmm_devmem_page_set_drvdata(struct page *page,
unsigned long data)
{
page->hmm_data = data;
}
/*
* hmm_devmem_page_get_drvdata - get per page driver data field
*
* @page: pointer to struct page
* Return: driver data value
*/
static inline unsigned long hmm_devmem_page_get_drvdata(const struct page *page)
{
return page->hmm_data;
}
/*
* struct hmm_device - fake device to hang device memory onto
*
* @device: device struct
* @minor: device minor number
*/
struct hmm_device {
struct device device;
unsigned int minor;
};
/*
* A device driver that wants to handle multiple devices memory through a
* single fake device can use hmm_device to do so. This is purely a helper and
* it is not strictly needed, in order to make use of any HMM functionality.
*/
struct hmm_device *hmm_device_new(void *drvdata);
void hmm_device_put(struct hmm_device *hmm_device);
#endif /* CONFIG_DEVICE_PRIVATE || CONFIG_DEVICE_PUBLIC */
#else /* IS_ENABLED(CONFIG_HMM) */
static inline void hmm_mm_destroy(struct mm_struct *mm) {}
static inline void hmm_mm_init(struct mm_struct *mm) {}
#endif /* IS_ENABLED(CONFIG_HMM) */
#endif /* LINUX_HMM_H */