09bd9e940e98acad0c854f75b31fc79a93cde981
113 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Greg Kroah-Hartman
|
194be71cc6 |
Merge 5.10.47 into android12-5.10-lts
Changes in 5.10.47 module: limit enabling module.sig_enforce Revert "drm/amdgpu/gfx9: fix the doorbell missing when in CGPG issue." Revert "drm/amdgpu/gfx10: enlarge CP_MEC_DOORBELL_RANGE_UPPER to cover full doorbell." drm: add a locked version of drm_is_current_master drm/nouveau: wait for moving fence after pinning v2 drm/radeon: wait for moving fence after pinning drm/amdgpu: wait for moving fence after pinning ARM: 9081/1: fix gcc-10 thumb2-kernel regression mmc: meson-gx: use memcpy_to/fromio for dram-access-quirk MIPS: generic: Update node names to avoid unit addresses arm64: Ignore any DMA offsets in the max_zone_phys() calculation arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required spi: spi-nxp-fspi: move the register operation after the clock enable Revert "PCI: PM: Do not read power state in pci_enable_device_flags()" drm/vc4: hdmi: Move the HSM clock enable to runtime_pm drm/vc4: hdmi: Make sure the controller is powered in detect x86/entry: Fix noinstr fail in __do_fast_syscall_32() x86/xen: Fix noinstr fail in exc_xen_unknown_trap() locking/lockdep: Improve noinstr vs errors perf/x86/lbr: Remove cpuc->lbr_xsave allocation from atomic context perf/x86/intel/lbr: Zero the xstate buffer on allocation dmaengine: zynqmp_dma: Fix PM reference leak in zynqmp_dma_alloc_chan_resourc() dmaengine: stm32-mdma: fix PM reference leak in stm32_mdma_alloc_chan_resourc() dmaengine: xilinx: dpdma: Add missing dependencies to Kconfig dmaengine: xilinx: dpdma: Limit descriptor IDs to 16 bits mac80211: remove warning in ieee80211_get_sband() mac80211_hwsim: drop pending frames on stop cfg80211: call cfg80211_leave_ocb when switching away from OCB dmaengine: rcar-dmac: Fix PM reference leak in rcar_dmac_probe() dmaengine: mediatek: free the proper desc in desc_free handler dmaengine: mediatek: do not issue a new desc if one is still current dmaengine: mediatek: use GFP_NOWAIT instead of GFP_ATOMIC in prep_dma net: ipv4: Remove unneed BUG() function mac80211: drop multicast fragments net: ethtool: clear heap allocations for ethtool function inet: annotate data race in inet_send_prepare() and inet_dgram_connect() ping: Check return value of function 'ping_queue_rcv_skb' net: annotate data race in sock_error() inet: annotate date races around sk->sk_txhash net/packet: annotate data race in packet_sendmsg() net: phy: dp83867: perform soft reset and retain established link riscv32: Use medany C model for modules net: caif: fix memory leak in ldisc_open net/packet: annotate accesses to po->bind net/packet: annotate accesses to po->ifindex r8152: Avoid memcpy() over-reading of ETH_SS_STATS sh_eth: Avoid memcpy() over-reading of ETH_SS_STATS r8169: Avoid memcpy() over-reading of ETH_SS_STATS KVM: selftests: Fix kvm_check_cap() assertion net: qed: Fix memcpy() overflow of qed_dcbx_params() mac80211: reset profile_periodicity/ema_ap mac80211: handle various extensible elements correctly recordmcount: Correct st_shndx handling PCI: Add AMD RS690 quirk to enable 64-bit DMA net: ll_temac: Add memory-barriers for TX BD access net: ll_temac: Avoid ndo_start_xmit returning NETDEV_TX_BUSY perf/x86: Track pmu in per-CPU cpu_hw_events pinctrl: stm32: fix the reported number of GPIO lines per bank i2c: i801: Ensure that SMBHSTSTS_INUSE_STS is cleared when leaving i801_access gpiolib: cdev: zero padding during conversion to gpioline_info_changed scsi: sd: Call sd_revalidate_disk() for ioctl(BLKRRPART) nilfs2: fix memory leak in nilfs_sysfs_delete_device_group s390/stack: fix possible register corruption with stack switch helper KVM: do not allow mapping valid but non-reference-counted pages i2c: robotfuzz-osif: fix control-request directions ceph: must hold snap_rwsem when filling inode for async create kthread_worker: split code for canceling the delayed work timer kthread: prevent deadlock when kthread_mod_delayed_work() races with kthread_cancel_delayed_work_sync() x86/fpu: Preserve supervisor states in sanitize_restored_user_xstate() x86/fpu: Make init_fpstate correct with optimized XSAVE mm: add VM_WARN_ON_ONCE_PAGE() macro mm/rmap: remove unneeded semicolon in page_not_mapped() mm/rmap: use page_not_mapped in try_to_unmap() mm, thp: use head page in __migration_entry_wait() mm/thp: fix __split_huge_pmd_locked() on shmem migration entry mm/thp: make is_huge_zero_pmd() safe and quicker mm/thp: try_to_unmap() use TTU_SYNC for safe splitting mm/thp: fix vma_address() if virtual address below file offset mm/thp: fix page_address_in_vma() on file THP tails mm/thp: unmap_mapping_page() to fix THP truncate_cleanup_page() mm: thp: replace DEBUG_VM BUG with VM_WARN when unmap fails for split mm: page_vma_mapped_walk(): use page for pvmw->page mm: page_vma_mapped_walk(): settle PageHuge on entry mm: page_vma_mapped_walk(): use pmde for *pvmw->pmd mm: page_vma_mapped_walk(): prettify PVMW_MIGRATION block mm: page_vma_mapped_walk(): crossing page table boundary mm: page_vma_mapped_walk(): add a level of indentation mm: page_vma_mapped_walk(): use goto instead of while (1) mm: page_vma_mapped_walk(): get vma_address_end() earlier mm/thp: fix page_vma_mapped_walk() if THP mapped by ptes mm/thp: another PVMW_SYNC fix in page_vma_mapped_walk() mm, futex: fix shared futex pgoff on shmem huge page KVM: SVM: Call SEV Guest Decommission if ASID binding fails swiotlb: manipulate orig_addr when tlb_addr has offset netfs: fix test for whether we can skip read when writing beyond EOF Revert "drm: add a locked version of drm_is_current_master" certs: Add EFI_CERT_X509_GUID support for dbx entries certs: Move load_system_certificate_list to a common function certs: Add ability to preload revocation certs integrity: Load mokx variables into the blacklist keyring Linux 5.10.47 Signed-off-by: Greg Kroah-Hartman <gregkh@google.com> Change-Id: I68f731ad78a5db003c41093e4faf59f6f9f2e446 |
||
|
Hugh Dickins
|
66be14a926 |
mm/thp: try_to_unmap() use TTU_SYNC for safe splitting
commit 732ed55823fc3ad998d43b86bf771887bcc5ec67 upstream.
Stressing huge tmpfs often crashed on unmap_page()'s VM_BUG_ON_PAGE
(!unmap_success): with dump_page() showing mapcount:1, but then its raw
struct page output showing _mapcount ffffffff i.e. mapcount 0.
And even if that particular VM_BUG_ON_PAGE(!unmap_success) is removed,
it is immediately followed by a VM_BUG_ON_PAGE(compound_mapcount(head)),
and further down an IS_ENABLED(CONFIG_DEBUG_VM) total_mapcount BUG():
all indicative of some mapcount difficulty in development here perhaps.
But the !CONFIG_DEBUG_VM path handles the failures correctly and
silently.
I believe the problem is that once a racing unmap has cleared pte or
pmd, try_to_unmap_one() may skip taking the page table lock, and emerge
from try_to_unmap() before the racing task has reached decrementing
mapcount.
Instead of abandoning the unsafe VM_BUG_ON_PAGE(), and the ones that
follow, use PVMW_SYNC in try_to_unmap_one() in this case: adding
TTU_SYNC to the options, and passing that from unmap_page().
When CONFIG_DEBUG_VM, or for non-debug too? Consensus is to do the same
for both: the slight overhead added should rarely matter, except perhaps
if splitting sparsely-populated multiply-mapped shmem. Once confident
that bugs are fixed, TTU_SYNC here can be removed, and the race
tolerated.
Link: https://lkml.kernel.org/r/c1e95853-8bcd-d8fd-55fa-e7f2488e78f@google.com
Fixes:
|
||
|
Greg Kroah-Hartman
|
d8c7f0a3cd |
Merge 5.10.20 into android12-5.10
Changes in 5.10.20 vmlinux.lds.h: add DWARF v5 sections vdpa/mlx5: fix param validation in mlx5_vdpa_get_config() debugfs: be more robust at handling improper input in debugfs_lookup() debugfs: do not attempt to create a new file before the filesystem is initalized scsi: libsas: docs: Remove notify_ha_event() scsi: qla2xxx: Fix mailbox Ch erroneous error kdb: Make memory allocations more robust w1: w1_therm: Fix conversion result for negative temperatures PCI: qcom: Use PHY_REFCLK_USE_PAD only for ipq8064 PCI: Decline to resize resources if boot config must be preserved virt: vbox: Do not use wait_event_interruptible when called from kernel context bfq: Avoid false bfq queue merging ALSA: usb-audio: Fix PCM buffer allocation in non-vmalloc mode MIPS: vmlinux.lds.S: add missing PAGE_ALIGNED_DATA() section vmlinux.lds.h: Define SANTIZER_DISCARDS with CONFIG_GCOV_KERNEL=y random: fix the RNDRESEEDCRNG ioctl ALSA: pcm: Call sync_stop at disconnection ALSA: pcm: Assure sync with the pending stop operation at suspend ALSA: pcm: Don't call sync_stop if it hasn't been stopped drm/i915/gt: One more flush for Baytrail clear residuals ath10k: Fix error handling in case of CE pipe init failure Bluetooth: btqcomsmd: Fix a resource leak in error handling paths in the probe function Bluetooth: hci_uart: Fix a race for write_work scheduling Bluetooth: Fix initializing response id after clearing struct arm64: dts: renesas: beacon kit: Fix choppy Bluetooth Audio arm64: dts: renesas: beacon: Fix audio-1.8V pin enable ARM: dts: exynos: correct PMIC interrupt trigger level on Artik 5 ARM: dts: exynos: correct PMIC interrupt trigger level on Monk ARM: dts: exynos: correct PMIC interrupt trigger level on Rinato ARM: dts: exynos: correct PMIC interrupt trigger level on Spring ARM: dts: exynos: correct PMIC interrupt trigger level on Arndale Octa ARM: dts: exynos: correct PMIC interrupt trigger level on Odroid XU3 family arm64: dts: exynos: correct PMIC interrupt trigger level on TM2 arm64: dts: exynos: correct PMIC interrupt trigger level on Espresso memory: mtk-smi: Fix PM usage counter unbalance in mtk_smi ops Bluetooth: hci_qca: Fix memleak in qca_controller_memdump staging: vchiq: Fix bulk userdata handling staging: vchiq: Fix bulk transfers on 64-bit builds arm64: dts: qcom: msm8916-samsung-a5u: Fix iris compatible net: stmmac: dwmac-meson8b: fix enabling the timing-adjustment clock bpf: Add bpf_patch_call_args prototype to include/linux/bpf.h bpf: Avoid warning when re-casting __bpf_call_base into __bpf_call_base_args firmware: arm_scmi: Fix call site of scmi_notification_exit arm64: dts: allwinner: A64: properly connect USB PHY to port 0 arm64: dts: allwinner: H6: properly connect USB PHY to port 0 arm64: dts: allwinner: Drop non-removable from SoPine/LTS SD card arm64: dts: allwinner: H6: Allow up to 150 MHz MMC bus frequency arm64: dts: allwinner: A64: Limit MMC2 bus frequency to 150 MHz arm64: dts: qcom: msm8916-samsung-a2015: Fix sensors cpufreq: brcmstb-avs-cpufreq: Free resources in error path cpufreq: brcmstb-avs-cpufreq: Fix resource leaks in ->remove() arm64: dts: rockchip: rk3328: Add clock_in_out property to gmac2phy node ACPICA: Fix exception code class checks usb: gadget: u_audio: Free requests only after callback arm64: dts: qcom: sdm845-db845c: Fix reset-pin of ov8856 node soc: qcom: socinfo: Fix an off by one in qcom_show_pmic_model() soc: ti: pm33xx: Fix some resource leak in the error handling paths of the probe function staging: media: atomisp: Fix size_t format specifier in hmm_alloc() debug statemenet Bluetooth: drop HCI device reference before return Bluetooth: Put HCI device if inquiry procedure interrupts memory: ti-aemif: Drop child node when jumping out loop ARM: dts: Configure missing thermal interrupt for 4430 usb: dwc2: Do not update data length if it is 0 on inbound transfers usb: dwc2: Abort transaction after errors with unknown reason usb: dwc2: Make "trimming xfer length" a debug message staging: rtl8723bs: wifi_regd.c: Fix incorrect number of regulatory rules x86/MSR: Filter MSR writes through X86_IOC_WRMSR_REGS ioctl too arm64: dts: renesas: beacon: Fix EEPROM compatible value can: mcp251xfd: mcp251xfd_probe(): fix errata reference ARM: dts: armada388-helios4: assign pinctrl to LEDs ARM: dts: armada388-helios4: assign pinctrl to each fan arm64: dts: armada-3720-turris-mox: rename u-boot mtd partition to a53-firmware opp: Correct debug message in _opp_add_static_v2() Bluetooth: btusb: Fix memory leak in btusb_mtk_wmt_recv soc: qcom: ocmem: don't return NULL in of_get_ocmem arm64: dts: msm8916: Fix reserved and rfsa nodes unit address arm64: dts: meson: fix broken wifi node for Khadas VIM3L iwlwifi: mvm: set enabled in the PPAG command properly ARM: s3c: fix fiq for clang IAS optee: simplify i2c access staging: wfx: fix possible panic with re-queued frames ARM: at91: use proper asm syntax in pm_suspend ath10k: Fix suspicious RCU usage warning in ath10k_wmi_tlv_parse_peer_stats_info() ath10k: Fix lockdep assertion warning in ath10k_sta_statistics ath11k: fix a locking bug in ath11k_mac_op_start() soc: aspeed: snoop: Add clock control logic iwlwifi: mvm: fix the type we use in the PPAG table validity checks iwlwifi: mvm: store PPAG enabled/disabled flag properly iwlwifi: mvm: send stored PPAG command instead of local iwlwifi: mvm: assign SAR table revision to the command later iwlwifi: mvm: don't check if CSA event is running before removing bpf_lru_list: Read double-checked variable once without lock iwlwifi: pnvm: set the PNVM again if it was already loaded iwlwifi: pnvm: increment the pointer before checking the TLV ath9k: fix data bus crash when setting nf_override via debugfs selftests/bpf: Convert test_xdp_redirect.sh to bash ibmvnic: Set to CLOSED state even on error bnxt_en: reverse order of TX disable and carrier off bnxt_en: Fix devlink info's stored fw.psid version format. xen/netback: fix spurious event detection for common event case dpaa2-eth: fix memory leak in XDP_REDIRECT net: phy: consider that suspend2ram may cut off PHY power net/mlx5e: Don't change interrupt moderation params when DIM is enabled net/mlx5e: Change interrupt moderation channel params also when channels are closed net/mlx5: Fix health error state handling net/mlx5e: Replace synchronize_rcu with synchronize_net net/mlx5e: kTLS, Use refcounts to free kTLS RX priv context net/mlx5: Disable devlink reload for multi port slave device net/mlx5: Disallow RoCE on multi port slave device net/mlx5: Disallow RoCE on lag device net/mlx5: Disable devlink reload for lag devices net/mlx5e: CT: manage the lifetime of the ct entry object net/mlx5e: Check tunnel offload is required before setting SWP mac80211: fix potential overflow when multiplying to u32 integers libbpf: Ignore non function pointer member in struct_ops bpf: Fix an unitialized value in bpf_iter bpf, devmap: Use GFP_KERNEL for xdp bulk queue allocation bpf: Fix bpf_fib_lookup helper MTU check for SKB ctx selftests: mptcp: fix ACKRX debug message tcp: fix SO_RCVLOWAT related hangs under mem pressure net: axienet: Handle deferred probe on clock properly cxgb4/chtls/cxgbit: Keeping the max ofld immediate data size same in cxgb4 and ulds b43: N-PHY: Fix the update of coef for the PHY revision >= 3case bpf: Clear subreg_def for global function return values ibmvnic: add memory barrier to protect long term buffer ibmvnic: skip send_request_unmap for timeout reset net: dsa: felix: perform teardown in reverse order of setup net: dsa: felix: don't deinitialize unused ports net: phy: mscc: adding LCPLL reset to VSC8514 net: amd-xgbe: Reset the PHY rx data path when mailbox command timeout net: amd-xgbe: Fix NETDEV WATCHDOG transmit queue timeout warning net: amd-xgbe: Reset link when the link never comes back net: amd-xgbe: Fix network fluctuations when using 1G BELFUSE SFP net: mvneta: Remove per-cpu queue mapping for Armada 3700 net: enetc: fix destroyed phylink dereference during unbind tty: convert tty_ldisc_ops 'read()' function to take a kernel pointer tty: implement read_iter fbdev: aty: SPARC64 requires FB_ATY_CT drm/gma500: Fix error return code in psb_driver_load() gma500: clean up error handling in init drm/fb-helper: Add missed unlocks in setcmap_legacy() drm/panel: mantix: Tweak init sequence drm/vc4: hdmi: Take into account the clock doubling flag in atomic_check crypto: sun4i-ss - linearize buffers content must be kept crypto: sun4i-ss - fix kmap usage crypto: arm64/aes-ce - really hide slower algos when faster ones are enabled hwrng: ingenic - Fix a resource leak in an error handling path media: allegro: Fix use after free on error kcsan: Rewrite kcsan_prandom_u32_max() without prandom_u32_state() drm: rcar-du: Fix PM reference leak in rcar_cmm_enable() drm: rcar-du: Fix crash when using LVDS1 clock for CRTC drm: rcar-du: Fix the return check of of_parse_phandle and of_find_device_by_node drm/amdgpu: Fix macro name _AMDGPU_TRACE_H_ in preprocessor if condition MIPS: c-r4k: Fix section mismatch for loongson2_sc_init MIPS: lantiq: Explicitly compare LTQ_EBU_PCC_ISTAT against 0 drm/virtio: make sure context is created in gem open drm/fourcc: fix Amlogic format modifier masks media: ipu3-cio2: Build only for x86 media: i2c: ov5670: Fix PIXEL_RATE minimum value media: imx: Unregister csc/scaler only if registered media: imx: Fix csc/scaler unregister media: mtk-vcodec: fix error return code in vdec_vp9_decode() media: camss: missing error code in msm_video_register() media: vsp1: Fix an error handling path in the probe function media: em28xx: Fix use-after-free in em28xx_alloc_urbs media: media/pci: Fix memleak in empress_init media: tm6000: Fix memleak in tm6000_start_stream media: aspeed: fix error return code in aspeed_video_setup_video() ASoC: cs42l56: fix up error handling in probe ASoC: qcom: qdsp6: Move frontend AIFs to q6asm-dai evm: Fix memleak in init_desc crypto: bcm - Rename struct device_private to bcm_device_private sched/fair: Avoid stale CPU util_est value for schedutil in task dequeue drm/sun4i: tcon: fix inverted DCLK polarity media: imx7: csi: Fix regression for parallel cameras on i.MX6UL media: imx7: csi: Fix pad link validation media: ti-vpe: cal: fix write to unallocated memory MIPS: properly stop .eh_frame generation MIPS: Compare __SYNC_loongson3_war against 0 drm/tegra: Fix reference leak when pm_runtime_get_sync() fails drm/amdgpu: toggle on DF Cstate after finishing xgmi injection bsg: free the request before return error code macintosh/adb-iop: Use big-endian autopoll mask drm/amd/display: Fix 10/12 bpc setup in DCE output bit depth reduction. drm/amd/display: Fix HDMI deep color output for DCE 6-11. media: software_node: Fix refcounts in software_node_get_next_child() media: lmedm04: Fix misuse of comma media: vidtv: psi: fix missing crc for PMT media: atomisp: Fix a buffer overflow in debug code media: qm1d1c0042: fix error return code in qm1d1c0042_init() media: cx25821: Fix a bug when reallocating some dma memory media: mtk-vcodec: fix argument used when DEBUG is defined media: pxa_camera: declare variable when DEBUG is defined media: uvcvideo: Accept invalid bFormatIndex and bFrameIndex values sched/eas: Don't update misfit status if the task is pinned f2fs: compress: fix potential deadlock ASoC: qcom: lpass-cpu: Remove bit clock state check ASoC: SOF: Intel: hda: cancel D0i3 work during runtime suspend perf/arm-cmn: Fix PMU instance naming perf/arm-cmn: Move IRQs when migrating context mtd: parser: imagetag: fix error codes in bcm963xx_parse_imagetag_partitions() crypto: talitos - Work around SEC6 ERRATA (AES-CTR mode data size error) crypto: talitos - Fix ctr(aes) on SEC1 drm/nouveau: bail out of nouveau_channel_new if channel init fails mm: proc: Invalidate TLB after clearing soft-dirty page state ata: ahci_brcm: Add back regulators management ASoC: cpcap: fix microphone timeslot mask ASoC: codecs: add missing max_register in regmap config mtd: parsers: afs: Fix freeing the part name memory in failure f2fs: fix to avoid inconsistent quota data drm/amdgpu: Prevent shift wrapping in amdgpu_read_mask() f2fs: fix a wrong condition in __submit_bio ASoC: qcom: Fix typo error in HDMI regmap config callbacks KVM: nSVM: Don't strip host's C-bit from guest's CR3 when reading PDPTRs drm/mediatek: Check if fb is null Drivers: hv: vmbus: Avoid use-after-free in vmbus_onoffer_rescind() ASoC: Intel: sof_sdw: add missing TGL_HDMI quirk for Dell SKU 0A5E ASoC: Intel: sof_sdw: add missing TGL_HDMI quirk for Dell SKU 0A3E locking/lockdep: Avoid unmatched unlock ASoC: qcom: lpass: Fix i2s ctl register bit map ASoC: rt5682: Fix panic in rt5682_jack_detect_handler happening during system shutdown ASoC: SOF: debug: Fix a potential issue on string buffer termination btrfs: clarify error returns values in __load_free_space_cache btrfs: fix double accounting of ordered extent for subpage case in btrfs_invalidapge KVM: x86: Restore all 64 bits of DR6 and DR7 during RSM on x86-64 s390/zcrypt: return EIO when msg retry limit reached drm/vc4: hdmi: Move hdmi reset to bind drm/vc4: hdmi: Fix register offset with longer CEC messages drm/vc4: hdmi: Fix up CEC registers drm/vc4: hdmi: Restore cec physical address on reconnect drm/vc4: hdmi: Compute the CEC clock divider from the clock rate drm/vc4: hdmi: Update the CEC clock divider on HSM rate change drm/lima: fix reference leak in lima_pm_busy drm/dp_mst: Don't cache EDIDs for physical ports hwrng: timeriomem - Fix cooldown period calculation crypto: ecdh_helper - Ensure 'len >= secret.len' in decode_key() io_uring: fix possible deadlock in io_uring_poll nvmet-tcp: fix receive data digest calculation for multiple h2cdata PDUs nvmet-tcp: fix potential race of tcp socket closing accept_work nvme-multipath: set nr_zones for zoned namespaces nvmet: remove extra variable in identify ns nvmet: set status to 0 in case for invalid nsid ASoC: SOF: sof-pci-dev: add missing Up-Extreme quirk ima: Free IMA measurement buffer on error ima: Free IMA measurement buffer after kexec syscall ASoC: simple-card-utils: Fix device module clock fs/jfs: fix potential integer overflow on shift of a int jffs2: fix use after free in jffs2_sum_write_data() ubifs: Fix memleak in ubifs_init_authentication ubifs: replay: Fix high stack usage, again ubifs: Fix error return code in alloc_wbufs() irqchip/imx: IMX_INTMUX should not default to y, unconditionally smp: Process pending softirqs in flush_smp_call_function_from_idle() drm/amdgpu/display: remove hdcp_srm sysfs on device removal capabilities: Don't allow writing ambiguous v3 file capabilities HSI: Fix PM usage counter unbalance in ssi_hw_init power: supply: cpcap: Add missing IRQF_ONESHOT to fix regression clk: meson: clk-pll: fix initializing the old rate (fallback) for a PLL clk: meson: clk-pll: make "ret" a signed integer clk: meson: clk-pll: propagate the error from meson_clk_pll_set_rate() selftests/powerpc: Make the test check in eeh-basic.sh posix compliant regulator: qcom-rpmh-regulator: add pm8009-1 chip revision arm64: dts: qcom: qrb5165-rb5: fix pm8009 regulators quota: Fix memory leak when handling corrupted quota file i2c: iproc: handle only slave interrupts which are enabled i2c: iproc: update slave isr mask (ISR_MASK_SLAVE) i2c: iproc: handle master read request spi: cadence-quadspi: Abort read if dummy cycles required are too many clk: sunxi-ng: h6: Fix CEC clock clk: renesas: r8a779a0: Remove non-existent S2 clock clk: renesas: r8a779a0: Fix parent of CBFUSA clock HID: core: detect and skip invalid inputs to snto32() RDMA/siw: Fix handling of zero-sized Read and Receive Queues. dmaengine: fsldma: Fix a resource leak in the remove function dmaengine: fsldma: Fix a resource leak in an error handling path of the probe function dmaengine: owl-dma: Fix a resource leak in the remove function dmaengine: hsu: disable spurious interrupt mfd: bd9571mwv: Use devm_mfd_add_devices() power: supply: cpcap-charger: Fix missing power_supply_put() power: supply: cpcap-battery: Fix missing power_supply_put() power: supply: cpcap-charger: Fix power_supply_put on null battery pointer fdt: Properly handle "no-map" field in the memory region of/fdt: Make sure no-map does not remove already reserved regions RDMA/rtrs: Extend ibtrs_cq_qp_create RDMA/rtrs-srv: Release lock before call into close_sess RDMA/rtrs-srv: Use sysfs_remove_file_self for disconnect RDMA/rtrs-clt: Set mininum limit when create QP RDMA/rtrs: Call kobject_put in the failure path RDMA/rtrs-srv: Fix missing wr_cqe RDMA/rtrs-clt: Refactor the failure cases in alloc_clt RDMA/rtrs-srv: Init wr_cnt as 1 power: reset: at91-sama5d2_shdwc: fix wkupdbc mask rtc: s5m: select REGMAP_I2C dmaengine: idxd: set DMA channel to be private power: supply: fix sbs-charger build, needs REGMAP_I2C clocksource/drivers/ixp4xx: Select TIMER_OF when needed clocksource/drivers/mxs_timer: Add missing semicolon when DEBUG is defined spi: imx: Don't print error on -EPROBEDEFER RDMA/mlx5: Use the correct obj_id upon DEVX TIR creation IB/mlx5: Add mutex destroy call to cap_mask_mutex mutex clk: sunxi-ng: h6: Fix clock divider range on some clocks platform/chrome: cros_ec_proto: Use EC_HOST_EVENT_MASK not BIT platform/chrome: cros_ec_proto: Add LID and BATTERY to default mask regulator: axp20x: Fix reference cout leak watch_queue: Drop references to /dev/watch_queue certs: Fix blacklist flag type confusion regulator: s5m8767: Fix reference count leak spi: atmel: Put allocated master before return regulator: s5m8767: Drop regulators OF node reference power: supply: axp20x_usb_power: Init work before enabling IRQs power: supply: smb347-charger: Fix interrupt usage if interrupt is unavailable regulator: core: Avoid debugfs: Directory ... already present! error isofs: release buffer head before return watchdog: intel-mid_wdt: Postpone IRQ handler registration till SCU is ready auxdisplay: ht16k33: Fix refresh rate handling objtool: Fix error handling for STD/CLD warnings objtool: Fix retpoline detection in asm code objtool: Fix ".cold" section suffix check for newer versions of GCC scsi: lpfc: Fix ancient double free iommu: Switch gather->end to the inclusive end IB/umad: Return EIO in case of when device disassociated IB/umad: Return EPOLLERR in case of when device disassociated KVM: PPC: Make the VMX instruction emulation routines static powerpc/47x: Disable 256k page size powerpc/time: Enable sched clock for irqtime mmc: owl-mmc: Fix a resource leak in an error handling path and in the remove function mmc: sdhci-sprd: Fix some resource leaks in the remove function mmc: usdhi6rol0: Fix a resource leak in the error handling path of the probe mmc: renesas_sdhi_internal_dmac: Fix DMA buffer alignment from 8 to 128-bytes ARM: 9046/1: decompressor: Do not clear SCTLR.nTLSMD for ARMv7+ cores i2c: qcom-geni: Store DMA mapping data in geni_i2c_dev struct amba: Fix resource leak for drivers without .remove iommu: Move iotlb_sync_map out from __iommu_map iommu: Properly pass gfp_t in _iommu_map() to avoid atomic sleeping IB/mlx5: Return appropriate error code instead of ENOMEM IB/cm: Avoid a loop when device has 255 ports tracepoint: Do not fail unregistering a probe due to memory failure rtc: zynqmp: depend on HAS_IOMEM perf tools: Fix DSO filtering when not finding a map for a sampled address perf vendor events arm64: Fix Ampere eMag event typo RDMA/rxe: Fix coding error in rxe_recv.c RDMA/rxe: Fix coding error in rxe_rcv_mcast_pkt RDMA/rxe: Correct skb on loopback path spi: stm32: properly handle 0 byte transfer mfd: altera-sysmgr: Fix physical address storing more mfd: wm831x-auxadc: Prevent use after free in wm831x_auxadc_read_irq() powerpc/pseries/dlpar: handle ibm, configure-connector delay status powerpc/8xx: Fix software emulation interrupt clk: qcom: gcc-msm8998: Fix Alpha PLL type for all GPLLs kunit: tool: fix unit test cleanup handling kselftests: dmabuf-heaps: Fix Makefile's inclusion of the kernel's usr/include dir RDMA/hns: Fixed wrong judgments in the goto branch RDMA/siw: Fix calculation of tx_valid_cpus size RDMA/hns: Fix type of sq_signal_bits RDMA/hns: Disable RQ inline by default clk: divider: fix initialization with parent_hw spi: pxa2xx: Fix the controller numbering for Wildcat Point powerpc/uaccess: Avoid might_fault() when user access is enabled powerpc/kuap: Restore AMR after replaying soft interrupts regulator: qcom-rpmh: fix pm8009 ldo7 clk: aspeed: Fix APLL calculate formula from ast2600-A2 selftests/ftrace: Update synthetic event syntax errors perf symbols: Use (long) for iterator for bfd symbols regulator: bd718x7, bd71828, Fix dvs voltage levels spi: dw: Avoid stack content exposure spi: Skip zero-length transfers in spi_transfer_one_message() printk: avoid prb_first_valid_seq() where possible perf symbols: Fix return value when loading PE DSO nfsd: register pernet ops last, unregister first svcrdma: Hold private mutex while invoking rdma_accept() ceph: fix flush_snap logic after putting caps RDMA/hns: Fixes missing error code of CMDQ RDMA/ucma: Fix use-after-free bug in ucma_create_uevent RDMA/rtrs-srv: Fix stack-out-of-bounds RDMA/rtrs: Only allow addition of path to an already established session RDMA/rtrs-srv: fix memory leak by missing kobject free RDMA/rtrs-srv-sysfs: fix missing put_device RDMA/rtrs-srv: Do not pass a valid pointer to PTR_ERR() Input: sur40 - fix an error code in sur40_probe() perf record: Fix continue profiling after draining the buffer perf intel-pt: Fix missing CYC processing in PSB perf intel-pt: Fix premature IPC perf intel-pt: Fix IPC with CYC threshold perf test: Fix unaligned access in sample parsing test Input: elo - fix an error code in elo_connect() sparc64: only select COMPAT_BINFMT_ELF if BINFMT_ELF is set sparc: fix led.c driver when PROC_FS is not enabled Input: zinitix - fix return type of zinitix_init_touch() ARM: 9065/1: OABI compat: fix build when EPOLL is not enabled misc: eeprom_93xx46: Fix module alias to enable module autoprobe phy: rockchip-emmc: emmc_phy_init() always return 0 phy: cadence-torrent: Fix error code in cdns_torrent_phy_probe() misc: eeprom_93xx46: Add module alias to avoid breaking support for non device tree users PCI: rcar: Always allocate MSI addresses in 32bit space soundwire: cadence: fix ACK/NAK handling pwm: rockchip: Enable APB clock during register access while probing pwm: rockchip: rockchip_pwm_probe(): Remove superfluous clk_unprepare() pwm: rockchip: Eliminate potential race condition when probing PCI: xilinx-cpm: Fix reference count leak on error path VMCI: Use set_page_dirty_lock() when unregistering guest memory PCI: Align checking of syscall user config accessors mei: hbm: call mei_set_devstate() on hbm stop response drm/msm: Fix MSM_INFO_GET_IOVA with carveout drm/msm/dsi: Correct io_start for MSM8994 (20nm PHY) drm/msm/mdp5: Fix wait-for-commit for cmd panels drm/msm: Fix race of GPU init vs timestamp power management. drm/msm: Fix races managing the OOB state for timestamp vs timestamps. drm/msm/dp: trigger unplug event in msm_dp_display_disable vfio/iommu_type1: Populate full dirty when detach non-pinned group vfio/iommu_type1: Fix some sanity checks in detach group vfio-pci/zdev: fix possible segmentation fault issue ext4: fix potential htree index checksum corruption phy: USB_LGM_PHY should depend on X86 coresight: etm4x: Skip accessing TRCPDCR in save/restore nvmem: core: Fix a resource leak on error in nvmem_add_cells_from_of() nvmem: core: skip child nodes not matching binding soundwire: bus: use sdw_update_no_pm when initializing a device soundwire: bus: use sdw_write_no_pm when setting the bus scale registers soundwire: export sdw_write/read_no_pm functions soundwire: bus: fix confusion on device used by pm_runtime misc: fastrpc: fix incorrect usage of dma_map_sgtable remoteproc/mediatek: acknowledge watchdog IRQ after handled regmap: sdw: use _no_pm functions in regmap_read/write ext: EXT4_KUNIT_TESTS should depend on EXT4_FS instead of selecting it mailbox: sprd: correct definition of SPRD_OUTBOX_FIFO_FULL device-dax: Fix default return code of range_parse() PCI: pci-bridge-emul: Fix array overruns, improve safety PCI: cadence: Fix DMA range mapping early return error i40e: Fix flow for IPv6 next header (extension header) i40e: Add zero-initialization of AQ command structures i40e: Fix overwriting flow control settings during driver loading i40e: Fix addition of RX filters after enabling FW LLDP agent i40e: Fix VFs not created Take mmap lock in cacheflush syscall nios2: fixed broken sys_clone syscall i40e: Fix add TC filter for IPv6 octeontx2-af: Fix an off by one in rvu_dbg_qsize_write() pwm: iqs620a: Fix overflow and optimize calculations vfio/type1: Use follow_pte() ice: report correct max number of TCs ice: Account for port VLAN in VF max packet size calculation ice: Fix state bits on LLDP mode switch ice: update the number of available RSS queues net: stmmac: fix CBS idleslope and sendslope calculation net/mlx4_core: Add missed mlx4_free_cmd_mailbox() PCI: rockchip: Make 'ep-gpios' DT property optional vxlan: move debug check after netdev unregister wireguard: device: do not generate ICMP for non-IP packets wireguard: kconfig: use arm chacha even with no neon ocfs2: fix a use after free on error mm: memcontrol: fix NR_ANON_THPS accounting in charge moving mm: memcontrol: fix slub memory accounting mm/memory.c: fix potential pte_unmap_unlock pte error mm/hugetlb: fix potential double free in hugetlb_register_node() error path mm/hugetlb: suppress wrong warning info when alloc gigantic page mm/compaction: fix misbehaviors of fast_find_migrateblock() r8169: fix jumbo packet handling on RTL8168e NFSv4: Fixes for nfs4_bitmask_adjust() KVM: SVM: Intercept INVPCID when it's disabled to inject #UD KVM: x86/mmu: Expand collapsible SPTE zap for TDP MMU to ZONE_DEVICE and HugeTLB pages arm64: Add missing ISB after invalidating TLB in __primary_switch i2c: brcmstb: Fix brcmstd_send_i2c_cmd condition i2c: exynos5: Preserve high speed master code mm,thp,shmem: make khugepaged obey tmpfs mount flags mm: fix memory_failure() handling of dax-namespace metadata mm/rmap: fix potential pte_unmap on an not mapped pte proc: use kvzalloc for our kernel buffer csky: Fix a size determination in gpr_get() scsi: bnx2fc: Fix Kconfig warning & CNIC build errors scsi: sd: sd_zbc: Don't pass GFP_NOIO to kvcalloc block: reopen the device in blkdev_reread_part ide/falconide: Fix module unload scsi: sd: Fix Opal support blk-settings: align max_sectors on "logical_block_size" boundary soundwire: intel: fix possible crash when no device is detected ACPI: property: Fix fwnode string properties matching ACPI: configfs: add missing check after configfs_register_default_group() cpufreq: ACPI: Set cpuinfo.max_freq directly if max boost is known HID: logitech-dj: add support for keyboard events in eQUAD step 4 Gaming HID: wacom: Ignore attempts to overwrite the touch_max value from HID Input: raydium_ts_i2c - do not send zero length Input: xpad - add support for PowerA Enhanced Wired Controller for Xbox Series X|S Input: joydev - prevent potential read overflow in ioctl Input: i8042 - add ASUS Zenbook Flip to noselftest list media: mceusb: Fix potential out-of-bounds shift USB: serial: option: update interface mapping for ZTE P685M usb: musb: Fix runtime PM race in musb_queue_resume_work usb: dwc3: gadget: Fix setting of DEPCFG.bInterval_m1 usb: dwc3: gadget: Fix dep->interval for fullspeed interrupt USB: serial: ftdi_sio: fix FTX sub-integer prescaler USB: serial: pl2303: fix line-speed handling on newer chips USB: serial: mos7840: fix error code in mos7840_write() USB: serial: mos7720: fix error code in mos7720_write() phy: lantiq: rcu-usb2: wait after clock enable ALSA: fireface: fix to parse sync status register of latter protocol ALSA: hda: Add another CometLake-H PCI ID ALSA: hda/hdmi: Drop bogus check at closing a stream ALSA: hda/realtek: modify EAPD in the ALC886 ALSA: hda/realtek: Quirk for HP Spectre x360 14 amp setup MIPS: Ingenic: Disable HPTLB for D0 XBurst CPUs too MIPS: Support binutils configured with --enable-mips-fix-loongson3-llsc=yes MIPS: VDSO: Use CLANG_FLAGS instead of filtering out '--target=' Revert "MIPS: Octeon: Remove special handling of CONFIG_MIPS_ELF_APPENDED_DTB=y" Revert "bcache: Kill btree_io_wq" bcache: Give btree_io_wq correct semantics again bcache: Move journal work to new flush wq Revert "drm/amd/display: Update NV1x SR latency values" drm/amd/display: Add FPU wrappers to dcn21_validate_bandwidth() drm/amd/display: Remove Assert from dcn10_get_dig_frontend drm/amd/display: Add vupdate_no_lock interrupts for DCN2.1 drm/amdkfd: Fix recursive lock warnings drm/amdgpu: Set reference clock to 100Mhz on Renoir (v2) drm/nouveau/kms: handle mDP connectors drm/modes: Switch to 64bit maths to avoid integer overflow drm/sched: Cancel and flush all outstanding jobs before finish. drm/panel: kd35t133: allow using non-continuous dsi clock drm/rockchip: Require the YTR modifier for AFBC ASoC: siu: Fix build error by a wrong const prefix selinux: fix inconsistency between inode_getxattr and inode_listsecurity erofs: initialized fields can only be observed after bit is set tpm_tis: Fix check_locality for correct locality acquisition tpm_tis: Clean up locality release KEYS: trusted: Fix incorrect handling of tpm_get_random() KEYS: trusted: Fix migratable=1 failing KEYS: trusted: Reserve TPM for seal and unseal operations btrfs: do not cleanup upper nodes in btrfs_backref_cleanup_node btrfs: do not warn if we can't find the reloc root when looking up backref btrfs: add asserts for deleting backref cache nodes btrfs: abort the transaction if we fail to inc ref in btrfs_copy_root btrfs: fix reloc root leak with 0 ref reloc roots on recovery btrfs: splice remaining dirty_bg's onto the transaction dirty bg list btrfs: handle space_info::total_bytes_pinned inside the delayed ref itself btrfs: account for new extents being deleted in total_bytes_pinned btrfs: fix extent buffer leak on failure to copy root drm/i915/gt: Flush before changing register state drm/i915/gt: Correct surface base address for renderclear crypto: arm64/sha - add missing module aliases crypto: aesni - prevent misaligned buffers on the stack crypto: michael_mic - fix broken misalignment handling crypto: sun4i-ss - checking sg length is not sufficient crypto: sun4i-ss - IV register does not work on A10 and A13 crypto: sun4i-ss - handle BigEndian for cipher crypto: sun4i-ss - initialize need_fallback soc: samsung: exynos-asv: don't defer early on not-supported SoCs soc: samsung: exynos-asv: handle reading revision register error seccomp: Add missing return in non-void function arm64: ptrace: Fix seccomp of traced syscall -1 (NO_SYSCALL) misc: rtsx: init of rts522a add OCP power off when no card is present drivers/misc/vmw_vmci: restrict too big queue size in qp_host_alloc_queue pstore: Fix typo in compression option name dts64: mt7622: fix slow sd card access arm64: dts: agilex: fix phy interface bit shift for gmac1 and gmac2 staging/mt7621-dma: mtk-hsdma.c->hsdma-mt7621.c staging: gdm724x: Fix DMA from stack staging: rtl8188eu: Add Edimax EW-7811UN V2 to device table floppy: reintroduce O_NDELAY fix media: i2c: max9286: fix access to unallocated memory media: ir_toy: add another IR Droid device media: ipu3-cio2: Fix mbus_code processing in cio2_subdev_set_fmt() media: marvell-ccic: power up the device on mclk enable media: smipcie: fix interrupt handling and IR timeout x86/virt: Eat faults on VMXOFF in reboot flows x86/reboot: Force all cpus to exit VMX root if VMX is supported x86/fault: Fix AMD erratum #91 errata fixup for user code x86/entry: Fix instrumentation annotation powerpc/prom: Fix "ibm,arch-vec-5-platform-support" scan rcu: Pull deferred rcuog wake up to rcu_eqs_enter() callers rcu/nocb: Perform deferred wake up before last idle's need_resched() check kprobes: Fix to delay the kprobes jump optimization arm64: Extend workaround for erratum 1024718 to all versions of Cortex-A55 iommu/arm-smmu-qcom: Fix mask extraction for bootloader programmed SMRs arm64: kexec_file: fix memory leakage in create_dtb() when fdt_open_into() fails arm64: uprobe: Return EOPNOTSUPP for AARCH32 instruction probing arm64 module: set plt* section addresses to 0x0 arm64: spectre: Prevent lockdep splat on v4 mitigation enable path riscv: Disable KSAN_SANITIZE for vDSO watchdog: qcom: Remove incorrect usage of QCOM_WDT_ENABLE_IRQ watchdog: mei_wdt: request stop on unregister coresight: etm4x: Handle accesses to TRCSTALLCTLR mtd: spi-nor: sfdp: Fix last erase region marking mtd: spi-nor: sfdp: Fix wrong erase type bitmask for overlaid region mtd: spi-nor: core: Fix erase type discovery for overlaid region mtd: spi-nor: core: Add erase size check for erase command initialization mtd: spi-nor: hisi-sfc: Put child node np on error path fs/affs: release old buffer head on error path seq_file: document how per-entry resources are managed. x86: fix seq_file iteration for pat/memtype.c mm: memcontrol: fix swap undercounting in cgroup2 mm: memcontrol: fix get_active_memcg return value hugetlb: fix update_and_free_page contig page struct assumption hugetlb: fix copy_huge_page_from_user contig page struct assumption mm/vmscan: restore zone_reclaim_mode ABI mm, compaction: make fast_isolate_freepages() stay within zone KVM: nSVM: fix running nested guests when npt=0 nvmem: qcom-spmi-sdam: Fix uninitialized pdev pointer module: Ignore _GLOBAL_OFFSET_TABLE_ when warning for undefined symbols mmc: sdhci-esdhc-imx: fix kernel panic when remove module mmc: sdhci-pci-o2micro: Bug fix for SDR104 HW tuning failure powerpc/32: Preserve cr1 in exception prolog stack check to fix build error powerpc/kexec_file: fix FDT size estimation for kdump kernel powerpc/32s: Add missing call to kuep_lock on syscall entry spmi: spmi-pmic-arb: Fix hw_irq overflow mei: fix transfer over dma with extended header mei: me: emmitsburg workstation DID mei: me: add adler lake point S DID mei: me: add adler lake point LP DID gpio: pcf857x: Fix missing first interrupt mfd: gateworks-gsc: Fix interrupt type printk: fix deadlock when kernel panic exfat: fix shift-out-of-bounds in exfat_fill_super() zonefs: Fix file size of zones in full condition kcmp: Support selection of SYS_kcmp without CHECKPOINT_RESTORE thermal: cpufreq_cooling: freq_qos_update_request() returns < 0 on error cpufreq: qcom-hw: drop devm_xxx() calls from init/exit hooks cpufreq: intel_pstate: Change intel_pstate_get_hwp_max() argument cpufreq: intel_pstate: Get per-CPU max freq via MSR_HWP_CAPABILITIES if available proc: don't allow async path resolution of /proc/thread-self components s390/vtime: fix inline assembly clobber list virtio/s390: implement virtio-ccw revision 2 correctly um: mm: check more comprehensively for stub changes um: defer killing userspace on page table update failures irqchip/loongson-pch-msi: Use bitmap_zalloc() to allocate bitmap f2fs: fix out-of-repair __setattr_copy() f2fs: enforce the immutable flag on open files f2fs: flush data when enabling checkpoint back sparc32: fix a user-triggerable oops in clear_user() spi: fsl: invert spisel_boot signal on MPC8309 spi: spi-synquacer: fix set_cs handling gfs2: fix glock confusion in function signal_our_withdraw gfs2: Don't skip dlm unlock if glock has an lvb gfs2: Lock imbalance on error path in gfs2_recover_one gfs2: Recursive gfs2_quota_hold in gfs2_iomap_end dm: fix deadlock when swapping to encrypted device dm table: fix iterate_devices based device capability checks dm table: fix DAX iterate_devices based device capability checks dm table: fix zoned iterate_devices based device capability checks dm writecache: fix performance degradation in ssd mode dm writecache: return the exact table values that were set dm writecache: fix writing beyond end of underlying device when shrinking dm era: Recover committed writeset after crash dm era: Update in-core bitset after committing the metadata dm era: Verify the data block size hasn't changed dm era: Fix bitset memory leaks dm era: Use correct value size in equality function of writeset tree dm era: Reinitialize bitset cache before digesting a new writeset dm era: only resize metadata in preresume drm/i915: Reject 446-480MHz HDMI clock on GLK kgdb: fix to kill breakpoints on initmem after boot ipv6: silence compilation warning for non-IPV6 builds net: icmp: pass zeroed opts from icmp{,v6}_ndo_send before sending wireguard: selftests: test multiple parallel streams wireguard: queueing: get rid of per-peer ring buffers net: sched: fix police ext initialization net: qrtr: Fix memory leak in qrtr_tun_open net_sched: fix RTNL deadlock again caused by request_module() ARM: dts: aspeed: Add LCLK to lpc-snoop Linux 5.10.20 Signed-off-by: Greg Kroah-Hartman <gregkh@google.com> Change-Id: I3fbcecd9413ce212dac68d5cc800c9457feba56a |
||
|
Miaohe Lin
|
d45f943ef8 |
mm/rmap: fix potential pte_unmap on an not mapped pte
[ Upstream commit 5d5d19eda6b0ee790af89c45e3f678345be6f50f ]
For PMD-mapped page (usually THP), pvmw->pte is NULL. For PTE-mapped THP,
pvmw->pte is mapped. But for HugeTLB pages, pvmw->pte is not mapped and
set to the relevant page table entry. So in page_vma_mapped_walk_done(),
we may do pte_unmap() for HugeTLB pte which is not mapped. Fix this by
checking pvmw->page against PageHuge before trying to do pte_unmap().
Link: https://lkml.kernel.org/r/20210127093349.39081-1-linmiaohe@huawei.com
Fixes:
|
||
|
Laurent Dufour
|
a1dbf20e8e |
FROMLIST: mm: introduce __page_add_new_anon_rmap()
When dealing with speculative page fault handler, we may race with VMA being split or merged. In this case the vma->vm_start and vm->vm_end fields may not match the address the page fault is occurring. This can only happens when the VMA is split but in that case, the anon_vma pointer of the new VMA will be the same as the original one, because in __split_vma the new->anon_vma is set to src->anon_vma when *new = *vma. So even if the VMA boundaries are not correct, the anon_vma pointer is still valid. If the VMA has been merged, then the VMA in which it has been merged must have the same anon_vma pointer otherwise the merge can't be done. So in all the case we know that the anon_vma is valid, since we have checked before starting the speculative page fault that the anon_vma pointer is valid for this VMA and since there is an anon_vma this means that at one time a page has been backed and that before the VMA is cleaned, the page table lock would have to be grab to clean the PTE, and the anon_vma field is checked once the PTE is locked. This patch introduce a new __page_add_new_anon_rmap() service which doesn't check for the VMA boundaries, and create a new inline one which do the check. When called from a page fault handler, if this is not a speculative one, there is a guarantee that vm_start and vm_end match the faulting address, so this check is useless. In the context of the speculative page fault handler, this check may be wrong but anon_vma is still valid as explained above. Change-Id: I72c47830181579f8c9618df879077d321653b5f1 Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> Link: https://lore.kernel.org/lkml/1523975611-15978-17-git-send-email-ldufour@linux.vnet.ibm.com/ Bug: 161210518 Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org> |
||
|
Shakeel Butt
|
dd156e3fca |
mm/rmap: always do TTU_IGNORE_ACCESS
[ Upstream commit 013339df116c2ee0d796dd8bfb8f293a2030c063 ] Since commit |
||
|
Michel Lespinasse
|
c1e8d7c6a7 |
mmap locking API: convert mmap_sem comments
Convert comments that reference mmap_sem to reference mmap_lock instead. [akpm@linux-foundation.org: fix up linux-next leftovers] [akpm@linux-foundation.org: s/lockaphore/lock/, per Vlastimil] [akpm@linux-foundation.org: more linux-next fixups, per Michel] Signed-off-by: Michel Lespinasse <walken@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Daniel Jordan <daniel.m.jordan@oracle.com> Cc: Davidlohr Bueso <dbueso@suse.de> Cc: David Rientjes <rientjes@google.com> Cc: Hugh Dickins <hughd@google.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Jerome Glisse <jglisse@redhat.com> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Laurent Dufour <ldufour@linux.ibm.com> Cc: Liam Howlett <Liam.Howlett@oracle.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ying Han <yinghan@google.com> Link: http://lkml.kernel.org/r/20200520052908.204642-13-walken@google.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Greg Kroah-Hartman
|
b24413180f |
License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
||
|
Davidlohr Bueso
|
f808c13fd3 |
lib/interval_tree: fast overlap detection
Allow interval trees to quickly check for overlaps to avoid unnecesary tree lookups in interval_tree_iter_first(). As of this patch, all interval tree flavors will require using a 'rb_root_cached' such that we can have the leftmost node easily available. While most users will make use of this feature, those with special functions (in addition to the generic insert, delete, search calls) will avoid using the cached option as they can do funky things with insertions -- for example, vma_interval_tree_insert_after(). [jglisse@redhat.com: fix deadlock from typo vm_lock_anon_vma()] Link: http://lkml.kernel.org/r/20170808225719.20723-1-jglisse@redhat.com Link: http://lkml.kernel.org/r/20170719014603.19029-12-dave@stgolabs.net Signed-off-by: Davidlohr Bueso <dbueso@suse.de> Signed-off-by: Jérôme Glisse <jglisse@redhat.com> Acked-by: Christian König <christian.koenig@amd.com> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Doug Ledford <dledford@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Cc: David Airlie <airlied@linux.ie> Cc: Jason Wang <jasowang@redhat.com> Cc: Christian Benvenuti <benve@cisco.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Naoya Horiguchi
|
b5ff8161e3 |
mm: thp: introduce separate TTU flag for thp freezing
TTU_MIGRATION is used to convert pte into migration entry until thp
split completes. This behavior conflicts with thp migration added later
patches, so let's introduce a new TTU flag specifically for freezing.
try_to_unmap() is used both for thp split (via freeze_page()) and page
migration (via __unmap_and_move()). In freeze_page(), ttu_flag given
for head page is like below (assuming anonymous thp):
(TTU_IGNORE_MLOCK | TTU_IGNORE_ACCESS | TTU_RMAP_LOCKED | \
TTU_MIGRATION | TTU_SPLIT_HUGE_PMD)
and ttu_flag given for tail pages is:
(TTU_IGNORE_MLOCK | TTU_IGNORE_ACCESS | TTU_RMAP_LOCKED | \
TTU_MIGRATION)
__unmap_and_move() calls try_to_unmap() with ttu_flag:
(TTU_MIGRATION | TTU_IGNORE_MLOCK | TTU_IGNORE_ACCESS)
Now I'm trying to insert a branch for thp migration at the top of
try_to_unmap_one() like below
static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
unsigned long address, void *arg)
{
...
/* PMD-mapped THP migration entry */
if (!pvmw.pte && (flags & TTU_MIGRATION)) {
if (!PageAnon(page))
continue;
set_pmd_migration_entry(&pvmw, page);
continue;
}
...
}
so try_to_unmap() for tail pages called by thp split can go into thp
migration code path (which converts *pmd* into migration entry), while
the expectation is to freeze thp (which converts *pte* into migration
entry.)
I detected this failure as a "bad page state" error in a testcase where
split_huge_page() is called from queue_pages_pte_range().
Link: http://lkml.kernel.org/r/20170717193955.20207-4-zi.yan@sent.com
Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: David Nellans <dnellans@nvidia.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
||
|
Minchan Kim
|
83612a948d |
mm: remove SWAP_[SUCCESS|AGAIN|FAIL]
There is no user for it. Remove it. [minchan@kernel.org: use false instead of SWAP_FAIL] Link: http://lkml.kernel.org/r/20170316053313.GA19241@bbox Link: http://lkml.kernel.org/r/1489555493-14659-11-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
e4b8222271 |
mm: make rmap_one boolean function
rmap_one's return value controls whether rmap_work should contine to scan other ptes or not so it's target for changing to boolean. Return true if the scan should be continued. Otherwise, return false to stop the scanning. This patch makes rmap_one's return value to boolean. Link: http://lkml.kernel.org/r/1489555493-14659-10-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
1df631ae19 |
mm: make rmap_walk() return void
There is no user of the return value from rmap_walk() and friends so this patch makes them void-returning functions. Link: http://lkml.kernel.org/r/1489555493-14659-9-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
666e5a406c |
mm: make ttu's return boolean
try_to_unmap() returns SWAP_SUCCESS or SWAP_FAIL so it's suitable for boolean return. This patch changes it. Link: http://lkml.kernel.org/r/1489555493-14659-8-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
ad6b67041a |
mm: remove SWAP_MLOCK in ttu
ttu doesn't need to return SWAP_MLOCK. Instead, just return SWAP_FAIL because it means the page is not-swappable so it should move to another LRU list(active or unevictable). putback friends will move it to right list depending on the page's LRU flag. Link: http://lkml.kernel.org/r/1489555493-14659-6-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
192d723256 |
mm: make try_to_munlock() return void
try_to_munlock returns SWAP_MLOCK if the one of VMAs mapped the page has VM_LOCKED flag. In that time, VM set PG_mlocked to the page if the page is not pte-mapped THP which cannot be mlocked, either. With that, __munlock_isolated_page can use PageMlocked to check whether try_to_munlock is successful or not without relying on try_to_munlock's retval. It helps to make try_to_unmap/try_to_unmap_one simple with upcoming patches. [minchan@kernel.org: remove PG_Mlocked VM_BUG_ON check] Link: http://lkml.kernel.org/r/20170411025615.GA6545@bbox Link: http://lkml.kernel.org/r/1489555493-14659-5-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Sasha Levin <alexander.levin@verizon.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
18863d3a3f |
mm: remove SWAP_DIRTY in ttu
If we found lazyfree page is dirty, try_to_unmap_one can just SetPageSwapBakced in there like PG_mlocked page and just return with SWAP_FAIL which is very natural because the page is not swappable right now so that vmscan can activate it. There is no point to introduce new return value SWAP_DIRTY in try_to_unmap at the moment. Link: http://lkml.kernel.org/r/1489555493-14659-3-git-send-email-minchan@kernel.org Signed-off-by: Minchan Kim <minchan@kernel.org> Acked-by: Hillf Danton <hillf.zj@alibaba-inc.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Shaohua Li
|
802a3a92ad |
mm: reclaim MADV_FREE pages
When memory pressure is high, we free MADV_FREE pages. If the pages are not dirty in pte, the pages could be freed immediately. Otherwise we can't reclaim them. We put the pages back to anonumous LRU list (by setting SwapBacked flag) and the pages will be reclaimed in normal swapout way. We use normal page reclaim policy. Since MADV_FREE pages are put into inactive file list, such pages and inactive file pages are reclaimed according to their age. This is expected, because we don't want to reclaim too many MADV_FREE pages before used once pages. Based on Minchan's original patch [minchan@kernel.org: clean up lazyfree page handling] Link: http://lkml.kernel.org/r/20170303025237.GB3503@bbox Link: http://lkml.kernel.org/r/14b8eb1d3f6bf6cc492833f183ac8c304e560484.1487965799.git.shli@fb.com Signed-off-by: Shaohua Li <shli@fb.com> Signed-off-by: Minchan Kim <minchan@kernel.org> Acked-by: Minchan Kim <minchan@kernel.org> Acked-by: Michal Hocko <mhocko@suse.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Mel Gorman <mgorman@techsingularity.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Shaohua Li
|
a128ca71fb |
mm: delete unnecessary TTU_* flags
Patch series "mm: fix some MADV_FREE issues", v5. We are trying to use MADV_FREE in jemalloc. Several issues are found. Without solving the issues, jemalloc can't use the MADV_FREE feature. - Doesn't support system without swap enabled. Because if swap is off, we can't or can't efficiently age anonymous pages. And since MADV_FREE pages are mixed with other anonymous pages, we can't reclaim MADV_FREE pages. In current implementation, MADV_FREE will fallback to MADV_DONTNEED without swap enabled. But in our environment, a lot of machines don't enable swap. This will prevent our setup using MADV_FREE. - Increases memory pressure. page reclaim bias file pages reclaim against anonymous pages. This doesn't make sense for MADV_FREE pages, because those pages could be freed easily and refilled with very slight penality. Even page reclaim doesn't bias file pages, there is still an issue, because MADV_FREE pages and other anonymous pages are mixed together. To reclaim a MADV_FREE page, we probably must scan a lot of other anonymous pages, which is inefficient. In our test, we usually see oom with MADV_FREE enabled and nothing without it. - Accounting. There are two accounting problems. We don't have a global accounting. If the system is abnormal, we don't know if it's a problem from MADV_FREE side. The other problem is RSS accounting. MADV_FREE pages are accounted as normal anon pages and reclaimed lazily, so application's RSS becomes bigger. This confuses our workloads. We have monitoring daemon running and if it finds applications' RSS becomes abnormal, the daemon will kill the applications even kernel can reclaim the memory easily. To address the first the two issues, we can either put MADV_FREE pages into a separate LRU list (Minchan's previous patches and V1 patches), or put them into LRU_INACTIVE_FILE list (suggested by Johannes). The patchset use the second idea. The reason is LRU_INACTIVE_FILE list is tiny nowadays and should be full of used once file pages. So we can still efficiently reclaim MADV_FREE pages there without interference with other anon and active file pages. Putting the pages into inactive file list also has an advantage which allows page reclaim to prioritize MADV_FREE pages and used once file pages. MADV_FREE pages are put into the lru list and clear SwapBacked flag, so PageAnon(page) && !PageSwapBacked(page) will indicate a MADV_FREE pages. These pages will directly freed without pageout if they are clean, otherwise normal swap will reclaim them. For the third issue, the previous post adds global accounting and a separate RSS count for MADV_FREE pages. The problem is we never get accurate accounting for MADV_FREE pages. The pages are mapped to userspace, can be dirtied without notice from kernel side. To get accurate accounting, we could write protect the page, but then there is extra page fault overhead, which people don't want to pay. Jemalloc guys have concerns about the inaccurate accounting, so this post drops the accounting patches temporarily. The info exported to /proc/pid/smaps for MADV_FREE pages are kept, which is the only place we can get accurate accounting right now. This patch (of 6): Johannes pointed out TTU_LZFREE is unnecessary. It's true because we always have the flag set if we want to do an unmap. For cases we don't do an unmap, the TTU_LZFREE part of code should never run. Also the TTU_UNMAP is unnecessary. If no other flags set (for example, TTU_MIGRATION), an unmap is implied. The patch includes Johannes's cleanup and dead TTU_ACTION macro removal code Link: http://lkml.kernel.org/r/4be3ea1bc56b26fd98a54d0a6f70bec63f6d8980.1487965799.git.shli@fb.com Signed-off-by: Shaohua Li <shli@fb.com> Suggested-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Minchan Kim <minchan@kernel.org> Acked-by: Hillf Danton <hillf.zj@alibaba-inc.com> Acked-by: Michal Hocko <mhocko@suse.com> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Mel Gorman <mgorman@techsingularity.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
d53a8b49a6 |
mm: drop page_check_address{,_transhuge}
All users are gone. Let's drop them. Link: http://lkml.kernel.org/r/20170129173858.45174-12-kirill.shutemov@linux.intel.com Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hillf Danton <hillf.zj@alibaba-inc.com> Cc: Hugh Dickins <hughd@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com> Cc: Vladimir Davydov <vdavydov.dev@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
ace71a19ce |
mm: introduce page_vma_mapped_walk()
Introduce a new interface to check if a page is mapped into a vma. It
aims to address shortcomings of page_check_address{,_transhuge}.
Existing interface is not able to handle PTE-mapped THPs: it only finds
the first PTE. The rest lefted unnoticed.
page_vma_mapped_walk() iterates over all possible mapping of the page in
the vma.
Link: http://lkml.kernel.org/r/20170129173858.45174-3-kirill.shutemov@linux.intel.com
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Hillf Danton <hillf.zj@alibaba-inc.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
||
|
Vlastimil Babka
|
d5a187daf5 |
mm, rmap: handle anon_vma_prepare() common case inline
anon_vma_prepare() is mostly a large "if (unlikely(...))" block, as the expected common case is that an anon_vma already exists. We could turn the condition around and return 0, but it also makes sense to do it inline and avoid a call for the common case. Bloat-o-meter naturally shows that inlining the check has some code size costs: add/remove: 1/1 grow/shrink: 4/0 up/down: 475/-373 (102) function old new delta __anon_vma_prepare - 359 +359 handle_mm_fault 2744 2796 +52 hugetlb_cow 1146 1170 +24 hugetlb_fault 2123 2145 +22 wp_page_copy 1469 1487 +18 anon_vma_prepare 373 - -373 Checking the asm however confirms that the hot paths now avoid a call, which is moved away. [akpm@linux-foundation.org: coding-style fixes] Link: http://lkml.kernel.org/r/20161116074005.22768-1-vbabka@suse.cz Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Rik van Riel <riel@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
dd78fedde4 |
rmap: support file thp
Naive approach: on mapping/unmapping the page as compound we update ->_mapcount on each 4k page. That's not efficient, but it's not obvious how we can optimize this. We can look into optimization later. PG_double_map optimization doesn't work for file pages since lifecycle of file pages is different comparing to anon pages: file page can be mapped again at any time. Link: http://lkml.kernel.org/r/1466021202-61880-11-git-send-email-kirill.shutemov@linux.intel.com Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Hugh Dickins
|
5a49973d71 |
mm: thp: refix false positive BUG in page_move_anon_rmap()
The VM_BUG_ON_PAGE in page_move_anon_rmap() is more trouble than it's
worth: the syzkaller fuzzer hit it again. It's still wrong for some THP
cases, because linear_page_index() was never intended to apply to
addresses before the start of a vma.
That's easily fixed with a signed long cast inside linear_page_index();
and Dmitry has tested such a patch, to verify the false positive. But
why extend linear_page_index() just for this case? when the avoidance in
page_move_anon_rmap() has already grown ugly, and there's no reason for
the check at all (nothing else there is using address or index).
Remove address arg from page_move_anon_rmap(), remove VM_BUG_ON_PAGE,
remove CONFIG_DEBUG_VM PageTransHuge adjustment.
And one more thing: should the compound_head(page) be done inside or
outside page_move_anon_rmap()? It's usually pushed down to the lowest
level nowadays (and mm/memory.c shows no other explicit use of it), so I
think it's better done in page_move_anon_rmap() than by caller.
Fixes:
|
||
|
Kirill A. Shutemov
|
e388466de4 |
mm: make remove_migration_ptes() beyond mm/migration.c
Make remove_migration_ptes() available to be used in split_huge_page(). New parameter 'locked' added: as with try_to_umap() we need a way to indicate that caller holds rmap lock. We also shouldn't try to mlock() pte-mapped huge pages: pte-mapeed THP pages are never mlocked. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
2a52bcbcc6 |
rmap: extend try_to_unmap() to be usable by split_huge_page()
Add support for two ttu_flags:
- TTU_SPLIT_HUGE_PMD would split PMD if it's there, before trying to
unmap page;
- TTU_RMAP_LOCKED indicates that caller holds relevant rmap lock;
Also, change rwc->done to !page_mapcount() instead of !page_mapped().
try_to_unmap() works on pte level, so we are really interested in the
mappedness of this small page rather than of the compound page it's a
part of.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
||
|
Kirill A. Shutemov
|
b97731992d |
rmap: introduce rmap_walk_locked()
This patchset rewrites freeze_page() and unfreeze_page() using try_to_unmap() and remove_migration_ptes(). Result is much simpler, but somewhat slower. Migration 8GiB worth of PMD-mapped THP: Baseline 20.21 +/- 0.393 Patched 20.73 +/- 0.082 Slowdown 1.03x It's 3% slower, comparing to 14% in v1. I don't it should be a stopper. Splitting of PTE-mapped pages slowed more. But this is not a common case. Migration 8GiB worth of PMD-mapped THP: Baseline 20.39 +/- 0.225 Patched 22.43 +/- 0.496 Slowdown 1.10x rmap_walk_locked() is the same as rmap_walk(), but the caller takes care of the relevant rmap lock. This is preparation for switching THP splitting from custom rmap walk in freeze_page()/unfreeze_page() to the generic one. There is no support for KSM pages for now: not clear which lock is implied. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Konstantin Khlebnikov
|
12352d3cae |
mm: replace vma_lock_anon_vma with anon_vma_lock_read/write
Sequence vma_lock_anon_vma() - vma_unlock_anon_vma() isn't safe if anon_vma appeared between lock and unlock. We have to check anon_vma first or call anon_vma_prepare() to be sure that it's here. There are only few users of these legacy helpers. Let's get rid of them. This patch fixes anon_vma lock imbalance in validate_mm(). Write lock isn't required here, read lock is enough. And reorders expand_downwards/expand_upwards: security_mmap_addr() and wrapping-around check don't have to be under anon vma lock. Link: https://lkml.kernel.org/r/CACT4Y+Y908EjM2z=706dv4rV6dWtxTLK9nFg9_7DhRMLppBo2g@mail.gmail.com Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com> Reported-by: Dmitry Vyukov <dvyukov@google.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Minchan Kim
|
854e9ed09d |
mm: support madvise(MADV_FREE)
Linux doesn't have an ability to free pages lazy while other OS already have been supported that named by madvise(MADV_FREE). The gain is clear that kernel can discard freed pages rather than swapping out or OOM if memory pressure happens. Without memory pressure, freed pages would be reused by userspace without another additional overhead(ex, page fault + allocation + zeroing). Jason Evans said: : Facebook has been using MAP_UNINITIALIZED : (https://lkml.org/lkml/2012/1/18/308) in some of its applications for : several years, but there are operational costs to maintaining this : out-of-tree in our kernel and in jemalloc, and we are anxious to retire it : in favor of MADV_FREE. When we first enabled MAP_UNINITIALIZED it : increased throughput for much of our workload by ~5%, and although the : benefit has decreased using newer hardware and kernels, there is still : enough benefit that we cannot reasonably retire it without a replacement. : : Aside from Facebook operations, there are numerous broadly used : applications that would benefit from MADV_FREE. The ones that immediately : come to mind are redis, varnish, and MariaDB. I don't have much insight : into Android internals and development process, but I would hope to see : MADV_FREE support eventually end up there as well to benefit applications : linked with the integrated jemalloc. : : jemalloc will use MADV_FREE once it becomes available in the Linux kernel. : In fact, jemalloc already uses MADV_FREE or equivalent everywhere it's : available: *BSD, OS X, Windows, and Solaris -- every platform except Linux : (and AIX, but I'm not sure it even compiles on AIX). The lack of : MADV_FREE on Linux forced me down a long series of increasingly : sophisticated heuristics for madvise() volume reduction, and even so this : remains a common performance issue for people using jemalloc on Linux. : Please integrate MADV_FREE; many people will benefit substantially. How it works: When madvise syscall is called, VM clears dirty bit of ptes of the range. If memory pressure happens, VM checks dirty bit of page table and if it found still "clean", it means it's a "lazyfree pages" so VM could discard the page instead of swapping out. Once there was store operation for the page before VM peek a page to reclaim, dirty bit is set so VM can swap out the page instead of discarding. One thing we should notice is that basically, MADV_FREE relies on dirty bit in page table entry to decide whether VM allows to discard the page or not. IOW, if page table entry includes marked dirty bit, VM shouldn't discard the page. However, as a example, if swap-in by read fault happens, page table entry doesn't have dirty bit so MADV_FREE could discard the page wrongly. For avoiding the problem, MADV_FREE did more checks with PageDirty and PageSwapCache. It worked out because swapped-in page lives on swap cache and since it is evicted from the swap cache, the page has PG_dirty flag. So both page flags check effectively prevent wrong discarding by MADV_FREE. However, a problem in above logic is that swapped-in page has PG_dirty still after they are removed from swap cache so VM cannot consider the page as freeable any more even if madvise_free is called in future. Look at below example for detail. ptr = malloc(); memset(ptr); .. .. .. heavy memory pressure so all of pages are swapped out .. .. var = *ptr; -> a page swapped-in and could be removed from swapcache. Then, page table doesn't mark dirty bit and page descriptor includes PG_dirty .. .. madvise_free(ptr); -> It doesn't clear PG_dirty of the page. .. .. .. .. heavy memory pressure again. .. In this time, VM cannot discard the page because the page .. has *PG_dirty* To solve the problem, this patch clears PG_dirty if only the page is owned exclusively by current process when madvise is called because PG_dirty represents ptes's dirtiness in several processes so we could clear it only if we own it exclusively. Firstly, heavy users would be general allocators(ex, jemalloc, tcmalloc and hope glibc supports it) and jemalloc/tcmalloc already have supported the feature for other OS(ex, FreeBSD) barrios@blaptop:~/benchmark/ebizzy$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 12 On-line CPU(s) list: 0-11 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 12 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 2 Stepping: 3 CPU MHz: 3200.185 BogoMIPS: 6400.53 Virtualization: VT-x Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 4096K NUMA node0 CPU(s): 0-11 ebizzy benchmark(./ebizzy -S 10 -n 512) Higher avg is better. vanilla-jemalloc MADV_free-jemalloc 1 thread records: 10 records: 10 avg: 2961.90 avg: 12069.70 std: 71.96(2.43%) std: 186.68(1.55%) max: 3070.00 max: 12385.00 min: 2796.00 min: 11746.00 2 thread records: 10 records: 10 avg: 5020.00 avg: 17827.00 std: 264.87(5.28%) std: 358.52(2.01%) max: 5244.00 max: 18760.00 min: 4251.00 min: 17382.00 4 thread records: 10 records: 10 avg: 8988.80 avg: 27930.80 std: 1175.33(13.08%) std: 3317.33(11.88%) max: 9508.00 max: 30879.00 min: 5477.00 min: 21024.00 8 thread records: 10 records: 10 avg: 13036.50 avg: 33739.40 std: 170.67(1.31%) std: 5146.22(15.25%) max: 13371.00 max: 40572.00 min: 12785.00 min: 24088.00 16 thread records: 10 records: 10 avg: 11092.40 avg: 31424.20 std: 710.60(6.41%) std: 3763.89(11.98%) max: 12446.00 max: 36635.00 min: 9949.00 min: 25669.00 32 thread records: 10 records: 10 avg: 11067.00 avg: 34495.80 std: 971.06(8.77%) std: 2721.36(7.89%) max: 12010.00 max: 38598.00 min: 9002.00 min: 30636.00 In summary, MADV_FREE is about much faster than MADV_DONTNEED. This patch (of 12): Add core MADV_FREE implementation. [akpm@linux-foundation.org: small cleanups] Signed-off-by: Minchan Kim <minchan@kernel.org> Acked-by: Michal Hocko <mhocko@suse.com> Acked-by: Hugh Dickins <hughd@google.com> Cc: Mika Penttil <mika.penttila@nextfour.com> Cc: Michael Kerrisk <mtk.manpages@gmail.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Rik van Riel <riel@redhat.com> Cc: Mel Gorman <mgorman@suse.de> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: Jason Evans <je@fb.com> Cc: Daniel Micay <danielmicay@gmail.com> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Shaohua Li <shli@kernel.org> Cc: <yalin.wang2010@gmail.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: "James E.J. Bottomley" <jejb@parisc-linux.org> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: "Shaohua Li" <shli@kernel.org> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Chen Gang <gang.chen.5i5j@gmail.com> Cc: Chris Zankel <chris@zankel.net> Cc: Darrick J. Wong <darrick.wong@oracle.com> Cc: David S. Miller <davem@davemloft.net> Cc: Helge Deller <deller@gmx.de> Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru> Cc: Matt Turner <mattst88@gmail.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Richard Henderson <rth@twiddle.net> Cc: Roland Dreier <roland@kernel.org> Cc: Russell King <rmk@arm.linux.org.uk> Cc: Shaohua Li <shli@kernel.org> Cc: Will Deacon <will.deacon@arm.com> Cc: Wu Fengguang <fengguang.wu@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Vladimir Davydov
|
8749cfea11 |
mm: add page_check_address_transhuge() helper
page_referenced_one() and page_idle_clear_pte_refs_one() duplicate the code for looking up pte of a (possibly transhuge) page. Move this code to a new helper function, page_check_address_transhuge(), and make the above mentioned functions use it. This is just a cleanup, no functional changes are intended. Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com> Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
53f9263bab |
mm: rework mapcount accounting to enable 4k mapping of THPs
We're going to allow mapping of individual 4k pages of THP compound. It means we need to track mapcount on per small page basis. Straight-forward approach is to use ->_mapcount in all subpages to track how many time this subpage is mapped with PMDs or PTEs combined. But this is rather expensive: mapping or unmapping of a THP page with PMD would require HPAGE_PMD_NR atomic operations instead of single we have now. The idea is to store separately how many times the page was mapped as whole -- compound_mapcount. This frees up ->_mapcount in subpages to track PTE mapcount. We use the same approach as with compound page destructor and compound order to store compound_mapcount: use space in first tail page, ->mapping this time. Any time we map/unmap whole compound page (THP or hugetlb) -- we increment/decrement compound_mapcount. When we map part of compound page with PTE we operate on ->_mapcount of the subpage. page_mapcount() counts both: PTE and PMD mappings of the page. Basically, we have mapcount for a subpage spread over two counters. It makes tricky to detect when last mapcount for a page goes away. We introduced PageDoubleMap() for this. When we split THP PMD for the first time and there's other PMD mapping left we offset up ->_mapcount in all subpages by one and set PG_double_map on the compound page. These additional references go away with last compound_mapcount. This approach provides a way to detect when last mapcount goes away on per small page basis without introducing new overhead for most common cases. [akpm@linux-foundation.org: fix typo in comment] [mhocko@suse.com: ignore partial THP when moving task] Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Acked-by: Jerome Marchand <jmarchan@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Cc: Jerome Marchand <jmarchan@redhat.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Steve Capper <steve.capper@linaro.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Signed-off-by: Michal Hocko <mhocko@suse.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
d281ee6145 |
rmap: add argument to charge compound page
We're going to allow mapping of individual 4k pages of THP compound page. It means we cannot rely on PageTransHuge() check to decide if map/unmap small page or THP. The patch adds new argument to rmap functions to indicate whether we want to operate on whole compound page or only the small page. [n-horiguchi@ah.jp.nec.com: fix mapcount mismatch in hugepage migration] Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Tested-by: Sasha Levin <sasha.levin@oracle.com> Tested-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> Acked-by: Jerome Marchand <jmarchan@redhat.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Steve Capper <steve.capper@linaro.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.cz> Cc: Christoph Lameter <cl@linux.com> Cc: David Rientjes <rientjes@google.com> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Mel Gorman
|
72b252aed5 |
mm: send one IPI per CPU to TLB flush all entries after unmapping pages
An IPI is sent to flush remote TLBs when a page is unmapped that was
potentially accesssed by other CPUs. There are many circumstances where
this happens but the obvious one is kswapd reclaiming pages belonging to a
running process as kswapd and the task are likely running on separate
CPUs.
On small machines, this is not a significant problem but as machine gets
larger with more cores and more memory, the cost of these IPIs can be
high. This patch uses a simple structure that tracks CPUs that
potentially have TLB entries for pages being unmapped. When the unmapping
is complete, the full TLB is flushed on the assumption that a refill cost
is lower than flushing individual entries.
Architectures wishing to do this must give the following guarantee.
If a clean page is unmapped and not immediately flushed, the
architecture must guarantee that a write to that linear address
from a CPU with a cached TLB entry will trap a page fault.
This is essentially what the kernel already depends on but the window is
much larger with this patch applied and is worth highlighting. The
architecture should consider whether the cost of the full TLB flush is
higher than sending an IPI to flush each individual entry. An additional
architecture helper called flush_tlb_local is required. It's a trivial
wrapper with some accounting in the x86 case.
The impact of this patch depends on the workload as measuring any benefit
requires both mapped pages co-located on the LRU and memory pressure. The
case with the biggest impact is multiple processes reading mapped pages
taken from the vm-scalability test suite. The test case uses NR_CPU
readers of mapped files that consume 10*RAM.
Linear mapped reader on a 4-node machine with 64G RAM and 48 CPUs
4.2.0-rc1 4.2.0-rc1
vanilla flushfull-v7
Ops lru-file-mmap-read-elapsed 159.62 ( 0.00%) 120.68 ( 24.40%)
Ops lru-file-mmap-read-time_range 30.59 ( 0.00%) 2.80 ( 90.85%)
Ops lru-file-mmap-read-time_stddv 6.70 ( 0.00%) 0.64 ( 90.38%)
4.2.0-rc1 4.2.0-rc1
vanilla flushfull-v7
User 581.00 611.43
System 5804.93 4111.76
Elapsed 161.03 122.12
This is showing that the readers completed 24.40% faster with 29% less
system CPU time. From vmstats, it is known that the vanilla kernel was
interrupted roughly 900K times per second during the steady phase of the
test and the patched kernel was interrupts 180K times per second.
The impact is lower on a single socket machine.
4.2.0-rc1 4.2.0-rc1
vanilla flushfull-v7
Ops lru-file-mmap-read-elapsed 25.33 ( 0.00%) 20.38 ( 19.54%)
Ops lru-file-mmap-read-time_range 0.91 ( 0.00%) 1.44 (-58.24%)
Ops lru-file-mmap-read-time_stddv 0.28 ( 0.00%) 0.47 (-65.34%)
4.2.0-rc1 4.2.0-rc1
vanilla flushfull-v7
User 58.09 57.64
System 111.82 76.56
Elapsed 27.29 22.55
It's still a noticeable improvement with vmstat showing interrupts went
from roughly 500K per second to 45K per second.
The patch will have no impact on workloads with no memory pressure or have
relatively few mapped pages. It will have an unpredictable impact on the
workload running on the CPU being flushed as it'll depend on how many TLB
entries need to be refilled and how long that takes. Worst case, the TLB
will be completely cleared of active entries when the target PFNs were not
resident at all.
[sasha.levin@oracle.com: trace tlb flush after disabling preemption in try_to_unmap_flush]
Signed-off-by: Mel Gorman <mgorman@suse.de>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
||
|
Kirill A. Shutemov
|
e39155ea11 |
mm: uninline and cleanup page-mapping related helpers
Most-used page->mapping helper -- page_mapping() -- has already uninlined.
Let's uninline also page_rmapping() and page_anon_vma(). It saves us
depending on configuration around 400 bytes in text:
text data bss dec hex filename
660318 99254 410000
|
||
|
Matthew Wilcox
|
e748dcd095 |
vfs: remove get_xip_mem
All callers of get_xip_mem() are now gone. Remove checks for it, initialisers of it, documentation of it and the only implementation of it. Also remove mm/filemap_xip.c as it is now empty. Also remove documentation of the long-gone get_xip_page(). Signed-off-by: Matthew Wilcox <matthew.r.wilcox@intel.com> Cc: Andreas Dilger <andreas.dilger@intel.com> Cc: Boaz Harrosh <boaz@plexistor.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Dave Chinner <david@fromorbit.com> Cc: Jan Kara <jack@suse.cz> Cc: Jens Axboe <axboe@kernel.dk> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Ross Zwisler <ross.zwisler@linux.intel.com> Cc: Theodore Ts'o <tytso@mit.edu> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
27ba0644ea |
rmap: drop support of non-linear mappings
We don't create non-linear mappings anymore. Let's drop code which handles them in rmap. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Konstantin Khlebnikov
|
7a3ef208e6 |
mm: prevent endless growth of anon_vma hierarchy
Constantly forking task causes unlimited grow of anon_vma chain. Each
next child allocates new level of anon_vmas and links vma to all
previous levels because pages might be inherited from any level.
This patch adds heuristic which decides to reuse existing anon_vma
instead of forking new one. It adds counter anon_vma->degree which
counts linked vmas and directly descending anon_vmas and reuses anon_vma
if counter is lower than two. As a result each anon_vma has either vma
or at least two descending anon_vmas. In such trees half of nodes are
leafs with alive vmas, thus count of anon_vmas is no more than two times
bigger than count of vmas.
This heuristic reuses anon_vmas as few as possible because each reuse
adds false aliasing among vmas and rmap walker ought to scan more ptes
when it searches where page is might be mapped.
Link: http://lkml.kernel.org/r/20120816024610.GA5350@evergreen.ssec.wisc.edu
Fixes:
|
||
|
Sasha Levin
|
81d1b09c6b |
mm: convert a few VM_BUG_ON callers to VM_BUG_ON_VMA
Trivially convert a few VM_BUG_ON calls to VM_BUG_ON_VMA to extract more information when they trigger. [akpm@linux-foundation.org: coding-style fixes] Signed-off-by: Sasha Levin <sasha.levin@oracle.com> Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Konstantin Khlebnikov <khlebnikov@openvz.org> Cc: Rik van Riel <riel@redhat.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.cz> Cc: Hugh Dickins <hughd@google.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Michel Lespinasse <walken@google.com> Cc: Minchan Kim <minchan@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Konstantin Khlebnikov
|
daa5ba768b |
mm/rmap.c: cleanup ttu_flags
Transform action part of ttu_flags into individiual bits. These flags aren't part of any uses-space visible api or even trace events. Signed-off-by: Konstantin Khlebnikov <koct9i@gmail.com> Cc: Rik van Riel <riel@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Kirill A. Shutemov
|
ac7695012a |
mm/rmap.c: make page_referenced_one() and try_to_unmap_one() static
KSM was converted to use rmap_walk() and now nobody uses these functions outside mm/rmap.c. Let's covert them back to static. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Hugh Dickins
|
7e09e738af |
mm: fix swapops.h:131 bug if remap_file_pages raced migration
Add remove_linear_migration_ptes_from_nonlinear(), to fix an interesting little include/linux/swapops.h:131 BUG_ON(!PageLocked) found by trinity: indicating that remove_migration_ptes() failed to find one of the migration entries that was temporarily inserted. The problem comes from remap_file_pages()'s switch from vma_interval_tree (good for inserting the migration entry) to i_mmap_nonlinear list (no good for locating it again); but can only be a problem if the remap_file_pages() range does not cover the whole of the vma (zap_pte() clears the range). remove_migration_ptes() needs a file_nonlinear method to go down the i_mmap_nonlinear list, applying linear location to look for migration entries in those vmas too, just in case there was this race. The file_nonlinear method does need rmap_walk_control.arg to do this; but it never needed vma passed in - vma comes from its own iteration. Reported-and-tested-by: Dave Jones <davej@redhat.com> Reported-and-tested-by: Sasha Levin <sasha.levin@oracle.com> Signed-off-by: Hugh Dickins <hughd@google.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Joonsoo Kim
|
9f32624be9 |
mm/rmap: use rmap_walk() in page_referenced()
Now, we have an infrastructure in rmap_walk() to handle difference from variants of rmap traversing functions. So, just use it in page_referenced(). In this patch, I change following things. 1. remove some variants of rmap traversing functions. cf> page_referenced_ksm, page_referenced_anon, page_referenced_file 2. introduce new struct page_referenced_arg and pass it to page_referenced_one(), main function of rmap_walk, in order to count reference, to store vm_flags and to check finish condition. 3. mechanical change to use rmap_walk() in page_referenced(). [liwanp@linux.vnet.ibm.com: fix BUG at rmap_walk] Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Hillf Danton <dhillf@gmail.com> Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com> Cc: Sasha Levin <sasha.levin@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Joonsoo Kim
|
5262950642 |
mm/rmap: use rmap_walk() in try_to_unmap()
Now, we have an infrastructure in rmap_walk() to handle difference from variants of rmap traversing functions. So, just use it in try_to_unmap(). In this patch, I change following things. 1. enable rmap_walk() if !CONFIG_MIGRATION. 2. mechanical change to use rmap_walk() in try_to_unmap(). Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Hillf Danton <dhillf@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Joonsoo Kim
|
0dd1c7bbce |
mm/rmap: extend rmap_walk_xxx() to cope with different cases
There are a lot of common parts in traversing functions, but there are also a little of uncommon parts in it. By assigning proper function pointer on each rmap_walker_control, we can handle these difference correctly. Following are differences we should handle. 1. difference of lock function in anon mapping case 2. nonlinear handling in file mapping case 3. prechecked condition: checking memcg in page_referenced(), checking VM_SHARE in page_mkclean() checking temporary vma in try_to_unmap() 4. exit condition: checking page_mapped() in try_to_unmap() So, in this patch, I introduce 4 function pointers to handle above differences. Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Hillf Danton <dhillf@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Joonsoo Kim
|
051ac83adf |
mm/rmap: make rmap_walk to get the rmap_walk_control argument
In each rmap traverse case, there is some difference so that we need function pointers and arguments to them in order to handle these For this purpose, struct rmap_walk_control is introduced in this patch, and will be extended in following patch. Introducing and extending are separate, because it clarify changes. Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Hugh Dickins <hughd@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Hillf Danton <dhillf@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Konstantin Khlebnikov
|
08b52706d5 |
mm/rmap: rename anon_vma_unlock() => anon_vma_unlock_write()
The comment in commit
|
||
|
Ingo Molnar
|
4fc3f1d66b |
mm/rmap, migration: Make rmap_walk_anon() and try_to_unmap_anon() more scalable
rmap_walk_anon() and try_to_unmap_anon() appears to be too
careful about locking the anon vma: while it needs protection
against anon vma list modifications, it does not need exclusive
access to the list itself.
Transforming this exclusive lock to a read-locked rwsem removes
a global lock from the hot path of page-migration intense
threaded workloads which can cause pathological performance like
this:
96.43% process 0 [kernel.kallsyms] [k] perf_trace_sched_switch
|
--- perf_trace_sched_switch
__schedule
schedule
schedule_preempt_disabled
__mutex_lock_common.isra.6
__mutex_lock_slowpath
mutex_lock
|
|--50.61%-- rmap_walk
| move_to_new_page
| migrate_pages
| migrate_misplaced_page
| __do_numa_page.isra.69
| handle_pte_fault
| handle_mm_fault
| __do_page_fault
| do_page_fault
| page_fault
| __memset_sse2
| |
| --100.00%-- worker_thread
| |
| --100.00%-- start_thread
|
--49.39%-- page_lock_anon_vma
try_to_unmap_anon
try_to_unmap
migrate_pages
migrate_misplaced_page
__do_numa_page.isra.69
handle_pte_fault
handle_mm_fault
__do_page_fault
do_page_fault
page_fault
__memset_sse2
|
--100.00%-- worker_thread
start_thread
With this change applied the profile is now nicely flat
and there's no anon-vma related scheduling/blocking.
Rename anon_vma_[un]lock() => anon_vma_[un]lock_write(),
to make it clearer that it's an exclusive write-lock in
that case - suggested by Rik van Riel.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Turner <pjt@google.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
|
||
|
Ingo Molnar
|
5a505085f0 |
mm/rmap: Convert the struct anon_vma::mutex to an rwsem
Convert the struct anon_vma::mutex to an rwsem, which will help in solving a page-migration scalability problem. (Addressed in a separate patch.) The conversion is simple and straightforward: in every case where we mutex_lock()ed we'll now down_write(). Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Reviewed-by: Rik van Riel <riel@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Turner <pjt@google.com> Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com> Cc: Christoph Lameter <cl@linux.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Hugh Dickins <hughd@google.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Mel Gorman <mgorman@suse.de> |
||
|
Minchan Kim
|
02c6de8d75 |
mm: cma: discard clean pages during contiguous allocation instead of migration
Drop clean cache pages instead of migration during alloc_contig_range() to minimise allocation latency by reducing the amount of migration that is necessary. It's useful for CMA because latency of migration is more important than evicting the background process's working set. In addition, as pages are reclaimed then fewer free pages for migration targets are required so it avoids memory reclaiming to get free pages, which is a contributory factor to increased latency. I measured elapsed time of __alloc_contig_migrate_range() which migrates 10M in 40M movable zone in QEMU machine. Before - 146ms, After - 7ms [akpm@linux-foundation.org: fix nommu build] Signed-off-by: Mel Gorman <mgorman@suse.de> Signed-off-by: Minchan Kim <minchan@kernel.org> Reviewed-by: Mel Gorman <mgorman@suse.de> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Michal Nazarewicz <mina86@mina86.com> Cc: Rik van Riel <riel@redhat.com> Tested-by: Kyungmin Park <kyungmin.park@samsung.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
||
|
Michel Lespinasse
|
ed8ea81501 |
mm: add CONFIG_DEBUG_VM_RB build option
Add a CONFIG_DEBUG_VM_RB build option for the previously existing DEBUG_MM_RB code. Now that Andi Kleen modified it to avoid using recursive algorithms, we can expose it a bit more. Also extend this code to validate_mm() after stack expansion, and to check that the vma's start and last pgoffs have not changed since the nodes were inserted on the anon vma interval tree (as it is important that the nodes be reindexed after each such update). Signed-off-by: Michel Lespinasse <walken@google.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Rik van Riel <riel@redhat.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Daniel Santos <daniel.santos@pobox.com> Cc: Hugh Dickins <hughd@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |