Commit Graph

154 Commits

Author SHA1 Message Date
Chao Yu
594835143a BACKPORT: f2fs: introduce F2FS_IPU_HONOR_OPU_WRITE ipu policy
Once F2FS_IPU_FORCE policy is enabled in some cases:
a) f2fs forces to use F2FS_IPU_FORCE in a small-sized volume
b) user sets F2FS_IPU_FORCE policy via sysfs

Then we may fail to defragment file due to IPU policy check, it doesn't
make sense, let's introduce a new IPU policy to allow OPU during file
defragmentation.

In small-sized volume, let's enable F2FS_IPU_HONOR_OPU_WRITE policy
by default.

Bug: 244657983

Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
(cherry picked from commit 1018a5463a063715365784704c4e8cdf2eec4b04)
Change-Id: I05dfa5a07a6a17dcda68f50a8f4a8260c2612dcc
2022-09-02 05:39:17 +00:00
Greg Kroah-Hartman
4ec3c2eea5 Merge 5.10.94 into android12-5.10-lts
Changes in 5.10.94
	KVM: VMX: switch blocked_vcpu_on_cpu_lock to raw spinlock
	HID: uhid: Fix worker destroying device without any protection
	HID: wacom: Reset expected and received contact counts at the same time
	HID: wacom: Ignore the confidence flag when a touch is removed
	HID: wacom: Avoid using stale array indicies to read contact count
	f2fs: fix to do sanity check in is_alive()
	nfc: llcp: fix NULL error pointer dereference on sendmsg() after failed bind()
	mtd: rawnand: gpmi: Add ERR007117 protection for nfc_apply_timings
	mtd: rawnand: gpmi: Remove explicit default gpmi clock setting for i.MX6
	mtd: Fixed breaking list in __mtd_del_partition.
	mtd: rawnand: davinci: Don't calculate ECC when reading page
	mtd: rawnand: davinci: Avoid duplicated page read
	mtd: rawnand: davinci: Rewrite function description
	x86/gpu: Reserve stolen memory for first integrated Intel GPU
	tools/nolibc: x86-64: Fix startup code bug
	tools/nolibc: i386: fix initial stack alignment
	tools/nolibc: fix incorrect truncation of exit code
	rtc: cmos: take rtc_lock while reading from CMOS
	media: v4l2-ioctl.c: readbuffers depends on V4L2_CAP_READWRITE
	media: flexcop-usb: fix control-message timeouts
	media: mceusb: fix control-message timeouts
	media: em28xx: fix control-message timeouts
	media: cpia2: fix control-message timeouts
	media: s2255: fix control-message timeouts
	media: dib0700: fix undefined behavior in tuner shutdown
	media: redrat3: fix control-message timeouts
	media: pvrusb2: fix control-message timeouts
	media: stk1160: fix control-message timeouts
	media: cec-pin: fix interrupt en/disable handling
	can: softing_cs: softingcs_probe(): fix memleak on registration failure
	iio: adc: ti-adc081c: Partial revert of removal of ACPI IDs
	lkdtm: Fix content of section containing lkdtm_rodata_do_nothing()
	iommu/io-pgtable-arm-v7s: Add error handle for page table allocation failure
	gpu: host1x: Add back arm_iommu_detach_device()
	dma_fence_array: Fix PENDING_ERROR leak in dma_fence_array_signaled()
	PCI: Add function 1 DMA alias quirk for Marvell 88SE9125 SATA controller
	mm_zone: add function to check if managed dma zone exists
	dma/pool: create dma atomic pool only if dma zone has managed pages
	mm/page_alloc.c: do not warn allocation failure on zone DMA if no managed pages
	shmem: fix a race between shmem_unused_huge_shrink and shmem_evict_inode
	drm/ttm: Put BO in its memory manager's lru list
	Bluetooth: L2CAP: Fix not initializing sk_peer_pid
	drm/bridge: display-connector: fix an uninitialized pointer in probe()
	drm: fix null-ptr-deref in drm_dev_init_release()
	drm/panel: kingdisplay-kd097d04: Delete panel on attach() failure
	drm/panel: innolux-p079zca: Delete panel on attach() failure
	drm/rockchip: dsi: Fix unbalanced clock on probe error
	drm/rockchip: dsi: Hold pm-runtime across bind/unbind
	drm/rockchip: dsi: Disable PLL clock on bind error
	drm/rockchip: dsi: Reconfigure hardware on resume()
	Bluetooth: cmtp: fix possible panic when cmtp_init_sockets() fails
	clk: bcm-2835: Pick the closest clock rate
	clk: bcm-2835: Remove rounding up the dividers
	drm/vc4: hdmi: Set a default HSM rate
	wcn36xx: ensure pairing of init_scan/finish_scan and start_scan/end_scan
	wcn36xx: Indicate beacon not connection loss on MISSED_BEACON_IND
	wcn36xx: Fix DMA channel enable/disable cycle
	wcn36xx: Release DMA channel descriptor allocations
	wcn36xx: Put DXE block into reset before freeing memory
	wcn36xx: populate band before determining rate on RX
	wcn36xx: fix RX BD rate mapping for 5GHz legacy rates
	ath11k: Send PPDU_STATS_CFG with proper pdev mask to firmware
	mtd: hyperbus: rpc-if: Check return value of rpcif_sw_init()
	media: videobuf2: Fix the size printk format
	media: atomisp: add missing media_device_cleanup() in atomisp_unregister_entities()
	media: atomisp: fix punit_ddr_dvfs_enable() argument for mrfld_power up case
	media: atomisp: fix inverted logic in buffers_needed()
	media: atomisp: do not use err var when checking port validity for ISP2400
	media: atomisp: fix inverted error check for ia_css_mipi_is_source_port_valid()
	media: atomisp: fix ifdefs in sh_css.c
	media: staging: media: atomisp: pci: Balance braces around conditional statements in file atomisp_cmd.c
	media: atomisp: add NULL check for asd obtained from atomisp_video_pipe
	media: atomisp: fix enum formats logic
	media: atomisp: fix uninitialized bug in gmin_get_pmic_id_and_addr()
	media: aspeed: fix mode-detect always time out at 2nd run
	media: em28xx: fix memory leak in em28xx_init_dev
	media: aspeed: Update signal status immediately to ensure sane hw state
	arm64: dts: amlogic: meson-g12: Fix GPU operating point table node name
	arm64: dts: amlogic: Fix SPI NOR flash node name for ODROID N2/N2+
	arm64: dts: meson-gxbb-wetek: fix HDMI in early boot
	arm64: dts: meson-gxbb-wetek: fix missing GPIO binding
	fs: dlm: use sk->sk_socket instead of con->sock
	fs: dlm: don't call kernel_getpeername() in error_report()
	memory: renesas-rpc-if: Return error in case devm_ioremap_resource() fails
	Bluetooth: stop proccessing malicious adv data
	ath11k: Fix ETSI regd with weather radar overlap
	ath11k: clear the keys properly via DISABLE_KEY
	ath11k: reset RSN/WPA present state for open BSS
	tee: fix put order in teedev_close_context()
	fs: dlm: fix build with CONFIG_IPV6 disabled
	drm/vboxvideo: fix a NULL vs IS_ERR() check
	arm64: dts: renesas: cat875: Add rx/tx delays
	media: dmxdev: fix UAF when dvb_register_device() fails
	crypto: qce - fix uaf on qce_ahash_register_one
	crypto: qce - fix uaf on qce_skcipher_register_one
	mtd: hyperbus: rpc-if: fix bug in rpcif_hb_remove
	ARM: dts: stm32: fix dtbs_check warning on ili9341 dts binding on stm32f429 disco
	crypto: qat - fix spelling mistake: "messge" -> "message"
	crypto: qat - remove unnecessary collision prevention step in PFVF
	crypto: qat - make pfvf send message direction agnostic
	crypto: qat - fix undetected PFVF timeout in ACK loop
	ath11k: Use host CE parameters for CE interrupts configuration
	arm64: dts: ti: k3-j721e: correct cache-sets info
	tty: serial: atmel: Check return code of dmaengine_submit()
	tty: serial: atmel: Call dma_async_issue_pending()
	mfd: atmel-flexcom: Remove #ifdef CONFIG_PM_SLEEP
	mfd: atmel-flexcom: Use .resume_noirq
	media: rcar-csi2: Correct the selection of hsfreqrange
	media: imx-pxp: Initialize the spinlock prior to using it
	media: si470x-i2c: fix possible memory leak in si470x_i2c_probe()
	media: mtk-vcodec: call v4l2_m2m_ctx_release first when file is released
	media: coda: fix CODA960 JPEG encoder buffer overflow
	media: venus: pm_helpers: Control core power domain manually
	media: venus: core, venc, vdec: Fix probe dependency error
	media: venus: core: Fix a potential NULL pointer dereference in an error handling path
	media: venus: core: Fix a resource leak in the error handling path of 'venus_probe()'
	thermal/drivers/imx: Implement runtime PM support
	netfilter: bridge: add support for pppoe filtering
	arm64: dts: qcom: msm8916: fix MMC controller aliases
	cgroup: Trace event cgroup id fields should be u64
	ACPI: EC: Rework flushing of EC work while suspended to idle
	thermal/drivers/imx8mm: Enable ADC when enabling monitor
	drm/amdgpu: Fix a NULL pointer dereference in amdgpu_connector_lcd_native_mode()
	drm/radeon/radeon_kms: Fix a NULL pointer dereference in radeon_driver_open_kms()
	arm64: dts: ti: k3-j7200: Fix the L2 cache sets
	arm64: dts: ti: k3-j721e: Fix the L2 cache sets
	arm64: dts: ti: k3-j7200: Correct the d-cache-sets info
	tty: serial: uartlite: allow 64 bit address
	serial: amba-pl011: do not request memory region twice
	floppy: Fix hang in watchdog when disk is ejected
	staging: rtl8192e: return error code from rtllib_softmac_init()
	staging: rtl8192e: rtllib_module: fix error handle case in alloc_rtllib()
	Bluetooth: btmtksdio: fix resume failure
	sched/fair: Fix detection of per-CPU kthreads waking a task
	sched/fair: Fix per-CPU kthread and wakee stacking for asym CPU capacity
	bpf: Adjust BTF log size limit.
	bpf: Disallow BPF_LOG_KERNEL log level for bpf(BPF_BTF_LOAD)
	bpf: Remove config check to enable bpf support for branch records
	arm64: lib: Annotate {clear, copy}_page() as position-independent
	arm64: clear_page() shouldn't use DC ZVA when DCZID_EL0.DZP == 1
	media: dib8000: Fix a memleak in dib8000_init()
	media: saa7146: mxb: Fix a NULL pointer dereference in mxb_attach()
	media: si2157: Fix "warm" tuner state detection
	wireless: iwlwifi: Fix a double free in iwl_txq_dyn_alloc_dma
	sched/rt: Try to restart rt period timer when rt runtime exceeded
	drm/msm/dp: displayPort driver need algorithm rational
	rcu/exp: Mark current CPU as exp-QS in IPI loop second pass
	mwifiex: Fix possible ABBA deadlock
	xfrm: fix a small bug in xfrm_sa_len()
	x86/uaccess: Move variable into switch case statement
	selftests: clone3: clone3: add case CLONE3_ARGS_NO_TEST
	selftests: harness: avoid false negatives if test has no ASSERTs
	crypto: stm32 - Fix last sparse warning in stm32_cryp_check_ctr_counter
	crypto: stm32/cryp - fix CTR counter carry
	crypto: stm32/cryp - fix xts and race condition in crypto_engine requests
	crypto: stm32/cryp - check early input data
	crypto: stm32/cryp - fix double pm exit
	crypto: stm32/cryp - fix lrw chaining mode
	crypto: stm32/cryp - fix bugs and crash in tests
	crypto: stm32 - Revert broken pm_runtime_resume_and_get changes
	ath11k: Fix deleting uninitialized kernel timer during fragment cache flush
	ARM: dts: gemini: NAS4220-B: fis-index-block with 128 KiB sectors
	media: dw2102: Fix use after free
	media: msi001: fix possible null-ptr-deref in msi001_probe()
	media: coda/imx-vdoa: Handle dma_set_coherent_mask error codes
	ath11k: Fix a NULL pointer dereference in ath11k_mac_op_hw_scan()
	arm64: dts: qcom: c630: Fix soundcard setup
	arm64: dts: qcom: ipq6018: Fix gpio-ranges property
	drm/msm/dpu: fix safe status debugfs file
	drm/bridge: ti-sn65dsi86: Set max register for regmap
	drm/tegra: vic: Fix DMA API misuse
	media: hantro: Fix probe func error path
	xfrm: interface with if_id 0 should return error
	xfrm: state and policy should fail if XFRMA_IF_ID 0
	ARM: 9159/1: decompressor: Avoid UNPREDICTABLE NOP encoding
	usb: ftdi-elan: fix memory leak on device disconnect
	arm64: dts: marvell: cn9130: add GPIO and SPI aliases
	arm64: dts: marvell: cn9130: enable CP0 GPIO controllers
	ARM: dts: armada-38x: Add generic compatible to UART nodes
	iwlwifi: mvm: fix 32-bit build in FTM
	iwlwifi: mvm: test roc running status bits before removing the sta
	mmc: meson-mx-sdhc: add IRQ check
	mmc: meson-mx-sdio: add IRQ check
	selinux: fix potential memleak in selinux_add_opt()
	um: fix ndelay/udelay defines
	um: virtio_uml: Fix time-travel external time propagation
	Bluetooth: L2CAP: Fix using wrong mode
	bpftool: Enable line buffering for stdout
	backlight: qcom-wled: Validate enabled string indices in DT
	backlight: qcom-wled: Pass number of elements to read to read_u32_array
	backlight: qcom-wled: Fix off-by-one maximum with default num_strings
	backlight: qcom-wled: Override default length with qcom,enabled-strings
	backlight: qcom-wled: Use cpu_to_le16 macro to perform conversion
	backlight: qcom-wled: Respect enabled-strings in set_brightness
	software node: fix wrong node passed to find nargs_prop
	Bluetooth: hci_qca: Stop IBS timer during BT OFF
	x86/boot/compressed: Move CLANG_FLAGS to beginning of KBUILD_CFLAGS
	hwmon: (mr75203) fix wrong power-up delay value
	x86/mce/inject: Avoid out-of-bounds write when setting flags
	ACPI: scan: Create platform device for BCM4752 and LNV4752 ACPI nodes
	pcmcia: rsrc_nonstatic: Fix a NULL pointer dereference in __nonstatic_find_io_region()
	pcmcia: rsrc_nonstatic: Fix a NULL pointer dereference in nonstatic_find_mem_region()
	power: reset: mt6397: Check for null res pointer
	netfilter: ipt_CLUSTERIP: fix refcount leak in clusterip_tg_check()
	bpf: Don't promote bogus looking registers after null check.
	bpf: Fix SO_RCVBUF/SO_SNDBUF handling in _bpf_setsockopt().
	netfilter: nft_set_pipapo: allocate pcpu scratch maps on clone
	ppp: ensure minimum packet size in ppp_write()
	rocker: fix a sleeping in atomic bug
	staging: greybus: audio: Check null pointer
	fsl/fman: Check for null pointer after calling devm_ioremap
	Bluetooth: hci_bcm: Check for error irq
	Bluetooth: hci_qca: Fix NULL vs IS_ERR_OR_NULL check in qca_serdev_probe
	usb: dwc3: qcom: Fix NULL vs IS_ERR checking in dwc3_qcom_probe
	HID: hid-uclogic-params: Invalid parameter check in uclogic_params_init
	HID: hid-uclogic-params: Invalid parameter check in uclogic_params_get_str_desc
	HID: hid-uclogic-params: Invalid parameter check in uclogic_params_huion_init
	HID: hid-uclogic-params: Invalid parameter check in uclogic_params_frame_init_v1_buttonpad
	debugfs: lockdown: Allow reading debugfs files that are not world readable
	net/mlx5e: Fix page DMA map/unmap attributes
	net/mlx5e: Don't block routes with nexthop objects in SW
	Revert "net/mlx5e: Block offload of outer header csum for UDP tunnels"
	net/mlx5: Set command entry semaphore up once got index free
	lib/mpi: Add the return value check of kcalloc()
	Bluetooth: L2CAP: uninitialized variables in l2cap_sock_setsockopt()
	spi: spi-meson-spifc: Add missing pm_runtime_disable() in meson_spifc_probe
	ax25: uninitialized variable in ax25_setsockopt()
	netrom: fix api breakage in nr_setsockopt()
	regmap: Call regmap_debugfs_exit() prior to _init()
	can: mcp251xfd: add missing newline to printed strings
	tpm: add request_locality before write TPM_INT_ENABLE
	tpm_tis: Fix an error handling path in 'tpm_tis_core_init()'
	can: softing: softing_startstop(): fix set but not used variable warning
	can: xilinx_can: xcan_probe(): check for error irq
	pcmcia: fix setting of kthread task states
	iwlwifi: mvm: Use div_s64 instead of do_div in iwl_mvm_ftm_rtt_smoothing()
	net: mcs7830: handle usb read errors properly
	ext4: avoid trim error on fs with small groups
	ALSA: jack: Add missing rwsem around snd_ctl_remove() calls
	ALSA: PCM: Add missing rwsem around snd_ctl_remove() calls
	ALSA: hda: Add missing rwsem around snd_ctl_remove() calls
	RDMA/bnxt_re: Scan the whole bitmap when checking if "disabling RCFW with pending cmd-bit"
	RDMA/hns: Validate the pkey index
	scsi: pm80xx: Update WARN_ON check in pm8001_mpi_build_cmd()
	clk: imx8mn: Fix imx8mn_clko1_sels
	powerpc/prom_init: Fix improper check of prom_getprop()
	ASoC: uniphier: drop selecting non-existing SND_SOC_UNIPHIER_AIO_DMA
	dt-bindings: thermal: Fix definition of cooling-maps contribution property
	powerpc/64s: Convert some cpu_setup() and cpu_restore() functions to C
	powerpc/perf: MMCR0 control for PMU registers under PMCC=00
	powerpc/perf: move perf irq/nmi handling details into traps.c
	powerpc/irq: Add helper to set regs->softe
	powerpc/perf: Fix PMU callbacks to clear pending PMI before resetting an overflown PMC
	powerpc/32s: Fix shift-out-of-bounds in KASAN init
	clocksource: Reduce clocksource-skew threshold
	clocksource: Avoid accidental unstable marking of clocksources
	ALSA: oss: fix compile error when OSS_DEBUG is enabled
	ALSA: usb-audio: Drop superfluous '0' in Presonus Studio 1810c's ID
	char/mwave: Adjust io port register size
	binder: fix handling of error during copy
	openrisc: Add clone3 ABI wrapper
	iommu/io-pgtable-arm: Fix table descriptor paddr formatting
	scsi: ufs: Fix race conditions related to driver data
	RDMA/qedr: Fix reporting max_{send/recv}_wr attrs
	PCI/MSI: Fix pci_irq_vector()/pci_irq_get_affinity()
	powerpc/powermac: Add additional missing lockdep_register_key()
	RDMA/core: Let ib_find_gid() continue search even after empty entry
	RDMA/cma: Let cma_resolve_ib_dev() continue search even after empty entry
	ASoC: rt5663: Handle device_property_read_u32_array error codes
	of: unittest: fix warning on PowerPC frame size warning
	of: unittest: 64 bit dma address test requires arch support
	clk: stm32: Fix ltdc's clock turn off by clk_disable_unused() after system enter shell
	mips: add SYS_HAS_CPU_MIPS64_R5 config for MIPS Release 5 support
	mips: fix Kconfig reference to PHYS_ADDR_T_64BIT
	dmaengine: pxa/mmp: stop referencing config->slave_id
	iommu/amd: Remove iommu_init_ga()
	iommu/amd: Restore GA log/tail pointer on host resume
	ASoC: Intel: catpt: Test dmaengine_submit() result before moving on
	iommu/iova: Fix race between FQ timeout and teardown
	scsi: block: pm: Always set request queue runtime active in blk_post_runtime_resume()
	phy: uniphier-usb3ss: fix unintended writing zeros to PHY register
	ASoC: mediatek: Check for error clk pointer
	ASoC: samsung: idma: Check of ioremap return value
	misc: lattice-ecp3-config: Fix task hung when firmware load failed
	counter: stm32-lptimer-cnt: remove iio counter abi
	arm64: tegra: Fix Tegra194 HDA {clock,reset}-names ordering
	arm64: tegra: Remove non existent Tegra194 reset
	mips: lantiq: add support for clk_set_parent()
	mips: bcm63xx: add support for clk_set_parent()
	powerpc/xive: Add missing null check after calling kmalloc
	ASoC: fsl_mqs: fix MODULE_ALIAS
	RDMA/cxgb4: Set queue pair state when being queried
	ASoC: fsl_asrc: refine the check of available clock divider
	clk: bm1880: remove kfrees on static allocations
	of: base: Fix phandle argument length mismatch error message
	ARM: dts: omap3-n900: Fix lp5523 for multi color
	Bluetooth: Fix debugfs entry leak in hci_register_dev()
	fs: dlm: filter user dlm messages for kernel locks
	drm/lima: fix warning when CONFIG_DEBUG_SG=y & CONFIG_DMA_API_DEBUG=y
	selftests/bpf: Fix bpf_object leak in skb_ctx selftest
	ar5523: Fix null-ptr-deref with unexpected WDCMSG_TARGET_START reply
	drm/bridge: dw-hdmi: handle ELD when DRM_BRIDGE_ATTACH_NO_CONNECTOR
	drm/nouveau/pmu/gm200-: avoid touching PMU outside of DEVINIT/PREOS/ACR
	media: atomisp: fix try_fmt logic
	media: atomisp: set per-device's default mode
	media: atomisp-ov2680: Fix ov2680_set_fmt() clobbering the exposure
	ARM: shmobile: rcar-gen2: Add missing of_node_put()
	batman-adv: allow netlink usage in unprivileged containers
	media: atomisp: handle errors at sh_css_create_isp_params()
	ath11k: Fix crash caused by uninitialized TX ring
	usb: gadget: f_fs: Use stream_open() for endpoint files
	drm: panel-orientation-quirks: Add quirk for the Lenovo Yoga Book X91F/L
	HID: apple: Do not reset quirks when the Fn key is not found
	media: b2c2: Add missing check in flexcop_pci_isr:
	EDAC/synopsys: Use the quirk for version instead of ddr version
	ARM: imx: rename DEBUG_IMX21_IMX27_UART to DEBUG_IMX27_UART
	drm/amd/display: check top_pipe_to_program pointer
	drm/amdgpu/display: set vblank_disable_immediate for DC
	soc: ti: pruss: fix referenced node in error message
	mlxsw: pci: Add shutdown method in PCI driver
	drm/bridge: megachips: Ensure both bridges are probed before registration
	tty: serial: imx: disable UCR4_OREN in .stop_rx() instead of .shutdown()
	gpiolib: acpi: Do not set the IRQ type if the IRQ is already in use
	HSI: core: Fix return freed object in hsi_new_client
	crypto: jitter - consider 32 LSB for APT
	mwifiex: Fix skb_over_panic in mwifiex_usb_recv()
	rsi: Fix use-after-free in rsi_rx_done_handler()
	rsi: Fix out-of-bounds read in rsi_read_pkt()
	ath11k: Avoid NULL ptr access during mgmt tx cleanup
	media: venus: avoid calling core_clk_setrate() concurrently during concurrent video sessions
	ACPI / x86: Drop PWM2 device on Lenovo Yoga Book from always present table
	ACPI: Change acpi_device_always_present() into acpi_device_override_status()
	ACPI / x86: Allow specifying acpi_device_override_status() quirks by path
	ACPI / x86: Add not-present quirk for the PCI0.SDHB.BRC1 device on the GPD win
	arm64: dts: ti: j7200-main: Fix 'dtbs_check' serdes_ln_ctrl node
	usb: uhci: add aspeed ast2600 uhci support
	floppy: Add max size check for user space request
	x86/mm: Flush global TLB when switching to trampoline page-table
	drm: rcar-du: Fix CRTC timings when CMM is used
	media: uvcvideo: Increase UVC_CTRL_CONTROL_TIMEOUT to 5 seconds.
	media: rcar-vin: Update format alignment constraints
	media: saa7146: hexium_orion: Fix a NULL pointer dereference in hexium_attach()
	media: m920x: don't use stack on USB reads
	thunderbolt: Runtime PM activate both ends of the device link
	iwlwifi: mvm: synchronize with FW after multicast commands
	iwlwifi: mvm: avoid clearing a just saved session protection id
	ath11k: avoid deadlock by change ieee80211_queue_work for regd_update_work
	ath10k: Fix tx hanging
	net-sysfs: update the queue counts in the unregistration path
	net: phy: prefer 1000baseT over 1000baseKX
	gpio: aspeed: Convert aspeed_gpio.lock to raw_spinlock
	selftests/ftrace: make kprobe profile testcase description unique
	ath11k: Avoid false DEADLOCK warning reported by lockdep
	x86/mce: Allow instrumentation during task work queueing
	x86/mce: Mark mce_panic() noinstr
	x86/mce: Mark mce_end() noinstr
	x86/mce: Mark mce_read_aux() noinstr
	net: bonding: debug: avoid printing debug logs when bond is not notifying peers
	bpf: Do not WARN in bpf_warn_invalid_xdp_action()
	HID: quirks: Allow inverting the absolute X/Y values
	media: igorplugusb: receiver overflow should be reported
	media: saa7146: hexium_gemini: Fix a NULL pointer dereference in hexium_attach()
	mmc: core: Fixup storing of OCR for MMC_QUIRK_NONSTD_SDIO
	audit: ensure userspace is penalized the same as the kernel when under pressure
	arm64: dts: ls1028a-qds: move rtc node to the correct i2c bus
	arm64: tegra: Adjust length of CCPLEX cluster MMIO region
	PM: runtime: Add safety net to supplier device release
	cpufreq: Fix initialization of min and max frequency QoS requests
	usb: hub: Add delay for SuperSpeed hub resume to let links transit to U0
	ath9k: Fix out-of-bound memcpy in ath9k_hif_usb_rx_stream
	rtw88: 8822c: update rx settings to prevent potential hw deadlock
	PM: AVS: qcom-cpr: Use div64_ul instead of do_div
	iwlwifi: fix leaks/bad data after failed firmware load
	iwlwifi: remove module loading failure message
	iwlwifi: mvm: Fix calculation of frame length
	iwlwifi: pcie: make sure prph_info is set when treating wakeup IRQ
	um: registers: Rename function names to avoid conflicts and build problems
	ath11k: Fix napi related hang
	Bluetooth: vhci: Set HCI_QUIRK_VALID_LE_STATES
	xfrm: rate limit SA mapping change message to user space
	drm/etnaviv: consider completed fence seqno in hang check
	jffs2: GC deadlock reading a page that is used in jffs2_write_begin()
	ACPICA: actypes.h: Expand the ACPI_ACCESS_ definitions
	ACPICA: Utilities: Avoid deleting the same object twice in a row
	ACPICA: Executer: Fix the REFCLASS_REFOF case in acpi_ex_opcode_1A_0T_1R()
	ACPICA: Fix wrong interpretation of PCC address
	ACPICA: Hardware: Do not flush CPU cache when entering S4 and S5
	drm/amdgpu: fixup bad vram size on gmc v8
	amdgpu/pm: Make sysfs pm attributes as read-only for VFs
	ACPI: battery: Add the ThinkPad "Not Charging" quirk
	btrfs: remove BUG_ON() in find_parent_nodes()
	btrfs: remove BUG_ON(!eie) in find_parent_nodes
	net: mdio: Demote probed message to debug print
	mac80211: allow non-standard VHT MCS-10/11
	dm btree: add a defensive bounds check to insert_at()
	dm space map common: add bounds check to sm_ll_lookup_bitmap()
	mlxsw: pci: Avoid flow control for EMAD packets
	net: phy: marvell: configure RGMII delays for 88E1118
	net: gemini: allow any RGMII interface mode
	regulator: qcom_smd: Align probe function with rpmh-regulator
	serial: pl010: Drop CR register reset on set_termios
	serial: core: Keep mctrl register state and cached copy in sync
	random: do not throw away excess input to crng_fast_load
	parisc: Avoid calling faulthandler_disabled() twice
	x86/kbuild: Enable CONFIG_KALLSYMS_ALL=y in the defconfigs
	powerpc/6xx: add missing of_node_put
	powerpc/powernv: add missing of_node_put
	powerpc/cell: add missing of_node_put
	powerpc/btext: add missing of_node_put
	powerpc/watchdog: Fix missed watchdog reset due to memory ordering race
	i2c: i801: Don't silently correct invalid transfer size
	powerpc/smp: Move setup_profiling_timer() under CONFIG_PROFILING
	i2c: mpc: Correct I2C reset procedure
	clk: meson: gxbb: Fix the SDM_EN bit for MPLL0 on GXBB
	powerpc/powermac: Add missing lockdep_register_key()
	KVM: PPC: Book3S: Suppress warnings when allocating too big memory slots
	KVM: PPC: Book3S: Suppress failed alloc warning in H_COPY_TOFROM_GUEST
	w1: Misuse of get_user()/put_user() reported by sparse
	nvmem: core: set size for sysfs bin file
	dm: fix alloc_dax error handling in alloc_dev
	scsi: lpfc: Trigger SLI4 firmware dump before doing driver cleanup
	ALSA: seq: Set upper limit of processed events
	MIPS: Loongson64: Use three arguments for slti
	powerpc/40x: Map 32Mbytes of memory at startup
	selftests/powerpc/spectre_v2: Return skip code when miss_percent is high
	powerpc: handle kdump appropriately with crash_kexec_post_notifiers option
	powerpc/fadump: Fix inaccurate CPU state info in vmcore generated with panic
	udf: Fix error handling in udf_new_inode()
	MIPS: OCTEON: add put_device() after of_find_device_by_node()
	irqchip/gic-v4: Disable redistributors' view of the VPE table at boot time
	i2c: designware-pci: Fix to change data types of hcnt and lcnt parameters
	MIPS: Octeon: Fix build errors using clang
	scsi: sr: Don't use GFP_DMA
	ASoC: mediatek: mt8173: fix device_node leak
	ASoC: mediatek: mt8183: fix device_node leak
	phy: mediatek: Fix missing check in mtk_mipi_tx_probe
	rpmsg: core: Clean up resources on announce_create failure.
	crypto: omap-aes - Fix broken pm_runtime_and_get() usage
	crypto: stm32/crc32 - Fix kernel BUG triggered in probe()
	crypto: caam - replace this_cpu_ptr with raw_cpu_ptr
	ubifs: Error path in ubifs_remount_rw() seems to wrongly free write buffers
	tpm: fix NPE on probe for missing device
	spi: uniphier: Fix a bug that doesn't point to private data correctly
	xen/gntdev: fix unmap notification order
	fuse: Pass correct lend value to filemap_write_and_wait_range()
	serial: Fix incorrect rs485 polarity on uart open
	cputime, cpuacct: Include guest time in user time in cpuacct.stat
	tracing/kprobes: 'nmissed' not showed correctly for kretprobe
	iwlwifi: mvm: Increase the scan timeout guard to 30 seconds
	s390/mm: fix 2KB pgtable release race
	device property: Fix fwnode_graph_devcon_match() fwnode leak
	drm/etnaviv: limit submit sizes
	drm/nouveau/kms/nv04: use vzalloc for nv04_display
	drm/bridge: analogix_dp: Make PSR-exit block less
	parisc: Fix lpa and lpa_user defines
	powerpc/64s/radix: Fix huge vmap false positive
	PCI: xgene: Fix IB window setup
	PCI: pciehp: Use down_read/write_nested(reset_lock) to fix lockdep errors
	PCI: pci-bridge-emul: Make expansion ROM Base Address register read-only
	PCI: pci-bridge-emul: Properly mark reserved PCIe bits in PCI config space
	PCI: pci-bridge-emul: Fix definitions of reserved bits
	PCI: pci-bridge-emul: Correctly set PCIe capabilities
	PCI: pci-bridge-emul: Set PCI_STATUS_CAP_LIST for PCIe device
	xfrm: fix policy lookup for ipv6 gre packets
	btrfs: fix deadlock between quota enable and other quota operations
	btrfs: check the root node for uptodate before returning it
	btrfs: respect the max size in the header when activating swap file
	ext4: make sure to reset inode lockdep class when quota enabling fails
	ext4: make sure quota gets properly shutdown on error
	ext4: fix a possible ABBA deadlock due to busy PA
	ext4: initialize err_blk before calling __ext4_get_inode_loc
	ext4: fix fast commit may miss tracking range for FALLOC_FL_ZERO_RANGE
	ext4: set csum seed in tmp inode while migrating to extents
	ext4: Fix BUG_ON in ext4_bread when write quota data
	ext4: use ext4_ext_remove_space() for fast commit replay delete range
	ext4: fast commit may miss tracking unwritten range during ftruncate
	ext4: destroy ext4_fc_dentry_cachep kmemcache on module removal
	ext4: fix null-ptr-deref in '__ext4_journal_ensure_credits'
	ext4: don't use the orphan list when migrating an inode
	drm/radeon: fix error handling in radeon_driver_open_kms
	of: base: Improve argument length mismatch error
	firmware: Update Kconfig help text for Google firmware
	can: mcp251xfd: mcp251xfd_tef_obj_read(): fix typo in error message
	media: rcar-csi2: Optimize the selection PHTW register
	drm/vc4: hdmi: Make sure the device is powered with CEC
	media: correct MEDIA_TEST_SUPPORT help text
	Documentation: dmaengine: Correctly describe dmatest with channel unset
	Documentation: ACPI: Fix data node reference documentation
	Documentation: refer to config RANDOMIZE_BASE for kernel address-space randomization
	Documentation: fix firewire.rst ABI file path error
	Bluetooth: hci_sync: Fix not setting adv set duration
	scsi: core: Show SCMD_LAST in text form
	dmaengine: uniphier-xdmac: Fix type of address variables
	RDMA/hns: Modify the mapping attribute of doorbell to device
	RDMA/rxe: Fix a typo in opcode name
	dmaengine: stm32-mdma: fix STM32_MDMA_CTBR_TSEL_MASK
	Revert "net/mlx5: Add retry mechanism to the command entry index allocation"
	powerpc/cell: Fix clang -Wimplicit-fallthrough warning
	powerpc/fsl/dts: Enable WA for erratum A-009885 on fman3l MDIO buses
	block: Fix fsync always failed if once failed
	bpftool: Remove inclusion of utilities.mak from Makefiles
	xdp: check prog type before updating BPF link
	perf evsel: Override attr->sample_period for non-libpfm4 events
	ipv4: update fib_info_cnt under spinlock protection
	ipv4: avoid quadratic behavior in netns dismantle
	net/fsl: xgmac_mdio: Add workaround for erratum A-009885
	net/fsl: xgmac_mdio: Fix incorrect iounmap when removing module
	parisc: pdc_stable: Fix memory leak in pdcs_register_pathentries
	f2fs: compress: fix potential deadlock of compress file
	f2fs: fix to reserve space for IO align feature
	af_unix: annote lockless accesses to unix_tot_inflight & gc_in_progress
	clk: Emit a stern warning with writable debugfs enabled
	clk: si5341: Fix clock HW provider cleanup
	net/smc: Fix hung_task when removing SMC-R devices
	net: axienet: increase reset timeout
	net: axienet: Wait for PhyRstCmplt after core reset
	net: axienet: reset core on initialization prior to MDIO access
	net: axienet: add missing memory barriers
	net: axienet: limit minimum TX ring size
	net: axienet: Fix TX ring slot available check
	net: axienet: fix number of TX ring slots for available check
	net: axienet: fix for TX busy handling
	net: axienet: increase default TX ring size to 128
	HID: vivaldi: fix handling devices not using numbered reports
	rtc: pxa: fix null pointer dereference
	vdpa/mlx5: Fix wrong configuration of virtio_version_1_0
	virtio_ring: mark ring unused on error
	taskstats: Cleanup the use of task->exit_code
	inet: frags: annotate races around fqdir->dead and fqdir->high_thresh
	netns: add schedule point in ops_exit_list()
	xfrm: Don't accidentally set RTO_ONLINK in decode_session4()
	gre: Don't accidentally set RTO_ONLINK in gre_fill_metadata_dst()
	libcxgb: Don't accidentally set RTO_ONLINK in cxgb_find_route()
	perf script: Fix hex dump character output
	dmaengine: at_xdmac: Don't start transactions at tx_submit level
	dmaengine: at_xdmac: Start transfer for cyclic channels in issue_pending
	dmaengine: at_xdmac: Print debug message after realeasing the lock
	dmaengine: at_xdmac: Fix concurrency over xfers_list
	dmaengine: at_xdmac: Fix lld view setting
	dmaengine: at_xdmac: Fix at_xdmac_lld struct definition
	perf probe: Fix ppc64 'perf probe add events failed' case
	devlink: Remove misleading internal_flags from health reporter dump
	arm64: dts: qcom: msm8996: drop not documented adreno properties
	net: bonding: fix bond_xmit_broadcast return value error bug
	net_sched: restore "mpu xxx" handling
	bcmgenet: add WOL IRQ check
	net: ethernet: mtk_eth_soc: fix error checking in mtk_mac_config()
	net: sfp: fix high power modules without diagnostic monitoring
	net: mscc: ocelot: fix using match before it is set
	dt-bindings: display: meson-dw-hdmi: add missing sound-name-prefix property
	dt-bindings: display: meson-vpu: Add missing amlogic,canvas property
	dt-bindings: watchdog: Require samsung,syscon-phandle for Exynos7
	scripts/dtc: dtx_diff: remove broken example from help text
	lib82596: Fix IRQ check in sni_82596_probe
	mm/hmm.c: allow VM_MIXEDMAP to work with hmm_range_fault
	lib/test_meminit: destroy cache in kmem_cache_alloc_bulk() test
	mtd: nand: bbt: Fix corner case in bad block table handling
	ath10k: Fix the MTU size on QCA9377 SDIO
	scripts: sphinx-pre-install: add required ctex dependency
	scripts: sphinx-pre-install: Fix ctex support on Debian
	Linux 5.10.94

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I857f2417c899508815a1ba13d1285fd400a1f133
2022-01-27 11:49:22 +01:00
Chao Yu
a49e402f23 f2fs: fix to reserve space for IO align feature
commit 300a842937fbcfb5a189cea9ba15374fdb0b5c6b upstream.

https://bugzilla.kernel.org/show_bug.cgi?id=204137

With below script, we will hit panic during new segment allocation:

DISK=bingo.img
MOUNT_DIR=/mnt/f2fs

dd if=/dev/zero of=$DISK bs=1M count=105
mkfs.f2fe -a 1 -o 19 -t 1 -z 1 -f -q $DISK

mount -t f2fs $DISK $MOUNT_DIR -o "noinline_dentry,flush_merge,noextent_cache,mode=lfs,io_bits=7,fsync_mode=strict"

for (( i = 0; i < 4096; i++ )); do
	name=`head /dev/urandom | tr -dc A-Za-z0-9 | head -c 10`
	mkdir $MOUNT_DIR/$name
done

umount $MOUNT_DIR
rm $DISK
2022-01-27 10:54:31 +01:00
Chao Yu
105155a814 f2fs: fix to avoid touching checkpointed data in get_victim()
[ Upstream commit 61461fc921b756ae16e64243f72af2bfc2e620db ]

In CP disabling mode, there are two issues when using LFS or SSR | AT_SSR
mode to select victim:

1. LFS is set to find source section during GC, the victim should have
no checkpointed data, since after GC, section could not be set free for
reuse.

Previously, we only check valid chpt blocks in current segment rather
than section, fix it.

2. SSR | AT_SSR are set to find target segment for writes which can be
fully filled by checkpointed and newly written blocks, we should never
select such segment, otherwise it can cause panic or data corruption
during allocation, potential case is described as below:

 a) target segment has 'n' (n < 512) ckpt valid blocks
 b) GC migrates 'n' valid blocks to other segment (segment is still
    in dirty list)
 c) GC migrates '512 - n' blocks to target segment (segment has 'n'
    cp_vblocks and '512 - n' vblocks)
 d) If GC selects target segment via {AT,}SSR allocator, however there
    is no free space in targe segment.

Fixes: 4354994f09 ("f2fs: checkpoint disabling")
Fixes: 093749e296 ("f2fs: support age threshold based garbage collection")
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-05-19 10:13:02 +02:00
Chao Yu
adbeec301c f2fs: fix to avoid touching checkpointed data in get_victim()
In CP disabling mode, there are two issues when using LFS or SSR | AT_SSR
mode to select victim:

1. LFS is set to find source section during GC, the victim should have
no checkpointed data, since after GC, section could not be set free for
reuse.

Previously, we only check valid chpt blocks in current segment rather
than section, fix it.

2. SSR | AT_SSR are set to find target segment for writes which can be
fully filled by checkpointed and newly written blocks, we should never
select such segment, otherwise it can cause panic or data corruption
during allocation, potential case is described as below:

 a) target segment has 128 ckpt valid blocks
 b) GC migrates 'n' (n < 512) valid blocks to other segment (segment is
    still in dirty list)
 c) GC migrates '512 - n' blocks to target segment (segment has 'n'
    cp_vblocks and '512 - n' vblocks)
 d) If GC selects target segment via {AT,}SSR allocator, however there
    is no free space in targe segment.

Fixes: 4354994f09 ("f2fs: checkpoint disabling")
Fixes: 093749e296 ("f2fs: support age threshold based garbage collection")
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2021-03-25 18:26:16 -07:00
Chao Yu
14d0fdac3c f2fs: remove unused FORCE_FG_GC macro
FORCE_FG_GC was introduced by commit 6aefd93b01 ("f2fs: introduce
background_gc=sync mount option"), but never be used, remove it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2021-03-24 15:50:09 -07:00
Jaegeuk Kim
c86df2b84b f2fs: handle unallocated section and zone on pinned/atgc
[ Upstream commit 632faca72938f9f63049e48a8c438913828ac7a9 ]

If we have large section/zone, unallocated segment makes them corrupted.

E.g.,

  - Pinned file:       -1 119304647 119304647
  - ATGC   data:       -1 119304647 119304647

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-03-07 12:34:12 +01:00
Jaegeuk Kim
4ff2fc7ee7 f2fs: handle unallocated section and zone on pinned/atgc
If we have large section/zone, unallocated segment makes them corrupted.

E.g.,

  - Pinned file:       -1 119304647 119304647
  - ATGC   data:       -1 119304647 119304647

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2021-01-06 12:50:51 -08:00
Wang Xiaojun
f99ba9add6 f2fs: fix wrong total_sections check and fsmeta check
Meta area is not included in section_count computation.
So the minimum number of total_sections is 1 meanwhile it cannot be
greater than segment_count_main.

The minimum number of meta segments is 8 (SB + 2 (CP + SIT + NAT) + SSA).

Signed-off-by: Wang Xiaojun <wangxiaojun11@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-09-29 01:48:34 -07:00
Xiaojun Wang
4470eb2873 f2fs: change return value of reserved_segments to unsigned int
The type of SM_I(sbi)->reserved_segments is unsigned int,
so change the return value to unsigned int.
The type cast can be removed in reserved_sections as a result.

Signed-off-by: Xiaojun Wang <wangxiaojun11@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-09-29 01:48:33 -07:00
Chao Yu
093749e296 f2fs: support age threshold based garbage collection
There are several issues in current background GC algorithm:
- valid blocks is one of key factors during cost overhead calculation,
so if segment has less valid block, however even its age is young or
it locates hot segment, CB algorithm will still choose the segment as
victim, it's not appropriate.
- GCed data/node will go to existing logs, no matter in-there datas'
update frequency is the same or not, it may mix hot and cold data
again.
- GC alloctor mainly use LFS type segment, it will cost free segment
more quickly.

This patch introduces a new algorithm named age threshold based
garbage collection to solve above issues, there are three steps
mainly:

1. select a source victim:
- set an age threshold, and select candidates beased threshold:
e.g.
 0 means youngest, 100 means oldest, if we set age threshold to 80
 then select dirty segments which has age in range of [80, 100] as
 candiddates;
- set candidate_ratio threshold, and select candidates based the
ratio, so that we can shrink candidates to those oldest segments;
- select target segment with fewest valid blocks in order to
migrate blocks with minimum cost;

2. select a target victim:
- select candidates beased age threshold;
- set candidate_radius threshold, search candidates whose age is
around source victims, searching radius should less than the
radius threshold.
- select target segment with most valid blocks in order to avoid
migrating current target segment.

3. merge valid blocks from source victim into target victim with
SSR alloctor.

Test steps:
- create 160 dirty segments:
 * half of them have 128 valid blocks per segment
 * left of them have 384 valid blocks per segment
- run background GC

Benefit: GC count and block movement count both decrease obviously:

- Before:
  - Valid: 86
  - Dirty: 1
  - Prefree: 11
  - Free: 6001 (6001)

GC calls: 162 (BG: 220)
  - data segments : 160 (160)
  - node segments : 2 (2)
Try to move 41454 blocks (BG: 41454)
  - data blocks : 40960 (40960)
  - node blocks : 494 (494)

IPU: 0 blocks
SSR: 0 blocks in 0 segments
LFS: 41364 blocks in 81 segments

- After:

  - Valid: 87
  - Dirty: 0
  - Prefree: 4
  - Free: 6008 (6008)

GC calls: 75 (BG: 76)
  - data segments : 74 (74)
  - node segments : 1 (1)
Try to move 12813 blocks (BG: 12813)
  - data blocks : 12544 (12544)
  - node blocks : 269 (269)

IPU: 0 blocks
SSR: 12032 blocks in 77 segments
LFS: 855 blocks in 2 segments

Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix a bug along with pinfile in-mem segment & clean up]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-09-11 11:11:15 -07:00
Chao Yu
d0b9e42ab6 f2fs: introduce inmem curseg
Previous implementation of aligned pinfile allocation will:
- allocate new segment on cold data log no matter whether last used
segment is partially used or not, it makes IOs more random;
- force concurrent cold data/GCed IO going into warm data area, it
can make a bad effect on hot/cold data separation;

In this patch, we introduce a new type of log named 'inmem curseg',
the differents from normal curseg is:
- it reuses existed segment type (CURSEG_XXX_NODE/DATA);
- it only exists in memory, its segno, blkofs, summary will not b
 persisted into checkpoint area;

With this new feature, we can enhance scalability of log, special
allocators can be created for purposes:
- pure lfs allocator for aligned pinfile allocation or file
defragmentation
- pure ssr allocator for later feature

So that, let's update aligned pinfile allocation to use this new
inmem curseg fwk.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-09-10 14:03:30 -07:00
Aravind Ramesh
de881df977 f2fs: support zone capacity less than zone size
NVMe Zoned Namespace devices can have zone-capacity less than zone-size.
Zone-capacity indicates the maximum number of sectors that are usable in
a zone beginning from the first sector of the zone. This makes the sectors
sectors after the zone-capacity till zone-size to be unusable.
This patch set tracks zone-size and zone-capacity in zoned devices and
calculate the usable blocks per segment and usable segments per section.

If zone-capacity is less than zone-size mark only those segments which
start before zone-capacity as free segments. All segments at and beyond
zone-capacity are treated as permanently used segments. In cases where
zone-capacity does not align with segment size the last segment will start
before zone-capacity and end beyond the zone-capacity of the zone. For
such spanning segments only sectors within the zone-capacity are used.

During writes and GC manage the usable segments in a section and usable
blocks per segment. Segments which are beyond zone-capacity are never
allocated, and do not need to be garbage collected, only the segments
which are before zone-capacity needs to garbage collected.
For spanning segments based on the number of usable blocks in that
segment, write to blocks only up to zone-capacity.

Zone-capacity is device specific and cannot be configured by the user.
Since NVMe ZNS device zones are sequentially write only, a block device
with conventional zones or any normal block device is needed along with
the ZNS device for the metadata operations of F2fs.

A typical nvme-cli output of a zoned device shows zone start and capacity
and write pointer as below:

SLBA: 0x0     WP: 0x0     Cap: 0x18800 State: EMPTY Type: SEQWRITE_REQ
SLBA: 0x20000 WP: 0x20000 Cap: 0x18800 State: EMPTY Type: SEQWRITE_REQ
SLBA: 0x40000 WP: 0x40000 Cap: 0x18800 State: EMPTY Type: SEQWRITE_REQ

Here zone size is 64MB, capacity is 49MB, WP is at zone start as the zones
are in EMPTY state. For each zone, only zone start + 49MB is usable area,
any lba/sector after 49MB cannot be read or written to, the drive will fail
any attempts to read/write. So, the second zone starts at 64MB and is
usable till 113MB (64 + 49) and the range between 113 and 128MB is
again unusable. The next zone starts at 128MB, and so on.

Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-09-10 14:03:29 -07:00
Randy Dunlap
887e037391 f2fs: segment.h: delete a duplicated word
Drop the repeated word "the" in a comment.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jaegeuk Kim <jaegeuk@kernel.org>
Cc: Chao Yu <chao@kernel.org>
Cc: linux-f2fs-devel@lists.sourceforge.net
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-07-20 15:47:38 -07:00
Jack Qiu
da52f8ade4 f2fs: get the right gc victim section when section has several segments
Assume each section has 4 segment:
     .___________________________.
     |_Segment0_|_..._|_Segment3_|
     .                          .
     .                  .
     .__________.
     |_section0_|

Segment 0~2 has 0 valid block, segment 3 has 512 valid blocks.
It will fail if we want to gc section0 in this scenes,
because all 4 segments in section0 is not dirty.
So we should use dirty section bitmap instead of dirty segment bitmap
to get right victim section.

Signed-off-by: Jack Qiu <jack.qiu@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-06-18 12:35:38 -07:00
Nishad Kamdar
d29fbcdb05 f2fs: Use the correct style for SPDX License Identifier
This patch corrects the SPDX License Identifier style in
header files related to F2FS File System support.
For C header files Documentation/process/license-rules.rst
mandates C-like comments (opposed to C source files where
C++ style should be used).

Changes made by using a script provided by Joe Perches here:
https://lkml.org/lkml/2019/2/7/46.

Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Nishad Kamdar <nishadkamdar@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-05-08 06:55:55 -07:00
Jaegeuk Kim
a7e679b533 f2fs: show mounted time
Let's show mounted time.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-03-19 11:41:25 -07:00
Jaegeuk Kim
d7b0a23d81 f2fs: update f2fs document regarding to fsync_mode
This patch adds missing fsync_mode entry in f2fs document.

Fixes: 04485987f0 ("f2fs: introduce async IPU policy")
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-17 16:48:44 -08:00
Jaegeuk Kim
0e7f41974e f2fs: add a way to turn off ipu bio cache
Setting 0x40 in /sys/fs/f2fs/dev/ipu_policy gives a way to turn off
bio cache, which is useufl to check whether block layer using hardware
encryption engine merges IOs correctly.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-17 16:48:43 -08:00
Chao Yu
4c8ff7095b f2fs: support data compression
This patch tries to support compression in f2fs.

- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.

- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.

- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.

- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext

Compress metadata layout:
                             [Dnode Structure]
             +-----------------------------------------------+
             | cluster 1 | cluster 2 | ......... | cluster N |
             +-----------------------------------------------+
             .           .                       .           .
       .                       .                .                      .
  .         Compressed Cluster       .        .        Normal Cluster            .
+----------+---------+---------+---------+  +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 |  | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+  +---------+---------+---------+---------+
           .                             .
         .                                           .
       .                                                           .
      +-------------+-------------+----------+----------------------------+
      | data length | data chksum | reserved |      compressed data       |
      +-------------+-------------+----------+----------------------------+

Changelog:

20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().

20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().

20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().

20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.

20190402
- don't preallocate blocks for compressed file.

- add lz4 compress algorithm
- process multiple post read works in one workqueue
  Now f2fs supports processing post read work in multiple workqueue,
  it shows low performance due to schedule overhead of multiple
  workqueue executing orderly.

20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR

One cluster contain 4 blocks

 before overwrite   after overwrite

- VVVV		->	CVNN
- CVNN		->	VVVV

- CVNN		->	CVNN
- CVNN		->	CVVV

- CVVV		->	CVNN
- CVVV		->	CVVV

20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.

20191101
- apply fixes from Jaegeuk

20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity

20191216
- apply fixes from Jaegeuk

20200117
- fix to avoid NULL pointer dereference

[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks

Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-17 16:48:07 -08:00
Jaegeuk Kim
f5a53edcf0 f2fs: support aligned pinned file
This patch supports 2MB-aligned pinned file, which can guarantee no GC at all
by allocating fully valid 2MB segment.

Check free segments by has_not_enough_free_secs() with large budget.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-11-07 10:40:59 -08:00
Chao Yu
00e09c0bcc f2fs: enhance f2fs_is_checkpoint_ready()'s readability
This patch changes sematics of f2fs_is_checkpoint_ready()'s return
value as: return true when checkpoint is ready, other return false,
it can improve readability of below conditions.

f2fs_submit_page_write()
...
	if (is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN) ||
				!f2fs_is_checkpoint_ready(sbi))
		__submit_merged_bio(io);

f2fs_balance_fs()
...
	if (!f2fs_is_checkpoint_ready(sbi))
		return;

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-09-06 16:18:26 -07:00
Sahitya Tummala
bbf9f7d90f f2fs: Fix indefinite loop in f2fs_gc()
Policy - Foreground GC, LFS and greedy GC mode.

Under this policy, f2fs_gc() loops forever to GC as it doesn't have
enough free segements to proceed and thus it keeps calling gc_more
for the same victim segment.  This can happen if the selected victim
segment could not be GC'd due to failed blkaddr validity check i.e.
is_alive() returns false for the blocks set in current validity map.

Fix this by keeping track of such invalid segments and skip those
segments for selection in get_victim_by_default() to avoid endless
GC loop under such error scenarios. Currently, add this logic under
CONFIG_F2FS_CHECK_FS to be able to root cause the issue in debug
version.

Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix wrong bitmap size]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-23 07:57:15 -07:00
Chao Yu
2fde3dd14e f2fs: allocate memory in batch in build_sit_info()
build_sit_info() allocate all bitmaps for each segment one by one,
it's quite low efficiency, this pach changes to allocate large
continuous memory at a time, and divide it and assign for each bitmaps
of segment. For large size image, it can expect improving its mount
speed.

Signed-off-by: Chen Gong <gongchen4@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-08-23 07:57:15 -07:00
Chao Yu
10f966bbf5 f2fs: use generic EFSBADCRC/EFSCORRUPTED
f2fs uses EFAULT as error number to indicate filesystem is corrupted
all the time, but generic filesystems use EUCLEAN for such condition,
we need to change to follow others.

This patch adds two new macros as below to wrap more generic error
code macros, and spread them in code.

EFSBADCRC	EBADMSG		/* Bad CRC detected */
EFSCORRUPTED	EUCLEAN		/* Filesystem is corrupted */

Reported-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Acked-by: Pavel Machek <pavel@ucw.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-07-02 15:40:41 -07:00
Geert Uytterhoeven
f91108b801 f2fs: Use DIV_ROUND_UP() instead of open-coding
Replace the open-coded divisions with round-up by calls to the
DIV_ROUND_UP() helper macro.

Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-07-02 15:40:41 -07:00
Joe Perches
dcbb4c10e6 f2fs: introduce f2fs_<level> macros to wrap f2fs_printk()
- Add and use f2fs_<level> macros
- Convert f2fs_msg to f2fs_printk
- Remove level from f2fs_printk and embed the level in the format
- Coalesce formats and align multi-line arguments
- Remove unnecessary duplicate extern f2fs_msg f2fs.h

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-07-02 15:40:40 -07:00
Chao Yu
93770ab7a6 f2fs: introduce DATA_GENERIC_ENHANCE
Previously, f2fs_is_valid_blkaddr(, blkaddr, DATA_GENERIC) will check
whether @blkaddr locates in main area or not.

That check is weak, since the block address in range of main area can
point to the address which is not valid in segment info table, and we
can not detect such condition, we may suffer worse corruption as system
continues running.

So this patch introduce DATA_GENERIC_ENHANCE to enhance the sanity check
which trigger SIT bitmap check rather than only range check.

This patch did below changes as wel:
- set SBI_NEED_FSCK in f2fs_is_valid_blkaddr().
- get rid of is_valid_data_blkaddr() to avoid panic if blkaddr is invalid.
- introduce verify_fio_blkaddr() to wrap fio {new,old}_blkaddr validation check.
- spread blkaddr check in:
 * f2fs_get_node_info()
 * __read_out_blkaddrs()
 * f2fs_submit_page_read()
 * ra_data_block()
 * do_recover_data()

This patch can fix bug reported from bugzilla below:

https://bugzilla.kernel.org/show_bug.cgi?id=203215
https://bugzilla.kernel.org/show_bug.cgi?id=203223
https://bugzilla.kernel.org/show_bug.cgi?id=203231
https://bugzilla.kernel.org/show_bug.cgi?id=203235
https://bugzilla.kernel.org/show_bug.cgi?id=203241

= Update by Jaegeuk Kim =

DATA_GENERIC_ENHANCE enhanced to validate block addresses on read/write paths.
But, xfstest/generic/446 compalins some generated kernel messages saying invalid
bitmap was detected when reading a block. The reaons is, when we get the
block addresses from extent_cache, there is no lock to synchronize it from
truncating the blocks in parallel.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-08 21:23:13 -07:00
Chao Yu
e95bcdb2fe f2fs: fix to do sanity check on valid block count of segment
As Jungyeon reported in bugzilla:

https://bugzilla.kernel.org/show_bug.cgi?id=203233

- Overview
When mounting the attached crafted image and running program, following errors are reported.
Additionally, it hangs on sync after running program.

The image is intentionally fuzzed from a normal f2fs image for testing.
Compile options for F2FS are as follows.
CONFIG_F2FS_FS=y
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_CHECK_FS=y

- Reproduces
cc poc_13.c
mkdir test
mount -t f2fs tmp.img test
cp a.out test
cd test
sudo ./a.out
sync

- Kernel messages
 F2FS-fs (sdb): Bitmap was wrongly set, blk:4608
 kernel BUG at fs/f2fs/segment.c:2102!
 RIP: 0010:update_sit_entry+0x394/0x410
 Call Trace:
  f2fs_allocate_data_block+0x16f/0x660
  do_write_page+0x62/0x170
  f2fs_do_write_node_page+0x33/0xa0
  __write_node_page+0x270/0x4e0
  f2fs_sync_node_pages+0x5df/0x670
  f2fs_write_checkpoint+0x372/0x1400
  f2fs_sync_fs+0xa3/0x130
  f2fs_do_sync_file+0x1a6/0x810
  do_fsync+0x33/0x60
  __x64_sys_fsync+0xb/0x10
  do_syscall_64+0x43/0xf0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9

sit.vblocks and sum valid block count in sit.valid_map may be
inconsistent, segment w/ zero vblocks will be treated as free
segment, while allocating in free segment, we may allocate a
free block, if its bitmap is valid previously, it can cause
kernel crash due to bitmap verification failure.

Anyway, to avoid further serious metadata inconsistence and
corruption, it is necessary and worth to detect SIT
inconsistence. So let's enable check_block_count() to verify
vblocks and valid_map all the time rather than do it only
CONFIG_F2FS_CHECK_FS is enabled.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-05-08 21:23:09 -07:00
Jaegeuk Kim
b460866d27 f2fs: don't wake up too frequently, if there is lots of IOs
Otherwise, it wakes up discard thread which will sleep again by busy IOs
in a loop.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2019-02-15 20:59:45 -08:00
Chao Yu
2c70c5e387 f2fs: introduce __is_large_section() for cleanup
Introduce a wrapper __is_large_section() to clean up codes.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-11-26 15:53:55 -08:00
Daniel Rosenberg
4354994f09 f2fs: checkpoint disabling
Note that, it requires "f2fs: return correct errno in f2fs_gc".

This adds a lightweight non-persistent snapshotting scheme to f2fs.

To use, mount with the option checkpoint=disable, and to return to
normal operation, remount with checkpoint=enable. If the filesystem
is shut down before remounting with checkpoint=enable, it will revert
back to its apparent state when it was first mounted with
checkpoint=disable. This is useful for situations where you wish to be
able to roll back the state of the disk in case of some critical
failure.

Signed-off-by: Daniel Rosenberg <drosen@google.com>
[Jaegeuk Kim: use SB_RDONLY instead of MS_RDONLY]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-10-16 09:36:39 -07:00
Chao Yu
7c1a000d46 f2fs: add SPDX license identifiers
Remove the verbose license text from f2fs files and replace them with
SPDX tags.  This does not change the license of any of the code.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-09-12 13:07:10 -07:00
Jaegeuk Kim
6f8d445506 f2fs: avoid fi->i_gc_rwsem[WRITE] lock in f2fs_gc
The f2fs_gc() called by f2fs_balance_fs() requires to be called outside of
fi->i_gc_rwsem[WRITE], since f2fs_gc() can try to grab it in a loop.

If it hits the miximum retrials in GC, let's give a chance to release
gc_mutex for a short time in order not to go into live lock in the worst
case.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-20 23:13:42 -07:00
Yunlong Song
3611ce9911 f2fs: do not set free of current section
For the case when sbi->segs_per_sec > 1, take section:segment = 5 for
example, if segment 1 is just used and allocate new segment 2, and the
blocks of segment 1 is invalidated, at this time, the previous code will
use __set_test_and_free to free the free_secmap and free_sections++,
this is not correct since it is still a current section, so fix it.

Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-01 11:52:36 -07:00
Chao Yu
c9b60788fc f2fs: fix to do sanity check with block address in main area
This patch add to do sanity check with below field:
- cp_pack_total_block_count
- blkaddr of data/node
- extent info

- Overview
BUG() in verify_block_addr() when writing to a corrupted f2fs image

- Reproduce (4.18 upstream kernel)

- POC (poc.c)

static void activity(char *mpoint) {

  char *foo_bar_baz;
  int err;

  static int buf[8192];
  memset(buf, 0, sizeof(buf));

  err = asprintf(&foo_bar_baz, "%s/foo/bar/baz", mpoint);

  int fd = open(foo_bar_baz, O_RDWR | O_TRUNC, 0777);
  if (fd >= 0) {
    write(fd, (char *)buf, sizeof(buf));
    fdatasync(fd);
    close(fd);
  }
}

int main(int argc, char *argv[]) {
  activity(argv[1]);
  return 0;
}

- Kernel message
[  689.349473] F2FS-fs (loop0): Mounted with checkpoint version = 3
[  699.728662] WARNING: CPU: 0 PID: 1309 at fs/f2fs/segment.c:2860 f2fs_inplace_write_data+0x232/0x240
[  699.728670] Modules linked in: snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core snd_pcm snd_timer snd mac_hid i2c_piix4 soundcore ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 multipath linear 8139too crct10dif_pclmul crc32_pclmul qxl drm_kms_helper syscopyarea aesni_intel sysfillrect sysimgblt fb_sys_fops ttm drm aes_x86_64 crypto_simd cryptd 8139cp glue_helper mii pata_acpi floppy
[  699.729056] CPU: 0 PID: 1309 Comm: a.out Not tainted 4.18.0-rc1+ #4
[  699.729064] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[  699.729074] RIP: 0010:f2fs_inplace_write_data+0x232/0x240
[  699.729076] Code: ff e9 cf fe ff ff 49 8d 7d 10 e8 39 45 ad ff 4d 8b 7d 10 be 04 00 00 00 49 8d 7f 48 e8 07 49 ad ff 45 8b 7f 48 e9 fb fe ff ff <0f> 0b f0 41 80 4d 48 04 e9 65 fe ff ff 90 66 66 66 66 90 55 48 8d
[  699.729130] RSP: 0018:ffff8801f43af568 EFLAGS: 00010202
[  699.729139] RAX: 000000000000003f RBX: ffff8801f43af7b8 RCX: ffffffffb88c9113
[  699.729142] RDX: 0000000000000003 RSI: dffffc0000000000 RDI: ffff8802024e5540
[  699.729144] RBP: ffff8801f43af590 R08: 0000000000000009 R09: ffffffffffffffe8
[  699.729147] R10: 0000000000000001 R11: ffffed0039b0596a R12: ffff8802024e5540
[  699.729149] R13: ffff8801f0335500 R14: ffff8801e3e7a700 R15: ffff8801e1ee4450
[  699.729154] FS:  00007f9bf97f5700(0000) GS:ffff8801f6e00000(0000) knlGS:0000000000000000
[  699.729156] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  699.729159] CR2: 00007f9bf925d170 CR3: 00000001f0c34000 CR4: 00000000000006f0
[  699.729171] Call Trace:
[  699.729192]  f2fs_do_write_data_page+0x2e2/0xe00
[  699.729203]  ? f2fs_should_update_outplace+0xd0/0xd0
[  699.729238]  ? memcg_drain_all_list_lrus+0x280/0x280
[  699.729269]  ? __radix_tree_replace+0xa3/0x120
[  699.729276]  __write_data_page+0x5c7/0xe30
[  699.729291]  ? kasan_check_read+0x11/0x20
[  699.729310]  ? page_mapped+0x8a/0x110
[  699.729321]  ? page_mkclean+0xe9/0x160
[  699.729327]  ? f2fs_do_write_data_page+0xe00/0xe00
[  699.729331]  ? invalid_page_referenced_vma+0x130/0x130
[  699.729345]  ? clear_page_dirty_for_io+0x332/0x450
[  699.729351]  f2fs_write_cache_pages+0x4ca/0x860
[  699.729358]  ? __write_data_page+0xe30/0xe30
[  699.729374]  ? percpu_counter_add_batch+0x22/0xa0
[  699.729380]  ? kasan_check_write+0x14/0x20
[  699.729391]  ? _raw_spin_lock+0x17/0x40
[  699.729403]  ? f2fs_mark_inode_dirty_sync.part.18+0x16/0x30
[  699.729413]  ? iov_iter_advance+0x113/0x640
[  699.729418]  ? f2fs_write_end+0x133/0x2e0
[  699.729423]  ? balance_dirty_pages_ratelimited+0x239/0x640
[  699.729428]  f2fs_write_data_pages+0x329/0x520
[  699.729433]  ? generic_perform_write+0x250/0x320
[  699.729438]  ? f2fs_write_cache_pages+0x860/0x860
[  699.729454]  ? current_time+0x110/0x110
[  699.729459]  ? f2fs_preallocate_blocks+0x1ef/0x370
[  699.729464]  do_writepages+0x37/0xb0
[  699.729468]  ? f2fs_write_cache_pages+0x860/0x860
[  699.729472]  ? do_writepages+0x37/0xb0
[  699.729478]  __filemap_fdatawrite_range+0x19a/0x1f0
[  699.729483]  ? delete_from_page_cache_batch+0x4e0/0x4e0
[  699.729496]  ? __vfs_write+0x2b2/0x410
[  699.729501]  file_write_and_wait_range+0x66/0xb0
[  699.729506]  f2fs_do_sync_file+0x1f9/0xd90
[  699.729511]  ? truncate_partial_data_page+0x290/0x290
[  699.729521]  ? __sb_end_write+0x30/0x50
[  699.729526]  ? vfs_write+0x20f/0x260
[  699.729530]  f2fs_sync_file+0x9a/0xb0
[  699.729534]  ? f2fs_do_sync_file+0xd90/0xd90
[  699.729548]  vfs_fsync_range+0x68/0x100
[  699.729554]  ? __fget_light+0xc9/0xe0
[  699.729558]  do_fsync+0x3d/0x70
[  699.729562]  __x64_sys_fdatasync+0x24/0x30
[  699.729585]  do_syscall_64+0x78/0x170
[  699.729595]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  699.729613] RIP: 0033:0x7f9bf930d800
[  699.729615] Code: 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 83 3d 49 bf 2c 00 00 75 10 b8 4b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 be 78 01 00 48 89 04 24
[  699.729668] RSP: 002b:00007ffee3606c68 EFLAGS: 00000246 ORIG_RAX: 000000000000004b
[  699.729673] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9bf930d800
[  699.729675] RDX: 0000000000008000 RSI: 00000000006010a0 RDI: 0000000000000003
[  699.729678] RBP: 00007ffee3606ca0 R08: 0000000001503010 R09: 0000000000000000
[  699.729680] R10: 00000000000002e8 R11: 0000000000000246 R12: 0000000000400610
[  699.729683] R13: 00007ffee3606da0 R14: 0000000000000000 R15: 0000000000000000
[  699.729687] ---[ end trace 4ce02f25ff7d3df5 ]---
[  699.729782] ------------[ cut here ]------------
[  699.729785] kernel BUG at fs/f2fs/segment.h:654!
[  699.731055] invalid opcode: 0000 [#1] SMP KASAN PTI
[  699.732104] CPU: 0 PID: 1309 Comm: a.out Tainted: G        W         4.18.0-rc1+ #4
[  699.733684] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[  699.735611] RIP: 0010:f2fs_submit_page_bio+0x29b/0x730
[  699.736649] Code: 54 49 8d bd 18 04 00 00 e8 b2 59 af ff 41 8b 8d 18 04 00 00 8b 45 b8 41 d3 e6 44 01 f0 4c 8d 73 14 41 39 c7 0f 82 37 fe ff ff <0f> 0b 65 8b 05 2c 04 77 47 89 c0 48 0f a3 05 52 c1 d5 01 0f 92 c0
[  699.740524] RSP: 0018:ffff8801f43af508 EFLAGS: 00010283
[  699.741573] RAX: 0000000000000000 RBX: ffff8801f43af7b8 RCX: ffffffffb88a7cef
[  699.743006] RDX: 0000000000000007 RSI: dffffc0000000000 RDI: ffff8801e3e7a64c
[  699.744426] RBP: ffff8801f43af558 R08: ffffed003e066b55 R09: ffffed003e066b55
[  699.745833] R10: 0000000000000001 R11: ffffed003e066b54 R12: ffffea0007876940
[  699.747256] R13: ffff8801f0335500 R14: ffff8801e3e7a600 R15: 0000000000000001
[  699.748683] FS:  00007f9bf97f5700(0000) GS:ffff8801f6e00000(0000) knlGS:0000000000000000
[  699.750293] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  699.751462] CR2: 00007f9bf925d170 CR3: 00000001f0c34000 CR4: 00000000000006f0
[  699.752874] Call Trace:
[  699.753386]  ? f2fs_inplace_write_data+0x93/0x240
[  699.754341]  f2fs_inplace_write_data+0xd2/0x240
[  699.755271]  f2fs_do_write_data_page+0x2e2/0xe00
[  699.756214]  ? f2fs_should_update_outplace+0xd0/0xd0
[  699.757215]  ? memcg_drain_all_list_lrus+0x280/0x280
[  699.758209]  ? __radix_tree_replace+0xa3/0x120
[  699.759164]  __write_data_page+0x5c7/0xe30
[  699.760002]  ? kasan_check_read+0x11/0x20
[  699.760823]  ? page_mapped+0x8a/0x110
[  699.761573]  ? page_mkclean+0xe9/0x160
[  699.762345]  ? f2fs_do_write_data_page+0xe00/0xe00
[  699.763332]  ? invalid_page_referenced_vma+0x130/0x130
[  699.764374]  ? clear_page_dirty_for_io+0x332/0x450
[  699.765347]  f2fs_write_cache_pages+0x4ca/0x860
[  699.766276]  ? __write_data_page+0xe30/0xe30
[  699.767161]  ? percpu_counter_add_batch+0x22/0xa0
[  699.768112]  ? kasan_check_write+0x14/0x20
[  699.768951]  ? _raw_spin_lock+0x17/0x40
[  699.769739]  ? f2fs_mark_inode_dirty_sync.part.18+0x16/0x30
[  699.770885]  ? iov_iter_advance+0x113/0x640
[  699.771743]  ? f2fs_write_end+0x133/0x2e0
[  699.772569]  ? balance_dirty_pages_ratelimited+0x239/0x640
[  699.773680]  f2fs_write_data_pages+0x329/0x520
[  699.774603]  ? generic_perform_write+0x250/0x320
[  699.775544]  ? f2fs_write_cache_pages+0x860/0x860
[  699.776510]  ? current_time+0x110/0x110
[  699.777299]  ? f2fs_preallocate_blocks+0x1ef/0x370
[  699.778279]  do_writepages+0x37/0xb0
[  699.779026]  ? f2fs_write_cache_pages+0x860/0x860
[  699.779978]  ? do_writepages+0x37/0xb0
[  699.780755]  __filemap_fdatawrite_range+0x19a/0x1f0
[  699.781746]  ? delete_from_page_cache_batch+0x4e0/0x4e0
[  699.782820]  ? __vfs_write+0x2b2/0x410
[  699.783597]  file_write_and_wait_range+0x66/0xb0
[  699.784540]  f2fs_do_sync_file+0x1f9/0xd90
[  699.785381]  ? truncate_partial_data_page+0x290/0x290
[  699.786415]  ? __sb_end_write+0x30/0x50
[  699.787204]  ? vfs_write+0x20f/0x260
[  699.787941]  f2fs_sync_file+0x9a/0xb0
[  699.788694]  ? f2fs_do_sync_file+0xd90/0xd90
[  699.789572]  vfs_fsync_range+0x68/0x100
[  699.790360]  ? __fget_light+0xc9/0xe0
[  699.791128]  do_fsync+0x3d/0x70
[  699.791779]  __x64_sys_fdatasync+0x24/0x30
[  699.792614]  do_syscall_64+0x78/0x170
[  699.793371]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  699.794406] RIP: 0033:0x7f9bf930d800
[  699.795134] Code: 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 83 3d 49 bf 2c 00 00 75 10 b8 4b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 be 78 01 00 48 89 04 24
[  699.798960] RSP: 002b:00007ffee3606c68 EFLAGS: 00000246 ORIG_RAX: 000000000000004b
[  699.800483] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9bf930d800
[  699.801923] RDX: 0000000000008000 RSI: 00000000006010a0 RDI: 0000000000000003
[  699.803373] RBP: 00007ffee3606ca0 R08: 0000000001503010 R09: 0000000000000000
[  699.804798] R10: 00000000000002e8 R11: 0000000000000246 R12: 0000000000400610
[  699.806233] R13: 00007ffee3606da0 R14: 0000000000000000 R15: 0000000000000000
[  699.807667] Modules linked in: snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core snd_pcm snd_timer snd mac_hid i2c_piix4 soundcore ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 multipath linear 8139too crct10dif_pclmul crc32_pclmul qxl drm_kms_helper syscopyarea aesni_intel sysfillrect sysimgblt fb_sys_fops ttm drm aes_x86_64 crypto_simd cryptd 8139cp glue_helper mii pata_acpi floppy
[  699.817079] ---[ end trace 4ce02f25ff7d3df6 ]---
[  699.818068] RIP: 0010:f2fs_submit_page_bio+0x29b/0x730
[  699.819114] Code: 54 49 8d bd 18 04 00 00 e8 b2 59 af ff 41 8b 8d 18 04 00 00 8b 45 b8 41 d3 e6 44 01 f0 4c 8d 73 14 41 39 c7 0f 82 37 fe ff ff <0f> 0b 65 8b 05 2c 04 77 47 89 c0 48 0f a3 05 52 c1 d5 01 0f 92 c0
[  699.822919] RSP: 0018:ffff8801f43af508 EFLAGS: 00010283
[  699.823977] RAX: 0000000000000000 RBX: ffff8801f43af7b8 RCX: ffffffffb88a7cef
[  699.825436] RDX: 0000000000000007 RSI: dffffc0000000000 RDI: ffff8801e3e7a64c
[  699.826881] RBP: ffff8801f43af558 R08: ffffed003e066b55 R09: ffffed003e066b55
[  699.828292] R10: 0000000000000001 R11: ffffed003e066b54 R12: ffffea0007876940
[  699.829750] R13: ffff8801f0335500 R14: ffff8801e3e7a600 R15: 0000000000000001
[  699.831192] FS:  00007f9bf97f5700(0000) GS:ffff8801f6e00000(0000) knlGS:0000000000000000
[  699.832793] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  699.833981] CR2: 00007f9bf925d170 CR3: 00000001f0c34000 CR4: 00000000000006f0
[  699.835556] ==================================================================
[  699.837029] BUG: KASAN: stack-out-of-bounds in update_stack_state+0x38c/0x3e0
[  699.838462] Read of size 8 at addr ffff8801f43af970 by task a.out/1309

[  699.840086] CPU: 0 PID: 1309 Comm: a.out Tainted: G      D W         4.18.0-rc1+ #4
[  699.841603] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[  699.843475] Call Trace:
[  699.843982]  dump_stack+0x7b/0xb5
[  699.844661]  print_address_description+0x70/0x290
[  699.845607]  kasan_report+0x291/0x390
[  699.846351]  ? update_stack_state+0x38c/0x3e0
[  699.853831]  __asan_load8+0x54/0x90
[  699.854569]  update_stack_state+0x38c/0x3e0
[  699.855428]  ? __read_once_size_nocheck.constprop.7+0x20/0x20
[  699.856601]  ? __save_stack_trace+0x5e/0x100
[  699.857476]  unwind_next_frame.part.5+0x18e/0x490
[  699.858448]  ? unwind_dump+0x290/0x290
[  699.859217]  ? clear_page_dirty_for_io+0x332/0x450
[  699.860185]  __unwind_start+0x106/0x190
[  699.860974]  __save_stack_trace+0x5e/0x100
[  699.861808]  ? __save_stack_trace+0x5e/0x100
[  699.862691]  ? unlink_anon_vmas+0xba/0x2c0
[  699.863525]  save_stack_trace+0x1f/0x30
[  699.864312]  save_stack+0x46/0xd0
[  699.864993]  ? __alloc_pages_slowpath+0x1420/0x1420
[  699.865990]  ? flush_tlb_mm_range+0x15e/0x220
[  699.866889]  ? kasan_check_write+0x14/0x20
[  699.867724]  ? __dec_node_state+0x92/0xb0
[  699.868543]  ? lock_page_memcg+0x85/0xf0
[  699.869350]  ? unlock_page_memcg+0x16/0x80
[  699.870185]  ? page_remove_rmap+0x198/0x520
[  699.871048]  ? mark_page_accessed+0x133/0x200
[  699.871930]  ? _cond_resched+0x1a/0x50
[  699.872700]  ? unmap_page_range+0xcd4/0xe50
[  699.873551]  ? rb_next+0x58/0x80
[  699.874217]  ? rb_next+0x58/0x80
[  699.874895]  __kasan_slab_free+0x13c/0x1a0
[  699.875734]  ? unlink_anon_vmas+0xba/0x2c0
[  699.876563]  kasan_slab_free+0xe/0x10
[  699.877315]  kmem_cache_free+0x89/0x1e0
[  699.878095]  unlink_anon_vmas+0xba/0x2c0
[  699.878913]  free_pgtables+0x101/0x1b0
[  699.879677]  exit_mmap+0x146/0x2a0
[  699.880378]  ? __ia32_sys_munmap+0x50/0x50
[  699.881214]  ? kasan_check_read+0x11/0x20
[  699.882052]  ? mm_update_next_owner+0x322/0x380
[  699.882985]  mmput+0x8b/0x1d0
[  699.883602]  do_exit+0x43a/0x1390
[  699.884288]  ? mm_update_next_owner+0x380/0x380
[  699.885212]  ? f2fs_sync_file+0x9a/0xb0
[  699.885995]  ? f2fs_do_sync_file+0xd90/0xd90
[  699.886877]  ? vfs_fsync_range+0x68/0x100
[  699.887694]  ? __fget_light+0xc9/0xe0
[  699.888442]  ? do_fsync+0x3d/0x70
[  699.889118]  ? __x64_sys_fdatasync+0x24/0x30
[  699.889996]  rewind_stack_do_exit+0x17/0x20
[  699.890860] RIP: 0033:0x7f9bf930d800
[  699.891585] Code: Bad RIP value.
[  699.892268] RSP: 002b:00007ffee3606c68 EFLAGS: 00000246 ORIG_RAX: 000000000000004b
[  699.893781] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9bf930d800
[  699.895220] RDX: 0000000000008000 RSI: 00000000006010a0 RDI: 0000000000000003
[  699.896643] RBP: 00007ffee3606ca0 R08: 0000000001503010 R09: 0000000000000000
[  699.898069] R10: 00000000000002e8 R11: 0000000000000246 R12: 0000000000400610
[  699.899505] R13: 00007ffee3606da0 R14: 0000000000000000 R15: 0000000000000000

[  699.901241] The buggy address belongs to the page:
[  699.902215] page:ffffea0007d0ebc0 count:0 mapcount:0 mapping:0000000000000000 index:0x0
[  699.903811] flags: 0x2ffff0000000000()
[  699.904585] raw: 02ffff0000000000 0000000000000000 ffffffff07d00101 0000000000000000
[  699.906125] raw: 0000000000000000 0000000000240000 00000000ffffffff 0000000000000000
[  699.907673] page dumped because: kasan: bad access detected

[  699.909108] Memory state around the buggy address:
[  699.910077]  ffff8801f43af800: 00 f1 f1 f1 f1 00 f4 f4 f4 f3 f3 f3 f3 00 00 00
[  699.911528]  ffff8801f43af880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  699.912953] >ffff8801f43af900: 00 00 00 00 00 00 00 00 f1 01 f4 f4 f4 f2 f2 f2
[  699.914392]                                                              ^
[  699.915758]  ffff8801f43af980: f2 00 f4 f4 00 00 00 00 f2 00 00 00 00 00 00 00
[  699.917193]  ffff8801f43afa00: 00 00 00 00 00 00 00 00 00 f3 f3 f3 00 00 00 00
[  699.918634] ==================================================================

- Location
https://elixir.bootlin.com/linux/v4.18-rc1/source/fs/f2fs/segment.h#L644

Reported-by Wen Xu <wen.xu@gatech.edu>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-08-01 11:52:32 -07:00
Chao Yu
e1da7872f6 f2fs: introduce and spread verify_blkaddr
This patch introduces verify_blkaddr to check meta/data block address
with valid range to detect bug earlier.

In addition, once we encounter an invalid blkaddr, notice user to run
fsck to fix, and let the kernel panic.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-07-27 18:03:59 +09:00
Chao Yu
a1f72ac2c0 f2fs: fix to update mtime correctly
If we change system time to the past, get_mtime() will return a
overflowed time, and SIT_I(sbi)->max_mtime will be udpated
incorrectly, this patch fixes the two issues.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-06-04 14:31:11 -07:00
Chao Yu
2ef79ecb5e f2fs: avoid stucking GC due to atomic write
f2fs doesn't allow abuse on atomic write class interface, so except
limiting in-mem pages' total memory usage capacity, we need to limit
atomic-write usage as well when filesystem is seriously fragmented,
otherwise we may run into infinite loop during foreground GC because
target blocks in victim segment are belong to atomic opened file for
long time.

Now, we will detect failure due to atomic write in foreground GC, if
the count exceeds threshold, we will drop all atomic written data in
cache, by this, I expect it can keep our system running safely to
prevent Dos attack.

In addition, his patch adds to show GC skip information in debugfs,
now it just shows count of skipped caused by atomic write.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-31 11:31:51 -07:00
Chao Yu
7b525dd013 f2fs: clean up with is_valid_blkaddr()
- rename is_valid_blkaddr() to is_valid_meta_blkaddr() for readability.
- introduce is_valid_blkaddr() for cleanup.

No logic change in this patch.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-31 11:31:50 -07:00
Chao Yu
299254d85d Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"
For extreme case:
10 section, op = 10%, no_fggc_threshold = 90%
All section usage: 85% 85% 85% 85% 90% 90% 95% 95% 95% 95%

During foreground GC, if we skip select dirty section whose usage
is larger than no_fggc_threshold, we can only recycle 80% invalid
space from four 85% usage sections and two 90% usage sections,
result in encountering out-of-space issue.

This reverts commit e93b986525 to
fix this issue, besides, we keep the logic that we scan all dirty
section when searching a victim, so that GC can select victim with
least valid blocks.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-31 11:31:50 -07:00
Chao Yu
81114baa83 f2fs: don't use GFP_ZERO for page caches
Related to https://lkml.org/lkml/2018/4/8/661

Sometimes, we need to write meta data to new allocated block address,
then we will allocate a zeroed page in inner inode's address space, and
fill partial data in it, and leave other place with zero value which means
some fields are initial status.

There are two inner inodes (meta inode and node inode) setting __GFP_ZERO,
I have just checked them, for both of them, we can avoid using __GFP_ZERO,
and do initialization by ourselves to avoid unneeded/redundant zeroing
from mm.

Cc: <stable@vger.kernel.org>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-05-31 11:31:45 -07:00
Yunlei He
0833721ec3 f2fs: check blkaddr more accuratly before issue a bio
This patch check blkaddr more accuratly before issue a
write or read bio.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-03-18 23:33:50 -07:00
Jaegeuk Kim
84b89e5d94 f2fs: add auto tuning for small devices
If f2fs is running on top of very small devices, it's worth to avoid abusing
free LBAs. In order to achieve that, this patch introduces some parameter
tuning.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-03-13 08:05:51 +09:00
Yunlei He
068c3cd858 f2fs: rebuild sit page from sit info in mem
This patch rebuild sit page from sit info in mem instead
of issue a read io.

I test this method and the result is as below:

Pre:
 mmc_perf_test-12061 [001] ...1   976.819992: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [001] ...1   976.856446: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [003] ...1   998.976946: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [003] ...1   999.023269: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [003] ...1  1022.060772: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [003] ...1  1022.111034: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [002] ...1  1070.127643: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [003] ...1  1070.187352: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [003] ...1  1095.942124: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [003] ...1  1095.995975: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [003] ...1  1122.535091: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [003] ...1  1122.586521: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [001] ...1  1147.897487: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [001] ...1  1147.959438: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [003] ...1  1177.926951: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [002] ...1  1177.976823: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
 mmc_perf_test-12061 [002] ...1  1204.176087: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
 mmc_perf_test-12061 [002] ...1  1204.239046: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit

Some sit flush consume more than 50ms.

Now:
mmc_perf_test-2187  [007] ...1   196.840684: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [007] ...1   196.841258: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [007] ...1   219.430582: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [007] ...1   219.431144: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [002] ...1   243.638678: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [000] ...1   243.638980: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [002] ...1   265.392180: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [002] ...1   265.392245: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [000] ...1   290.309051: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [000] ...1   290.309116: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [003] ...1   317.144209: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [003] ...1   317.145913: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [005] ...1   343.224954: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [005] ...1   343.225574: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [000] ...1   370.239846: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [000] ...1   370.241138: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [001] ...1   397.029043: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [001] ...1   397.030750: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit
mmc_perf_test-2187  [003] ...1   425.386377: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = start flush sit
mmc_perf_test-2187  [003] ...1   425.387735: f2fs_write_checkpoint: dev = (259,44), checkpoint for Sync, state = end flush sit

Most sit flush consume no more than 1ms.

Signed-off-by: Yunlei He <heyunlei@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-25 10:44:34 -08:00
Chao Yu
bb9e3bb8db f2fs: split need_inplace_update
This patch splits need_inplace_update to two functions:
a. should_update_inplace() includes all conditions that we must use IPU.
b. should_update_outplace() includes all conditions that we must use OPU.

So that, in f2fs_ioc_set_pin_file() and f2fs_defragment_range(), we can
use corresponding function to check whether we can trigger OPU/IPU or not.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-22 14:56:53 -08:00
Jaegeuk Kim
c39a1b348c f2fs: return error during fill_super
Let's avoid BUG_ON during fill_super, when on-disk was totall corrupted.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2018-01-02 19:27:31 -08:00
Chao Yu
bf34c93d26 f2fs: check curseg space before foreground GC
When we are closing to trigger foreground GC, if there are only a few
of dirty metas, we can log these dirty metas in left space of opened
segments instead of triggering foreground GC.

With this patch, total count of foreground GC triggered by
test/generic/* of fstest suit reduce from 254 to 184.

So let's do the check before foreground GC anyway.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-05 16:42:04 -08:00
Chao Yu
3d26fa6be3 f2fs: use rw_semaphore to protect SIT cache
There are some cases user didn't update SIT cache under this lock,
so let's use rw_semaphore instead of mutex to enhance concurrently
accessing.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-11-05 16:42:03 -08:00
Chao Yu
7e515b31d4 f2fs: fix to correct no_fggc_candidate
There may be extreme case as below:

For one section contains one segment, and there are total 100 segments
with 10% over-privision ratio in f2fs partition, fggc_threshold will
be rounded down to 460 instead of 460.8 as below caclulation:

sbi->fggc_threshold = div_u64((u64)(main_count - ovp_count) *
			BLKS_PER_SEC(sbi), (main_count - resv_count));

If section usage is as:
60 segments which contain 460 valid blocks
40 segments which contain 462 valid blocks

As valid block number in all sections is large than fggc_threshold, so
none of them will be chosen as candidate due to incorrect fggc_threshold.

Let's just soften the term of choosing foreground GC candidates.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2017-10-26 10:44:24 +02:00