[Deepin-Kernel-SIG] [linux 6.6.y] [Upstream] workload-specific and memory pressure-driven zswap writeback - #2028
Open
opsiff wants to merge 5 commits into
Open
Conversation
Reviewer's GuideIntroduces a memory-pressure-driven, memcg/NUMA-aware zswap shrinker by making zswap LRU per-memcg/NUMA, tracking per-lruvec zswap protection state, wiring it into swap readahead, and exposing per-memcg writeback stats and tests, with supporting list_lru and memcg APIs plus a Kconfig toggle. Sequence diagram for memcg/NUMA-aware zswap shrinker under memory pressuresequenceDiagram
actor vmscan
participant zswap_shrinker
participant zswap_pool
participant memcg
participant lruvec
participant list_lru
participant shrink_memcg_cb
vmscan->>zswap_shrinker: zswap_shrinker_count(shrinker, shrink_control)
zswap_shrinker->>memcg: memcg_page_state(memcg, MEMCG_ZSWAP_B / MEMCG_ZSWAPPED)
zswap_shrinker->>zswap_pool: get_zswap_pool_size(pool)
zswap_shrinker->>lruvec: mem_cgroup_lruvec(memcg, NODE_DATA(nid))
zswap_shrinker->>list_lru: list_lru_shrink_count(&pool.list_lru, shrink_control)
zswap_shrinker-->>vmscan: mult_frac(nr_freeable, nr_backing, nr_stored)
vmscan->>zswap_shrinker: zswap_shrinker_scan(shrinker, shrink_control)
zswap_shrinker->>lruvec: atomic_long_read(lruvec.zswap_lruvec_state.nr_zswap_protected)
zswap_shrinker->>list_lru: list_lru_shrink_count(&pool.list_lru, shrink_control)
alt [nr_protected >= lru_size - nr_to_scan]
zswap_shrinker-->>vmscan: return SHRINK_STOP
else [nr_protected < lru_size - nr_to_scan]
zswap_shrinker->>list_lru: list_lru_shrink_walk(&pool.list_lru, shrink_control, shrink_memcg_cb, &encountered_page_in_swapcache)
list_lru->>shrink_memcg_cb: shrink_memcg_cb(item, list_lru_one, lock, &encountered_page_in_swapcache)
alt [writeback_result == -EEXIST]
shrink_memcg_cb->>list_lru: zswap_lru_putback(&entry.pool.list_lru, entry)
shrink_memcg_cb->>zswap_shrinker: *encountered_page_in_swapcache = true
zswap_shrinker-->>vmscan: return SHRINK_STOP
else [writeback_result == 0]
shrink_memcg_cb-->>list_lru: LRU_REMOVED_RETRY
zswap_shrinker-->>vmscan: return shrink_ret
end
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In zswap_shrinker_count(), the comment notes that sc->memcg can be NULL when memcg is runtime-disabled, but the code still unconditionally passes memcg to memcg_page_state() in the !mem_cgroup_disabled() path; consider explicitly handling the sc->memcg == NULL case (e.g., falling back to pool-wide stats) to avoid dereferencing NULL.
- In the cgroup selftest, test_no_invasive_cgroup_shrink() allocates a large buffer with malloc() inside a 1M-limited cgroup without checking for allocation failure, which can lead to crashes rather than a clean test failure; guard the allocation and handle ENOMEM so the test remains robust under tighter limits.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In zswap_shrinker_count(), the comment notes that sc->memcg can be NULL when memcg is runtime-disabled, but the code still unconditionally passes memcg to memcg_page_state() in the !mem_cgroup_disabled() path; consider explicitly handling the sc->memcg == NULL case (e.g., falling back to pool-wide stats) to avoid dereferencing NULL.
- In the cgroup selftest, test_no_invasive_cgroup_shrink() allocates a large buffer with malloc() inside a 1M-limited cgroup without checking for allocation failure, which can lead to crashes rather than a clean test failure; guard the allocation and handle ENOMEM so the test remains robust under tighter limits.
## Individual Comments
### Comment 1
<location path="Documentation/admin-guide/mm/zswap.rst" line_range="166" />
<code_context>
+This can be enabled at the boot time if ``CONFIG_ZSWAP_SHRINKER_DEFAULT_ON`` is
+selected.
+
A debugfs interface is provided for various statistic about pool size, number
of pages stored, same-value filled pages and various counters for the reasons
pages are rejected.
</code_context>
<issue_to_address>
**issue (typo):** Use the plural "statistics" instead of "statistic".
Because this covers multiple metrics (pool size, number of pages, etc.), "various statistic" should be "various statistics".
```suggestion
A debugfs interface is provided for various statistics about pool size, number
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| This can be enabled at the boot time if ``CONFIG_ZSWAP_SHRINKER_DEFAULT_ON`` is | ||
| selected. | ||
|
|
||
| A debugfs interface is provided for various statistic about pool size, number |
There was a problem hiding this comment.
issue (typo): Use the plural "statistics" instead of "statistic".
Because this covers multiple metrics (pool size, number of pages, etc.), "various statistic" should be "various statistics".
Suggested change
| A debugfs interface is provided for various statistic about pool size, number | |
| A debugfs interface is provided for various statistics about pool size, number |
| #ifdef CONFIG_MEMCG | ||
| struct pglist_data *pgdat; | ||
| #endif | ||
| struct zswap_lruvec_state zswap_lruvec_state; |
Member
There was a problem hiding this comment.
竟然没破坏白名单内的KABI，说明当初白预留了......
Member
Author
There was a problem hiding this comment.
竟然没破坏白名单内的KABI，说明当初白预留了......
有破坏再修复
| #ifdef CONFIG_MEMCG | ||
| struct pglist_data *pgdat; | ||
| #endif | ||
| struct zswap_lruvec_state zswap_lruvec_state; |
Member
There was a problem hiding this comment.
竟然没破坏白名单内的KABI，说明当初白预留了......
There was a problem hiding this comment.
Pull request overview
This PR backports an upstream zswap change set to Linux 6.6.y, introducing a memory-pressure-driven zswap shrinker that is memcg- and NUMA-aware to enable workload-specific writeback and reduce cold data retention in the zswap pool.
Changes:
- Add a per-memcg/per-NUMA zswap shrinker and per-lruvec “protected” accounting to avoid reclaiming warm zswap entries.
- Hook swap readahead/swapin paths and lruvec initialization to feed the protection heuristics.
- Update admin documentation and extend cgroup zswap selftests for per-memcg writeback accounting.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| Documentation/admin-guide/mm/zswap.rst | Documents the new shrinker enablement path and default-on Kconfig option. |
| include/linux/mmzone.h | Adds per-lruvec zswap state storage needed by the new protection heuristics. |
| include/linux/zswap.h | Introduces zswap lruvec state + helper APIs for initialization and swapin tracking. |
| mm/Kconfig | Adds CONFIG_ZSWAP_SHRINKER_DEFAULT_ON to control the default enablement of the shrinker. |
| mm/mmzone.c | Initializes the new zswap lruvec state during lruvec setup. |
| mm/swap_state.c | Calls into zswap swapin tracking from swap readahead paths. |
| mm/zswap.c | Implements shrinker logic, pool accounting helpers, and protected-region heuristics. |
| tools/testing/selftests/cgroup/test_zswap.c | Updates the selftest to validate non-invasive, per-memcg zswap writeback via memory.stat. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
96
to
100
| int ret = KSFT_FAIL; | ||
| char *test_group; | ||
| size_t control_allocation_size = MB(10); | ||
| char *control_allocation, *wb_group = NULL, *control_group = NULL; | ||
|
|
||
| /* Set up */ |
Comment on lines
+53
to
56
| static int get_cg_wb_count(const char *cg) | ||
| { | ||
| return read_int("/sys/kernel/debug/zswap/written_back_pages", value); | ||
| return cg_read_key_long(cg, "memory.stat", "zswp_wb"); | ||
| } |
| if (get_zswap_written_back_pages(&written_back_after)) | ||
| goto out; | ||
| if (written_back_after == written_back_before) | ||
| /* Verify that only zswapped memory from gwb_group has been written back */ |
Comment on lines
+615
to
+618
| if (nr_protected >= lru_size - sc->nr_to_scan) { | ||
| sc->nr_scanned = 0; | ||
| return SHRINK_STOP; | ||
| } |
Comment on lines
+24
to
+25
| * These pages are likely to be warm, and might incur IO if the are written | ||
| * to swap. |
Comment on lines
+158
to
+164
| the memory for other use cases. By default, the zswap shrinker is disabled. | ||
| User can enable it as follows: | ||
|
|
||
| echo Y > /sys/module/zswap/parameters/shrinker_enabled | ||
|
|
||
| This can be enabled at the boot time if ``CONFIG_ZSWAP_SHRINKER_DEFAULT_ON`` is | ||
| selected. |
Comment on lines
+410
to
+417
| old = atomic_long_inc_return(nr_zswap_protected); | ||
| /* | ||
| * Decay to avoid overflow and adapt to changing workloads. | ||
| * This is based on LRU reclaim cost decaying heuristics. | ||
| */ | ||
| do { | ||
| new = old > lru_size / 4 ? old / 2 : old; | ||
| } while (!atomic_long_try_cmpxchg(nr_zswap_protected, &old, new)); |
mainline inclusion
from mainline-v6.8-rc1
category: performance
The memcg-zswap self test is updated to adjust to the behavior change
implemented by commit 87730b165089 ("zswap: make shrinking memcg-aware"),
where zswap performs writeback for specific memcg.
Link: https://lkml.kernel.org/r/20231130194023.4102148-6-nphamcs@gmail.com
Signed-off-by: Domenico Cerasuolo <cerasuolodomenico@gmail.com>
Signed-off-by: Nhat Pham <nphamcs@gmail.com>
Tested-by: Bagas Sanjaya <bagasdotme@gmail.com>
Acked-by: Chris Li <chrisl@kernel.org> (Google)
Cc: Dan Streetman <ddstreet@ieee.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Seth Jennings <sjenning@redhat.com>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Vitaly Wool <vitaly.wool@konsulko.com>
Cc: Yosry Ahmed <yosryahmed@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
(cherry picked from commit a697dc2)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
mainline inclusion from mainline-v6.8-rc1 category: performance Currently, we only shrink the zswap pool when the user-defined limit is hit. This means that if we set the limit too high, cold data that are unlikely to be used again will reside in the pool, wasting precious memory. It is hard to predict how much zswap space will be needed ahead of time, as this depends on the workload (specifically, on factors such as memory access patterns and compressibility of the memory pages). This patch implements a memcg- and NUMA-aware shrinker for zswap, that is initiated when there is memory pressure. The shrinker does not have any parameter that must be tuned by the user, and can be opted in or out on a per-memcg basis. Furthermore, to make it more robust for many workloads and prevent overshrinking (i.e evicting warm pages that might be refaulted into memory), we build in the following heuristics: * Estimate the number of warm pages residing in zswap, and attempt to protect this region of the zswap LRU. * Scale the number of freeable objects by an estimate of the memory saving factor. The better zswap compresses the data, the fewer pages we will evict to swap (as we will otherwise incur IO for relatively small memory saving). * During reclaim, if the shrinker encounters a page that is also being brought into memory, the shrinker will cautiously terminate its shrinking action, as this is a sign that it is touching the warmer region of the zswap LRU. As a proof of concept, we ran the following synthetic benchmark: build the linux kernel in a memory-limited cgroup, and allocate some cold data in tmpfs to see if the shrinker could write them out and improved the overall performance. Depending on the amount of cold data generated, we observe from 14% to 35% reduction in kernel CPU time used in the kernel builds. [nphamcs@gmail.com: check shrinker enablement early, use less costly stat flushing] Link: https://lkml.kernel.org/r/20231206194456.3234203-1-nphamcs@gmail.com Link: https://lkml.kernel.org/r/20231130194023.4102148-7-nphamcs@gmail.com Signed-off-by: Nhat Pham <nphamcs@gmail.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Tested-by: Bagas Sanjaya <bagasdotme@gmail.com> Cc: Chris Li <chrisl@kernel.org> Cc: Dan Streetman <ddstreet@ieee.org> Cc: Domenico Cerasuolo <cerasuolodomenico@gmail.com> Cc: Michal Hocko <mhocko@kernel.org> Cc: Muchun Song <muchun.song@linux.dev> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Seth Jennings <sjenning@redhat.com> Cc: Shakeel Butt <shakeelb@google.com> Cc: Shuah Khan <shuah@kernel.org> Cc: Vitaly Wool <vitaly.wool@konsulko.com> Cc: Yosry Ahmed <yosryahmed@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> (cherry picked from commit b5ba474) Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
mainline inclusion from mainline-v6.8-rc4 category: bugfix LRU_SKIP can only be returned if we don't ever dropped lru lock, or we need to return LRU_RETRY to restart from the head of lru list. Otherwise, the iteration might continue from a cursor position that was freed while the locks were dropped. Actually we may need to introduce another LRU_STOP to really terminate the ongoing shrinking scan process, when we encounter a warm page already in the swap cache. The current list_lru implementation doesn't have this function to early break from __list_lru_walk_one. Link: https://lkml.kernel.org/r/20240126-zswap-writeback-race-v2-1-b10479847099@bytedance.com Fixes: b5ba474 ("zswap: shrink zswap pool based on memory pressure") Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: Nhat Pham <nphamcs@gmail.com> Cc: Chris Li <chriscli@google.com> Cc: Yosry Ahmed <yosryahmed@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> (cherry picked from commit 27d3969) Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
mainline inclusion from mainline-v6.9-rc2 category: bugfix Kent forwards this bug report of zswap re-entering the block layer from an IO request allocation and locking up: [10264.128242] sysrq: Show Blocked State [10264.128268] task:kworker/20:0H state:D stack:0 pid:143 tgid:143 ppid:2 flags:0x00004000 [10264.128271] Workqueue: bcachefs_io btree_write_submit [bcachefs] [10264.128295] Call Trace: [10264.128295] <TASK> [10264.128297] __schedule+0x3e6/0x1520 [10264.128303] schedule+0x32/0xd0 [10264.128304] schedule_timeout+0x98/0x160 [10264.128308] io_schedule_timeout+0x50/0x80 [10264.128309] wait_for_completion_io_timeout+0x7f/0x180 [10264.128310] submit_bio_wait+0x78/0xb0 [10264.128313] swap_writepage_bdev_sync+0xf6/0x150 [10264.128317] zswap_writeback_entry+0xf2/0x180 [10264.128319] shrink_memcg_cb+0xe7/0x2f0 [10264.128322] __list_lru_walk_one+0xb9/0x1d0 [10264.128325] list_lru_walk_one+0x5d/0x90 [10264.128326] zswap_shrinker_scan+0xc4/0x130 [10264.128327] do_shrink_slab+0x13f/0x360 [10264.128328] shrink_slab+0x28e/0x3c0 [10264.128329] shrink_one+0x123/0x1b0 [10264.128331] shrink_node+0x97e/0xbc0 [10264.128332] do_try_to_free_pages+0xe7/0x5b0 [10264.128333] try_to_free_pages+0xe1/0x200 [10264.128334] __alloc_pages_slowpath.constprop.0+0x343/0xde0 [10264.128337] __alloc_pages+0x32d/0x350 [10264.128338] allocate_slab+0x400/0x460 [10264.128339] ___slab_alloc+0x40d/0xa40 [10264.128345] kmem_cache_alloc+0x2e7/0x330 [10264.128348] mempool_alloc+0x86/0x1b0 [10264.128349] bio_alloc_bioset+0x200/0x4f0 [10264.128352] bio_alloc_clone+0x23/0x60 [10264.128354] alloc_io+0x26/0xf0 [dm_mod 7e9e6b44df4927f93fb3e4b5c782767396f58382] [10264.128361] dm_submit_bio+0xb8/0x580 [dm_mod 7e9e6b44df4927f93fb3e4b5c782767396f58382] [10264.128366] __submit_bio+0xb0/0x170 [10264.128367] submit_bio_noacct_nocheck+0x159/0x370 [10264.128368] bch2_submit_wbio_replicas+0x21c/0x3a0 [bcachefs 85f1b9a7a824f272eff794653a06dde1a94439f2] [10264.128391] btree_write_submit+0x1cf/0x220 [bcachefs 85f1b9a7a824f272eff794653a06dde1a94439f2] [10264.128406] process_one_work+0x178/0x350 [10264.128408] worker_thread+0x30f/0x450 [10264.128409] kthread+0xe5/0x120 The zswap shrinker resumes the swap_writepage()s that were intercepted by the zswap store. This will enter the block layer, and may even enter the filesystem depending on the swap backing file. Make it respect GFP_NOIO and GFP_NOFS. Link: https://lore.kernel.org/linux-mm/rc4pk2r42oyvjo4dc62z6sovquyllq56i5cdgcaqbd7wy3hfzr@n4nbxido3fme/ Link: https://lkml.kernel.org/r/20240321182532.60000-1-hannes@cmpxchg.org Fixes: b5ba474 ("zswap: shrink zswap pool based on memory pressure") Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Reported-by: Kent Overstreet <kent.overstreet@linux.dev> Acked-by: Yosry Ahmed <yosryahmed@google.com> Reported-by: Jérôme Poulin <jeromepoulin@gmail.com> Reviewed-by: Nhat Pham <nphamcs@gmail.com> Reviewed-by: Chengming Zhou <chengming.zhou@linux.dev> Cc: stable@vger.kernel.org [v6.8] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> (cherry picked from commit 30fb6a8) Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
mainline inclusion from mainline-v6.9-rc6 category: bugfix Christian reports a NULL deref in zswap that he bisected down to the zswap shrinker. The issue also cropped up in the bug trackers of libguestfs [1] and the Red Hat bugzilla [2]. The problem is that when memcg is disabled with the boot time flag, the zswap shrinker might get called with sc->memcg == NULL. This is okay in many places, like the lruvec operations. But it crashes in memcg_page_state() - which is only used due to the non-node accounting of cgroup's the zswap memory to begin with. Nhat spotted that the memcg can be NULL in the memcg-disabled case, and I was then able to reproduce the crash locally as well. [1] libguestfs/libguestfs#139 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2275252 Link: https://lkml.kernel.org/r/20240418124043.GC1055428@cmpxchg.org Link: https://lkml.kernel.org/r/20240417143324.GA1055428@cmpxchg.org Fixes: b5ba474 ("zswap: shrink zswap pool based on memory pressure") Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Reported-by: Christian Heusel <christian@heusel.eu> Debugged-by: Nhat Pham <nphamcs@gmail.com> Suggested-by: Nhat Pham <nphamcs@gmail.com> Tested-by: Christian Heusel <christian@heusel.eu> Acked-by: Yosry Ahmed <yosryahmed@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Dan Streetman <ddstreet@ieee.org> Cc: Richard W.M. Jones <rjones@redhat.com> Cc: Seth Jennings <sjenning@redhat.com> Cc: Vitaly Wool <vitaly.wool@konsulko.com> Cc: <stable@vger.kernel.org> [v6.8] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Conflicts: mm/zswap.c (cherry picked from commit 682886e) Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff
force-pushed
the
linux-6.6.y-2026-07-30-mm-2
branch
from
July 31, 2026 14:41
4b4f5e9 to
2087348
Compare
|
@opsiff: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link: https://lore.kernel.org/all/20231130194023.4102148-1-nphamcs@gmail.com/T/
Changelog:
v8:
limit reclaim (suggested by Michal Hocko and Johannes Weiner)
(patch 2 and patch 3)
default. (suggested by Johannes Weiner) (patch 6)
v7:
behavior (suggested by Andrew Morton) (patch 2)
v6:
manipulation functions (patch 1)
v5:
zswap lru modifications (suggested by Yosry)
online cgroup.
offlined (suggested by Yosry Ahmed, Johannes Weiner)
v4:
list_lru_add (patch 1) (suggested by Johannes Weiner and
Yosry Ahmed)
(suggested by Yosry Ahmed)
(patch 3) (suggested by Yosry Ahmed)
zswap_lruvec_state (patch 5) (suggested by Yosry Ahmed)
v3:
that is not online for the global limit zswap writeback (patch 2)
(suggested by Yosry Ahmed)
__read_swapcache_async() (patch 2) (suggested by Yosry Ahmed)
(reported by Ryan Roberts)
(patch 5) (suggested by Yosry Ahmed)
(suggested by Yosry Ahmed)
v2:
There are currently several issues with zswap writeback:
There is only a single global LRU for zswap, making it impossible to
perform worload-specific shrinking - an memcg under memory pressure
cannot determine which pages in the pool it owns, and often ends up
writing pages from other memcgs. This issue has been previously
observed in practice and mitigated by simply disabling
memcg-initiated shrinking:
https://lore.kernel.org/all/20230530232435.3097106-1-nphamcs@gmail.com/T/#u
But this solution leaves a lot to be desired, as we still do not
have an avenue for an memcg to free up its own memory locked up in
the zswap pool.
We only shrink the zswap pool when the user-defined limit is hit.
This means that if we set the limit too high, cold data that are
unlikely to be used again will reside in the pool, wasting precious
memory. It is hard to predict how much zswap space will be needed
ahead of time, as this depends on the workload (specifically, on
factors such as memory access patterns and compressibility of the
memory pages).
This patch series solves these issues by separating the global zswap
LRU into per-memcg and per-NUMA LRUs, and performs workload-specific
(i.e memcg- and NUMA-aware) zswap writeback under memory pressure. The
new shrinker does not have any parameter that must be tuned by the
user, and can be opted in or out on a per-memcg basis.
As a proof of concept, we ran the following synthetic benchmark:
build the linux kernel in a memory-limited cgroup, and allocate some
cold data in tmpfs to see if the shrinker could write them out and
improved the overall performance. Depending on the amount of cold data
generated, we observe from 14% to 35% reduction in kernel CPU time used
in the kernel builds.
Domenico Cerasuolo (3):
zswap: make shrinking memcg-aware
mm: memcg: add per-memcg zswap writeback stat
selftests: cgroup: update per-memcg zswap writeback selftest
Nhat Pham (3):
list_lru: allows explicit memcg and NUMA node selection
memcontrol: implement mem_cgroup_tryget_online()
zswap: shrinks zswap pool based on memory pressure
Documentation/admin-guide/mm/zswap.rst | 10 +
drivers/android/binder_alloc.c | 7 +-
fs/dcache.c | 8 +-
fs/gfs2/quota.c | 6 +-
fs/inode.c | 4 +-
fs/nfs/nfs42xattr.c | 8 +-
fs/nfsd/filecache.c | 4 +-
fs/xfs/xfs_buf.c | 6 +-
fs/xfs/xfs_dquot.c | 2 +-
fs/xfs/xfs_qm.c | 2 +-
include/linux/list_lru.h | 54 ++-
include/linux/memcontrol.h | 15 +
include/linux/mmzone.h | 2 +
include/linux/vm_event_item.h | 1 +
include/linux/zswap.h | 27 +-
mm/Kconfig | 14 +
mm/list_lru.c | 48 ++-
mm/memcontrol.c | 3 +
mm/mmzone.c | 1 +
mm/swap.h | 3 +-
mm/swap_state.c | 26 +-
mm/vmstat.c | 1 +
mm/workingset.c | 4 +-
mm/zswap.c | 456 +++++++++++++++++---
tools/testing/selftests/cgroup/test_zswap.c | 74 ++--
25 files changed, 661 insertions(+), 125 deletions(-)
base-commit: 5cdba94229e58a39ca389ad99763af29e6b0c5a5
2.34.1
Summary by Sourcery
Introduce a memory-pressure-driven, memcg- and NUMA-aware zswap shrinker that performs targeted writeback and avoids reclaiming recently used or warm pages.
New Features:
Enhancements:
Tests: