Vladimir Davydov 60d3fd32a7
list_lru: introduce per-memcg lists ...
There are several FS shrinkers, including super_block::s_shrink, that
keep reclaimable objects in the list_lru structure. Hence to turn them
to memcg-aware shrinkers, it is enough to make list_lru per-memcg.
This patch does the trick. It adds an array of lru lists to the
list_lru_node structure (per-node part of the list_lru), one for each
kmem-active memcg, and dispatches every item addition or removal to the
list corresponding to the memcg which the item is accounted to. So now
the list_lru structure is not just per node, but per node and per memcg.
Not all list_lrus need this feature, so this patch also adds a new
method, list_lru_init_memcg, which initializes a list_lru as memcg
aware. Otherwise (i.e. if initialized with old list_lru_init), the
list_lru won't have per memcg lists.
Just like per memcg caches arrays, the arrays of per-memcg lists are
indexed by memcg_cache_id, so we must grow them whenever
memcg_nr_cache_ids is increased. So we introduce a callback,
memcg_update_all_list_lrus, invoked by memcg_alloc_cache_id if the id
space is full.
The locking is implemented in a manner similar to lruvecs, i.e. we have
one lock per node that protects all lists (both global and per cgroup) on
the node.
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com >
Cc: Dave Chinner <david@fromorbit.com >
Cc: Johannes Weiner <hannes@cmpxchg.org >
Cc: Michal Hocko <mhocko@suse.cz >
Cc: Greg Thelen <gthelen@google.com >
Cc: Glauber Costa <glommer@gmail.com >
Cc: Alexander Viro <viro@zeniv.linux.org.uk >
Cc: Christoph Lameter <cl@linux.com >
Cc: Pekka Enberg <penberg@kernel.org >
Cc: David Rientjes <rientjes@google.com >
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com >
Cc: Tejun Heo <tj@kernel.org >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2015-02-12 18:54:09 -08:00
2015-02-10 10:23:15 +00:00
2015-01-29 10:54:43 +02:00
2014-08-06 18:01:23 -07:00
2014-12-07 21:22:09 +01:00
2015-01-08 20:36:57 +03:00
2014-12-15 17:05:08 -08:00
2014-08-08 15:57:26 -07:00
2014-10-15 20:31:05 +05:30
2014-09-22 16:48:47 +09:00
2015-01-15 17:23:22 +01:00
2014-12-14 14:57:16 -08:00
2014-12-02 16:20:59 -08:00
2015-02-01 11:50:35 -08:00
2014-12-15 17:30:09 -08:00
2014-11-24 17:24:08 -05:00
2015-02-11 11:17:34 -08:00
2015-02-09 14:03:53 -08:00
2014-12-15 18:19:03 -08:00
2015-02-11 10:56:48 -08:00
2014-11-25 20:11:12 -08:00
2014-09-15 22:20:21 +02:00
2015-02-11 10:28:45 -08:00
2015-01-14 14:21:51 +01:00
2015-02-12 09:16:56 -08:00
2015-01-21 20:52:07 +01:00
2015-02-08 11:16:27 +08:00
2014-09-24 11:53:39 -07:00
2015-02-08 11:16:55 +08:00
2015-01-29 10:49:28 +02:00
2015-02-12 10:39:41 -08:00
2014-12-14 14:57:16 -08:00
2014-08-07 14:40:09 -04:00
2015-02-05 15:09:26 +01:00
2014-09-16 16:28:34 -06:00
2015-01-28 18:45:23 -05:00
2014-07-09 16:02:41 +02:00
2014-07-07 21:34:43 -07:00
2015-01-28 18:45:23 -05:00
2014-12-25 15:27:16 -05:00
2014-10-27 14:16:19 -04:00
2014-09-08 10:54:03 +02:00
2014-11-10 12:40:48 +01:00
2014-06-21 11:37:35 +01:00
2014-08-13 10:31:57 +02:00
2015-02-11 20:07:47 -08:00
2014-09-08 10:00:43 -06:00
2014-10-09 22:26:01 -04:00
2014-12-13 12:42:51 -08:00
2014-11-24 08:04:44 -07:00
2014-12-14 16:10:09 -08:00
2014-11-16 09:55:39 +01:00
2014-12-22 16:43:06 +00:00
2015-01-08 08:59:01 -07:00
2015-01-08 08:55:53 -07:00
2014-12-13 14:14:23 -08:00
2014-11-13 16:17:06 -08:00
2014-12-05 21:47:32 -08:00
2014-10-01 22:12:48 -04:00
2014-09-04 22:04:42 -04:00
2014-11-07 11:45:00 -08:00
2014-07-24 21:53:47 +10:00
2014-09-24 14:23:34 +08:00
2015-01-06 12:02:46 -05:00
2015-01-06 12:02:46 -05:00
2014-09-27 12:52:33 -07:00
2014-12-03 16:21:37 -08:00
2014-10-15 07:05:03 +02:00
2014-11-20 10:43:16 -04:00
2014-12-30 18:29:25 -05:00
2014-12-18 19:08:10 -08:00
2014-09-29 14:36:26 -04:00
2015-02-11 17:06:04 -08:00
2015-01-27 09:37:53 +00:00
2014-10-28 11:03:40 +01:00
2014-10-28 11:03:40 +01:00
2015-02-11 17:42:32 -08:00
2014-07-03 11:32:43 +01:00
2014-11-26 19:28:11 -08:00
2014-06-25 13:26:47 -07:00
2014-12-17 09:00:04 -04:00
2014-11-07 11:45:00 -08:00
2014-10-21 00:51:01 +02:00
2015-01-23 23:06:44 +01:00
2014-12-17 02:26:28 +01:00
2014-11-07 11:45:00 -08:00
2014-10-27 11:15:27 -04:00
2014-10-22 16:14:04 -06:00
2014-06-25 16:04:00 -07:00
2014-09-27 09:14:59 -06:00
2014-12-05 17:19:27 -06:00
2014-11-24 22:43:57 +08:00
2014-11-19 13:01:20 -05:00
2014-12-14 16:10:09 -08:00
2014-09-23 22:53:15 -07:00
2015-01-30 17:56:40 +09:00
2014-12-19 22:55:06 +01:00
2014-11-19 11:24:59 -05:00
2014-12-14 16:10:09 -08:00
2014-07-08 13:36:52 -07:00
2014-08-06 18:01:16 -07:00
2014-12-01 16:50:45 +00:00
2014-12-05 20:55:27 +05:30
2014-11-18 11:18:35 +01:00
2015-01-21 19:21:30 +01:00
2014-07-10 18:35:01 +02:00
2014-07-10 18:35:01 +02:00
2014-11-10 09:27:30 -07:00
2014-10-03 14:55:48 -07:00
2014-09-29 00:04:55 -04:00
2014-10-20 14:22:22 +02:00
2014-11-25 15:42:53 +01:00
2015-02-11 18:03:54 -08:00
2014-06-22 16:34:11 -06:00
2014-11-24 07:45:02 +01:00
2015-01-09 15:44:19 +01:00
2015-01-02 15:46:41 -05:00
2014-12-08 21:07:10 -05:00
2015-02-05 14:35:17 +01:00
2014-11-04 17:34:15 -08:00
2014-12-13 12:42:52 -08:00
2015-01-15 13:33:41 +02:00
2014-12-31 13:06:50 -05:00
2014-11-03 11:53:34 -05:00
2014-12-10 17:41:10 -08:00
2014-12-05 21:47:32 -08:00
2014-07-08 15:28:16 -07:00
2014-09-08 09:51:30 +09:00
2014-10-09 11:35:48 +03:00
2014-11-04 07:17:46 +01:00
2014-10-08 16:01:41 -04:00
2014-08-07 14:40:08 -04:00
2015-02-12 18:54:08 -08:00
2014-11-05 14:45:53 -08:00
2014-10-15 21:31:27 +05:30
2014-12-13 12:42:53 -08:00
2015-02-10 14:30:28 -08:00
2015-02-12 08:37:41 -08:00
2014-12-15 10:17:38 -05:00
2014-10-09 22:25:52 -04:00
2015-01-16 17:04:25 -05:00
2014-10-14 17:03:53 -04:00
2015-02-11 17:06:04 -08:00
2014-08-06 18:01:24 -07:00
2014-11-04 21:58:23 +01:00
2014-10-28 17:30:59 +01:00
2014-12-10 15:17:46 -05:00
2015-01-27 10:19:12 -02:00
2014-12-17 09:13:13 +01:00
2014-08-06 18:01:22 -07:00
2014-08-04 10:07:36 +02:00
2015-01-23 12:13:20 +01:00
2015-02-12 18:54:08 -08:00
2014-12-10 17:41:04 -08:00
2015-02-11 17:06:01 -08:00
2014-07-15 04:49:40 -04:00
2014-12-08 20:24:11 -05:00
2015-01-26 22:05:20 +01:00
2015-01-27 11:09:13 +01:00
2014-11-17 09:49:17 +01:00
2015-01-18 00:23:57 -05:00
2014-09-29 15:37:01 -04:00
2015-02-05 14:33:28 -08:00
2014-09-04 22:26:14 -07:00
2014-09-09 10:28:43 -04:00
2014-11-14 17:08:58 -05:00
2014-12-09 21:21:34 -08:00
2014-11-17 23:12:01 -05:00
2014-07-04 12:35:59 +02:00
2014-12-12 15:15:41 -05:00
2014-11-07 10:09:07 -08:00
2014-12-16 14:53:01 -08:00
2015-01-19 16:11:59 +00:00
2014-09-04 14:41:43 -06:00
2015-01-19 14:55:22 +01:00
2014-09-23 23:13:13 -07:00
2014-12-16 15:53:03 -08:00
2014-12-11 15:04:11 -06:00
2014-12-11 15:04:06 -06:00
2015-02-02 19:28:04 -08:00
2014-09-13 18:38:15 +02:00
2014-12-10 09:01:01 -08:00
2014-10-09 16:07:43 -04:00
2014-11-23 13:01:46 +01:00
2014-11-23 13:01:46 +01:00
2014-08-02 15:03:58 -07:00
2015-01-15 10:34:54 +01:00
2015-01-15 10:34:54 +01:00
2014-09-12 13:59:03 -07:00
2014-08-10 09:00:16 +02:00
2014-11-11 09:31:52 -06:00
2014-12-10 17:41:11 -08:00
2014-12-13 13:33:07 +01:00
2015-02-10 18:35:40 -08:00
2014-10-14 15:05:34 -04:00
2014-11-07 10:53:25 -08:00
2014-10-14 02:18:21 +02:00
2014-09-16 17:36:08 +01:00
2014-07-22 21:55:45 +01:00
2014-09-09 10:34:56 +02:00
2014-10-22 16:14:04 -06:00
2014-10-29 16:33:14 -07:00
2014-08-06 18:01:24 -07:00
2014-12-13 12:42:47 -08:00
2015-01-13 16:10:16 +00:00
2014-07-28 14:16:30 -04:00
2015-01-24 10:11:12 +01:00
2015-02-11 17:06:05 -08:00
2014-12-18 09:39:51 +01:00
2014-12-15 18:28:25 -08:00
2015-02-09 07:54:41 -05:00
2015-02-12 18:54:09 -08:00
2015-01-04 23:11:43 -05:00
2014-11-20 14:45:15 +01:00
2015-02-04 11:25:51 +01:00
2014-10-13 15:51:40 +02:00
2014-11-27 12:50:59 +05:30
2014-10-08 10:39:41 +05:30
2014-10-24 15:49:20 -04:00
2014-11-30 16:40:11 +00:00
2014-09-29 11:56:02 -04:00
2014-08-06 18:01:15 -07:00
2015-02-12 18:54:09 -08:00
2014-10-09 22:25:52 -04:00
2014-10-22 16:14:04 -06:00
2014-10-09 22:25:56 -04:00
2014-07-11 18:27:56 -07:00
2014-11-21 14:08:56 -05:00
2015-02-12 18:54:08 -08:00
2015-02-11 17:06:04 -08:00
2015-02-12 18:54:09 -08:00
2014-10-09 22:25:58 -04:00
2014-12-15 15:52:01 -08:00
2015-02-11 17:06:02 -08:00
2015-02-03 20:31:05 -06:00
2015-01-22 11:15:54 +10:30
2015-01-20 11:38:33 +10:30
2014-10-14 02:18:28 +02:00
2014-10-24 00:14:36 +02:00
2014-11-23 17:14:43 +01:00
2014-08-13 10:32:03 +02:00
2014-07-16 12:58:36 +00:00
2014-12-10 21:32:13 -05:00
2015-02-01 23:16:33 -08:00
2015-02-08 21:54:09 -08:00
2014-09-26 18:42:31 +02:00
2014-08-25 10:45:28 +02:00
2014-12-27 03:07:50 -05:00
2015-02-12 10:39:41 -08:00
2015-02-03 11:06:34 -08:00
2014-11-24 17:00:42 -05:00
2015-02-03 11:06:52 -08:00
2015-02-03 11:06:45 -08:00
2015-02-05 22:16:45 -05:00
2014-10-25 08:07:30 +02:00
2014-10-14 02:18:27 +02:00
2014-08-06 18:01:21 -07:00
2014-12-10 21:30:20 -05:00
2014-07-29 18:08:50 -07:00
2014-11-04 13:18:52 -07:00
2014-11-04 16:43:08 +00:00
2014-07-26 00:21:41 +05:30
2014-07-29 21:26:45 -06:00
2015-01-15 17:23:17 +01:00
2014-12-05 14:35:52 +00:00
2014-07-02 00:24:14 -07:00
2014-11-12 07:49:38 -07:00
2014-11-04 13:29:38 +00:00
2014-11-24 22:25:06 +00:00
2014-10-29 16:33:14 -07:00
2014-12-11 13:06:58 -08:00
2014-07-09 14:58:37 +01:00
2014-09-18 09:53:25 -07:00
2014-11-20 12:11:25 -08:00
2014-07-29 12:38:07 +02:00
2014-11-27 12:51:04 +05:30
2015-02-11 17:06:03 -08:00
2015-01-14 15:07:32 +01:00
2015-02-11 17:06:02 -08:00
2015-02-11 17:06:07 -08:00
2014-12-13 12:42:48 -08:00
2015-01-28 14:03:09 +00:00
2014-11-13 16:17:05 -08:00
2014-12-29 12:45:45 -08:00
2014-11-13 10:35:25 -07:00
2014-12-14 14:57:16 -08:00
2014-11-05 13:06:16 -07:00
2015-02-10 15:09:41 -08:00
2014-07-09 14:58:37 +01:00
2014-09-08 09:51:29 +09:00
2014-12-13 12:42:53 -08:00
2015-01-06 10:26:10 -05:00
2014-11-21 11:19:15 -05:00
2015-02-11 18:15:38 -08:00
2015-01-09 11:12:28 +01:00
2014-12-16 15:02:23 -05:00
2015-01-27 00:16:51 -08:00
2014-12-04 14:31:00 -05:00
2014-10-08 10:39:33 +05:30
2014-11-06 15:16:02 -08:00
2014-11-20 14:45:15 +01:00
2014-11-07 01:46:51 +01:00
2015-02-03 22:56:53 +01:00
2014-11-29 23:53:20 +01:00
2014-12-04 00:46:58 +01:00
2014-12-12 15:26:48 -08:00
2015-02-03 22:59:25 +01:00
2014-10-19 20:22:32 +03:00
2014-10-28 03:30:19 +01:00
2015-01-26 13:37:18 -08:00
2014-09-17 16:33:11 -04:00
2014-12-10 21:30:20 -05:00
2014-11-04 21:58:23 +01:00
2014-09-08 09:51:30 +09:00
2015-01-16 16:01:10 -08:00
2015-01-16 16:01:10 -08:00
2014-07-07 16:57:18 -07:00
2014-12-10 17:41:10 -08:00
2014-12-22 20:17:23 +00:00
2014-10-28 16:48:02 -04:00
2015-01-30 12:51:21 +01:00
2015-01-30 12:50:32 +01:00
2014-08-24 18:36:01 -07:00
2014-06-25 11:22:03 -07:00
2014-12-13 12:42:52 -08:00
2014-10-14 02:18:21 +02:00
2015-01-06 11:01:13 -08:00
2015-01-15 23:34:34 -08:00
2015-01-15 23:34:34 -08:00
2015-01-15 23:34:34 -08:00
2014-09-26 00:00:06 -07:00
2015-01-27 13:13:04 +00:00
2014-07-08 13:41:08 -07:00
2014-10-20 10:11:29 +02:00
2014-10-20 10:11:29 +02:00
2015-02-05 15:09:25 +01:00
2015-02-04 20:34:52 -08:00
2014-11-10 16:45:43 -05:00
2014-08-08 15:57:24 -07:00
2015-02-10 14:30:31 -08:00
2014-06-19 21:17:37 -07:00
2015-01-23 17:21:57 -08:00
2014-06-21 22:05:30 +02:00
2014-12-09 13:36:57 -05:00
2014-07-16 14:56:55 +02:00
2014-08-13 10:32:03 +02:00
2014-08-08 15:57:26 -07:00
2014-12-13 12:42:51 -08:00
2014-10-09 22:26:04 -04:00
2014-09-03 14:58:17 -07:00
2014-10-12 10:13:55 -04:00
2014-11-19 22:01:19 -05:00
2014-10-29 20:26:06 -04:00
2014-09-19 12:35:16 +02:00
2014-08-28 11:59:38 +05:30
2014-11-06 14:59:21 -08:00
2014-11-06 14:57:24 -08:00
2014-11-25 17:06:40 -08:00
2014-07-23 01:10:45 +02:00
2014-07-09 10:55:57 +02:00
2014-07-02 16:01:51 +02:00
2014-08-08 15:57:26 -07:00
2014-08-08 15:57:31 -07:00
2015-02-12 18:54:09 -08:00
2014-10-14 02:18:20 +02:00
2015-02-08 16:28:26 -08:00
2014-10-09 22:25:51 -04:00
2015-02-10 14:30:34 -08:00
2015-01-22 15:10:56 +01:00
2015-02-04 01:34:15 -05:00
2015-01-03 14:32:57 -05:00
2015-01-03 14:32:57 -05:00
2015-01-03 14:32:57 -05:00
2014-06-12 00:21:11 -04:00
2014-11-26 19:33:57 -08:00
2015-01-06 11:04:27 -08:00
2014-12-13 12:42:48 -08:00
2014-07-31 14:13:29 -07:00
2014-10-14 02:18:26 +02:00
2014-10-24 12:33:32 -07:00
2014-09-30 21:06:07 +02:00
2014-12-10 17:41:09 -08:00
2015-02-11 17:06:00 -08:00
2015-02-12 18:54:08 -08:00
2015-01-27 09:38:00 +00:00
2014-08-08 15:57:24 -07:00
2014-11-07 10:53:25 -08:00
2014-11-17 10:28:04 -08:00
2014-09-27 09:14:59 -06:00
2015-02-08 01:03:13 -08:00
2014-12-21 22:49:12 +08:00
2014-09-26 08:15:29 +01:00
2014-10-14 03:47:00 +02:00
2014-07-23 10:17:53 -07:00
2015-01-22 12:28:02 +01:00
2015-01-02 16:47:35 -05:00
2014-10-29 15:15:40 +01:00
2015-01-23 17:21:55 -08:00
2014-07-18 11:49:57 +02:00
2014-10-09 22:25:51 -04:00
2014-09-16 13:41:06 -07:00
2014-07-29 23:10:56 +02:00
2014-11-19 22:01:15 -05:00
2014-08-06 13:03:43 +02:00
2015-02-07 19:34:25 -05:00
2014-09-23 21:19:35 -07:00
2014-11-06 14:57:27 -08:00
2014-12-30 18:29:25 -05:00
2015-01-05 22:44:46 -05:00
2014-11-07 10:09:07 -08:00
2015-02-04 01:34:16 -05:00
2014-10-22 16:14:04 -06:00
2014-09-23 21:40:48 -07:00
2014-12-04 00:51:54 +01:00
2014-12-17 12:31:40 -08:00
2014-12-04 14:31:00 -05:00
2014-07-09 14:58:37 +01:00
2014-11-28 16:08:16 +01:00
2014-08-08 10:39:16 -06:00
2014-09-18 19:21:20 -04:00
2014-09-16 15:02:55 -06:00
2014-12-09 12:05:24 +02:00
2014-12-14 18:21:31 +02:00
2014-12-17 10:37:56 -08:00
2014-12-13 12:42:48 -08:00
2014-08-06 18:01:19 -07:00
2015-02-04 01:34:14 -05:00
2014-12-15 23:49:28 +02:00
2015-02-04 07:52:32 +01:00
2014-10-20 20:46:25 +02:00
2015-01-06 18:16:30 -05:00
2015-01-08 15:10:51 -08:00
2014-08-06 18:01:23 -07:00
2014-08-08 15:57:26 -07:00
2014-08-06 18:01:23 -07:00
2014-10-09 22:26:02 -04:00
60d3fd32a7