Jason Baron d430d3d7e6
jump label: Introduce static_branch() interface ...
Introduce:
static __always_inline bool static_branch(struct jump_label_key *key);
instead of the old JUMP_LABEL(key, label) macro.
In this way, jump labels become really easy to use:
Define:
struct jump_label_key jump_key;
Can be used as:
if (static_branch(&jump_key))
do unlikely code
enable/disale via:
jump_label_inc(&jump_key);
jump_label_dec(&jump_key);
that's it!
For the jump labels disabled case, the static_branch() becomes an
atomic_read(), and jump_label_inc()/dec() are simply atomic_inc(),
atomic_dec() operations. We show testing results for this change below.
Thanks to H. Peter Anvin for suggesting the 'static_branch()' construct.
Since we now require a 'struct jump_label_key *key', we can store a pointer into
the jump table addresses. In this way, we can enable/disable jump labels, in
basically constant time. This change allows us to completely remove the previous
hashtable scheme. Thanks to Peter Zijlstra for this re-write.
Testing:
I ran a series of 'tbench 20' runs 5 times (with reboots) for 3
configurations, where tracepoints were disabled.
jump label configured in
avg: 815.6
jump label *not* configured in (using atomic reads)
avg: 800.1
jump label *not* configured in (regular reads)
avg: 803.4
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl >
LKML-Reference: <20110316212947.GA8792@redhat.com >
Signed-off-by: Jason Baron <jbaron@redhat.com >
Suggested-by: H. Peter Anvin <hpa@linux.intel.com >
Tested-by: David Daney <ddaney@caviumnetworks.com >
Acked-by: Ralf Baechle <ralf@linux-mips.org >
Acked-by: David S. Miller <davem@davemloft.net >
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com >
Signed-off-by: Steven Rostedt <rostedt@goodmis.org >
2011-04-04 12:48:08 -04:00
2011-03-20 09:32:12 +00:00
2010-08-14 22:26:51 +02:00
2011-01-30 01:14:14 -08:00
2010-10-18 15:16:08 +02:00
2011-03-22 11:33:55 -07:00
2010-10-20 15:37:28 -07:00
2011-01-13 08:03:25 -08:00
2010-08-14 22:26:51 +02:00
2011-03-23 10:42:00 +01:00
2011-03-18 23:38:50 -07:00
2010-12-16 12:37:26 -05:00
2011-03-28 08:47:17 +02:00
2011-03-23 12:24:23 -07:00
2011-03-25 01:24:57 +09:00
2011-03-25 17:41:20 +02:00
2011-03-20 15:33:26 +01:00
2010-10-13 18:00:51 +02:00
2011-01-20 17:50:17 +01:00
2011-01-20 17:50:17 +01:00
2011-01-20 17:50:17 +01:00
2011-01-13 08:03:19 -08:00
2011-01-13 21:04:07 -05:00
2011-03-16 15:11:04 -07:00
2011-03-01 22:24:19 +03:00
2010-08-12 06:38:24 +10:00
2011-03-26 14:15:06 +00:00
2011-03-19 22:27:06 -07:00
2011-02-18 16:54:54 -05:00
2011-03-24 13:52:42 -04:00
2010-08-20 01:42:59 -07:00
2010-08-24 14:48:10 -07:00
2011-01-13 08:03:08 -08:00
2011-03-16 15:11:04 -07:00
2011-02-24 19:58:42 +01:00
2010-10-15 21:18:59 +02:00
2011-01-20 18:30:17 -08:00
2011-03-21 22:59:08 -04:00
2010-11-23 20:14:46 +00:00
2010-10-22 10:20:08 -07:00
2011-03-15 02:42:32 -04:00
2011-01-31 14:03:00 -08:00
2010-12-10 15:45:05 -08:00
2010-11-12 07:55:30 -08:00
2011-01-19 23:51:37 +01:00
2011-01-15 20:07:45 -05:00
2010-08-09 00:13:34 +02:00
2010-12-06 15:58:43 -05:00
2011-03-10 08:52:07 +01:00
2011-03-22 17:43:59 -07:00
2010-10-27 18:03:06 -07:00
2011-03-11 14:25:50 +00:00
2011-01-10 13:31:14 -08:00
2011-01-13 08:03:12 -08:00
2011-03-08 08:28:01 +01:00
2011-01-07 17:50:31 +11:00
2010-11-15 13:27:04 -05:00
2011-03-10 08:52:27 +01:00
2011-03-11 20:17:08 +01:00
2011-03-03 10:53:20 -05:00
2011-03-23 19:47:19 -07:00
2011-03-10 08:52:07 +01:00
2011-03-23 19:47:06 -07:00
2011-01-13 08:03:17 -08:00
2010-12-16 17:53:38 +01:00
2011-02-16 13:30:48 +01:00
2011-02-16 13:30:48 +01:00
2010-11-26 10:51:04 +00:00
2010-12-20 09:07:35 -08:00
2010-10-25 08:02:40 -07:00
2011-03-22 17:44:00 -07:00
2010-09-14 16:08:45 -07:00
2011-03-22 17:44:10 -07:00
2011-03-22 17:44:10 -07:00
2011-03-22 17:44:10 -07:00
2010-08-19 17:17:59 -07:00
2011-01-05 14:15:50 +01:00
2011-02-23 13:22:26 -08:00
2010-08-10 13:47:42 -07:00
2011-01-26 10:50:06 +10:00
2010-10-14 14:32:06 -07:00
2011-03-21 22:59:08 -04:00
2011-01-24 14:51:56 -08:00
2010-11-10 16:57:11 -08:00
2011-03-16 17:54:33 -04:00
2011-01-12 12:47:34 -05:00
2011-01-13 08:03:22 -08:00
2010-11-30 13:39:55 +00:00
2011-03-22 17:44:15 -07:00
2011-03-23 19:47:02 -07:00
2011-01-13 08:03:13 -08:00
2011-03-22 19:25:05 -07:00
2011-03-21 12:13:10 -04:00
2011-03-14 17:02:43 -07:00
2011-01-20 17:01:09 -08:00
2011-01-10 08:51:44 -08:00
2010-08-17 09:11:10 +02:00
2011-03-08 16:10:38 +01:00
2011-03-10 08:52:07 +01:00
2011-03-23 22:16:41 +01:00
2010-09-07 14:17:10 -05:00
2011-03-24 13:54:30 +00:00
2011-01-13 19:59:52 +00:00
2010-09-22 17:22:38 -07:00
2011-01-17 10:54:41 -08:00
2010-11-26 09:57:36 +01:00
2011-02-25 12:00:34 -08:00
2011-03-10 11:45:31 +01:00
2011-03-10 11:35:17 +01:00
2011-03-10 11:35:15 +01:00
2011-03-10 11:48:15 +01:00
2011-03-07 01:12:28 +05:30
2011-04-04 12:48:08 -04:00
2010-10-21 14:47:59 +02:00
2011-03-14 08:40:51 -07:00
2011-03-21 10:14:27 +01:00
2011-03-22 17:44:11 -07:00
2011-01-13 21:49:56 -08:00
2011-03-22 01:00:26 -07:00
2011-03-15 02:21:37 -04:00
2011-03-23 19:46:17 -07:00
2011-01-12 20:16:43 -05:00
2010-12-15 13:56:33 -05:00
2011-03-17 09:19:20 +08:00
2011-03-14 19:12:20 -04:00
2010-10-18 11:03:14 +02:00
2010-08-19 17:18:03 -07:00
2011-01-09 15:42:55 -08:00
2011-03-15 02:21:45 -04:00
2010-12-08 10:30:34 -08:00
2010-11-01 15:38:34 -04:00
2011-03-21 10:05:22 -07:00
2011-01-10 08:51:44 -08:00
2011-02-03 15:39:17 -08:00
2010-11-30 17:28:57 -05:00
2011-02-16 17:48:59 +01:00
2011-01-07 17:50:27 +11:00
2011-03-28 15:31:05 -07:00
2010-10-22 10:22:17 -07:00
2010-11-15 13:49:51 +09:00
2011-01-07 17:50:23 +11:00
2011-01-07 17:50:23 +11:00
2011-03-10 10:31:48 -05:00
2011-02-11 16:23:33 -05:00
2010-12-07 20:16:56 +01:00
2010-09-14 00:23:59 -07:00
2011-01-07 17:50:29 +11:00
2011-03-22 08:35:35 +01:00
2011-03-22 17:44:05 -07:00
2010-12-06 17:15:15 -08:00
2010-10-25 14:11:37 -07:00
2011-01-10 22:11:23 +01:00
2011-01-14 08:06:39 -08:00
2011-03-05 10:56:00 +01:00
2011-02-03 16:37:28 +01:00
2011-03-17 14:31:46 +01:00
2010-09-24 14:03:44 +02:00
2011-02-11 15:05:49 +01:00
2010-12-17 15:18:04 +01:00
2011-03-10 19:15:58 +01:00
2010-10-12 16:53:37 +02:00
2011-02-15 15:21:11 -08:00
2010-08-11 09:23:22 +02:00
2010-10-08 09:32:46 +02:00
2010-11-12 14:51:55 +01:00
2011-02-17 09:52:03 -08:00
2010-08-11 18:21:02 +02:00
2010-11-09 09:31:10 -08:00
2011-02-23 00:53:26 +00:00
2011-03-20 14:50:53 +01:00
2011-03-12 15:08:54 -08:00
2010-10-26 17:40:56 -07:00
2011-03-09 16:10:56 -05:00
2010-11-19 17:47:57 +08:00
2010-10-05 20:26:58 -07:00
2011-01-11 23:55:51 +01:00
2011-01-06 11:10:05 -08:00
2010-08-22 21:15:39 -07:00
2010-08-22 21:15:39 -07:00
2010-10-18 11:03:14 +02:00
2011-01-19 23:31:09 -08:00
2010-11-16 10:58:30 -08:00
2011-03-18 15:14:45 -07:00
2010-09-21 18:04:47 -07:00
2010-11-12 12:30:57 -08:00
2011-02-27 16:11:51 -08:00
2011-03-10 16:37:26 -08:00
2011-02-10 07:51:44 -05:00
2010-10-21 16:08:28 +02:00
2010-08-19 17:16:23 -07:00
2011-02-18 12:42:28 -08:00
2011-01-06 10:23:33 -08:00
2010-10-22 10:21:26 -07:00
2011-01-31 21:17:41 -08:00
2011-03-28 06:05:21 -04:00
2010-10-07 15:03:44 -07:00
2011-03-15 19:23:40 -07:00
2010-10-26 16:52:08 -07:00
2010-11-11 13:40:11 +01:00
2010-08-09 20:45:05 -07:00
2010-12-17 10:01:09 -08:00
2010-11-25 10:42:59 +09:00
2011-03-23 19:47:08 -07:00
2010-12-14 00:21:51 -05:00
2010-12-14 00:21:51 -05:00
2010-11-24 11:16:42 -08:00
2010-10-18 19:58:50 +02:00
2011-03-28 20:13:57 +02:00
2011-03-23 20:22:06 +01:00
2010-10-07 14:08:55 +01:00
2010-10-28 11:58:39 +02:00
2010-08-10 13:47:40 -07:00
2011-03-20 21:13:43 -04:00
2010-08-10 11:49:21 -07:00
2010-12-09 20:17:07 -08:00
2011-01-26 12:33:20 +01:00
2011-03-20 20:08:48 -04:00
2011-04-04 12:48:08 -04:00
2010-08-12 11:27:58 +02:00
2011-03-24 08:36:10 +01:00
2011-03-22 16:17:32 -07:00
2011-03-22 04:53:54 -03:00
2011-02-17 11:12:40 -08:00
2010-10-22 15:34:12 -05:00
2011-01-13 17:32:31 -08:00
2011-03-22 17:44:14 -07:00
2011-03-08 11:17:18 +11:00
2011-03-08 11:17:18 +11:00
2011-03-08 11:17:22 +11:00
2010-10-27 18:03:18 -07:00
2011-03-18 16:54:31 +00:00
2011-01-13 17:32:47 -08:00
2011-02-13 16:54:24 -08:00
2011-01-24 14:45:11 +10:30
2011-01-13 08:03:07 -08:00
2010-08-23 18:12:46 -07:00
2011-02-18 08:02:10 -08:00
2011-01-07 17:02:58 -08:00
2011-01-07 08:43:37 +01:00
2010-09-09 18:57:24 -07:00
2011-03-22 17:44:01 -07:00
2011-03-17 13:08:30 -03:00
2010-10-24 10:50:49 +02:00
2011-01-12 11:23:08 +02:00
2011-01-12 11:23:12 +02:00
2011-03-22 17:43:59 -07:00
2011-01-13 08:03:06 -08:00
2011-01-13 08:03:05 -08:00
2011-03-25 21:00:29 -07:00
2010-09-09 09:09:43 -07:00
2011-03-14 02:52:41 -04:00
2010-10-25 14:11:39 -07:00
2011-01-14 13:12:45 +00:00
2011-02-18 11:32:28 -08:00
2011-01-21 13:38:57 -08:00
2011-03-18 22:33:38 -07:00
2010-11-22 08:34:23 -08:00
2010-10-26 16:52:19 -07:00
2011-01-10 19:04:08 +01:00
2010-12-06 11:03:46 -08:00
2011-03-22 04:53:56 -03:00
2010-10-12 15:37:51 -07:00
2011-03-23 19:46:25 -07:00
2011-01-15 13:28:17 +02:00
2011-02-03 16:08:57 -08:00
2010-08-09 20:44:56 -07:00
2011-02-14 17:38:30 -08:00
2011-01-13 17:32:34 -08:00
2010-08-21 00:37:40 -07:00
2011-01-13 17:32:46 -08:00
2011-03-22 17:44:03 -07:00
2011-03-24 17:49:37 -07:00
2010-08-09 20:44:58 -07:00
2011-01-13 17:32:46 -08:00
2011-02-15 10:24:31 +01:00
2011-02-21 15:21:53 -08:00
2011-01-24 14:32:52 +10:30
2011-01-16 13:47:07 -05:00
2011-02-03 18:05:29 -08:00
2011-01-30 01:14:38 -08:00
2010-10-12 16:53:34 +02:00
2011-03-02 13:43:15 -08:00
2010-10-08 17:16:22 -05:00
2010-11-26 15:05:34 +01:00
2010-10-18 11:03:14 +02:00
2011-03-18 09:04:20 -04:00
2010-08-22 21:15:39 -07:00
2011-01-12 20:03:43 -05:00
2011-01-12 20:03:43 -05:00
2010-08-22 21:15:39 -07:00
2011-02-22 10:19:31 -08:00
2011-03-16 12:53:54 -07:00
2011-01-18 16:08:30 +01:00
2011-03-03 13:32:07 -08:00
2010-10-18 08:32:07 -07:00
2010-12-16 12:37:23 -05:00
2011-03-24 17:03:14 -04:00
2011-03-11 15:39:27 -05:00
2011-03-23 15:29:04 -04:00
2011-03-11 15:39:26 -05:00
2011-03-11 15:38:44 -05:00
2010-09-23 08:55:58 -04:00
2011-03-27 17:48:07 +02:00
2011-03-24 17:03:14 -04:00
2011-01-25 15:24:47 -05:00
2011-03-10 17:29:40 +09:00
2011-03-01 13:48:21 -05:00
2010-12-22 22:15:32 +01:00
2011-02-03 16:08:57 -08:00
2010-08-19 17:18:02 -07:00
2010-12-23 15:57:48 -07:00
2011-02-28 01:36:38 -07:00
2011-01-15 22:01:58 -07:00
2010-10-21 11:34:55 -06:00
2010-11-01 01:08:14 -04:00
2011-02-04 11:46:50 -07:00
2010-10-12 21:58:08 -06:00
2011-03-23 14:55:56 -06:00
2011-03-16 17:28:10 -07:00
2011-03-22 04:53:46 -03:00
2010-08-08 10:02:59 -07:00
2010-08-11 08:59:19 -07:00
2010-10-17 01:57:50 +02:00
2011-02-15 11:10:20 +01:00
2010-09-03 19:09:46 +08:00
2011-03-23 19:46:28 -07:00
2011-03-22 17:44:03 -07:00
2010-10-26 16:52:05 -07:00
2011-03-24 10:16:26 -07:00
2011-03-14 02:58:03 -04:00
2011-01-16 13:47:07 -05:00
2011-03-23 10:42:08 +01:00
2010-12-23 12:54:03 -08:00
2011-01-16 11:56:26 -08:00
2011-03-21 09:40:43 -07:00
2011-03-21 09:38:02 -07:00
2010-10-27 21:30:13 -04:00
2010-10-27 17:53:25 +02:00
2011-02-28 11:20:03 +01:00
2011-04-04 12:48:08 -04:00
2011-03-09 11:59:33 -08:00
2010-10-24 15:07:11 -07:00
2011-03-23 19:46:57 -07:00
2011-01-10 07:38:54 -08:00
2010-08-21 23:40:14 -07:00
2011-02-24 22:11:56 -08:00
2011-02-18 08:15:51 -08:00
2011-03-11 15:13:26 -05:00
2011-03-15 00:43:14 +01:00
2011-03-15 00:43:14 +01:00
2011-03-23 22:16:41 +01:00
2010-08-11 09:21:15 +02:00
2010-12-10 15:06:43 +01:00
2011-01-25 15:24:47 -05:00
2011-02-02 15:28:20 +01:00
2011-02-02 15:28:20 +01:00
2011-03-01 22:27:26 +03:00
2011-01-13 08:03:21 -08:00
2011-01-13 08:03:21 -08:00
2011-01-13 08:03:10 -08:00
2011-03-23 19:46:37 -07:00
2010-12-28 14:25:21 -08:00
2011-03-04 09:23:30 -08:00
2011-03-22 17:44:00 -07:00
2010-12-02 17:55:12 +01:00
2010-08-18 23:30:42 -07:00
2011-01-12 19:14:55 +01:00
2011-03-24 21:16:31 -04:00
2011-01-13 17:32:48 -08:00
2010-10-29 04:16:31 -04:00
2010-10-27 18:03:18 -07:00
2010-11-29 11:17:22 -08:00
2010-10-26 16:52:16 -07:00
2011-01-14 02:36:43 +00:00
2010-08-19 17:18:00 -07:00
2010-12-17 12:34:59 -08:00
2010-12-17 12:34:08 -08:00
2010-11-29 22:01:58 -08:00
2010-11-29 22:01:58 -08:00
2010-09-08 18:16:55 -07:00
2010-08-09 16:47:27 -04:00
2011-03-23 19:46:18 -07:00
2011-02-01 11:12:29 -05:00
2011-03-23 19:46:23 -07:00
2010-11-12 07:55:30 -08:00
2010-10-17 01:57:50 +02:00
2010-11-15 13:24:06 -05:00
2011-03-09 13:52:27 -05:00
2011-03-23 19:46:41 -07:00
2011-01-13 08:03:18 -08:00
2011-02-25 15:07:37 -08:00
2011-03-23 19:46:43 -07:00
2011-03-22 17:44:03 -07:00
2011-01-13 08:03:23 -08:00
2011-03-09 11:23:37 -08:00
2010-11-15 11:29:30 -08:00
2011-01-27 12:30:38 +01:00
2011-01-27 12:30:39 +01:00
2011-01-27 12:30:40 +01:00
2010-12-22 02:39:57 +03:00
2011-03-25 17:59:38 -07:00
2011-03-23 19:47:02 -07:00
2010-10-22 10:20:06 -07:00
2010-10-21 10:12:48 +11:00
2010-10-30 12:12:50 +02:00
2011-01-07 17:50:27 +11:00
2010-10-22 10:20:10 -07:00
2011-01-13 08:03:19 -08:00
2010-08-10 13:47:46 -07:00
2010-11-30 14:11:49 -08:00
2011-01-19 15:37:31 +09:00
2010-09-03 17:29:04 -07:00
2010-10-18 09:33:31 -07:00
2010-11-09 14:45:52 +01:00
2010-11-15 18:48:25 +09:00
2010-11-09 16:38:20 +09:00
2010-10-04 03:54:56 +09:00
2010-10-31 10:40:39 -04:00
2010-08-09 20:44:58 -07:00
2011-03-22 17:44:15 -07:00
2010-10-08 09:32:15 +02:00
2011-03-16 12:53:54 -07:00
2010-11-28 21:16:28 +02:00
2011-01-23 21:00:05 +02:00
2010-08-11 08:59:21 -07:00
2011-03-22 20:48:04 +02:00
2011-03-22 17:16:34 +09:00
2011-03-23 07:48:42 -07:00
2010-12-08 12:16:33 -08:00
2011-01-13 10:25:58 -08:00
2011-03-09 14:08:09 -08:00
2011-03-28 06:05:24 -04:00
2011-01-27 12:30:38 +01:00
2010-10-07 14:08:55 +01:00
2010-08-20 09:00:15 -07:00
2010-11-18 14:37:34 +01:00
2010-08-09 16:48:44 -04:00
2010-11-24 11:14:24 -08:00
2010-10-14 08:55:28 +02:00
2011-03-15 00:43:16 +01:00
2011-03-22 15:47:22 +09:00
2011-03-24 10:16:26 -07:00
2010-10-27 18:03:14 -07:00
2011-03-21 00:40:29 -04:00
2011-03-15 00:43:46 +01:00
2011-03-09 16:43:24 -08:00
2011-03-23 22:16:41 +01:00
2010-10-17 01:57:44 +02:00
2011-01-24 09:33:36 -08:00
2010-08-11 08:59:22 -07:00
2010-12-22 19:43:34 -08:00
2010-08-30 13:23:33 -07:00
2011-02-28 18:00:31 -08:00
2011-02-02 15:28:13 +01:00
2011-02-04 12:41:21 -08:00
2011-02-21 12:53:08 -08:00
2010-10-22 14:46:25 +02:00
2010-12-11 12:34:34 +01:00
2011-02-02 15:28:18 +01:00
2011-03-13 16:35:18 -04:00
2011-02-23 18:05:11 -05:00
2010-09-09 20:41:20 +02:00
2011-01-13 08:03:08 -08:00
2010-11-29 08:55:25 +11:00
2010-11-29 08:55:22 +11:00
2010-10-27 18:03:12 -07:00
2011-04-04 12:48:08 -04:00
2011-02-17 11:59:56 -08:00
2011-01-13 08:03:20 -08:00
2011-03-22 16:17:32 -07:00
2011-03-22 17:44:09 -07:00
2010-11-28 23:08:04 +01:00
2010-12-20 09:37:33 +01:00
2010-11-10 16:57:11 -08:00
2010-10-22 10:22:05 -07:00
2011-02-04 11:48:55 -08:00
2011-01-13 08:03:18 -08:00
2011-03-23 19:47:03 -07:00
2011-03-22 04:53:37 -03:00
2011-03-22 04:53:33 -03:00
2011-01-06 13:05:04 +10:00
2010-08-27 09:10:44 +10:00
2010-11-10 12:04:41 +09:00
2010-12-11 02:01:35 -05:00
2011-03-22 04:54:51 -03:00
2010-10-27 18:03:18 -07:00
2011-01-24 14:45:11 +10:30
2011-02-09 08:19:55 +10:30
2011-01-13 17:32:34 -08:00
2011-03-22 17:44:05 -07:00
2011-02-17 11:13:19 -08:00
2010-10-05 11:47:18 -07:00
2010-10-13 15:45:21 -04:00
2010-11-15 13:25:02 -05:00
2011-02-21 10:07:23 +01:00
2011-03-24 21:17:51 -04:00
2011-01-17 08:05:27 -08:00
2011-03-13 20:22:28 -07:00
2011-01-13 08:03:24 -08:00
2011-03-22 17:44:17 -07:00
d430d3d7e6