tracing, perf: Adjust code layout in get_recursion_context()

netoptimizer · Ingo Molnar · commit d0618410eced · 2017-08-25T11:04:18.000+02:00
In an XDP redirect applications using tracepoint xdp:xdp_redirect to diagnose TX overrun, I noticed perf_swevent_get_recursion_context() was consuming 2% CPU. This was reduced to 1.85% with this simple change. Looking at the annotated asm code, it was clear that the unlikely case in_nmi() test was chosen (by the compiler) as the most likely event/branch. This small adjustment makes the compiler (GCC version 7.1.1 20170622 (Red Hat 7.1.1-3)) put in_nmi() as an unlikely branch. Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/150342256382.16595.986861478681783732.stgit@firesoul Signed-off-by: Ingo Molnar <mingo@kernel.org>
diff --git a/kernel/events/internal.h b/kernel/events/internal.h
@@ -208,7 +208,7 @@ static inline int get_recursion_context(int *recursion)
 {
 	int rctx;
 
-	if (in_nmi())
+	if (unlikely(in_nmi()))
 		rctx = 3;
 	else if (in_irq())
 		rctx = 2;

Original file line number	Diff line number	Diff line change
`@@ -208,7 +208,7 @@ static inline int get_recursion_context(int *recursion)`
`208`	`208`	`{`
`209`	`209`	`int rctx;`
`210`	`210`
`211`		`- if (in_nmi())`
	`211`	`+ if (unlikely(in_nmi()))`
`212`	`212`	`rctx = 3;`
`213`	`213`	`else if (in_irq())`
`214`	`214`	`rctx = 2;`