Maybe needed? https://stackoverflow.com/questions/466377/how-to-detect-what-cpu-is-being-used-during-runtime

We do not currently target BMI2, see this line...

We do not do so because we don't use BMI2 in the haswell kernel.

Note that if we are going to use pdep/pext, we are going to have to omit pre-Zen3 AMD processors were these instructions are super slow. It may require an entirely new kernel which is a bit complicated.

It is entirely safe to use pdep/pext in the icelake kernel however.

Why not target BMI2 for Haswell? Even if you want to avoid PDEP/PEXT because of AMD, don't you still think the other instructions are nice? Does this mean that the compiler can't insert BMI2 instructions at all or does it just mean that you don't use BMI2 intrinsics?

Why not target BMI2 for Haswell? Even if you want to avoid PDEP/PEXT because of AMD, don't you still think the other instructions are nice? Does this mean that the compiler can't insert BMI2 instructions at all or does it just mean that you don't use BMI2 intrinsics?

Currently, GCC and LLVM won't use BMI2 in the haswell kernel. The reason we do not do it is because we do not expect benefits. Of course, that's an empirical issue. I will investigate later. :-)

Note that the initial versions of simdjson used BMI2 (pdep/pext). At least the way we were using it was not clearly beneficial on Intel processors, and, at the time, quite bad for AMD processors. So we progressively removed it.

(We do use BMI2 in the icelake kernel.)

Well, according to Agner Fog's instruction tables, BZHI has 1 cycle latency and a reciprocal throughput of 0.25 (i.e. you can start up to 4 per cycle if the ports are available) for Zen 1 and Zen 2. For every other machine, it's 1 cycle latency with 0.5 reciprocal throughput. uops.info confirms this, although it includes the relatively recent Alder Lake, which appears to have regressed its performance to 3 cycles of latency and a throughput of 1.

(I assume you know what reciprocal throughput is, but honestly I forget which is which sometimes)

BZHI can be called directly through an intrinsic due to its better handling of 64 as a second input (if the 2nd input has a bottom byte of 64-255, i.e. 0x40-0xff, it will return the first input unchanged), or it can be generated by the compiler when the idiom is used, although obviously that code has undefined behavior when index is 64 or greater, so for many non-BMI2-optimized builds it would likely truncate the second input to just the lower 6 bits (not that Clang would warn you about that by default, but don't worry, zig cc has better defaults).

The other instructions MULX, RORX, SARX, SHRX, SHLX are just special versions of normal operations we already had support for, but these don't affect flags and have a separate destination register from the two input operands, so these are potential wins that the compiler can give you for free. (although they can't use immediates)

So it seems to me the only drawback of BMI2 would be pre-Zen3 AMD processors being really slow at PDEP/PEXT, but I don't think compilers know how to generate those automatically [1, 2, 3], nor would they for pre-Zen3 AMD, if their cost model was correct. (Update: I just checked, PDEP/PEXT are not emitted at all in my build of simdjson without the intrinsics asking for them)

So I don't really see the point of disabling BMI2. Just because the custom PDEP/PEXT routines you had before didn't work out doesn't mean it's bad to enable BMI2 as an option for compilers.

Update: I added bmi2 to that haswell target like you said. This now compiles. Is it faster? Well, maybe. I can't really tell by just running it on the twitter.json benchmark and doing eyeball statistics (definition). However, according to LLVM's cost model, my version is faster [haswell, znver3] (I am running a Zen 3 machine). It also means we don't have to include 2576 bytes for the 3 tables, so that seems like a win. However, I am not sure how you would prefer to account for the AMD problem [znver2]. Let me know what you think.

So I don't really see the point of disabling BMI2

It is not enabled by default. So we are not disabling it. However, it is interesting to check whether enabling it would be beneficial, and I plan to investigate. As your analysis suggests, the important instruction is BZHI.

It also means we don't have to include 2576 bytes for the 3 tables,

We need the tables because we can't assume that BMI2 is present.

@Validark

This now compiles. Is it faster?

Thanks. I will review and assess.

We might introduce a kernel between icelake and haswell if the performance gains are sufficiently clear.

Of course, a Zen 4 or Zen 5 will use the icelake kernel (where we do use pdep and pext).

However, it is interesting to check whether enabling it would be beneficial, and I plan to investigate. As your analysis suggests, the important instruction is BZHI.

My message was an attempt to explain everything that is included in BMI2, not say which instructions are most important to simdjson. Based on a objdump, here are the number of occurrences for each instruction:

They aren't used much, but they do appear in simdjson. And they can eliminate unnecessary internal flag register allocation and possibly eliminate an unnecessary mov instruction as well.

So I don't really see the point of disabling BMI2

It is not enabled by default. So we are not disabling it.

It also means we don't have to include 2576 bytes for the 3 tables,

We need the tables because we can't assume that BMI2 is present.

Haswell has BMI2 though, right? So you can assume that BMI2 is present. LLVM gives you BMI2 for the Haswell cpu target.

They aren't used much, but they do appear in simdjson.

It is an empirical question. By how much does enabling BMI2 in LLVM or GCC improves performance. As I wrote, I plan to review this issue experimentally.

Haswell has BMI2 though, right? So you can assume that BMI2 is present.

We do not assume that the processor supports the equivalent of Haswell. This is determined at runtime by CPU detection.

They aren't used much, but they do appear in simdjson.

It is an empirical question. By how much does enabling BMI2 in LLVM or GCC improves performance. As I wrote, I plan to review this issue experimentally.

Feel free, but I think you're unlikely to notice a big difference in practice, these are sort of micro-optimizations. Yes, they save the CPU a little work, but it might be able to do that extra work without adding to the runtime.

Haswell has BMI2 though, right? So you can assume that BMI2 is present.

We do not assume that the processor supports the equivalent of Haswell. This is determined at runtime by CPU detection.

Right, but I'm only talking about the Haswell kernel? I added BMI2 to the list of features that Haswell supports, so if we dispatch to the Haswell kernel, that code can use BMI2 instructions, especially MULX, SHLX, and SHRX.

I am trying to get you to answer the question of why BMI2 is not enabled for the Haswell kernel. You said you would do it if it's beneficial, which you would test empirically. But my question isn't "Should it be enabled?", my question is, "Why not?" I'm asking what's the harm in saving the CPU a small amount of work or power consumption? I'm basically implying that I think the default position is to have BMI2 enabled for the Haswell kernel, whereas you're saying that the default position is to have BMI2 disabled for the Haswell kernel. I'm questioning why you think that way.

I think that these instructions were added to the CPU as an optimization, so if the compiler agrees, i.e., wants to insert it in your Haswell specific code, why not just let it? Again, I am not sure that the specific circumstances in which those instructions are inserted are sufficient to see a performance improvement, but the potential is there, so why not? I definitely don't think you'll see performance degradation. Why not just enable it for the Haswell kernel?

@Validark At least under some versions of GCC, and some hardware, enabling BMI2 appears to lower the performance. It does increase slightly the instruction count.

See #2243

For LLVM, it is beneficial so I am currently considering enabling it only under LLVM.

@Validark At least under some versions of GCC, and some hardware, enabling BMI2 appears to lower the performance. It does increase slightly the instruction count.

See #2243

For LLVM, it is beneficial so I am currently considering enabling it only under LLVM.

That's sad that GCC does worse with it on. Well, c'est la vie.

@Validark

Even here, in this PR, you can see that our CI reports a performance degradation:

However, I am not giving up. I think that if we are sufficiently careful, we could get more performance in some cases with BMI2.