There are already benchmarks for block in another file - can extend those instead of making a new suite?

Sorry, I'm really bad at navigating this codebase. Thanks!.

I'd recommend that before adding a new benchmark / test, you search the existing benchmarks / tests folders for places that already call the function you want to profile/test.

I had found the ones for masked arrays, but not the ones for dense arrays. :/

Thanks for having me extend this benchmark @eric-wieser I think we can essentially use this benchmark's algorithm with a reshape to get block done in 1 shot.

I think we can essentially use this benchmark's algorithm with a reshape

I find it very unlikely that that would work. You're right that we could do it in one allocation, but it will require some careful juggling of slices.

@pv: Is there any easy way to make this continue the existing benchmark graphs, rather than resulting in a benchmark with a new name?

I don't think there is.

Apparently there is a way to build history for a nee benchmark. But I’m not too sure how

@eric-wieser I would almost like to change this from calling concatenate to calling arr.copy() on all the arrays provided.

The goal is to show that blocking is a memory bandwidth limited algorithm.

Anyway, PR #11971 gets pretty close to the copy limit.

===== ============= ========== ========== ========== ==========
--                                     dtype                   
------------------- -------------------------------------------
n        mode       uint8      uint16     uint32     uint64  
===== ============= ========== ========== ========== ==========
1       block      180±0μs    172±0μs    183±0μs    171±0μs  
1    concatenate   47.7±0μs   50.1±0μs   47.2±0μs   45.9±0μs 
1        copy      30.4±0μs   34.6±0μs   31.5±0μs   36.1±0μs 

10      block      253±0μs    277±0μs    390±0μs    582±0μs  
10   concatenate   88.9±0μs   128±0μs    217±0μs    404±0μs  
10       copy      65.7±0μs   96.0±0μs   150±0μs    287±0μs  

50      block      6.77±0ms   14.6±0ms   27.6±0ms   52.3±0ms 
50   concatenate   6.31±0ms   12.7±0ms   26.8±0ms   53.7±0ms 
50       copy      6.50±0ms   13.2±0ms   27.1±0ms   54.6±0ms 

75      block      23.5±0ms   45.4±0ms   91.1±0ms   181±0ms  
75   concatenate   22.8±0ms   46.5±0ms   91.8±0ms   181±0ms  
75       copy      29.7±0ms   52.2±0ms   95.7±0ms   188±0ms  

100      block      57.8±0ms   121±0ms    208±0ms    394±0ms  
100   concatenate   55.8±0ms   110±0ms    236±0ms    420±0ms  
100       copy      63.2±0ms   115±0ms    212±0ms    426±0ms  

150      block      181±0ms    352±0ms    700±0ms    1.44±0s  
150   concatenate   181±0ms    350±0ms    707±0ms    1.41±0s  
150       copy      183±0ms    355±0ms    706±0ms    1.47±0s  

200      block      435±0ms    851±0ms    1.62±0s    3.45±0s  
200   concatenate   439±0ms    881±0ms    1.73±0s    3.50±0s  
200       copy      442±0ms    890±0ms    1.75±0s    3.54±0s  
===== ============= ========== ========== ========== ==========

Is this ready to be merged?

@mattip: My worry here is that we break contiguity in our benchmark history by doing this - something we deliberately considered when we introduced the Block3D test.

@eric-wieser why is there going to be a new name for the benchmark? Do additional parameters trigger a new name?

@hmaarrfk: Yes, I believe they do

:(

An easy workaround would be to just expose the test twice, once under the old name and once under the new name. Failing that, we might want to look into patching asv to allow a params -> name function to be provided.

What happens when we add a new benchmark? Is history created automatically or will it start from "this new commit"

@eric-wieser, If we don't want to create a new benchmark, we would need to run the command:

asv run -E conda:3.6 -b Block.time_3d EXISTING 

to rebuild the history for all the commits that currently have benchmarks. I'm not sure if that is a possiility.

rebased off master

As discussed at the Oct 17 weekly status meeting, the general opinion was that breaking continuity is not a PR blocker. It is relatively easy to rerun newer benchmarks on older code to determine a baseline and compare across the gap, especially since this PR only touches the benchmark itself.

Thanks @hmaarrfk