It seems the log output is now too large for travis, it exceeds 50,000 lines. Is there a way to tweak the benchmark run to be less verbose?

@mattip, I guess the only way is by providing argument options to control of generated tests, same as in #15987, example --strides, --sizes(array sizes), --nsamples(number of collected samples), and different filter mechanism beyond ASV that allow us to disable deep tests by default unless user provided a regex that hit some matches. I think it can be handle it through runtests.py. what do u think?

It seems quite complicated. How will anyone track the changes over time when the full suite takes so long to run? What is the goal of the added benchmarks: perhaps there is another way to achieve the same goal.

@mattip,

What is the goal of the added benchmarks

I was simply want to detect the performance changes, so I can reply to @rgommers comment. also for the upcoming patches.

How will anyone track the changes?

simply you don't need to run all the tests only filter what u target through the regex.

full suite takes so long to run?

ASV kinda slow even with disabling ASV's warmup process

perhaps there is another way to achieve the same goal.

This patch #15987 is more effective and faster. currently, I'm doing some final tests on it and
it seems I'm going to rely on it instead of using ASV.

it seems I'm going to rely on it instead of using ASV.

Are you sure you cannot do what you wish with ASV and using your C code from gh-15987 (although that code looks fragile: what happens if we add more ufuncs)?

@mattip,

Are you sure you cannot do what you wish with ASV and using your C code from gh-15987

There's no way to integrate the C code from gh-15987 since ASV doesn't provide a method to specify the elapsed time of each sample.

(although that code looks fragile

The python code still needs more improvements, but the C code looks good to me.

what happens if we add more ufuncs)?

The python script collects all ufuncs via dir() and fetches the input types from ufunc's attribute types same as what this patch does. so the code is frinedly with any new ufuncs.

I still prefer this as an alternative to gh-15987. @seiko2plus do you now see a way to make the output nicer/less verbose?

@mattip, yes, I see away. I'm going to count on gh-15987 for now, then I will work on it after getting done from my other pull-requests.