Skip to content

Latest commit

 

History

History

benchmark

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Makefile
Makefile
 
 
README.md
README.md
 
 
c.mk
c.mk
 
 
cpp.mk
cpp.mk
 
 
fortran.mk
fortran.mk
 
 
javascript.mk
javascript.mk
 
 
julia.mk
julia.mk
 
 
python.mk
python.mk
 
 
r.mk
r.mk
 
 

README.md

Benchmark

Benchmark commands.

This directory contains make rules for running language benchmarks.

Usage

Usage: make <command> [<ENV_VAR>=<value> <ENV_VAR>=<value> ...]

Commands

benchmark

Runs benchmarks.

$ make benchmark

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/blas/base/dasum/.*.

This command is an alias for benchmark-javascript, which is documented below.

benchmark-lang

Runs cross-language benchmarks.

$ make benchmark-lang

The command supports the environment variables supported by each language-specific (benchmark-<lang>) command documented below.

This command is useful when wanting to glob for benchmark files, irrespective of language, for a particular package in order to compare cross-language performance.

C

Note: C benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

benchmark-c

Runs C benchmarks.

$ make benchmark-c

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/math/base/special/abs/.*.
  • BLAS: BLAS library name; e.g., openblas.
  • BLAS_DIR: BLAS directory.
  • C_COMPILER: C compiler; e.g., gcc.

This command is useful when wanting to glob for C benchmark files (e.g., run all C benchmarks for a particular package).

benchmark-c-files

Runs a specified list of C benchmark files.

$ make benchmark-c-files FILES='/foo/benchmark.c /bar/benchmark.c'

The command supports the following environment variables:

  • FILES: list of C benchmark files.
  • BLAS: BLAS library name; e.g., openblas.
  • BLAS_DIR: BLAS directory.
  • C_COMPILER: C compiler; e.g., gcc.

This command is useful when wanting to run a list of C benchmark files generated by some other command (e.g., a filtered list of changed C benchmark files obtained via git diff).

C++

Note: C++ benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

benchmark-cpp

Runs C++ benchmarks.

$ make benchmark-cpp

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/math/base/special/beta/.*.
  • CXX_COMPILER: C++ compiler; e.g., g++.

This command is useful when wanting to glob for C++ benchmark files (e.g., run all C++ benchmarks for a particular package).

benchmark-cpp-files

Runs a specified list of C++ benchmark files.

$ make benchmark-cpp-files FILES='/foo/benchmark.cpp /bar/benchmark.cpp'

The command supports the following environment variables:

  • FILES: list of C++ benchmark files.
  • CXX_COMPILER: C++ compiler; e.g., g++.

This command is useful when wanting to run a list of C++ benchmark files generated by some other command (e.g., a filtered list of changed C++ benchmark files obtained via git diff).

Fortran

Note: Fortran benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

benchmark-fortran

Runs Fortran benchmarks.

$ make benchmark-fortran

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/blas/base/daxpy/.*.
  • FORTRAN_COMPILER: Fortran compiler; e.g., gfortran.

This command is useful when wanting to glob for Fortran benchmark files (e.g., run all Fortran benchmarks for a particular package).

benchmark-fortran-files

Runs a specified list of Fortran benchmark files.

$ make benchmark-fortran-files FILES='/foo/benchmark.f /bar/benchmark.f'

The command supports the following environment variables:

  • FILES: list of Fortran benchmark files.
  • FORTRAN_COMPILER: Fortran compiler; e.g., gfortran.

This command is useful when wanting to run a list of Fortran benchmark files generated by some other command (e.g., a filtered list of changed Fortran benchmark files obtained via git diff).

JavaScript

benchmark-javascript

Runs JavaScript benchmarks.

$ make benchmark-javascript

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/utils/group-by/.*.

This command is useful when wanting to glob for JavaScript benchmark files (e.g., run all JavaScript benchmarks for a particular package).

benchmark-javascript-files

Runs a specified list of JavaScript benchmark files.

$ make benchmark-javascript-files FILES='/foo/benchmark.js /bar/benchmark.js'

The command supports the following environment variables:

  • FILES: list of JavaScript benchmark files.

This command is useful when wanting to run a list of JavaScript benchmark files generated by some other command (e.g., a filtered list of changed JavaScript benchmark files obtained via git diff).

Julia

benchmark-julia

Runs Julia benchmarks.

$ make benchmark-julia

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/math/base/special/erf/.*.

This command is useful when wanting to glob for Julia benchmark files (e.g., run all Julia benchmarks for a particular package).

benchmark-julia-files

Runs a specified list of Julia benchmark files.

$ make benchmark-julia-files FILES='/foo/benchmark.jl /bar/benchmark.jl'

The command supports the following environment variables:

  • FILES: list of Julia benchmark files.

This command is useful when wanting to run a list of Julia benchmark files generated by some other command (e.g., a filtered list of changed Julia benchmark files obtained via git diff).

Python

benchmark-python

Runs Python benchmarks.

$ make benchmark-python

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/math/base/special/digamma/.*.

This command is useful when wanting to glob for Python benchmark files (e.g., run all Python benchmarks for a particular package).

benchmark-python-files

Runs a specified list of Python benchmark files.

$ make benchmark-python-files FILES='/foo/benchmark.py /bar/benchmark.py'

The command supports the following environment variables:

  • FILES: list of Python benchmark files.

This command is useful when wanting to run a list of Python benchmark files generated by some other command (e.g., a filtered list of changed Python benchmark files obtained via git diff).

R

benchmark-r

Runs R benchmarks.

$ make benchmark-r

The command supports the following environment variables:

  • BENCHMARKS_FILTER: file path pattern; e.g., .*/math/base/special/expm1/.*.

This command is useful when wanting to glob for R benchmark files (e.g., run all R benchmarks for a particular package).

benchmark-r-files

Runs a specified list of R benchmark files.

$ make benchmark-r-files FILES='/foo/benchmark.R /bar/benchmark.R'

The command supports the following environment variables:

  • FILES: list of R benchmark files.

This command is useful when wanting to run a list of R benchmark files generated by some other command (e.g., a filtered list of changed R benchmark files obtained via git diff).