MCPcopy Create free account
hub / github.com/apache/arrow / benchmark_diff

Function benchmark_diff

dev/archery/archery/cli.py:533–659  ·  view source on GitHub ↗

Compare (diff) benchmark runs. This command acts like git-diff but for benchmark results. The caller can optionally specify both the contender and the baseline. If unspecified, the contender will default to the current workspace (like git) and the baseline will default to the mainl

(ctx, src, preserve, preserve_dir, output, language,
                   cmake_extras, suite_filter, benchmark_filter, repetitions,
                   no_counters, java_home, java_options, build_extras,
                   benchmark_extras, cpp_benchmark_extras, threshold,
                   contender, baseline, **kwargs)

Source from the content-addressed store, hash-verified

531 required=False)
532@click.pass_context
533def benchmark_diff(ctx, src, preserve, preserve_dir, output, language,
534 cmake_extras, suite_filter, benchmark_filter, repetitions,
535 no_counters, java_home, java_options, build_extras,
536 benchmark_extras, cpp_benchmark_extras, threshold,
537 contender, baseline, **kwargs):
538 """Compare (diff) benchmark runs.
539
540 This command acts like git-diff but for benchmark results.
541
542 The caller can optionally specify both the contender and the baseline. If
543 unspecified, the contender will default to the current workspace (like git)
544 and the baseline will default to the mainline development branch (i.e.
545 default git branch).
546
547 Each target (contender or baseline) can either be a git revision
548 (commit, tag, special values like HEAD) or a cmake build directory. This
549 allow comparing git commits, and/or different compilers and/or compiler
550 flags.
551
552 When a commit is referenced, a local clone of the arrow sources (specified
553 via --src) is performed and the proper branch is created. This is done in
554 a temporary directory which can be left intact with the `--preserve` flag.
555
556 The special token "WORKSPACE" is reserved to specify the current git
557 workspace. This imply that no clone will be performed.
558
559 Examples:
560
561 \b
562 # Compare workspace (contender) against the mainline development branch
563 # (baseline)
564 \b
565 archery benchmark diff
566
567 \b
568 # Compare the mainline development branch (contender) against the latest
569 # version (baseline)
570 \b
571 export LAST=$(git tag -l "apache-arrow-[0-9]*" | sort -rV | head -1)
572 \b
573 archery benchmark diff <default-branch> "$LAST"
574
575 \b
576 # Compare g++7 (contender) with clang++-8 (baseline) builds
577 \b
578 archery build --with-benchmarks=true \\
579 --cxx-flags=-ftree-vectorize \\
580 --cc=gcc-7 --cxx=g++-7 gcc7-build
581 \b
582 archery build --with-benchmarks=true \\
583 --cxx-flags=-flax-vector-conversions \\
584 --cc=clang-8 --cxx=clang++-8 clang8-build
585 \b
586 archery benchmark diff gcc7-build clang8-build
587
588 \b
589 # Compare default targets but scoped to the suites matching
590 # `^arrow-compute-aggregate` and benchmarks matching `(Sum|Mean)Kernel`.

Callers

nothing calls this directly

Calls 6

tmpdirFunction · 0.90
RunnerComparatorClass · 0.85
_get_comparisons_as_jsonFunction · 0.85
default_configurationMethod · 0.45
from_rev_or_pathMethod · 0.45

Tested by

no test coverage detected