Compare (diff) benchmark runs. This command acts like git-diff but for benchmark results. The caller can optionally specify both the contender and the baseline. If unspecified, the contender will default to the current workspace (like git) and the baseline will default to the mainl
(ctx, src, preserve, preserve_dir, output, language,
cmake_extras, suite_filter, benchmark_filter, repetitions,
no_counters, java_home, java_options, build_extras,
benchmark_extras, cpp_benchmark_extras, threshold,
contender, baseline, **kwargs)
| 531 | required=False) |
| 532 | @click.pass_context |
| 533 | def benchmark_diff(ctx, src, preserve, preserve_dir, output, language, |
| 534 | cmake_extras, suite_filter, benchmark_filter, repetitions, |
| 535 | no_counters, java_home, java_options, build_extras, |
| 536 | benchmark_extras, cpp_benchmark_extras, threshold, |
| 537 | contender, baseline, **kwargs): |
| 538 | """Compare (diff) benchmark runs. |
| 539 | |
| 540 | This command acts like git-diff but for benchmark results. |
| 541 | |
| 542 | The caller can optionally specify both the contender and the baseline. If |
| 543 | unspecified, the contender will default to the current workspace (like git) |
| 544 | and the baseline will default to the mainline development branch (i.e. |
| 545 | default git branch). |
| 546 | |
| 547 | Each target (contender or baseline) can either be a git revision |
| 548 | (commit, tag, special values like HEAD) or a cmake build directory. This |
| 549 | allow comparing git commits, and/or different compilers and/or compiler |
| 550 | flags. |
| 551 | |
| 552 | When a commit is referenced, a local clone of the arrow sources (specified |
| 553 | via --src) is performed and the proper branch is created. This is done in |
| 554 | a temporary directory which can be left intact with the `--preserve` flag. |
| 555 | |
| 556 | The special token "WORKSPACE" is reserved to specify the current git |
| 557 | workspace. This imply that no clone will be performed. |
| 558 | |
| 559 | Examples: |
| 560 | |
| 561 | \b |
| 562 | # Compare workspace (contender) against the mainline development branch |
| 563 | # (baseline) |
| 564 | \b |
| 565 | archery benchmark diff |
| 566 | |
| 567 | \b |
| 568 | # Compare the mainline development branch (contender) against the latest |
| 569 | # version (baseline) |
| 570 | \b |
| 571 | export LAST=$(git tag -l "apache-arrow-[0-9]*" | sort -rV | head -1) |
| 572 | \b |
| 573 | archery benchmark diff <default-branch> "$LAST" |
| 574 | |
| 575 | \b |
| 576 | # Compare g++7 (contender) with clang++-8 (baseline) builds |
| 577 | \b |
| 578 | archery build --with-benchmarks=true \\ |
| 579 | --cxx-flags=-ftree-vectorize \\ |
| 580 | --cc=gcc-7 --cxx=g++-7 gcc7-build |
| 581 | \b |
| 582 | archery build --with-benchmarks=true \\ |
| 583 | --cxx-flags=-flax-vector-conversions \\ |
| 584 | --cc=clang-8 --cxx=clang++-8 clang8-build |
| 585 | \b |
| 586 | archery benchmark diff gcc7-build clang8-build |
| 587 | |
| 588 | \b |
| 589 | # Compare default targets but scoped to the suites matching |
| 590 | # `^arrow-compute-aggregate` and benchmarks matching `(Sum|Mean)Kernel`. |
nothing calls this directly
no test coverage detected