"If you liked it then you "should have put a"_test on it", Beyonce rule

UT / μt

| Motivation | Quick Start | Overview | Tutorial | Examples | User Guide | FAQ | Benchmarks |

C++ single header/single module, macro-free μ(micro)/Unit Testing Framework

#include <boost/ut.hpp> // import boost.ut;

constexpr auto sum(auto... values) { return (values + ...); }

int main() {
  using namespace boost::ut;

  "sum"_test = [] {
    expect(sum(0) == 0_i);
    expect(sum(1, 2) == 3_i);
    expect(sum(1, 2) > 0_i and 41_i == sum(40, 2));
  };
}

Running "sum"...
  sum.cpp:11:FAILED [(3 > 0 and 41 == 42)]
FAILED

===============================================================================
tests:   1 | 1 failed
asserts: 3 | 2 passed | 1 failed

https://godbolt.org/z/f4jEcv9vo

Motivation

Testing is a very important part of the Software Development, however, C++ doesn't provide any good testing facilities out of the box, which often leads into a poor testing experience for develops and/or lack of tests/coverage in general.

One should treat testing code as production code!

Additionally, well established testing practises such as Test Driven Development (TDD)/Behaviour Driven Development (BDD) are often not followed due to the same reasons.

The following snippet is a common example of testing with projects in C++.

int main() {
  // should sum numbers
  {
    assert(3 == sum(1, 2));
  }
}

There are quite a few problems with the approach above

• No names for tests (Hard to follow intentions by further readers)
• No automatic registration of tests (No way to run specific tests)
• Hard to debug (Assertions don't provide any information why it failed)
• Hard to scale (No easy path forward for parameterized tests, multiple suites, parallel execution, etc...)
• Hard to integrate (No easy way to have a custom output such as XML for CI integration)
• Easy to make mistakes (With implicit casting, floating point comparison, pointer comparison for strings, etc...)
• Hard to follow good practises such as TDD/BDD (Lack of support for sections and declarative expressions)
• ...

UT is trying to address these issues by simplifying testing experience with a few simple steps:

• Just get a single header/module
• Integrate it into your project
• Learn a few simple concepts (expect, test, suite)

And you good to go!

Okay, great, but why I would use UT over other/similar testing frameworks already available in C++?

• Boost.Test
• GoogleTest
• Catch
• ...

Great question! There are a few unique features which makes UT worth trying

• Firstly, it supports all the basic Unit Testing Framework features (automatic registration of tests, assertions, suites, etc...)
• It's easy to integrate (it's just one header/module)
• It's macro free which makes testing experience that much nicer (it uses modern C++ features instead, macros are opt-in rather than being compulsory - Can I still use macros?)
• It's flexible (all parts of the framework such as: runner, reporter, printer can be customized, basically most other Unit Testing Frameworks can be implemented on top of UT primitives)
• It has smaller learning curve (just a few simple concepts (expect, test, suite))
• It leverages C++ features to support more complex testing (parameterized)
• It's faster to compile and execute than similar frameworks which makes it suitable for bigger projects without additional hassle (Benchmarks)
• It supports TDD/BDD workflows
• It supports Gherkin specification
• It supports Spec
• ...

Sounds intriguing/interesting? Learn more at

• Tutorial
• Examples
• User-Guide

Quick Start

https://bit.ly/ut-quick-start (slides)

Overview

• No dependencies (C++20, Tested Compilers: GCC-9+, Clang-9.0+, Apple Clang-11.0.0+, MSVC-2019+*, Clang-cl-9.0+
• Single header/module (boost/ut.hpp)
• Macro-free (How does it work?)
• Easy to use (Minimal API - test, suite, operators, literals, [expect])
• Fast to compile/execute (Benchmarks)
• Features (Assertions, Suites, Tests, Sections, Parameterized, BDD, Gherkin, Spec, Matchers, Logging, Runners, Reporters, ...)
• Integrations (ApprovalTests.cpp)

Tutorial

    Step 0: Get it...

Get the latest latest header/module from here!

Include/Import

// #include <boost/ut.hpp> // single header
// import boost.ut;        // single module (C++20)

int main() { }

Compile & Run

$CXX main.cpp && ./a.out

All tests passed (0 assert in 0 test)

[Optional] Install it

cmake -Bbuild -H.
cd build && make         # run tests
cd build && make install # install

[Optional] CMake integration

This project provides a CMake config and target. Just load ut with find_package to import the Boost::ut target. Linking against this target will add the necessary include directory for the single header file. This is demonstrated in the following example.

find_package(ut REQUIRED)
add_library(my_test my_test.cpp)
target_link_libraries(my_test PRIVATE Boost::ut)

[Optional] Conan integration

The boost-ext-ut package is available from Conan Center. Just include it in your project's Conanfile with boost-ext-ut/2.3.1.

    Step 1: Expect it...

Let's write our first assertion, shall we?

int main() {
  boost::ut::expect(true);
}

All tests passed (1 asserts in 0 test)

https://godbolt.org/z/vfx-eB

Okay, let's make it fail now?

int main() {
  boost::ut::expect(1 == 2);
}

main.cpp:4:FAILED [false]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/7qTePx

Notice that expression 1 == 2 hasn't been printed. Instead we got false?

Let's print it then?

int main() {
  using namespace boost::ut;
  expect(1_i == 2);
}

main.cpp:4:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/7MXVzu

Okay, now we have it! 1 == 2 has been printed as expected. Notice the User Defined Literal (UDL) 1_i was used. _i is a compile-time constant integer value

• It allows to override comparison operators 👍
• It disallow comparison of different types 👍

See the User-guide for more details.

Alternatively, a terse notation (no expect required) can be used.

int main() {
  using namespace boost::ut::literals;
  using namespace boost::ut::operators::terse;

  1_i == 2; // terse notation
}

main.cpp:7:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/s77GSm

Other expression syntaxes are also available.

expect(1_i == 2);       // UDL syntax
expect(1 == 2_i);       // UDL syntax
expect(that % 1 == 2);  // Matcher syntax
expect(eq(1, 2));       // eq/neq/gt/ge/lt/le

main.cpp:6:FAILED [1 == 2]
main.cpp:7:FAILED [1 == 2]
main.cpp:8:FAILED [1 == 2]
main.cpp:9:FAILED [1 == 2]
===============================================================================
tests:   0 | 0 failed
asserts: 4 | 0 passed | 4 failed

https://godbolt.org/z/QbgGtc

Okay, but what about the case if my assertion is fatal. Meaning that the program will crash unless the processing will be terminated. Nothing easier, let's just add fatal call to make the test fail immediately.

expect(fatal(1 == 2_i)); // fatal assertion
expect(1_i == 2);        // not executed

main.cpp:6:FAILED [1 == 2]
===============================================================================
tests:   1 | 1 failed
asserts: 2 | 0 passed | 2 failed

https://godbolt.org/z/WMe8Y1

But my expression is more complex than just simple comparisons. Not a problem, logic operators are also supported in the expect 👍.

expect(42l == 42_l and 1 == 2_i); // compound expression

main.cpp:5:FAILED [(42 == 42 and 1 == 2)]
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/aEhX4t

Can I add a custom message though? Sure, expect calls are streamable!

int main() {
  expect(42l == 42_l and 1 == 2_i) << "additional info";
}

main.cpp:5:FAILED [(42 == 42 and 1 == 2)] additional info
===============================================================================
tests:   0 | 0 failed
asserts: 1 | 0 passed | 1 failed

That's nice, can I use custom messages and fatal assertions? Yes, stream the fatal!

expect(fatal(1 == 2_i)) << "fatal assertion";
expect(1_i == 2);

FAILED
in: main.cpp:6 - test condition:  [1 == 2]

 fatal assertion
===============================================================================
tests:   0 | 2 failed
asserts: 0 | 0 passed | 2 failed

I use std::expected, can I stream its error() upon failure? Yes, since std::expected's error() can only be called when there is no value it requires lazy evaluation.

"lazy log"_test = [] {
  std::expected<bool, std::string> e = std::unexpected("lazy evaluated");
  expect(e.has_value()) << [&] { return e.error(); } << fatal;
  expect(e.value() == true);
};

Running test "lazy log"... FAILED
in: main.cpp:12 - test condition:  [false]

 lazy evaluated
===============================================================================
tests:   1 | 2 failed
asserts: 0 | 0 passed | 2 failed

https://godbolt.org/z/v2PDuU

    Step 2: Group it...

Assertions are great, but how to combine them into more cohesive units? Test cases are the way to go! They allow to group expectations for the same functionality into coherent units.

"hello world"_test = [] { };

Alternatively test("hello world") = [] {} can be used.

All tests passed (0 asserts in 1 tests)

https://godbolt.org/z/Bh-EmY

Notice 1 tests but 0 asserts.

Let's make our first end-2-end test case, shall we?

int main() {
  "hello world"_test = [] {
    int i = 43;
    expect(42_i == i);
  };
}

Running "hello world"...
  main.cpp:8:FAILED [42 == 43]
FAILED
===============================================================================
tests:   1 | 1 failed
asserts: 1 | 0 passed | 1 failed

https://godbolt.org/z/Y43mXz

👍 We are done here!

I'd like to nest my tests, though and