llvm-tutor

Example LLVM passes - based on LLVM 22

llvm-tutor is a collection of self-contained reference LLVM passes. It's a tutorial that targets novice and aspiring LLVM developers. Key features:

• Out-of-tree - builds against a binary LLVM installation (no need to build LLVM from sources)
• Complete - includes CMake build scripts, LIT tests, CI set-up and documentation
• Modern - based on the latest version of LLVM (and updated with every release)

Overview

LLVM implements a very rich, powerful and popular API. However, like many complex technologies, it can be quite daunting and overwhelming to learn and master. The goal of this LLVM tutorial is to showcase that LLVM can in fact be easy and fun to work with. This is demonstrated through a range of self-contained, testable LLVM passes, which are implemented using idiomatic LLVM.

This document explains how to set-up your environment, build and run the examples, and go about debugging. It contains a high-level overview of the implemented examples and contains some background information on writing LLVM passes. The source files, apart from the code itself, contain comments that will guide you through the implementation. All examples are complemented with LIT tests and reference input files.

Visit clang-tutor if you are interested in similar tutorial for Clang.

Table of Contents

• HelloWorld: Your First Pass
• Part 1: llvm-tutor in more detail
• Development Environment
• Building & Testing
• Overview of the Passes
• Debugging
• Part 2: Passes In LLVM
• Analysis vs Transformation Pass
• Dynamic vs Static Plugins
• Optimisation Passes Inside LLVM
• Optimisation Passes Inside LLVM
• Contributing
• References

HelloWorld: Your First Pass

The HelloWorld pass from HelloWorld.cpp is a self-contained reference example. The corresponding CMakeLists.txt implements the minimum set-up for an out-of-source pass.

For every function defined in the input module, HelloWorld prints its name and the number of arguments that it takes. You can build it like this:

export LLVM_DIR=<installation/dir/of/llvm/22>
mkdir build
cd build
cmake -DLT_LLVM_INSTALL_DIR=$LLVM_DIR 

/HelloWorld/
make

Before you can test it, you need to prepare an input file:

# Generate an LLVM test file
$LLVM_DIR/bin/clang -O1 -S -emit-llvm 

/inputs/input_for_hello.c -o input_for_hello.ll

Finally, run HelloWorld with opt (use libHelloWorld.so on Linux and libHelloWorld.dylib on Mac OS):

# Run the pass
$LLVM_DIR/bin/opt -load-pass-plugin ./libHelloWorld.{so|dylib} -passes=hello-world -disable-output input_for_hello.ll
# Expected output
(llvm-tutor) Hello from: foo
(llvm-tutor)   number of arguments: 1
(llvm-tutor) Hello from: bar
(llvm-tutor)   number of arguments: 2
(llvm-tutor) Hello from: fez
(llvm-tutor)   number of arguments: 3
(llvm-tutor) Hello from: main
(llvm-tutor)   number of arguments: 2

The HelloWorld pass doesn't modify the input module. The -disable-output flag is used to prevent opt from printing the output bitcode file.

Development Environment

Platform Support And Requirements

This project has been tested on Ubuntu 22.04 and Mac OS X 11.7. In order to build llvm-tutor you will need: * LLVM 22 * C++ compiler that supports C++17 * CMake 3.20 or higher

In order to run the passes, you will need: * clang-22 (to generate input LLVM files) * opt (to run the passes)

There are additional requirements for tests (these will be satisfied by installing LLVM 22): * lit (aka llvm-lit, LLVM tool for executing the tests) * FileCheck (LIT requirement, it's used to check whether tests generate the expected output)

Installing LLVM 22 on Mac OS X

On Darwin you can install LLVM 22 with Homebrew:

brew install llvm@22

If you already have an older version of LLVM installed, you can upgrade it to LLVM 22 like this:

brew upgrade llvm

Once the installation (or upgrade) is complete, all the required header files, libraries and tools will be located in /opt/homebrew/opt/llvm/.

Installing LLVM 22 on Ubuntu

On Ubuntu Jammy Jellyfish, you can install modern LLVM from the official repository:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo apt-add-repository "deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-22 main"
sudo apt-get update
sudo apt-get install -y llvm-22 llvm-22-dev llvm-22-tools clang-22

This will install all the required header files, libraries and tools in /usr/lib/llvm-22/.

Building LLVM 22 From Sources

Building from sources can be slow and tricky to debug. It is not necessary, but might be your preferred way of obtaining LLVM 22. The following steps will work on Linux and Mac OS X:

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout release/22.x
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=host -DLLVM_ENABLE_PROJECTS=clang <llvm-project/root/dir>/llvm/
cmake --build .

For more details read the official documentation.

Building & Testing

Building

You can build llvm-tutor (and all the provided pass plugins) as follows:

cd <build/dir>
cmake -DLT_LLVM_INSTALL_DIR=<installation/dir/of/llvm/22> 


make

The LT_LLVM_INSTALL_DIR variable should be set to the root of either the installation or build directory of LLVM 22. It is used to locate the corresponding LLVMConfig.cmake script that is used to set the include and library paths.

Testing

In order to run llvm-tutor tests, you need to install llvm-lit (aka lit). It's not bundled with LLVM 22 packages, but you can install it with pip:

# Install lit - note that this installs lit globally
pip install lit

Running the tests is as simple as:

$ lit <build_dir>/test

Voilà! You should see all tests passing.

LLVM Plugins as shared objects

In llvm-tutor every LLVM pass is implemented in a separate shared object (you can learn more about shared objects here). These shared objects are essentially dynamically loadable plugins for opt. All plugins are built in the <build/dir>/lib directory.

Note that the extension of dynamically loaded shared objects differs between Linux and Mac OS. For example, for the HelloWorld pass you will get:

• libHelloWorld.so on Linux
• libHelloWorld.dylib on MacOS.

For the sake of consistency, in this README.md file all examples use the *.so extension. When working on Mac OS, use *.dylib instead.

Overview of The Passes

The available passes are categorised as either Analysis, Transformation or CFG. The difference between Analysis and Transformation passes is rather self-explanatory (here is a more technical breakdown). A CFG pass is simply a Transformation pass that modifies the Control Flow Graph. This is frequently a bit more complex and requires some extra bookkeeping, hence a dedicated category.

In the following table the passes are grouped thematically and ordered by the level of complexity.

Name	Description	Category
HelloWorld	visits all functions and prints their names	Analysis
OpcodeCounter	prints a summary of LLVM IR opcodes in the input module	Analysis
InjectFuncCall	instruments the input module by inserting calls to printf	Transformation
StaticCallCounter	counts direct function calls at compile-time (static analysis)	Analysis
DynamicCallCounter	counts direct function calls at run-time (dynamic analysis)	Transformation
MBASub	obfuscate integer sub instructions	Transformation
MBAAdd	obfuscate 8-bit integer add instructions	Transformation
FindFCmpEq	finds floating-point equality comparisons	Analysis
ConvertFCmpEq	converts direct floating-point equality comparisons to difference comparisons	Transformation
RIV	finds reachable integer values for each basic block	Analysis
DuplicateBB	duplicates basic blocks, requires RIV analysis results	CFG
MergeBB	merges duplicated basic blocks	CFG

Once you've built this project, you can experiment with every pass separately. All passes, except for HelloWorld, are described in more details below.

LLVM passes work with LLVM IR files. You can generate one like this:

export LLVM_DIR=<installation/dir/of/llvm/22>
# Textual form
$LLVM_DIR/bin/clang -O1 -emit-llvm input.c -S -o out.ll
# Binary/bit-code form
$LLVM_DIR/bin/clang -O1 -emit-llvm input.c -c -o out.bc

It doesn't matter whether you choose the binary, *.bc (default), or textual/LLVM assembly form (.ll, requires the -S flag). Obviously, the latter is more human-readable. Similar logic applies to opt - by default it generates *.bc files. You can use -S to have the output written as *.ll files instead.

Note that clang adds the optnone function attribute if either

• no optimization level is specified, or
• -O0 is specified.

If you want to compile at -O0, you need to specify -O0 -Xclang -disable-O0-optnone or define a static isRequired method in your pass. Alternatively, you can specify -O1 or higher. Otherwise the new pass manager will register the pass but your pass will not be executed.

As noted earlier, all examples in this file use the *.so extension for pass plugins. When working on Mac OS, use *.dylib instead.

OpcodeCounter

OpcodeCounter is an Analysis pass that prints a summary of the LLVM IR opcodes encountered in every function in the input module. This pass can be run automatically with one of the pre-defined optimisation pipelines. However, let's use our tried and tested method first.

Run the pass

We will use input_for_cc.c to test OpcodeCounter. Since OpcodeCounter is an Analysis pass, we want opt to print its results. To this end, we will use a printing pass that corresponds to OpcodeCounter. This pass is called print<opcode-counter>. No extra arguments are needed, but it's a good idea to add -disable-output to prevent opt from printing the output LLVM IR module - we are only interested in the results of the analysis rather than the module itself. In fact, as this pass does not modify the input IR, the output module would be identical to the input anyway.

export LLVM_DIR=<installation/dir/of/llvm/22>
# Generate an LLVM file to analyze
$LLVM_DIR/bin/clang -emit-llvm -c <source_dir>/inputs/input_for_cc.c -o input_for_cc.bc
# Run the pass through opt
$LLVM_DIR/bin/opt -load-pass-plugin <build_dir>/lib/libOpcodeCounter.so --passes="print<opcode-counter>" -disable-output input_for_cc.bc

For main, OpcodeCounter prints the following summary (note that when running the pass, a summary for other functions defined in input_for_cc.bc is also printed):

=================================================
LLVM-TUTOR: OpcodeCounter results for `main`
=================================================
OPCODE               #N TIMES USED
-------------------------------------------------
load                 2
br                   4
icmp                 1
add                  1
ret                  1
alloca               2
store                4
call                 4
-------------------------------------------------

Auto-registration with optimisation pipelines

You can run OpcodeCounter by simply specifying an optimisation level (e.g. -O{1|2|3|s}). This is achieved through auto-registration with the existing optimisation pass pipelines. Note that you still have to specify the plugin file to be loaded:

```bash $LLVM_DI