Fast Binary Encoding (FBE)

Fast Binary Encoding allows to describe any domain models, business objects, complex data structures, client/server requests & responses and generate native code for different programming languages and platforms.

Fast Binary Encoding documentation

Fast Binary Encoding downloads

Fast Binary Encoding specification

Performance comparison to other protocols can be found here:

Protocol	Message size	Serialization time	Deserialization time
Cap'n'Proto	208 bytes	558 ns	359 ns
FastBinaryEncoding	234 bytes	66 ns	82 ns
FlatBuffers	280 bytes	830 ns	290 ns
Protobuf	120 bytes	628 ns	759 ns
JSON	301 bytes	740 ns	500 ns

Typical usage workflow is the following: 1. Create domain model using base types, enums, flags and structs 2. Generate domain model for any supported programming languages (C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift) 3. Build domain model library 4. Serialize/Deserialize objects from the domain model in unified, fast and compact FastBinaryEncoding (FBE) format 5. JSON convert objects from the domain model in order to use them in Web API 6. Implement Sender/Receiver interfaces to create a communication protocol

Sample projects: * C++ sample * C# sample * Go sample * Java sample * JavaScript sample * Kotlin sample * Python sample * Ruby sample * Swift sample

Contents

• Features
• Requirements
• How to build?
• Create domain model
• Generate domain model
• Build domain model
• FBE serialization
• FBE final serialization
• JSON serialization
• Packages and import
• Struct keys
• Struct numeration
• Struct inheritance
• Versioning
• Sender/Receiver protocol
• Performance benchmarks
  • Benchmark 1: Serialization
  • Benchmark 2: Deserialization
  • Benchmark 3: Verify

Features

• Cross platform (Linux, MacOS, Windows)
• Generators for C++, C#, Go, Java, JavaScript, Kotlin, Python, Ruby, Swift
• Fast binary encoding format
• Supported base types (byte, bool, char, wchar, int8, int16, int32, int64, float, double)
• Supported complex types (bytes, decimal, string, timestamp, uuid)
• Supported collections (array, vector, list, map, hash)
• Supported nullable optional types, enums, flags and structs
• Serialization/Deserialization to/from binary format
• Serialization/Deserialization to/from JSON
• Sender/Receiver interfaces for communication protocols
• Versioning solution
• Excellent performance

Requirements

• Linux
• MacOS
• Windows
• cmake
• gcc
• git
• gil
• python3

Optional: * clang * CLion * Cygwin * MSYS2 * MinGW * Visual Studio * WinFlexBison

How to build?

Linux: install required packages

sudo apt-get install -y binutils-dev uuid-dev flex bison

MacOS: install required packages

brew install flex bison

Windows: install required packages

choco install winflexbison3

Install gil (git links) tool

pip3 install gil

Setup repository

git clone https://github.com/chronoxor/FastBinaryEncoding.git
cd FastBinaryEncoding
gil update

Linux

cd build
./unix.sh

MacOS

cd build
./unix.sh

Windows (Cygwin)

cd build
unix.bat

Windows (MSYS2)

cd build
unix.bat

Windows (MinGW)

cd build
mingw.bat

Windows (Visual Studio)

cd build
vs.bat

Create domain model

To use Fast Binary Encoding you should provide a domain model (aka business objects). A domain model is a set of enums, flags and structures that relate to each other and might be aggregated in some hierarchy.

Fast Binary Encoding (FBE) format specification

There is a sample domain model which describes Account-Balance-Orders relation of some abstract trading platform:

// Package declaration
package proto

// Domain declaration
domain com.chronoxor

// Order side declaration
enum OrderSide : byte
{
    buy;
    sell;
}

// Order type declaration
enum OrderType : byte
{
    market;
    limit;
    stop;
}

// Order declaration
struct Order
{
    [key] int32 uid;
    string symbol;
    OrderSide side;
    OrderType type;
    double price = 0.0;
    double volume = 0.0;
}

// Account balance declaration
struct Balance
{
    [key] string currency;
    double amount = 0.0;
}

// Account state declaration
flags State : byte
{
    unknown = 0x00;
    invalid = 0x01;
    initialized = 0x02;
    calculated = 0x04;
    broken = 0x08;
    good = initialized | calculated;
    bad = unknown | invalid | broken;
}

// Account declaration
struct Account
{
    [key] int32 uid;
    string name;
    State state = State.initialized | State.bad;
    Balance wallet;
    Balance? asset;
    Order[] orders;
}

Generate domain model

The next step is a domain model compilation using 'fbec' compiler which will create a generated code for required programming language.

The following command will create a C++ generated code:

fbec --c++ --input=proto.fbe --output=.

All possible options for the 'fbec' compiler are the following:

Usage: fbec [options]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -h HELP, --help=HELP  Show help
  -i INPUT, --input=INPUT
                        Input path
  -o OUTPUT, --output=OUTPUT
                        Output path
  -q, --quiet           Launch in quiet mode. No progress will be shown!
  -n INDENT, --indent=INDENT
                        Format indent. Default: 0
  -t, --tabs            Format with tabs. Default: off
  --cpp                 Generate C++ code
  --cpp-logging         Generate C++ logging code
  --csharp              Generate C# code
  --go                  Generate Go code
  --java                Generate Java code
  --javascript          Generate JavaScript code
  --kotlin              Generate Kotlin code
  --python              Generate Python code
  --ruby                Generate Ruby code
  --swift               Generate Swift code
  --final               Generate Final serialization code
  --json                Generate JSON serialization code
  --proto               Generate Sender/Receiver protocol code

Build domain model

Generated domain model is represented with source code for the particular language. Just add it to your project and build it. There are several issues and dependencies that should be mentioned:

C++

• C++ standard is limited to C++17 in order to have the implementation of std::optional;
• C++ has no native support for decimal type. Currently decimal type is emulated with a double type. FBE does not use GMPLib because of heavy dependency in generated source code;
• C++ formatting is supported with {fmt} library of version started from 9.0.0. Required include headers are <fmt/format.h> and <fmt/ostream.h>;
• JSON serialization is implemented using RapidJSON library;

C

• JSON serialization is implemented using Json.NET library. Therefore it should be imported using NuGet;
• Fast JSON serialization libraty is also available - Utf8Json . If you want to try it, you should import is with NuGet and build domain model with 'UTF8JSON' definition;

Go

• Assert testing is based on stretchr/testify package (go get github.com/stretchr/testify);
• JSON serialization is based on jsoniter package (go get github.com/json-iterator/go);
• Decimal type is based on shopspring/decimal package (go get github.com/shopspring/decimal);
• UUID type is based on google/uuid package (go get github.com/google/uuid);

Java

• JSON serialization is implemented using Gson package. Therefore it should be imported using Maven;

JavaScript

• JavaScript domain model is implemented using ECMAScript 6 (classes, etc.);
• JSON serialization of set, map and hash types is limited to key with string type;

Kotlin

• Starting from the version 1.3 Kotlin supports unsigned integer numbers (UByte, UShort, UInt, ULong). This gives ability to represent FBE domain model more accurately than Java language does;
• JSON serialization is implemented using Gson package. Therefore it should be imported using Maven;

Python

• Python 3.7 is required because of time.time_ns();

Ruby

• Some Ruby dependencies should be installed from Gems:

gem install json
gem install uuidtools

Swift

• Swift domain model is implemented using uses SwiftPM as its build tool;
• JSON serialization is implemented using Codable protocol;
• JSON serialization of s