hub / github.com/babybear-labs/benchmark
github.com/babybear-labs/benchmark @main
Chat with this repo
129 symbols
257 edges
59 files
17 documented · 13%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README
ZK Benchmark
Benchmark Date: Nov 25, 2024
Overview
This repository hosts the source code and resources for benchmarking various Zero-Knowledge (ZK) proofs. It builds upon the original repositories of Halo2, RISC0, Nexus, SP1, Jolt, Circom, and Powdr with our implementations of proofs including but not limited to SHA256, Fibonacci, and Poseidon operations.
Our aim is to benchmark the performance of these proofs, comparing them in terms of time, space, and other metrics. This comprehensive comparison will not only highlight the most popular ZK proofs and their implementations but also deepen our understanding of their inner workings, developer friendliness, and proving systems.
Goals
- Benchmark Operations:
- Fibonacci: Variants 1, 10, 100, 1000, 10000
- SHA256: Variants 32b, 1k, 10k Bytes
-
Poseidon: Variants 32b, 100b
-
Additional Benchmarks:
- Arithmetic Operations: Addition, Subtraction, Multiplication, Division
- Loop: Variants 10, 100, 1000
- Hash Functions: Pederson, RPO, Keccak (Variants 1k, 10k Bytes)
- Merkle Tree: Generation, Membership/Inclusion, Merge
- nth Prime: Variants 1, 10, 100, 1000, 10000
- Cryptographic Verifications: ECDSA, Elliptic Curve, BLS Verification, BLS Aggregation
General overview of features of proving schemes
| Proof System | Setup Complexity | Post-Quantum Resistance | Scalability | Parallel Execution | Features |
|---|---|---|---|---|---|
| Halo2 | Transparent | No (ECC based) | High | Moderate | Recursive SNARKs, Custom gates, Plonkish arithmetization |
| Circom (Groth16) | Trusted Setup | No (Pairing-based) | Moderate | High | Efficient SNARK generation for circuits, good for small proof |
| Risc Zero | Transparent | Yes | High | High | General-purpose zkVM, RISC-V architecture, supports arbitrary computations (STARK + AIR + FRI + RISC V) |
| Jolt | Both | Yes | Very High | Very High | Sumcheck and lookup arguments, optimized for high-performance proof generation, upcoming twist and shout ( Spartan Sumcheck + R1CS + Zero morph + RISC V ) |
| Nexus zkVM | Transparent | Partial | Moderate | High | Privacy-focused zkVM, Ethereum compatibility, supports smart contract verification (R1CS + Relazxed R1CS + Nova + Hyper Nova + Risetto255 + RISC V ) |
| SP1 | Transparent | Yes | Very High | Very High | Optimized for rollups, efficient parallel proving, GPU-optimised, supports high-throughput applications, supports arbitrary computations, whitepaper. ZK: Plonky3 (STARK + AIR + FRI + RISC V), Multi set hashing, towards real-time proving with jagged PCS |
| Powdr | Transparent | Yes | High | High | Developer-friendly, multiple proof systems, and zk-continuations for unbounded execution, minimal setup |
| Polygon Miden | Transparent | Yes | High | High | STARK-based (Winterfell) zkVM, designed for program execution proofs, high scalability |
| Aleo SnarkVM | Trusted Setup | No (Pairing-based) | Moderate | High | Privacy-preserving SNARK-based zkVM, supports private smart contracts |
| Zokrates | Trusted Setup | No (Pairing-based) | Moderate | High | High-level DSL for zk-SNARKs, extensive developer tooling, supports Groth16 and PLONK |
| Delphinus ZKWASM | Transparent | Yes | High | High | WebAssembly-based zkVM, supports proving WASM execution, ideal for cross-platform apps |
| Lita Valida | Transparent | Yes | Moderate | Moderate | Modular chip, ensures the correctness of zk circuits, supports custom proofs (STARK + AIR + FRI + LITA ISA) |
| Eigen ZKVM | Transparent | Yes | Very High | High | High-performance zkVM, supports recursive proofsm has SIMD optimization. |
| CairoVM | Transparent | Yes | Very High | High | STARK-based VM, designed for StarkNet, supports general-purpose computation |
| Noir | Transparent | Yes (STARK-based) | High | High | High-level zkDSL, supports general-purpose zk circuits, STARK-based backend |
| Ola VM | Transparent | Yes | High | High | STARK-based, Privacy-preserving zkVM, supports custom circuits, programmable, scalable, and private proof |
| Triton VM | Transparent | Yes | High | High | Efficient recursive verification, Algebraic Execution Tables (AET) and Arithmetic Intermediate Representations (AIR) with a STARK proof system. |
| Lurk | Transparent | Yes | Moderate | Moderate | zk-LISP interpreter, supports verifiable computations, ideal for symbolic execution |
| Ceno | Transparent | Yes | High | High | Non-uniform prover based on GKR Protocol, Segment and Parallel Zero-knowledge Virtual Machine |
| Expander | Transparent | Yes | Very High | Very High | GKR + Libra, Parallel computing |
| OpenVM | Transparent | Yes | High | High | Modular architecture, supports general-purpose zkVM, adaptable to various use cases |
| Brevis | Transparent | Yes | Very High | High | Focused on proof aggregation and recursion, modular, glue-and-coprocessor architecture, multiple proving backends, Coprocessor Integration (STARK / CIRCLESTARK + AIR + FRI + RISC V) |
| PetraVM | Transparent | Yes | High | High | Binius based |
The key differences between these proof systems include their setup complexity, proof features, post-quantum resistance, scalability, and parallel execution capabilities. Some use recursive proofs, while others focus on efficiency or general-purpose functionality. The systems also vary in their post-quantum resistance and ability to scale and parallelize.
Proving Schemes
| Name | Type | Benchmark Code | Reports |
|---|---|---|---|
| Jolt | zkVM | code | report |
| Powdr | zkVM | code | report |
| Circom | HDL | code | report |
| Nexus | zkVM | code | report |
| Halo2 | Proof System | code | report |
| Risc Zero | zkVM | code | report |
| SP1 | zkVM | code | report |
| Polygon Miden | zkVM | TBA | TBA |
| Aleo SnarkVM | zkVM | TBA | TBA |
| Zokrates | DSL | TBA | TBA |
| Delphinus Lab ZKWASM | zkVM | TBA | TBA |
| Lita Valida | zkVM | TBA | TBA |
| Eigen ZKVM | zkVM | TBA | TBA |
| CairoVM | zkVM | TBA | TBA |
| Noir | DSL | TBA | TBA |
| Ola VM | zkVM | TBA | TBA |
| Triton VM | zkVM | TBA | TBA |
| Lurk | zkVM | TBA | TBA |
| Ceno | zkVM | TBA | TBA |
| Expander | zkVM | TBA | TBA |
| OpenVM | zkVM | TBA | TBA |
| Brevis Pico | zkVM | TBA | TBA |
- zkVM: Zero-Knowledge Virtual Machine, a virtual machine designed to execute programs written in standard programming languages while generating zero-knowledge proofs of their correctness.
- HDL: Hardware Description Language, used for circuit descriptions.
- DSL: Domain-Specific Language, designed specifically for writing ZKP programs.
- Proof System: Framework for creating and verifying zk-proofs
- TBA: To Be Added
Result Summary - as of Nov 2024
Only selected operations are shown. The detailed reports have metrics for other variants and additional operations.
Fibonacci
10000 th number
| Proof System | Prover Time (s) | Cycles | Verifier Time (s) | Prover Memory (KB) | Proof Size (B) |
|---|---|---|---|---|---|
| Halo2 | 0.196 | - | 0.004 | 9.8 | 1664 |
| Circom | 1.75 | 9999 | 0.81 | 466280 | 805 |
| Risc Zero | 6.37 | 65536 | - | - | 206182 |
| Jolt | 21.73 | 280287 | 0.01 | - | 452398 |
| Nexus (max 100) | 35.2 | - | 2.4 | - | 47.9 MB |
| SP1 | 18.87 | 69101 | 0.174 | - | 2656912 |
| Powdr | 8.64 | 2990 | - | - | - |
SHA256
1 KB Input
| Proof System | Prover Time (s) | Cycles | Verifier Time (s) | Prover Memory (KB) | Proof Size (B) |
|---|---|---|---|---|---|
| Halo2 | 14.78 | - | 0.13 | 1134 | 4064 |
| Circom | 46.07 | 540736 | 1.14 | 3920848 | 805 |
| Risc Zero | 2.5 | 65536 | - | - | 210157 |
| Jolt | 2.199 | 62231 | 0.052 | - | 401116 |
| Nexus | 30+ mins | - | - | - | - |
| SP1 | 17.6 | 71249 | 0.172 | - | 265691 |
| Powdr | 9.07 | 73731 | - | - | - |
Poseidon
Extension points exported contracts — how you extend this code
HashFn (Interface)(no doc) [2 implementers]
risc-zero/shared/src/hash.rs
Core symbols most depended-on inside this repo
measure_commandcalled by 16
circom/scripts/util.py
iter
called by 11
risc-zero/shared/src/lib.rs
digest
called by 4
risc-zero/shared/src/lib.rs
generate_circuit
called by 4
circom/scripts/util.py
extract_contraints
called by 4
circom/scripts/util.py
log_session
called by 3
risc-zero/host/benches/bench.rs
sha256_hash
called by 2
powdr/merkle-leaf-proof/guest/src/main.rs
sha256_hash
called by 2
jolt/algos/merkle-leaf-proof/guest/src/lib.rs
Shape
Languages
Rust84%
Python16%
Modules by API surface
circom/scripts/util.py16 symbols
risc-zero/shared/src/lib.rs12 symbols
nexus/src/sha256/sha256.rs9 symbols
powdr/merkle-leaf-proof/guest/src/main.rs8 symbols
jolt/algos/merkle-leaf-proof/guest/src/lib.rs7 symbols
risc-zero/shared/src/hash.rs6 symbols
risc-zero/host/benches/bench.rs4 symbols
powdr/poseidon/guest/src/main.rs3 symbols
nexus/src/main.rs3 symbols
sp1/algos/sha256/script/src/bin/main.rs2 symbols
sp1/algos/poseidon/script/src/bin/main.rs2 symbols
sp1/algos/poseidon/lib/src/lib.rs2 symbols
For agents
$ claude mcp add benchmark \
-- python -m otcore.mcp_server <graph>