MCPcopy Index your code
hub / github.com/Naveenaidu/rust-1brc

github.com/Naveenaidu/rust-1brc @main

Chat with this repo
repository ↗ · DeepWiki ↗ · + Follow
10 symbols 19 edges 1 files 0 documented · 0%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

1BRC

1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated. This repo contains my implementation in Rust!

I wrote a detailed blog about my implementation approach, you can check it out here.

Record of iterations

Final implementation approach looks like this: final iteration visualised

Here is a more detailed record of each individual iteration:

Attempt Number Approach Execution Time (sec) Diff Commit Flamegraph
1 Naive Implementation: Read temperatures into a map of cities. Iterate serially over each key (city) in map to find min, max and average temperatures. 253 455609a flamegraph
2 Remove redundant vector creation 241 -12 fb2dda8 flamegraph
3 Use BufReader for reading the file 137 -104 1f411b6 flamegraph
4 Faster float parsing 134 -3 df29672 flamegraph
5 Use a faster hashing algorithm and hashmap(FxHashMap) 123 -11 afc73a1 flamegraph
6 Use read_line instead of read_lines. This maintains a single buffer to store each line we read 105 -18 d4b60cf flamegraph
7 Use bytes [u8] instead of String 83 -22 2bc8cc3 flamegraph
8 Use mmap to read the file, this gives us data in &[u8] 78 -5 38bdd01 flamegraph
9 Use memchr to split the string, memchr has SIMD optimizations 71 -7 d24e56f flamegraph
10 Parallelization - 1, A single producer and multiple receiver with an additional unprocessed_buffer to store the data that cannot be sent in a chunk 12 -59 14f33a0 flamegraph
11 Parallelization - 2, uses only a single buffer to store the chunks 9 -3 ad0ea99 flamegraph

Core symbols most depended-on inside this repo

round_off
called by 3
src/main.rs
calculate_station_values
called by 2
src/main.rs
process_chunk
called by 1
src/main.rs
write_result_stdout
called by 1
src/main.rs
find_new_line_pos
called by 1
src/main.rs
read_test_output_file
called by 1
src/main.rs
main
called by 0
src/main.rs

Shape

Function 8
Class 2

Languages

Rust100%

Modules by API surface

src/main.rs10 symbols

For agents

$ claude mcp add rust-1brc \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact