MCPcopy Index your code
hub / github.com/twitter/the-algorithm-ml

github.com/twitter/the-algorithm-ml @main

repository ↗ · DeepWiki ↗ · + Follow
477 symbols 1,618 edges 91 files 138 documented · 29% ● updated 1y ago★ 10,59120 open issues
README

This project open sources some of the ML models used at Twitter.

Currently these are:

  1. The "For You" Heavy Ranker (projects/home/recap).

  2. TwHIN embeddings (projects/twhin) https://arxiv.org/abs/2202.05387

This project can be run inside a python virtualenv. We have only tried this on Linux machines and because we use torchrec it works best with an Nvidia GPU. To setup run

./images/init_venv.sh (Linux only).

The READMEs of each project contain instructions about how to run each project.

Core symbols most depended-on inside this repo

compute
called by 18
metrics/rce.py
update
called by 18
core/metric_mixin.py
to
called by 10
common/batch.py
reset
called by 9
metrics/rce.py
update
called by 8
metrics/rce.py
on_kf
called by 8
machines/environment.py
maybe_apply_stratification
called by 6
core/metric_mixin.py
as_dict
called by 6
common/batch.py

Shape

Function 168
Method 166
Class 142
Route 1

Languages

Python100%

Modules by API surface

core/train_pipeline.py29 symbols
core/metrics.py25 symbols
projects/home/recap/model/feature_transform.py20 symbols
projects/home/recap/model/config.py20 symbols
projects/home/recap/data/preprocessors.py20 symbols
projects/home/recap/data/dataset.py20 symbols
common/checkpointing/snapshot.py18 symbols
projects/home/recap/data/config.py17 symbols
common/batch.py15 symbols
machines/environment.py14 symbols
reader/dataset.py12 symbols
metrics/rce.py12 symbols

Dependencies from manifests, versioned

Cython0.29.32 · 1×
GitPython3.1.31 · 1×
Jinja23.1.2 · 1×
Keras-Preprocessing1.1.2 · 1×
Markdown3.4.1 · 1×
MarkupSafe2.1.1 · 1×
PyYAML6.0.0 · 1×
Pygments2.14.0 · 1×
Werkzeug2.2.3 · 1×
absl-py1.4.0 · 1×
aiofiles22.1.0 · 1×
aiohttp3.8.3 · 1×

For agents

$ claude mcp add the-algorithm-ml \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact