MCPcopy
hub / github.com/baaivision/Emu

github.com/baaivision/Emu @main sqlite

repository ↗ · DeepWiki ↗
656 symbols 2,268 edges 91 files 127 documented · 19%
README

Emu: Generative Multimodal Models from BAAI


comparison_fewshot.

  • Emu1 (ICLR 2024, 2023/07) - Generative Pretraining in Multimodality

  • Emu2 (CVPR 2024, 2023/12) - Generative Multimodal Models are In-Context Learners

  • Emu3 (arXiv 2024, 2024/09) - Next-Token Prediction is All You Need 🔥🔥🔥

News

  • 2024.9 We introduce Emu3, a new suite of state-of-the-art multimodal models trained solely with next-token prediction. 🔥🔥🔥
  • 2024.2 Emu1 and Emu2 are accepted by ICLR 2024 and CVPR 2024 respectively! 🎉
  • 2023.12 Inference code, model and demo of Emu2 are available. Enjoy the demo.
  • 2023.12 We have released Emu2, open and largest generative multimodal models that achieve new state of the art on multimodal understanding and generation tasks.
  • 2023.7 Inference code and model of Emu are available.
  • 2023.7 We have released Emu, a multimodal generalist that can seamlessly generate images and texts in multimodal context.

Hightlights

  • State-of-the-art performance
  • Next-generation capabilities
  • A base model for diverse tasks

We hope to foster the growth of our community through open-sourcing and promoting collaboration👬. Let's step towards multimodal intelligence together🍻.

Contact

  • We are hiring at all levels at BAAI Vision Team, including full-time researchers, engineers and interns. If you are interested in working with us on foundation model, visual perception and multimodal learning, please contact Xinlong Wang (wangxinlong@baai.ac.cn).

Misc

Stargazers repo roster for @baaivision/Emu

Forkers repo roster for @baaivision/Emu

Star History Chart

Core symbols most depended-on inside this repo

join
called by 91
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/logger.py
append
called by 80
Emu1/mm_eval/models/trie.py
append
called by 58
Emu2/demo/frontend/libs/meta.py
info
called by 41
Emu1/mm_eval/datasets/vqa_tools/vqa.py
pop
called by 35
Emu2/demo/frontend/libs/meta.py
write
called by 26
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/data_writer.py
params_num
called by 24
Emu2/demo/backend/hf_model/emu/chat.py
build
called by 23
Emu2/demo/frontend/libs/meta.py

Shape

Method 412
Class 120
Function 112
Route 12

Languages

Python100%

Modules by API surface

Emu1/models/modeling_t5.py83 symbols
Emu1/models/eva_vit_model.py43 symbols
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/data_writer.py32 symbols
Emu2/emu/eva_vit.py30 symbols
Emu2/demo/frontend/libs/meta.py28 symbols
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/logger.py26 symbols
Emu1/mm_eval/models/trie.py24 symbols
Emu1/models/transformer.py15 symbols
Emu2/emu/diffusion.py14 symbols
Emu1/models/rope.py13 symbols
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/dataloader/video_decode.py13 symbols
Emu1/data/yt-sb-1b/video2dataset-1.1.0/video2dataset/data_reader.py13 symbols

Dependencies from manifests, versioned

accelerate0.25.0 · 1×
black22.3.0 · 1×
diffusers0.15.1 · 1×
fire0.4.0 · 1×
gradio3.40.1 · 1×
mypy0.942 · 1×
numpy1.19.5 · 1×
pylint2.13.4 · 1×
pytest7.0.1 · 1×
pytest-cov3.0.0 · 1×
pytest-xdist2.5.0 · 1×
requests2.27.1 · 1×

For agents

$ claude mcp add Emu \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact