Colossal-AI

Colossal-AI: Making large AI models cheaper, faster, and more accessible

Paper | Documentation | Examples | Forum | GPU Cloud Playground | Blog

| English | 中文 |

Get Started with Colossal-AI Without Setup

Access high-end, on-demand compute for your research instantly—no setup needed.

Sign up now and get $10 in credits!

Limited Academic Bonuses:

• Top up $1,000 and receive 300 credits
• Top up $500 and receive 100 credits

Latest News

• [2025/02] DeepSeek 671B Fine-Tuning Guide Revealed—Unlock the Upgraded DeepSeek Suite with One Click, AI Players Ecstatic!
• [2024/12] The development cost of video generation models has saved by 50%! Open-source solutions are now available with H200 GPU vouchers [code] [vouchers]
• [2024/10] How to build a low-cost Sora-like app? Solutions for you
• [2024/09] Singapore Startup HPC-AI Tech Secures 50 Million USD in Series A Funding to Build the Video Generation AI Model and GPU Platform
• [2024/09] Reducing AI Large Model Training Costs by 30% Requires Just a Single Line of Code From FP8 Mixed Precision Training Upgrades
• [2024/06] Open-Sora Continues Open Source: Generate Any 16-Second 720p HD Video with One Click, Model Weights Ready to Use
• [2024/05] Large AI Models Inference Speed Doubled, Colossal-Inference Open Source Release
• [2024/04] Open-Sora Unveils Major Upgrade: Embracing Open Source with Single-Shot 16-Second Video Generation and 720p Resolution
• [2024/04] Most cost-effective solutions for inference, fine-tuning and pretraining, tailored to LLaMA3 series

Table of Contents

• Why Colossal-AI
• Features
• Colossal-AI for Real World Applications
  • Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models
  • Colossal-LLaMA-2: One Half-Day of Training Using a Few Hundred Dollars Yields Similar Results to Mainstream Large Models, Open-Source and Commercial-Free Domain-Specific Llm Solution
  • ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline
  • AIGC: Acceleration of Stable Diffusion
  • Biomedicine: Acceleration of AlphaFold Protein Structure
• Parallel Training Demo
  • LLaMA 1/2/3
  • MoE
  • GPT-3
  • GPT-2
  • BERT
  • PaLM
  • OPT
  • ViT
  • Recommendation System Models
• Single GPU Training Demo
  • GPT-2
  • PaLM
• Inference
  • Colossal-Inference: Large AI Models Inference Speed Doubled
  • Grok-1: 314B model of PyTorch + HuggingFace Inference
  • SwiftInfer:Breaks the Length Limit of LLM for Multi-Round Conversations with 46% Acceleration
• Installation
  • PyPI
  • Install From Source
• Use Docker
• Community
• Contributing
• Cite Us

Why Colossal-AI

Prof. James Demmel (UC Berkeley): Colossal-AI makes training AI models efficient, easy, and scalable.

(back to top)

Features

Colossal-AI provides a collection of parallel components for you. We aim to support you to write your distributed deep learning models just like how you write your model on your laptop. We provide user-friendly tools to kickstart distributed training and inference in a few lines.

• Parallelism strategies
• Data Parallelism
• Pipeline Parallelism
• 1D, 2D, 2.5D, 3D Tensor Parallelism
• Sequence Parallelism
• Zero Redundancy Optimizer (ZeRO)

• Auto-Parallelism

• Heterogeneous Memory Management

• PatrickStar

• Friendly Usage

• Parallelism based on the configuration file

(back to top)

Colossal-AI in the Real World

Open-Sora

Open-Sora：Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models [code] [blog] [Model weights] [Demo] [GPU Cloud Playground] [OpenSora Image]

(back to top)

Colossal-LLaMA-2

[GPU Cloud Playground] [LLaMA3 Image]

• 7B: One half-day of training using a few hundred dollars yields similar results to mainstream large models, open-source and commercial-free domain-specific LLM solution. [code] [blog] [HuggingFace model weights] [Modelscope model weights]

• 13B: Construct refined 13B private model with just $5000 USD. [code] [blog] [HuggingFace model weights] [Modelscope model weights]