<img alt="Bespoke Labs Logo" width="100px" src="https://github.com/bespokelabsai/curator/blob/main/docs/Bespoke-Labs-Logomark-Red-crop.png">

Bespoke Curator

Bulk Inference and Scalable Data Curation for Post-Training

[ English | 中文 ]

🎉 What's New

• [2026.06.09] Fireworks AI integration for fine-tuning: Run managed supervised fine-tuning (SFT) on Fireworks AI with FireworksTrainer — upload curated data, train a LoRA, and sample from the deployed model behind the same trainer interface as Tinker.
• [2026.03.14] Tinker integration for fine-tuning: Go from curated data to a LoRA fine-tuned model in a few lines of Python using the Tinker SDK.
• [2025.12.05] Launched OpenThoughts-Agents whose data was curated using Curator.
• [2025.04.09] Launching Reasoning Datasets Competition with HuggingFace and Together.ai. Win $5000 USD worth of prizes!
• [2025.04.03] We used Bespoke Curator to create OpenThoughts2-1M dataset, which was used to train OpenThinker2-32B that outperforms DeepSeek-R1-32B. The dataset started trending on HuggingFace.
• [2025.03.12] Gemini Batch support added: Gemini batch API is extremely challenging, and we made it much simpler! :)
• [2025.03.05] Claude 3.7 Sonnet Thinking and batch mode support added.
• [2025.02.26] Code Execution Support added: You can now run code (generated by Curator) using CodeExecutor. We support four backends: local (called multiprocessing), Ray, Docker and e2b.
• [2025.02.06] We used Bespoke Curator to create s1K-1.1, a high-quality sample-efficient reasoning dataset.
• [2025.01.30] Batch Processing Support for OpenAI, Anthropic, and other compatible APIs: Cut Token Costs in Half 🔥🔥🔥. Through our partnership with kluster.ai, new users using Curator can access open-source models like DeepSeek-R1 and receive a $25 credit (limits apply). EDIT: Promotion has come to an end.
• [2025.01.27] We used Bespoke Curator to create OpenThoughts-114k, a high-quality reasoning dataset (trending on HuggingFace).
• [2025.01.22] We used Bespoke Curator to create Bespoke-Stratos-17k, a high-quality reasoning dataset (trending on HuggingFace).
• [2025.01.15] Curator launched 🎉

Overview

Bespoke Curator makes it easy to create synthetic data pipelines. Whether you are training a model or extracting structured data, Curator will prepare high-quality data quickly and robustly.

• Rich Python based library for generating and curating synthetic data.
• Viewer to monitor data while it is being generated.
• First class support for structured outputs.
• Built-in performance optimizations for asynchronous operations, caching, and fault recovery at every scale.
• Support for a wide range of inference options via LiteLLM, vLLM, and popular batch APIs.

Check out our full documentation for getting started, tutorials, guides and detailed reference.

🛠️ Installation

pip install bespokelabs-curator

📕 Examples

Finetuning/Distillation

Task	Link(s)	Goal
Product feature extraction		Finetuning a model to identify features of a product
Sentiment analysis		Aspect-based sentiment analysis of restaurant reviews and finetuning using Together.ai
RAFT for domain-specific RAG	Code	Implement Retrieval Augmented Fine-Tuning (RAFT) that processes domain-specific documents, generates questions, and prepares data for fine-tuning LLMs.
Poem generation & LoRA fine-tuning	Code	End-to-end pipeline: curate poem data with Curator, then LoRA fine-tune with TinkerTrainer
Managed fine-tuning on Fireworks AI	Code	Run a managed SFT job on Fireworks AI with FireworksTrainer, then sample from the deployed LoRA model

Data Generation

Task	Link(s)	Goal
Reasoning dataset generation (Bespoke Stratos)	Code	Generate the Bespoke-Stratos-17k dataset, focusing on reasoning traces from math, coding, and problem-solving datasets.
Reasoning dataset generation (Open Thoughts)	Code	Generate the Open-Thoughts-114k dataset, focusing on reasoning traces from math, coding, and problem-solving datasets.
Multimodal	Code	Demonstrates multimodal capabilities by generating recipes from food images
Ungrounded Question Answer generation	Code	Generate diverse question-answer pairs using techniques similar to the CAMEL paper
Code Execution		Execute code generated with Curator
3Blue1Brown video generation	Code	Generate videos similar to 3Blue1Brown and render them using code execution!
Synthetic charts	Code	Generate charts synthetically.
Function calling	Code	Generate data for finetuning for function calling.

🚀 Quickstart

Using curator.LLM for Bulk Inference

from typing import Dict
from bespokelabs import curator
from datasets import Dataset
from pydantic import BaseModel, Field
from typing import Literal

class Sentiment(BaseModel):
  sentiment: Literal["positive", "negative", "neutral"] = Field(
    description="Sentiment of the review")

class SentimentAnalyzer(curator.LLM):

  def prompt(self, product: Dict):
    return f"Determine the sentiment of the product from the review: {product['review']}"

  def parse(self, product: Dict, response: Sentiment):
    return [{"name": product["name"], "sentiment": response.sentiment}]

# You can easily have a million rows here. 
# Curator takes care of parallelism, retries, and caches responses.
dataset = [{"name": "Curator", "review": "Already saved hours in one day of use."},
           {"name": "Bespoke MiniCheck", "review": "Hallucination rates dropped by 90%."}]

# You can set batch=True, and instantly uses batch mode to save 50% of the costs.
analyzer = SentimentAnalyzer(
    model_name="gpt-4o-mini", response_format=Sentiment, batch=False)
reviews = analyzer(dataset)
print(reviews.to_pandas())

Output:

                name sentiment
0            Curator  positive
1  Bespoke MiniCheck  positive

In the SentimentAnalyzer class: * prompt takes the input (product) and returns the prompt for the LLM. * parse takes the input (product) and the structured output (response) and converts it to a list of dictionaries. This is so that we can easily convert the output to a HuggingFace Dataset object.

Instead of a list, you can pass a HuggingFace Dataset object as well (see below for more details).

Using curator.LLM for data generation

Here's an example of using structured outputs and chaing together two curator.LLM blocks to generate diverse poems.

from typing import Dict, List
from bespokelabs import curator
from pydantic import BaseModel, Field

class Topics(BaseModel):
    topics_list: List[str] = Field(description="A list of topics.")

class TopicGenerator(curator.LLM):
  response_format = Topics

  def prompt(self, subject):
    return f"Return 3 topics related to {subject}"

  def parse(self, input: str, response: Topics):
    return [{"topic": t} for t in response.topics_list]


class Poem(BaseModel):
    title: str = Field(description="The title of the poem.")
    poem: str = Field(description="The content of the poem.")

class Poet(curator.LLM):
    response_format = Poem

    def prompt(self, input: Dict) -> str:
        return f"Write two poems about {input['topic']}."

    def parse(self, input: Dict, response: Poem) -> Dict:
        return [{"title": response.title, "poem": response.poem}]

topic_generator = TopicGenerator(model_name="gpt-4o-mini")
poet = Poet(model_name="gpt-4o-mini")
# Start generation
topics = topic_generator("Mathematics")
poems = poet(topics)

Output:

    title                     poem
0   The Language of Algebra   In symbols and signs, truths intertwine,..
1   The Geometry of Space     In the world around us, shapes do collide,..
2   The Language of Logic     In circuits and wires where silence speaks,..

You can see more examples in the examples directory.

See the docs for more details as well as for troubleshooting information.

[!TIP] If you are generating large datasets, you may want to use batch mode to save costs. Currently batch APIs from OpenAI and Anthropic are supported. With curator this is as simple as setting batch=True in the LLM class. [!NOTE] Retries and caching are enabled by default to help you rapidly iterate your data pipelines. So now if you run the same prompt again, you will get the same response, pretty much instantly. You can delete the cache at ~/.cache/curator or disable it with export CURATOR_DISABLE_CACHE=true.

[!IMPORTANT] Make sure to set your API keys as environment variables for the model you are calling. For example running export OPENAI_API_KEY=sk-... and export ANTHROPIC_API_KEY=ant-... will allow you to run the previous two examples. A full list of supported models and their associated environment variable names can be found in the litellm docs.

Anonymized Telemetry

We collect minimal, anonymized usage telemetry to help prioritize new features and improvements that benefit the Curator community. You can opt out by setting the TELEMETRY_ENABLED environment variable to False.

📖 Providers

Curator supports a wide range of providers, including OpenAI, Anthro