Clarifai Python SDK

This is the official Python client for interacting with our powerful API. The Clarifai Python SDK offers a comprehensive set of tools to integrate Clarifai's AI platform to leverage computer vision capabilities like classification , detection ,segementation and natural language capabilities like classification , summarisation , generation , Q&A ,etc into your applications. With just a few lines of code, you can leverage cutting-edge artificial intelligence to unlock valuable insights from visual and textual content.

Website | Schedule Demo | Signup for a Free Account | API Docs | Clarifai Community | Python SDK Docs | Examples | Colab Notebooks | Discord

Give the repo a star ⭐

Table Of Contents

• Installation
• Getting Started
• Compute Orchestration
• Cluster Operations
• Nodepool Operations
• Depolyment Operations
• Interacting with Datasets
• Interacting with Inputs
• Input Upload
• Input Listing
• Interacting with Models
• Model Predict
• Model Training
• Model Listing
• Interacting with Workflows
• Workflow Predict
• Workflow Listing
• Workflow Create
• Workflow Export
• Search
• Smart Image Search
• Smart Text Search
• Filters
• Pagination
• Retrieval Augmented Generation (RAG)
• More Examples

:rocket: Installation

Install from PyPi:

pip install -U clarifai

Install from Source:

git clone https://github.com/Clarifai/clarifai-python.git
cd clarifai-python
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Linting

For developers, use the precommit hook .pre-commit-config.yaml to automate linting.

pip install -r requirements-dev.txt
pre-commit install

Now every time you run git commit your code will be automatically linted and won't commit if it fails.

You can also manually trigger linting using:

pre-commit run --all-files

:memo: Getting started

Clarifai uses Personal Access Tokens(PATs) to validate requests. You can create and manage PATs under your Clarifai account security settings.

• 🔗 Create PAT: Log into Portal → Profile Icon → Security Settings → Create Personal Access Token → Set the scopes → Confirm

• 🔗 Get User ID: Log into Portal → Profile Icon → Account → Profile → User-ID

Export your PAT as an environment variable. Then, import and initialize the API Client.

Set PAT as environment variable through terminal:

export CLARIFAI_PAT={your personal access token}

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.user import User
client = User(user_id="user_id")

# Get all apps
apps_generator = client.list_apps()
apps = list(apps_generator)

OR

PAT can be passed as constructor argument

from clarifai.client.user import User
client = User(user_id="user_id", pat="your personal access token")

:rocket: Compute Orchestration

Clarifai’s Compute Orchestration offers a streamlined solution for managing the infrastructure required for training, deploying, and scaling machine learning models and workflows.

This flexible system supports any compute instance — across various hardware providers and deployment methods — and provides automatic scaling to match workload demands. More Details

Cluster Operations

from clarifai.client.user import User
client = User(user_id="user_id",base_url="https://api.clarifai.com")

# Create a new compute cluster
compute_cluster = client.create_compute_cluster(compute_cluster_id="demo-id",config_filepath="computer_cluster_config.yaml")

# List Clusters
all_compute_clusters = list(client.list_compute_clusters())
print(all_compute_clusters)

Example Cluster Config

Nodepool Operations

from clarifai.client.compute_cluster import ComputeCluster

# Initialize the ComputeCluster instance
compute_cluster = ComputeCluster(user_id="user_id",compute_cluster_id="demo-id")

# Create a new nodepool
nodepool = compute_cluster.create_nodepool(nodepool_id="demo-nodepool-id",config_filepath="nodepool_config.yaml")

#Get a nodepool
nodepool = compute_cluster.nodepool(nodepool_id="demo-nodepool-id")
print(nodepool)

# List nodepools
all_nodepools = list(compute_cluster.list_nodepools())
print(all_nodepools)

Example Nodepool config

Deployment Operations

from clarifai.client.nodepool import Nodepool

# Initialize the Nodepool instance
nodepool = Nodepool(user_id="user_id",nodepool_id="demo-nodepool-id")

# Create a new deployment
deployment = nodepool.create_deployment(deployment_id="demo-deployment-id",config_filepath="deployment_config.yaml")

#Get a deployment
deployment = nodepool.deployment(nodepool_id="demo-deployment-id")
print(deployment)

# List deployments
all_deployments = list(nodepool.list_deployments())
print(all_deployments)

Example Deployment config

Compute Orchestration CLI Operations

Refer Here: https://github.com/Clarifai/clarifai-python/tree/master/clarifai/cli

:floppy_disk: Interacting with Datasets

Clarifai datasets help in managing datasets used for model training and evaluation. It provides functionalities like creating datasets,uploading datasets, retrying failed uploads from logs and exporting datasets as .zip files.

# Note: CLARIFAI_PAT must be set as env variable.

# Create app and dataset
app = client.create_app(app_id="demo_app", base_workflow="Universal")
dataset = app.create_dataset(dataset_id="demo_dataset")

# execute data upload to Clarifai app dataset
from clarifai.datasets.upload.loaders.coco_detection import COCODetectionDataLoader
coco_dataloader = COCODetectionDataLoader("images_dir", "coco_annotation_filepath")
dataset.upload_dataset(dataloader=coco_dataloader, get_upload_status=True)


#Try upload and record the failed outputs in log file.
from clarifai.datasets.upload.utils import load_module_dataloader
cifar_dataloader = load_module_dataloader('./image_classification/cifar10')
dataset.upload_dataset(dataloader=cifar_dataloader,
                       get_upload_status=True,
                       log_warnings =True)

#Retry upload from logs for `upload_dataset`
# Set retry_duplicates to True if you want to ingest failed inputs due to duplication issues. by default it is set to 'False'.
dataset.retry_upload_from_logs(dataloader=cifar_dataloader, log_file_path='log_file.log',
                               retry_duplicates=True,
                               log_warnings=True)

#upload text from csv
dataset.upload_from_csv(csv_path='csv_path', input_type='text', csv_type='raw', labels=True)

#upload data from folder
dataset.upload_from_folder(folder_path='folder_path', input_type='text', labels=True)

# Export Dataset
dataset.export(save_path='output.zip')

:floppy_disk: Interacting with Inputs

You can use inputs() for adding and interacting with input data. Inputs can be uploaded directly from a URL or a file. You can also view input annotations and concepts.

Input Upload

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.user import User
app = User(user_id="user_id").app(app_id="app_id")
input_obj = app.inputs()

#input upload from url
input_obj.upload_from_url(input_id = 'demo', image_url='https://samples.clarifai.com/metro-north.jpg')

#input upload from filename
input_obj.upload_from_file(input_id = 'demo', video_file='demo.mp4')

# text upload
input_obj.upload_text(input_id = 'demo', raw_text = 'This is a test')

Input Listing

#listing inputs
input_generator = input_obj.list_inputs(page_no=1,per_page=10,input_type='image')
inputs_list = list(input_generator)

#listing annotations
annotation_generator = input_obj.list_annotations(batch_input=inputs_list)
annotations_list = list(annotation_generator)

#listing concepts
all_concepts = list(app.list_concepts())

Input Download

#listing inputs
input_generator = input_obj.list_inputs(page_no=1,per_page=1,input_type='image')
inputs_list = list(input_generator)

#downloading_inputs
input_bytes = input_obj.download_inputs(inputs_list)
with open('demo.jpg','wb') as f:
  f.write(input_bytes[0])

:brain: Interacting with Models

The Model Class allows you to perform predictions using Clarifai models. You can specify which model to use by providing the model URL or ID. This gives you flexibility in choosing models. The App Class also allows listing of all available Clarifai models for discovery. For greater control over model predictions, you can pass in an output_config to modify the model output as demonstrated below.

Model Predict

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.model import Model

"""
Get Model information on details of model(description, usecases..etc) and info on training or
# other inference parameters(eg: temperature, top_k, max_tokens..etc for LLMs)
"""
gpt_4_model = Model("https://clarifai.com/openai/chat-completion/models/GPT-4")
print(gpt_4_model)


# Model Predict
model_prediction = Model("https://clarifai.com/anthropic/completion/models/claude-v2").predict_by_bytes(b"Write a tweet on future of AI")

# Customizing Model Inference Output
model_prediction = gpt_4_model.predict_by_bytes(b"Write a tweet on future of AI", inference_params=dict(temperature=str(0.7), max_tokens=30))
# Return predictions having prediction confidence > 0.98
model_prediction = model.predict_by_filepath(filepath="local_filepath", output_config={"min_value": 0.98}) # Supports image, text, audio, video

# Supports prediction by url
model_prediction = model.predict_by_url(url="url") # Supports image, text, audio, video

# Return predictions for specified interval of video
video_input_proto = [input_obj.get_input_from_url("Input_id", video_url=BEER_VIDEO_URL)]
model_prediction = model.predict(video_input_proto, output_config={"sample_ms": 2000})

Model Training

# Note: CLARIFAI_PAT must be set as env variable.
from clarifai.client.app import App
from clarifai.client.model import Model

"""
Create model with trainable model_type
"""
app = App(user_id="user_id", app_id="app_id")
model = app.create_model(model_id="model_id", model_type_id="visual-classifier")
               (or)
model = Model('url')

"""
List training templates for the model_type
"""
templates = model.list_training_templates()
print(templates)

"""
Get parameters for the model.
"""
params = model.get_params(template='classification_basemodel_v1', save_to='model_params.yaml')

"""
Update the model params yaml and pass it to model.train()
"""
model_version_id = model.train('model_params.yaml')

"""
Training status and saving logs
"""
status = model.training_status(version_id=model_version_id,training_logs=True)
print(status)

Export your trained model

Model Export feature enables you to package your trained model into a model.tar file. This file enables deploying your model within a Triton Inference Server deployment.

from clarifai.client.model import Model

model = Model('url')
model.export('output/folder/')

Evaluate your trained model

When your model is trained and ready, you can evaluate by the following code

from clarifai.client.model import Model

model = Model('url')
model.evaluate(dataset_id='your-dataset-id')

Compare the evaluation results of your models.

```python from clarifai.client.model import Model from clarifai.client.dataset import Dataset from clarifai.utils.evaluation import EvalResultCompare

models = ['model url1', 'model url2'] # or [Model(url1), Model(url2)] dataset = 'dataset url' # or Data