Observability and DevTool platform for AI Agents

AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.

Key Integrations 🔌

<a href="https://docs.agentops.ai/v2/integrations/openai_agents_python"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/openai/agents-sdk.svg" height="45" alt="OpenAI Agents SDK"></a>
<a href="https://docs.agentops.ai/v1/integrations/crewai"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/v1/img/docs-icons/crew-banner.png" height="45" alt="CrewAI"></a>
<a href="https://docs.ag2.ai/docs/ecosystem/agentops"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/ag2/ag2-logo.svg" height="45" alt="AG2 (AutoGen)"></a>
<a href="https://docs.agentops.ai/v1/integrations/microsoft"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/microsoft/microsoft_logo.svg" height="45" alt="Microsoft"></a>







<a href="https://docs.agentops.ai/v1/integrations/langchain"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/langchain/langchain-logo.svg" height="45" alt="LangChain"></a>
<a href="https://docs.agentops.ai/v1/integrations/camel"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/camel/camel.png" height="45" alt="Camel AI"></a>
<a href="https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/ollama/ollama-icon.png" height="45" alt="LlamaIndex"></a>
<a href="https://docs.agentops.ai/v1/integrations/cohere"><img src="https://github.com/AgentOps-AI/agentops/raw/0.4.21/docs/images/external/cohere/cohere-logo.svg" height="45" alt="Cohere"></a>

📊 Replay Analytics and Debugging	Step-by-step agent execution graphs
💸 LLM Cost Management	Track spend with LLM foundation model providers
🤝 Framework Integrations	Native Integrations with CrewAI, AG2 (AutoGen), Agno, LangGraph, & more
⚒️ Self-Host	Want to run AgentOps on your own cloud? You're covered

Quick Start ⌨️

pip install agentops

Session replays in 2 lines of code

Initialize the AgentOps client and automatically get analytics on all your LLM calls.

Get an API key

import agentops

# Beginning of your program (i.e. main.py, __init__.py)
agentops.init( < INSERT YOUR API KEY HERE >)

...

# End of program
agentops.end_session('Success')

All your sessions can be viewed on the AgentOps dashboard

Self-Hosting

Looking to run the full AgentOps app (Dashboard + API backend) on your machine? Follow the setup guide in app/README.md:

• Run the App and Backend (Dashboard + API)

Agent Debugging

Session Replays

Summary Analytics

First class Developer Experience

Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.

Refer to our documentation

# Create a session span (root for all other spans)
from agentops.sdk.decorators import session

@session
def my_workflow():
    # Your session code here
    return result

# Create an agent span for tracking agent operations
from agentops.sdk.decorators import agent

@agent
class MyAgent:
    def __init__(self, name):
        self.name = name

    # Agent methods here

# Create operation/task spans for tracking specific operations
from agentops.sdk.decorators import operation, task

@operation  # or @task
def process_data(data):
    # Process the data
    return result

# Create workflow spans for tracking multi-operation workflows
from agentops.sdk.decorators import workflow

@workflow
def my_workflow(data):
    # Workflow implementation
    return result

# Nest decorators for proper span hierarchy
from agentops.sdk.decorators import session, agent, operation

@agent
class MyAgent:
    @operation
    def nested_operation(self, message):
        return f"Processed: {message}"

    @operation
    def main_operation(self):
        result = self.nested_operation("test message")
        return result

@session
def my_session():
    agent = MyAgent()
    return agent.main_operation()

All decorators support: - Input/Output Recording - Exception Handling - Async/await functions - Generator functions - Custom attributes and names

Integrations 🦾

OpenAI Agents SDK 🖇️

Build multi-agent systems with tools, handoffs, and guardrails. AgentOps natively integrates with the OpenAI Agents SDKs for both Python and TypeScript.

Python

pip install openai-agents

• Python integration guide
• OpenAI Agents Python documentation

TypeScript

npm install agentops @openai/agents

• TypeScript integration guide
• OpenAI Agents JS documentation

CrewAI 🛶

Build Crew agents with observability in just 2 lines of code. Simply set an AGENTOPS_API_KEY in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.

pip install 'crewai[agentops]'

• AgentOps integration example
• Official CrewAI documentation

AG2 🤖

With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an AGENTOPS_API_KEY in your environment and call agentops.init()

• AG2 Observability Example
• AG2 - AgentOps Documentation

Camel AI 🐪

Track and analyze CAMEL agents with full observability. Set an AGENTOPS_API_KEY in your environment and initialize AgentOps to get started.

• Camel AI - Advanced agent communication framework
• AgentOps integration example
• Official Camel AI documentation

Installation

pip install "camel-ai[all]==0.2.11"
pip install agentops

import os
import agentops
from camel.agents import ChatAgent
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType

# Initialize AgentOps
agentops.init(os.getenv("AGENTOPS_API_KEY"), tags=["CAMEL Example"])

# Import toolkits after AgentOps init for tracking
from camel.toolkits import SearchToolkit

# Set up the agent with search tools
sys_msg = BaseMessage.make_assistant_message(
    role_name='Tools calling operator',
    content='You are a helpful assistant'
)

# Configure tools and model
tools = [*SearchToolkit().get_tools()]
model = ModelFactory.create(
    model_platform=ModelPlatformType.OPENAI,
    model_type=ModelType.GPT_4O_MINI,
)

# Create and run the agent
camel_agent = ChatAgent(
    system_message=sys_msg,
    model=model,
    tools=tools,
)

response = camel_agent.step("What is AgentOps?")
print(response)

agentops.end_session("Success")

Check out our Camel integration guide for more examples including multi-agent scenarios.

Langchain 🦜🔗

AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:

Installation

pip install agentops[langchain]

To use the handler, import and set

import os
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent, AgentType
from agentops.integration.callbacks.langchain import LangchainCallbackHandler

AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']
handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])

llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
                 callbacks=[handler],
                 model='gpt-3.5-turbo')

agent = initialize_agent(tools,
                         llm,
                         agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
                         verbose=True,
                         callbacks=[handler], # You must pass in a callback handler to record your agent
                         handle_parsing_errors=True)

Check out the Langchain Examples Notebook for more details including Async handlers.

Cohere ⌨️

First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!

• AgentOps integration example
• Official Cohere documentation

Installation

pip install cohere

```python python import cohere import agentops

Beginning of program's code (i.e. main.py, init.py)

agentops.init() co = cohere.Client()

chat = co.chat( message="Is it pronounced ceaux-hear or co-hehray?" )

print(chat)

agentops.end_session('Success')

```python python
import cohere
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)

co = cohere.Client()

stream = co.chat_stream(
    message="Write me a haiku about the synergies between Cohere and AgentOps"
)

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')

agentops.end_session('Success')

Anthropic ﹨

Track agents built with the Anthropic Python SDK (>=0.32.0).

• AgentOps integration guide
• Official Anthropic documentation

Installation

pip install anthropic

``