Overview:
OpenLLMetry is an open-source set of extensions built on top of OpenTelemetry, designed to provide complete observability over LLM applications. By leveraging OpenTelemetry under the hood, it allows developers to trace LLM calls, vector database interactions, and framework usage, then send that telemetry data to existing observability solutions like Datadog, Honeycomb, and others. It is built and maintained by Traceloop under the Apache 2.0 license.
Core Features:
OpenTelemetry-based instrumentation: Extends standard OpenTelemetry instrumentations to cover LLM providers (OpenAI, Anthropic, Mistral AI, etc.), Vector DBs (Chroma, Pinecone, Qdrant, etc.), and frameworks (LangChain, LlamaIndex, CrewAI, etc.).
Wide destination support: Telemetry data can be exported to over 20 observability backends, including Datadog, Grafana, New Relic, SigNoz, and Google Cloud.
Simple SDK setup: Developers can instrument their LLM application by adding a single line of code to their project.
Direct instrumentation usage: Users already instrumented with OpenTelemetry can add OpenLLMetry's instrumentations directly without the SDK.
No telemetry collection: Starting from version 0.49.2, the SDK and instrumentations do not log or collect any telemetry data.
Use Cases:
Developers tracing LLM calls: Instrument calls to OpenAI, Anthropic, or other LLM providers to monitor latency, token usage, and errors.
Observability engineers connecting to existing stacks: Export telemetry from LLM applications to existing observability platforms like Datadog or Grafana.
Teams using LLM frameworks: Gain visibility into LangChain, LlamaIndex, or CrewAI workflows by automatically tracing framework operations.
System administrators monitoring vector databases: Track interactions with Chroma, Pinecone, or Qdrant to debug retrieval performance.
Why It Matters:
OpenLLMetry makes LLM application observability accessible by building on the OpenTelemetry standard, avoiding vendor lock-in for telemetry data. Developers can use the simple SDK to get started quickly or integrate directly with existing OpenTelemetry setups. Its support for major LLM providers, frameworks, and vector databases covers the key components of modern LLM stacks, and the project does not log or collect telemetry data from users.
