opik

Opik is an open-source LLM observability platform (Apache-2.0) built by Comet ML, designed to tackle the black-box challenges developers face when building and maintaining RAG systems and agent workflows—traditionally requiring manual log and trace inspection without systematic failure analysis.

Through automatic structured span trees, Opik connects each LLM call, tool invocation, and retrieval step, combining them with automated evaluations and production dashboards to enable real-time monitoring of latency, accuracy, and behavioral anomalies. Teams can also use human feedback annotations to continuously calibrate model performance and catch regressions during development via reproducible evaluation pipelines. It integrates with popular frameworks like LangChain and LlamaIndex, lowering the integration barrier.

As an open-source tool, Opik offers full code transparency and no licensing fees, providing flexibility without vendor lock-in; however, self-hosting means teams must invest effort in managing infrastructure (compute, storage, dashboards) and ensuring production reliability.