Independent developer Remdore launched Otterscope, an open-source observability tool designed for tracking LLM agent execution. The software addresses the need for monitoring tool calls, model responses, and cost metrics without relying on third-party hosting services. By using a single Go binary and a SQLite database, Otterscope avoids the infrastructure complexity of enterprise-scale monitoring tools that typically require multiple stateful services like Postgres or ClickHouse.

Otterscope functions by listening for OpenTelemetry traces from local agents. It provides a visual interface for viewing agent runs, including individual steps, LLM calls, and nested tool calls. The tool includes a cost estimation feature that identifies the current pricing for major model providers and displays the dollar amount associated with each run. Additionally, it integrates evaluation checks directly into the trace data, allowing users to run comparisons between different prompts or agent versions to assess performance over time.

A core component of the project is a translation layer that standardizes conflicting AI trace formats. Because various frameworks, such as the OpenAI Agents SDK, LangChain, and Vercel AI SDK, utilize differing attribute layouts for OpenTelemetry, Otterscope converts these disparate formats into a unified model. This approach allows users to store raw payloads and retroactively process them if translation logic improves. The software is currently designed for single-user, local environments, binding to localhost by default to ensure privacy and security without requiring complex authentication or team-based access controls.

For implementation, the developer provides a Docker container that exposes ports 8317 for the web interface and 4318 for OpenTelemetry data. The repository includes setup documentation for frameworks including Pydantic AI, LangGraph, and the Vercel AI SDK. To facilitate testing, the application includes a sample command that populates the database with simulated runs. The tool is optimized for low-resource environments and can be hosted on a standard small VPS, such as a $6 DigitalOcean Droplet, requiring no additional database management.

Independent developer Remdore launched Otterscope, an open-source observability tool designed for tracking LLM agent execution. The software addresses the need for monitoring tool calls, model responses, and cost metrics without relying on third-party hosting services. By using a single Go binary and a SQLite database, Otterscope avoids the infrastructure complexity of enterprise-scale monitoring tools that typically require multiple stateful services like Postgres or ClickHouse.

Otterscope functions by listening for OpenTelemetry traces from local agents. It provides a visual interface for viewing agent runs, including individual steps, LLM calls, and nested tool calls. The tool includes a cost estimation feature that identifies the current pricing for major model providers and displays the dollar amount associated with each run. Additionally, it integrates evaluation checks directly into the trace data, allowing users to run comparisons between different prompts or agent versions to assess performance over time.

A core component of the project is a translation layer that standardizes conflicting AI trace formats. Because various frameworks, such as the OpenAI Agents SDK, LangChain, and Vercel AI SDK, utilize differing attribute layouts for OpenTelemetry, Otterscope converts these disparate formats into a unified model. This approach allows users to store raw payloads and retroactively process them if translation logic improves. The software is currently designed for single-user, local environments, binding to localhost by default to ensure privacy and security without requiring complex authentication or team-based access controls.

For implementation, the developer provides a Docker container that exposes ports 8317 for the web interface and 4318 for OpenTelemetry data. The repository includes setup documentation for frameworks including Pydantic AI, LangGraph, and the Vercel AI SDK. To facilitate testing, the application includes a sample command that populates the database with simulated runs. The tool is optimized for low-resource environments and can be hosted on a standard small VPS, such as a $6 DigitalOcean Droplet, requiring no additional database management.