The Allen Institute for AI (Ai2) has developed Shippy, an AI agent designed to assist maritime analysts in high-stakes operational decision-making. Built by the Skylight team, Shippy interprets live data streams to identify vessel behaviors, such as fishing activity or transshipments, across global maritime territories. To ensure operational reliability, the system is structured into three components: a soul, which sets behavioral boundaries via system prompts; skills, which are versioned markdown files defining specific tasks; and config, which manages runtime settings and model selection. Shippy currently utilizes Claude Opus 4.6 as its underlying language model and operates within the OpenClaw agent framework.

The system prioritizes deterministic interaction by funneling requests through a custom CLI. Instead of allowing the model to generate raw API calls, the CLI handles complex tasks like pagination and geometry filtering, outputting results to local JSON files to avoid shell-related errors. Security is managed via Mothership, an internal platform that provisions an isolated Kubernetes deployment for each user session. This architecture ensures that user-specific data, such as vessel watchlists and Area of Interest (AOI) configurations, remains private and siloed. Each session functions as an ephemeral sandbox, allowing the agent to execute multi-step analyses while restricted to authorized network services.

Evaluation of Shippy relies on a proprietary framework built on the Harbor open evaluation platform. Rather than using static benchmarks, the team assesses the agent by tasking subject-matter experts with writing scenarios that grade the entire system—model, skills, and sandbox—against real-time data. An LLM-based judge assigns scores based on weighted rubrics, providing written reasoning for each evaluation criterion to identify specific failure points. Current testing has highlighted challenges in patrol-planning, where the agent occasionally makes tactical recommendations, and geometry-sensitive queries. The team plans to introduce agent-driven UI control, model routing to optimize performance, and cross-thread memory capabilities to improve user experience. These architectural lessons are also being applied to Ai2's other environmental platforms, including EarthRanger.

The Allen Institute for AI (Ai2) has developed Shippy, an AI agent designed to assist maritime analysts in high-stakes operational decision-making. Built by the Skylight team, Shippy interprets live data streams to identify vessel behaviors, such as fishing activity or transshipments, across global maritime territories. To ensure operational reliability, the system is structured into three components: a soul, which sets behavioral boundaries via system prompts; skills, which are versioned markdown files defining specific tasks; and config, which manages runtime settings and model selection. Shippy currently utilizes Claude Opus 4.6 as its underlying language model and operates within the OpenClaw agent framework.

The system prioritizes deterministic interaction by funneling requests through a custom CLI. Instead of allowing the model to generate raw API calls, the CLI handles complex tasks like pagination and geometry filtering, outputting results to local JSON files to avoid shell-related errors. Security is managed via Mothership, an internal platform that provisions an isolated Kubernetes deployment for each user session. This architecture ensures that user-specific data, such as vessel watchlists and Area of Interest (AOI) configurations, remains private and siloed. Each session functions as an ephemeral sandbox, allowing the agent to execute multi-step analyses while restricted to authorized network services.

Evaluation of Shippy relies on a proprietary framework built on the Harbor open evaluation platform. Rather than using static benchmarks, the team assesses the agent by tasking subject-matter experts with writing scenarios that grade the entire system—model, skills, and sandbox—against real-time data. An LLM-based judge assigns scores based on weighted rubrics, providing written reasoning for each evaluation criterion to identify specific failure points. Current testing has highlighted challenges in patrol-planning, where the agent occasionally makes tactical recommendations, and geometry-sensitive queries. The team plans to introduce agent-driven UI control, model routing to optimize performance, and cross-thread memory capabilities to improve user experience. These architectural lessons are also being applied to Ai2's other environmental platforms, including EarthRanger.