A developer discovered a production API-contract bug in an AI tool server after a live demo script revealed that the system was rejecting default parameter values. The server, which wraps the 'gemini-3.1-flash-lite-image' model using the Model Context Protocol (MCP), had 10 out of 10 unit tests passing and was considered stable. However, the demo script attempted to generate an image using a 'minimal' thinking level, triggering an HTTP 400 error. The investigation revealed that the server's validation logic for 'thinking_level' was outdated, referencing four allowed levels ('minimal', 'low', 'medium', 'high') while the live Gemini API only supported 'low' and 'high'. Furthermore, the server's default parameter was set to 'medium', meaning every live call that relied on the default value failed.

The bug remained undetected by the test suite because the unit tests relied on mocks that simulated successful responses for any input, including unsupported values. These tests verified that the server could faithfully forward parameter values, but they failed to validate the contract with the actual remote API. Additionally, because previous users of the server consistently overrode the default thinking level in their requests, the faulty default value was never triggered during normal operation. This highlighted a significant drift between the local allowlist in the code and the current requirements enforced by the remote API.

To resolve the issue, the developer updated the supported levels to only include 'low' and 'high' and set the default to 'low'. A new regression test was added to explicitly reject 'medium' locally, providing a more readable error message before a request ever reaches the API. The entire server was rebuilt and pushed to Docker Hub after the fix was verified. The developer concluded that while unit tests are useful for checking local logic, they cannot substitute for end-to-end smoke tests that use real API credentials. Maintaining a simple demo script that executes a real-world happy path serves as an effective, low-cost method to identify contract drift that mock-based tests often miss.

A developer discovered a production API-contract bug in an AI tool server after a live demo script revealed that the system was rejecting default parameter values. The server, which wraps the 'gemini-3.1-flash-lite-image' model using the Model Context Protocol (MCP), had 10 out of 10 unit tests passing and was considered stable. However, the demo script attempted to generate an image using a 'minimal' thinking level, triggering an HTTP 400 error. The investigation revealed that the server's validation logic for 'thinking_level' was outdated, referencing four allowed levels ('minimal', 'low', 'medium', 'high') while the live Gemini API only supported 'low' and 'high'. Furthermore, the server's default parameter was set to 'medium', meaning every live call that relied on the default value failed.

The bug remained undetected by the test suite because the unit tests relied on mocks that simulated successful responses for any input, including unsupported values. These tests verified that the server could faithfully forward parameter values, but they failed to validate the contract with the actual remote API. Additionally, because previous users of the server consistently overrode the default thinking level in their requests, the faulty default value was never triggered during normal operation. This highlighted a significant drift between the local allowlist in the code and the current requirements enforced by the remote API.

To resolve the issue, the developer updated the supported levels to only include 'low' and 'high' and set the default to 'low'. A new regression test was added to explicitly reject 'medium' locally, providing a more readable error message before a request ever reaches the API. The entire server was rebuilt and pushed to Docker Hub after the fix was verified. The developer concluded that while unit tests are useful for checking local logic, they cannot substitute for end-to-end smoke tests that use real API credentials. Maintaining a simple demo script that executes a real-world happy path serves as an effective, low-cost method to identify contract drift that mock-based tests often miss.