Rodrigo Diego reported on July 31, 2026, that a document-search copilot for a quality-management product gave a wrong answer when a user asked how many documents a specific person authored. The database’s real count was 84, but the RAG copilot answered from the documents it could see after retrieval, filtering and display processing, not from the system of record.

The copilot was a multi-agent system built with LangGraph and Bedrock. For a metadata question such as “documents authored by X,” a search agent queried a documents service, reranked results, applied view-permission gates, collapsed multiple document versions into one card and surfaced at most 30 results. Those steps were appropriate for search, but they changed the meaning of “how many”: SQL had the true total of 84, retrieval returned capped top-k rows, permission checks reduced visible rows, version collapse turned 5 versions of one document into 1 card, and the final surfaced set was always at most 30.

The failure was harder to detect than a crash because the answer looked plausible. A capped and deduplicated count such as 27 documents could appear credible in a quality-management workflow, where someone might place the number in a report. Diego argued that an LLM should not compute aggregates such as counts, sums or “most recent” values; a system of record should compute them, and the model should only phrase the result.

The article said the documents service already returned the true total with each filtered query, and the search node stored it as `total_available_results`. A downstream permission-gate node then reused the same field for a different meaning: the number of view-gated, version-collapsed cards shown to the user. Its comment said the UI count should match displayed cards, such as 5 versions becoming 4 documents, but the shared state field meant “matches in the database” upstream and “cards on screen” downstream.

Diego’s fix added a separate authoritative field, `total_matching_records`, for the documents service’s true match total before the 30-card cap and version collapse. The existing `total_available_results` field remained tied to surfaced cards and pagination. The outcome node then used `total_matching_records` to produce a user-facing headline such as “Showing 5 of 84 matching documents” and a tool message saying 84 documents matched in total.

The agent prompt also instructed the model to answer “how many” questions with the authoritative total reported by the tool, never the number of displayed cards, which is capped. Diego described the structure as placing the true number in both the agent context and the UI headline, so even if the model ignored the instruction, the interface still showed the database count beside the answer.

Three tests checked the change: a count note appears when more documents match than are shown, disappears when all documents are shown and never appears for content-only semantic search. The semantic-search case lacks an exact match set because similarity ranking (ordering by closeness of meaning) does not produce a crisp total, so presenting an exact number would be misleading.

The same release fixed a related date bug for questions such as “documents created in the last 10 days.” The model needed the current date to convert relative dates into filters, so Diego added LangGraph middleware that injects today’s UTC date into every model call and tells the model to resolve “today,” “yesterday,” “the last 10 days” and “since March” into concrete ISO-8601 dates before building a filter.

The middleware does not append the date to persisted conversation history, because a saved line such as “today is 2026-07-30” would become stale if the conversation resumed next week. Diego listed remaining limits: nothing enforces that the model’s prose uses the authoritative count, content-only searches still fall back to surfaced counts, and sums, averages or “which author has the most” would need real aggregation endpoints on the service side.

Rodrigo Diego reported on July 31, 2026, that a document-search copilot for a quality-management product gave a wrong answer when a user asked how many documents a specific person authored. The database’s real count was 84, but the RAG copilot answered from the documents it could see after retrieval, filtering and display processing, not from the system of record.

The copilot was a multi-agent system built with LangGraph and Bedrock. For a metadata question such as “documents authored by X,” a search agent queried a documents service, reranked results, applied view-permission gates, collapsed multiple document versions into one card and surfaced at most 30 results. Those steps were appropriate for search, but they changed the meaning of “how many”: SQL had the true total of 84, retrieval returned capped top-k rows, permission checks reduced visible rows, version collapse turned 5 versions of one document into 1 card, and the final surfaced set was always at most 30.

The failure was harder to detect than a crash because the answer looked plausible. A capped and deduplicated count such as 27 documents could appear credible in a quality-management workflow, where someone might place the number in a report. Diego argued that an LLM should not compute aggregates such as counts, sums or “most recent” values; a system of record should compute them, and the model should only phrase the result.

The article said the documents service already returned the true total with each filtered query, and the search node stored it as `total_available_results`. A downstream permission-gate node then reused the same field for a different meaning: the number of view-gated, version-collapsed cards shown to the user. Its comment said the UI count should match displayed cards, such as 5 versions becoming 4 documents, but the shared state field meant “matches in the database” upstream and “cards on screen” downstream.

Diego’s fix added a separate authoritative field, `total_matching_records`, for the documents service’s true match total before the 30-card cap and version collapse. The existing `total_available_results` field remained tied to surfaced cards and pagination. The outcome node then used `total_matching_records` to produce a user-facing headline such as “Showing 5 of 84 matching documents” and a tool message saying 84 documents matched in total.

The agent prompt also instructed the model to answer “how many” questions with the authoritative total reported by the tool, never the number of displayed cards, which is capped. Diego described the structure as placing the true number in both the agent context and the UI headline, so even if the model ignored the instruction, the interface still showed the database count beside the answer.

Three tests checked the change: a count note appears when more documents match than are shown, disappears when all documents are shown and never appears for content-only semantic search. The semantic-search case lacks an exact match set because similarity ranking (ordering by closeness of meaning) does not produce a crisp total, so presenting an exact number would be misleading.

The same release fixed a related date bug for questions such as “documents created in the last 10 days.” The model needed the current date to convert relative dates into filters, so Diego added LangGraph middleware that injects today’s UTC date into every model call and tells the model to resolve “today,” “yesterday,” “the last 10 days” and “since March” into concrete ISO-8601 dates before building a filter.

The middleware does not append the date to persisted conversation history, because a saved line such as “today is 2026-07-30” would become stale if the conversation resumed next week. Diego listed remaining limits: nothing enforces that the model’s prose uses the authoritative count, content-only searches still fall back to surfaced counts, and sums, averages or “which author has the most” would need real aggregation endpoints on the service side.