AQuA watches live agent failures without becoming another tool in the agent
Google Developers describes AQuA, an ambient quality agent positioned outside an application agent’s live request path. Instead of asking the production agent to grade itself, AQuA takes a bounded sample of recent execution trajectories from Cloud Trace, Cloud Logging, or BigQuery, checks them against a nine-point quality rubric, groups the suspicious behaviors into recurring patterns, and verifies representative transcripts before reporting actionable issues. A targeted diagnosis can then compare a confirmed pattern with the deployed source snapshot. The architecture is deliberately read-only: underlying customer traces and source code stay within the organization’s Google Cloud project, and the observer does not change the running agent. In its reported scripted test, 32 replayed sessions yielded 27 failures, 42 initial findings, nine candidate clusters, and six remaining issues after three were rejected on inspection. Those figures describe the authors’ bounded example, not a universal accuracy benchmark. AQuA is best understood as an external quality-investigation pattern that produces human-reviewable evidence, not a guarantee that silent failures are eliminated.
AQuA watches live agent failures without becoming another tool in the agent
Focus: Technical AI Engineering Date: October 8, 2026 Topics: agent observability, failure clustering, production diagnosis, trace evidence Evidence: Publisher Authored Availability: Available
Summary: Google Developers describes AQuA, an ambient quality agent positioned outside an application agent’s live request path. Instead of asking the production agent to grade itself, AQuA takes a bounded sample of recent execution trajectories from Cloud Trace, Cloud Logging, or BigQuery, checks them against a nine-point quality rubric, groups the suspicious behaviors into recurring patterns, and verifies representative transcripts before reporting actionable issues. A targeted diagnosis can then compare a confirmed pattern with the deployed source snapshot. The architecture is deliberately read-only: underlying customer traces and source code stay within the organization’s Google Cloud project, and the observer does not change the running agent. In its reported scripted test, 32 replayed sessions yielded 27 failures, 42 initial findings, nine candidate clusters, and six remaining issues after three were rejected on inspection. Those figures describe the authors’ bounded example, not a universal accuracy benchmark. AQuA is best understood as an external quality-investigation pattern that produces human-reviewable evidence, not a guarantee that silent failures are eliminated.
Why it matters: An agent can return an apparently helpful answer while losing a requirement, taking a wrong tool path, or repeating a costly mistake. Separating the diagnostic loop from production prevents the evaluator from silently changing the system it measures. For a pilot, choose a small representative trace sample, define quality criteria, require reviewed examples for each error class, and track whether a confirmed issue recurs after a release.
