Which failures hide behind a correct-looking answer?
Superficial checks reward fluency. The expensive failures are quieter: a policy document that was out of date, a tool call that errored and was summarized as success, a path that depended on a model release you no longer run. If you only store the final message, those failures are invisible until a customer reports them.
What the rubric will measure
Groundedness: did the answer use the sources that were retrieved. Instruction adherence: did the agent stay inside its constraints. Tool reliability: did granted calls succeed with valid arguments. Latency and cost: was the path acceptable, not only the prose. Each of those can fail while the sentence still sounds finished.
Method, and the current limitation
This page is a research scaffold. A later revision will name the dataset, the rubric, and the limitations next to any figure. Until that exists, do not cite a number from this URL, and do not treat marketing-interface sample metrics as peer-reviewed results.
What you can inspect now
Open an execution. Check the sources, the tool results, and the model step before you judge the paragraph. Replay the input if you need to know whether the path is stable. That inspection is the practice this research is built to evaluate.
Questions
What is AI agent reliability?
Reliability is whether the system repeats a good outcome: the right sources, acceptable latency and cost, successful tool calls, and an answer that still holds on the next run. A fluent sentence is not that property.
Why do agent answers look correct and still fail?
The failure is often upstream of the sentence. Retrieval used a stale or conflicting source, a tool returned something the model papered over, or the same input will not take the same path tomorrow.
Are there published Obliq reliability benchmarks on this page?
No. The page defines the question and the rubric we intend to use—groundedness, tool reliability, latency, and cost. It does not invent a score. Product UI samples stay illustrative until a measurement is published.