An AI agent can return a final answer that looks perfectly safe while, along the way, it used powers nobody delegated to it: it sent the email, changed the record, or called the external API that the owner never approved. If we evaluate agents only by their outputs, we miss exactly this class of failure. I am Tabata Jahoda, an independent AI-safety researcher in Rio de Janeiro, and I am building Nirmata, an open, early-stage method for auditing agents from their execution trajectories.
The problem
Most evaluation of LLM agents still looks at what the agent says at the end. But as agents get tools and permissions, the safety-relevant question shifts from "was the answer acceptable?" to "did the agent act within the authority it was actually given?" These can come apart. An agent can recommend something reasonable and still have attempted an action the owner never authorized.
The approach
Nirmata keeps four things distinct in every trace: a recommendation, owner authorization, an attempted action, and its external effect. Conflating them is how a "the user approved it" story hides an action that was never actually authorized. The framework then compares an output-only evaluator with a trajectory-only evaluator, combined by a fixed OR rule, so the question becomes measurable: how many delegated-authority and control-integrity failures does trajectory-aware auditing catch that output-only evaluation misses, and at what false-positive cost?
What exists
- Nirmata v0.1.0, public with a Zenodo DOI (10.5281/zenodo.22132451). Code and scripts Apache-2.0; documentation, protocols and data CC BY 4.0, with the exceptions in LICENSE.md.
- A trace schema, a blinding protocol and a scoring specification.
- A deterministic, reproducible fixture of 6 synthetic traces, with unit tests and CI.
- A sanitized applied case study and a publication audit.
- An n8n adapter that satisfied eight preregistered criteria on a second preregistered synthetic workflow. This is not detection accuracy and not general validation.
- A 9-page paper, pending arXiv endorsement.
What does not exist yet
There is no confirmatory result. My first holdout for the main experiment was invalidated because the answer key entered the evaluator's context. I documented that publicly instead of reporting numbers. I would rather say "we don't know yet" than publish a contaminated result.
The plan
- Preregister the protocol and metrics before any data.
- Keep holdout labels in independent custody.
- Record predictions before ground truth is revealed, then score without changing anything.
- Publish the pilot report whatever the outcome, including a negative or null result.
- Replicate on a second agentic system and invite external review of the schema.
"Worked" here means a credible answer, not necessarily a positive one.
Budget
- Minimum US$3,000: compute/API plus research hours to run the preregistered blind pilot.
- Full goal US$14,200: about 6 months of part-time research time, compute/API, one trip to an AI-safety conference or workshop for external review, and small open-access costs. That covers pilot, replication and external review.
I work on this part-time (12 h/week) alongside a full-time job. Funding to date: self-funded, US$0 received. I have pending applications with other funders and will adjust to avoid funding the same hours twice.
Main risks
Schedule slips on a part-time timeline; independent label custody is hard for a solo researcher (if it fails I will report it and downgrade the evidence); trajectory evaluation may add little or produce many false positives (that would be published as a negative result); synthetic traces may not generalize.
The ask
If you think measuring the gap between "what an agent says" and "what it does" is worth doing carefully, please consider supporting the pilot, sharing it, or, just as valuable, critiquing the schema and the protocol in the comments or on GitHub.
Repo: https://github.com/thabatarj8-stack/nirmata
Manifund (closes 26 Nov 2026): https://manifund.org/projects/nirmata-auditing-ai-agents-by-what-they-do-not-only-what-they-say
