Suppose an LLM produces a factual answer with five citations. You want to know if the answer is reliable so you ask another LLM to check it.
What has changed?
You may now have a second opinion but you do not necessarily have verification. The second model can misread the source, accept a fabricated citation, infer support that the document does not provide or repeat the first model’s assumptions. If the models share similar training data and failure patterns, apparent agreement may be correlated error rather than independent confirmation.
This does not mean LLM judges are useless as they can compare text, identify apparent contradictions, classify relationships between claims and passages and process volumes of material that would overwhelm a human reviewer. But we often give generative models responsibility for checks that ordinary software can perform more consistently and explain more precisely.
A DOI either resolves or it does not, a quoted passage either appears in the cited document or it does not, a publication date either falls within a defined period or it does not and a document hash either matches the registered file or it does not. These are not questions of judgment because asking an LLM to answer them adds uncertainty where none is needed.
My argument is simple, use deterministic verification for deterministic questions. Use model judgment only where judgment is required.
Deterministic verification applies stated rules to the same inputs and produces the same output. The rules may still be incomplete or badly designed but the system does not improvise them during each run.
For an AI generated claim and its citations, a deterministic layer could check
These checks do not establish that the claim is true instead they establish narrower facts about the evidence chain. That distinction is the point. Verification becomes easier to inspect when it is separated into claims that software can settle and claims that still require interpretation.
I explored this approach through Ancestor, a deterministic trust verification framework (see the report here). The research question was can a rule based trust scoring system influence citation behaviour in a multi-agent setting without relying on another generative model?
The simulation used three roles
The simplified verifier started each citation at 100 and applied penalties based on source age, domain characteristics, known problematic domains and content indicators. Each decision produced an audit record showing the input, penalties, final score, threshold and decision.
I ran two 100 episode simulations. The first used 100 manually checked citations from government portals, research repositories, academic institutions, international organisations, standards bodies, scientific journals and other established sources. All sampled citations were accepted, with an average trust score of 96.59.
The second simulation used a mixed dataset containing 50 verified citations and 50 fabricated or conspiratorial entries. Because citations were sampled randomly, the run selected approximately 48 verified and 52 fabricated entries. The system rejected 19 citations. Every rejected citation was fabricated, giving rejection precision of 1.00. It rejected none of the verified citations.
That sounds impressive until we examine recall. The system rejected only 19 of the 52 fabricated citations. Thirty three fabricated entries passed the threshold. Recall for fabricated source detection was therefore 0.37 with an F1 score of 0.54.
This is the most informative result in the study.
The deterministic layer was good at saying "I have explicit grounds to reject this." It was much worse at saying "Everything I did not reject is reliable." The main rejection triggers were obvious domain and conspiracy indicators. Fabricated health claims without those indicators often passed.
The experiment was also deliberately artificial. The fabricated sources were manually constructed, their age distribution differed from that of verified sources and the keyword rules targeted conspicuous forms of misinformation. It was a proof of mechanism not a production benchmark. Larger datasets, adversarial examples, consistent age distributions, domain specific calibration and comparisons with model based and hybrid evaluators are still needed.
The system’s low recall is not an argument for abandoning deterministic verification. It shows why its role must be defined carefully.
When a generative verifier makes a bad decision, explaining the cause can be difficult. Was the evidence missing from context? Did the model misunderstand the claim? Did a prompt change shift the result? Did the model rely on prior knowledge? Would the same input receive the same judgment tomorrow?
With a deterministic verifier, the failure can be located. In my simulation, accepted fabricated sources usually passed because domain and age penalties did not cross the rejection threshold and no content rule fired. The audit trail made the gap visible cos the rules had high precision and low coverage.
That gives a researcher or engineer something specific to improve. Add a registry check, verify the citation identifier, compare quoted text against the document, route unsupported health claims to a domain evaluator, change the threshold and measure the resulting false positive and false negative rates. The system may remain incomplete but its incompleteness can be measured.
The choice is not rules versus LLMs. Different questions call for different methods.
| Layer | Question | Suitable method |
|---|---|---|
| Citation integrity | Does the source exist and does its metadata match? | Deterministic lookup and identifier validation |
| Provenance | Where did the document come from and has it changed? | Registries, signatures, hashes and audit logs |
| Source policy | Does the source meet stated recency, retraction or domain requirements? | Configurable rules |
| Evidence support | Does the cited passage support the claim? | Retrieval, NLI, structured model evaluation and calibrated scoring |
| Interpretation | How should ambiguous or conflicting evidence be understood? | Domain experts, possibly assisted by LLMs |
| Consequential decision | Is the claim acceptable for use in medicine, law, policy or finance? | Risk based human review with the full audit record |
This layered design changes what an LLM judge is asked to do. Instead of giving it an open request to verify this answer, the system can first remove broken citations, retrieve the cited passages, expose metadata conflicts and identify retracted or stale sources. The model then receives a narrower question with better evidence: does this passage support this specific claim?
Even that judgment should not automatically become ground truth. It can be tested against labelled examples, compared across models, calibrated against human reviewers and routed to manual review when confidence is low or disagreement is high.
AI systems are often rewarded for producing an answer. Verification systems should sometimes be rewarded for refusing to approve one.
A fail closed verifier does not convert missing evidence into acceptance. If a citation cannot be retrieved, metadata conflicts or support cannot be established, the system returns unverified rather than guessing. This is less satisfying than a fluent explanation but in consequential settings it is often the more honest output.
This also requires separating three labels that are commonly collapsed
- Verified: the required checks passed.
- Rejected: one or more stated checks failed.
- Unresolved: the system lacks enough information or authority to decide.
A binary true or false label encourages false confidence. An unresolved category makes the boundary of the system visible and creates a queue for targeted human review.
Much of the discussion about AI epistemics focuses on better reasoning, forecasting, debate, model critics and scalable oversight and those questions matter. But epistemic systems also depend on infrastructure that may appear intellectually modest with source registries, version histories, retrieval logs, citation parsers, threshold policies, labelled evaluation sets,and interfaces for disputing automated decisions.
This work is easy to dismiss as ordinary engineering. That may be why it deserves more attention.
As AI generated research expands, the bottleneck may not only be producing better arguments. It may be maintaining a traceable chain from each material claim to the evidence offered for it, recording what was checked and showing which parts still depend on model or human judgment.
Deterministic systems will not settle contested facts. They can however, reduce the number of avoidable failures passed to probabilistic systems. They can also make later judgments easier to audit.
The next question is not Can rules verify everything? They cannot.
The more useful questions are
A useful benchmark could compare three systems on the same claim-evidence dataset with an LLM judge, a deterministic verifier and a layered system combining the two. Evaluation should measure precision, recall, calibration, stability across repeated runs, cost, auditability and performance under adversarial citation construction.
My expectation is not that the deterministic system would win overall. I expect it would perform well on narrow integrity checks, poorly on semantic ambiguity and improve the hybrid system by removing avoidable uncertainty before model judgment begins. That expectation should be tested rather than assumed.
LLMs can assist verification but they should not be the entire verification architecture.
When a question has an exact answer, we should use an exact check. When a question requires interpretation, we should use probabilistic judgment with measured performance. When the consequences are serious or the evidence remains disputed, a human should make the final decision with access to the complete record.
The aim is not to replace model evaluators with brittle lists of rules but to stop asking probabilistic systems to perform every epistemic function simply because they can produce an answer.
Before asking an LLM to verify another LLM, we should first ask which parts of this claim can be checked without an LLM at all?
---
The underlying technical report is available here. It is a preliminary study based on controlled simulations and has not been peer reviewed.
AI use note: I developed the argument and underlying research. I used an LLM to help structure and edit this Forum draft, then reviewed the claims and source material myself.
