Watching the Wrong Thing: an interpretability audit of two audio deepfake detectors

Project report. Full code, artifacts, and the append-only experiment log are in the project repository.

Can you tell, by looking inside a model, when it is being fooled?

I started this project expecting the answer to be yes. The plan was to build an internal-representation monitor for adversarial audio and show that internals were more trustworthy than the model's own confidence.

That headline did not survive contact with the experiments.

The project ended up testing nine hypotheses across two related audio anti-spoofing models, two corpora, clean and adversarial conditions, and several kinds of interpretability tools. The result I care about most is not that one particular monitor failed. It is that only three of nine interpretability findings transferred cleanly between the two models. Three inverted, one held only conditionally, and two stopped being meaningful on the second model.

That is a demonstration, not a population estimate. Two models are still two models. But the pair is unusually favorable for transfer: they share the same graph-attention back-end implementation and labelled training corpus while using very different front-ends. If an interpretability result can fail there, I think that should lower our confidence in single-model mechanistic claims more generally.

A few other results survived the audit:

• On natural in-domain errors, softmax confidence beats every internal monitor I built.
• Under domain shift, confidence can look fine in aggregate while becoming anti-predictive within the class where the errors actually occur.
• Internal-monitor results depend sharply on what negatives and what operating point you report. On AASIST, the flattering clean-negative AUROC of 0.987 falls to 0.851 against perturbed benign audio; at a fixed 5% false-alarm budget, the best tap catches about 61% of attacks. SSL-AASIST ranks better under the deployment-relevant comparison.
• A 98.8%-accurate linear probe can point in a direction that is causally inert even when a simple class-mean direction flips the model.
• Cross-corpus failure can come from a misplaced threshold, stale readout weights, or genuine representation loss. Looking inside separated those cases and changed the repair bill from one number, to 161 parameters, to retraining.
• A monitor can be worth keeping even when an adaptive attacker drives its detection AUROC below chance, if evading it makes the attack much harder.

The project also changed its own conclusions several times. I kept an append-only experiment log, so those reversals are still visible rather than cleaned away. In retrospect, that was probably the most useful methodological decision in the whole project.

The setup

Audio anti-spoofing models are binary classifiers. They take a few seconds of waveform and decide whether it is:

• bonafide — real human speech, or
• spoof — synthetic or manipulated speech.

These models rarely sit alone. A deployment usually wraps them in a policy:

• confidently real → accept,
• confidently fake → reject,
• uncertain → escalate.

That third branch is often treated as the safety valve. The whole policy assumes the model's own confidence or some internal signal is trustworthy precisely when the model is under stress.

I wanted to test that assumption directly.

The two models

The models are AASIST and SSL-AASIST.

Both take roughly four seconds of audio and eventually feed a 160-dimensional embedding into a Linear(160,2) readout. Both use the same graph-attention back-end design, with spectral and temporal branches that exchange information before being pooled into the final embedding.

The major difference is the front-end:

AASIST reports 0.83% EER on ASVspoof2019 LA. SSL-AASIST replaces the tiny fixed front-end with a large pretrained speech encoder, while leaving the graph-attention back-end implementation and final readout structure closely matched.

This is a strong portability test, but not a perfect one-variable experiment: SSL-AASIST also uses RawBoost augmentation. I therefore do not treat every difference downstream as causally attributable to the front-end alone. The narrower claim is that the back-end is held unusually stable while the representation entering it changes dramatically.

That representational change matters for interpretability itself. In AASIST, the spectral nodes correspond to actual frequency bands. In SSL-AASIST, the corresponding dimensions are abstract projections. So a question like which frequencies carry spoof evidence? is meaningful in one model and literally has no referent in the other.

Why this project exists

This started as a BlueDot Impact Technical AI Safety project sprint, following their Technical AI Safety course.

I chose audio deepfake detection because it sits in an unusually useful middle ground for interpretability work: the models are small enough to inspect end-to-end, open enough to patch and attack directly, and tied to a real deployment problem rather than a toy benchmark. AASIST has only about 298K parameters, so I could run probes, causal interventions, adversarial attacks, and whole-model audits on the actual checkpoint rather than on a simplified proxy.

The original plan was narrower: build an internal-representation monitor that detects adversarial audio. That plan did not turn into a defence. After correcting how I evaluated the monitor, the best AASIST tap still catches only about 61% of attacks at a 5% false-alarm budget, and that is against processed studio audio rather than real recordings. The two models also behave differently enough that one model's monitoring conclusion does not predict the other's.

At that point the project changed from building a defence to auditing the assumptions behind the defence: which signals are trustworthy, which interpretability tools actually track mechanism, which failures are cheap to repair, and which conclusions survive a model change.

That reframing is why the post is organized around things I initially watched in the wrong way — confidence, internal monitors, probe directions, and finally the assumption that a mechanistic finding from one model will transfer to another.

Four conditions, not one

Most numbers below live in one of four cells:

This distinction matters because several of the strongest findings have no adversary involved at all. All three of the transfer inversions I discuss later are clean-data results.

For the adversarial experiments, every headline attack result is paired with an energy-matched benign control. On AASIST, white-box PGD successfully flips the model’s prediction on 91.9% of the clips it attacks at the ~40 dB perturbation setting. As a control, I add random Gaussian noise with exactly the same perturbation energy; that changes the model’s prediction on only 0.09% of clips — about 1 in 1,100. So the result is not simply that AASIST is sensitive to perturbations of this size. The perturbation becomes effective when it is optimized in an adversarial direction.

1. The wrong signal: confidence

My original hypothesis was that the model's own confidence would be uninformative about its errors, and that internal representations would do better.

I was wrong.

The first reason is almost embarrassingly simple: attack success is defined by pushing the model confidently across the decision boundary. If you evaluate confidence only on successful attacks, you have built the answer into the subset you chose.

Once I split the analysis into natural errors, adversarial errors, and pooled errors, the result reversed. On AASIST's natural in-domain errors, confidence predicts failure at AUROC 0.954, while the best internal tap reaches only 0.659.

So the first lesson was not “confidence is bad.” It was:

Do not evaluate a metric on a subset selected using that same metric.

In my original analysis, I evaluated confidence only on successful attacks. But an attack counts as successful only after it has pushed the model across its decision boundary — exactly the behavior confidence is supposed to summarize. That makes the evaluation circular: I first use the model's score to help define which examples enter the subset, then ask whether that same score distinguishes the examples in it.

Once I separated natural errors from adversarially selected errors, the conclusion reversed: on natural in-domain errors, confidence actually outperformed every internal monitor I built.

Domain shift is worse because it can look normal

The more worrying failure needs no attack.

Train on ASVspoof2019, evaluate on ASVspoof5, and the aggregate error-prediction AUROC for AASIST confidence is 0.656. Mediocre, but not obviously disastrous.

The problem is that the errors are almost all one class.

For AASIST, there are 702 false rejects of bonafide speech against 64 false accepts. On SSL-AASIST, the clean cross-corpus errors are even more one-sided: 576 of 576 are bonafide clips being rejected.

If errors concentrate in one class, an “error prediction” AUROC can get credit for doing the original classification task. So I recomputed the metric within the class where the errors occur.

Here, “within class” means: among bonafide clips only, can confidence distinguish the false rejects from the correctly accepted bonafide clips?

The in-domain row is an important control: confidence genuinely works there. The collapse is specific to domain shift.

For AASIST, 0.339 is below chance. The bonafide clips that the model wrongly rejects are, on average, held at higher confidence than the bonafide clips it accepts correctly.

Measured directly:

• mean confidence on falsely rejected bonafide clips: 0.929
• 78.5% have confidence above 0.9
• median p(bonafide) is 0.013

So AASIST does not merely reject about 70% of bonafide speech out of domain. It rejects it confidently.

A policy that escalates only low-confidence inputs would escalate almost none of these failures.

The mechanism is simple: domain shift can move an entire class across the decision threshold. Once the errors are confidently wrong rather than borderline, confidence cannot help.

2. The wrong place: internal monitors

If the output cannot be trusted, the next move is obvious: monitor the internals.

I fitted class-conditional Mahalanobis monitors at multiple internal “taps” — points where I read intermediate activations. AASIST has 12 taps in this analysis; SSL-AASIST has 36.

The central issue is that monitor quality depends on which negative distribution you compare against, and false-alarm rate at one inherited threshold is not the same thing as separability.

What “benign” means here

A useful attack monitor should not merely distinguish modified audio from untouched audio. It should distinguish an adversarial modification from an innocuous modification.

So the relevant benign set is not pristine speech. It contains 13,685 legitimate clips that have all been perturbed or re-encoded:

• roughly 87% have Gaussian noise matched to the attacked clip's perturbation energy;
• roughly 7% are MP3-compressed;
• roughly 7% are resampled.

Scoring only against untouched clips would make the monitors look better while answering an easier question.

And even these controls are friendlier than deployment: they are software transforms of studio-clean ASVspoof audio, not phone calls, microphones, rooms, packet loss, or background environments. I return to that limitation below.

Three numbers, three different questions

Here is the same AASIST monitor reported three ways:

These columns answer different questions.

AUROC vs clean asks whether the monitor can separate adversarial audio from untouched audio. It is the easiest comparison, and the one that makes embedding look nearly perfect at 0.987.

AUROC vs perturbed benign asks the harder question: can the monitor distinguish adversarial perturbations from innocuous perturbations? The numbers fall, but embedding still reaches 0.851. That means the representation contains real attack-specific signal; it is not merely responding to the presence of any perturbation.

FAR at a clean-fitted threshold asks yet another question: what happens if I choose a threshold on pristine audio and then apply it to processed audio? For embedding, 73% of processed benign clips cross that threshold.

That 73% is real, but it is mostly a calibration result, not evidence that the representation contains no useful attack signal. The threshold was placed on one distribution and used on another.

This distinction matters because a single false-alarm number can make a capable-but-miscalibrated monitor look incapable.

What operating point does AASIST actually achieve?

A cleaner comparison is to fix the false-alarm budget and ask how many attacks the monitor catches.

So the AASIST conclusion is:

At a 5% false-alarm budget, the best monitor catches about 61% of attacks.

The 73% false-alarm figure and the 0.851 perturbed-negative AUROC are not contradictory: the first measures calibration at one particular threshold, while the second measures separability across all thresholds.

But it is still not a convincing defence. Missing roughly two attacks in five on the model and attack family used to build the experiment is a long way from where I would want a deployment monitor to be — especially because the negatives here are still cleaner than real deployment audio.

The control choice is not what creates the high FAR

One concern is that the benign pool is dominated by Gaussian noise, which is a deliberately synthetic control.

I split the false alarms by processing type at the same clean-fitted threshold. For the AASIST embedding, the rates are:

• Gaussian: 72.7%
• MP3: 75.5%
• resampling: 75.8%

The realistic software transforms are slightly worse, not better.

At a fixed 5% FAR the pattern also survives: embedding catches 60.8% of attacks against MP3 negatives versus 54.8% against matched Gaussian noise. So the Gaussian control was not chosen to manufacture the failure; if anything, it is the harder negative in Mahalanobis space.

There is one useful oddity: the fixed SincConv front-end behaves differently, firing on 90.3% of MP3 but only 3.0% of Gaussian at the inherited threshold. That is a reminder that “benign perturbation” is not one distribution, and different layers can be sensitive to very different processing artifacts.

The two models rank oppositely depending on the metric

The two-model comparison becomes especially informative when the metrics are separated.

If I report only the conventional clean-negative AUROC, AASIST wins.

If I ask the more deployment-relevant question — adversarial versus already-processed benign audio — SSL-AASIST wins.

Reporting the easy metric alone ranks the models backwards.

SSL's perturbed-negative separability is also unusually stable: 0.920 in-domain and 0.919 cross-corpus, even though its clean-negative AUROC drops under corpus shift.

So the updated lesson is not “AASIST has no monitor and SSL does.” It is:

Separability, calibration, and operating point are three different things. Report which one you measured.

Where SSL-AASIST's monitorable signal lives

There is still a striking architectural contrast inside SSL-AASIST:

All 24 XLS-R transformer taps false-alarm on 100% of the perturbed-benign pool at this threshold. For those taps, “detects perturbation rather than attack” really is the right description.

The useful monitoring signal appears later, in the roughly 447K-parameter graph back-end — around 0.1% of SSL-AASIST's parameters.

The 300M front-end buys a lot elsewhere:

• in-domain EER improves roughly 1.50% → 0.20%;
• cross-corpus EER improves 29.6% → 4.20%;
• in-domain random-noise flip rate drops 9.7% → 0.0%.

But it does not itself provide the useful monitoring site.

And the compute increase is smaller than the parameter count suggests: in this setup, 1,067× more parameters costs about 1.6× the forward pass.

What I have not tested is whether a much smaller learned front-end would get the same benefits. The experiment compares essentially 0 learned front-end parameters against 300M, with nothing in between.

3. Three interpretability instruments that answer three different questions

Before the next result, it helps to distinguish three tools that are easy to conflate:

These form a hierarchy.

A probe can say the information is present nearly everywhere. Cross-class patching can show only a few sites actually affect the verdict. Direction patching can then show that even at a causally important site, the probe's own preferred axis is not the axis the model responds to.

The controls also differ:

Every one of these controls changed at least one conclusion in the project.

4. A 98.8%-accurate probe can point in a causally useless direction

This result is on clean, in-domain audio. No attack is involved.

At AASIST's gat_S tap, define a simple class-mean direction:

d = μ_spoof − μ_bonafide

No training. Just average the activations for spoof clips, average the activations for bonafide clips, and subtract.

Now intervene on the model.

Sufficiency

Take a bonafide clip and add d.

p(bonafide): 0.997 → 0.015

The model flips to “spoof.”

Necessity

Take a spoof clip and subtract d.

p(bonafide): 0.107 → 0.885

The model flips to “bonafide.”

A same-norm random direction barely moves either result.

So d is not merely correlated with the class. It is causally effective in both directions.

Now train a logistic regression probe on the same activations and take its weight vector w.

The probe is excellent:

• 98.8% held-out accuracy
• 0.9993 AUROC

By the usual standards, this is a very strong probe.

Because logistic regression scores σ(w·z + b), w is the direction in activation space along which the probe's confidence rises fastest. If probe weights describe the model's own mechanism, pushing the model along w should matter.

It does not.

In this cell, the probe direction is causally inert.

The probe was not wrong about prediction. Prediction was the part that worked.

It was wrong as a description of the model's own computation.

The cheap geometric check also fails

A natural response is: perhaps w and d simply point in different directions, so compute their cosine.

For AASIST gat_S:

cos(w, d) = 0.391

Against a random-direction null of roughly 0.090 ± 0.069, this is z ≈ 4.4 and even exceeds the maximum of 200 random draws.

So the cosine test would tell you the two directions genuinely align.

The intervention still says the probe direction does nothing.

Across taps the cosine is unreliable in two threshold-independent ways:

1. it ranks AASIST's gat_S and gat_T branches in the wrong causal order;
2. at SSL gat_T, the same cosine corresponds to opposite answers depending on whether the intervention is sufficiency or necessity.

A single geometric statistic cannot encode a variable it never sees.

How far does the result generalize?

The headline cell was too neat, so I ran the same intervention battery across both models, three admissible taps, and both arms where possible.

Every probe here is 98.8–99.8% accurate, with AUROC 0.9987–1.0000.

The conclusion I would publish now is weaker than my first one:

• “causally inert” holds in 2 of 6 valid cells, not everywhere;
• what holds in every valid cell is d beats w;
• the random-direction control invalidates an apparent probe success at SSL's embedding.

That weaker result is also not new. Marks & Tegmark (2023) report that difference-in-means directions are more causally implicated than logistic-probe directions for truth representations in LLMs, and derive the relation between the two.

The mechanism is roughly:

w ≈ Σ⁻¹(μ₁ − μ₀)

The probe whitens the class-mean difference by feature covariance. That can improve classification while rotating the direction away from the axis the downstream computation actually uses.

What this project adds is a different regime — audio, graph attention, a 297K-parameter model — and a case where the probe is not merely weaker but indistinguishable from random under intervention.

My current rule is:

Use probes as detectors. Do not read probe weights as causal directions without an intervention.

5. Where interpretability actually earns its compute

So far this sounds mostly negative:

• confidence fails under shift,
• the attack monitor needs careful recalibration and still catches only about 61% of attacks at a 5% false-alarm budget,
• probe weights can misdescribe mechanism.

The next question is whether looking inside the model buys anything a benchmark number cannot.

Here it does.

Both models degrade when moved to a corpus they were not trained on. From the outside, the failures look similar: high error rates and lots of bonafide speech being rejected.

But “the model fails out of domain” is not yet a diagnosis. The failure can live in three different places, and those places have very different repair costs.

Three things can break, at three very different prices

The model can fail at three different stages:

raw audio → [ representation ] → embedding(160) → [ Linear(160,2) → softmax ]
                 ③ retrain                            ② refit readout
                 297 K / 318 M                        161 effective params

          → p(bonafide) → threshold → verdict
                           ① move threshold
                           1 number

The three repairs change progressively more of the system:

All three can produce the same visible symptom: a bad error rate.

That is why EER or accuracy alone cannot tell you what to repair. A system that needs one threshold changed and a system whose representation has collapsed can both look like “the model fails cross-corpus.”

Why I need two tests

The two tests answer different questions.

Test 1 diagnoses where the information is lost. Test 2 measures what can actually be repaired cheaply.

They are complementary rather than redundant.

The distinction matters.

Suppose the embedding still separates bonafide from spoof almost perfectly, but the deployed model performs badly. Then the representation has done its job: the information reached the boundary between the representation and the readout. The failure must be downstream.

But Test 1 alone cannot tell whether downstream means:

• the threshold is misplaced, or
• the readout weights are stale.

Both failures can coexist with a perfectly informative embedding.

That is what Test 2 resolves. First move only the threshold. Then refit only the linear readout. Because each intervention changes a different part of the pipeline, the amount of error recovered tells us which downstream component was actually broken.

Conversely, if Test 1 already shows that the embedding no longer separates the classes well, then no threshold or linear readout can fully recover information that never reached them. Test 2 can still recover whatever signal remains, but any substantial residual error points back upstream to the representation.

So the logic is:

Test 1: Is the information still present at the embedding?
          │
          ├─ yes → failure is downstream
          │          │
          │          └─ Test 2 separates threshold vs readout weights
          │
          └─ no  → representation has degraded
                     │
                     └─ Test 2 measures how much of the remaining error is still cheaply recoverable

Test 1: does the embedding still separate the classes?

I probe the 160-dimensional embedding, because it is exactly what the released classifier's final Linear(160,2) layer receives.

This tap is important. An earlier layer could contain useful information that the network later throws away. Probing the embedding gives the strictest possible question:

Given exactly the information available to the shipped readout, are bonafide and spoof still linearly separable?

Cross-corpus:

The in-domain probes are essentially perfect in both models, so the protocol itself is not the source of the gap.

Cross-corpus, the models are very different:

• SSL-AASIST: 0.996 — the representation still separates the classes almost perfectly.
• AASIST: 0.859 — substantial class information has already been lost by the embedding.

That immediately changes the diagnosis.

For SSL-AASIST, a full retrain should be suspicious: the information is still there. Something downstream is using it badly.

For AASIST, some retraining pressure is real: the downstream layers cannot recover information that the representation no longer provides.

But Test 1 still does not tell me whether SSL-AASIST's downstream problem is the threshold or the readout weights, nor how much of AASIST's remaining error is still cheap to fix.

That is Test 2.

Test 2: how much do the cheap fixes actually recover?

Now I intervene on the two downstream components separately.

The first repair — move the threshold — is not a new anti-spoofing technique. Threshold-swept evaluation is already standard in this field. In particular, EER (equal error rate) evaluates the detector at the threshold where the false-accept and false-reject rates are equal, and EER is a standard metric reported for ASVspoof systems.

So the point of moving the threshold here is mainly diagnostic:

If choosing an appropriate threshold makes most of the errors disappear, then the model's score still ranks the two classes well; the deployed cut was simply in the wrong place.

I use calibration data to choose the threshold and then evaluate it on held-out test data. This avoids choosing a threshold on the same examples used to report its performance.

One terminology caveat: I do not mean that the EER threshold is universally the “best” deployment threshold. A real system may choose a different operating point depending on the relative costs of false accepts and false rejects. Here, the EER-style threshold sweep is useful because it demonstrates how much performance is available without changing the representation or readout weights.

The second repair asks a different question. Refitting the readout changes the scoring function itself — it changes how the 160 embedding dimensions are weighted. That can recover failures that no threshold move can fix.

Everything below is fitted on calibration data and scored on held-out test data.

These two interventions tell us different things.

Repair ①: move the threshold

A threshold cannot change the ordering of examples. It can only move the cut.

This is why threshold sweeping is such a useful diagnostic. If a model has poor accuracy at the shipped threshold but a low EER, the score ordering may still be good: the classes are largely in the right order, but the chosen cut no longer matches the shifted score distribution.

So when SSL-AASIST drops from 28.7% error to 4.2% after choosing the threshold on calibration data, that says the model was already ranking bonafide and spoof very well; its operating threshold was simply in the wrong place for the new corpus.

That removes 85% of its errors without touching a model weight. In this section I use that result to demonstrate the diagnosis — threshold placement rather than representation failure — rather than to propose a new way of choosing thresholds for deployment.

Repair ②: refit the readout weights

Changing the threshold still uses exactly the same scalar score. It cannot change how the 160 embedding dimensions contribute to that score.

Refitting Linear(160,2) can.

On AASIST:

• threshold only: 28.9% error
• refit readout: 23.0% error

The extra improvement means some information survived into the embedding but the shipped readout was weighting it poorly under the new corpus.

So AASIST has two downstream problems on top of its representation loss:

• 23% of its original errors disappear by moving the threshold;
• another 16% disappear by refitting the readout weights;
• a 23.0% absolute error rate remains even after the best cheap downstream repair.

That remaining error is consistent with what Test 1 already told us: the embedding itself has degraded.

The distinction between the two repairs is therefore simple:

• move the threshold → keep the score function fixed and ask whether a better cut is enough;
• refit the readout → keep the representation fixed but learn a better score function from the same 160 embedding dimensions.

If the threshold fixes almost everything, the representation and ranking largely survived. If refitting the readout adds substantial recovery, the information survived but the old weights use it poorly. If large error remains after both, the missing information is upstream in the representation.

Putting Test 1 and Test 2 together

This is the diagnosis:

This is why I need both tests.

Test 1 tells me where the information survives. Test 2 tells me what repair actually pays off.

Without Test 1, I could see that a threshold or readout refit helps, but I would not know whether the residual failure reflects a deeper representation problem.

Without Test 2, I could see that SSL-AASIST's embedding is healthy, but I could not tell whether the problem is one misplaced threshold or 161 stale readout parameters.

A benchmark number would have told me only that both models degraded.

The internal diagnosis says something much more actionable:

• SSL-AASIST mostly needs one number moved.
• AASIST needs deeper repair, but even there 39% of the errors can be removed without retraining the representation.

That is the point in this project where interpretability stops being descriptive and changes what I would actually do to the model.

Why EER can be 4.2% while deployed error is 28.8%

EER is threshold-free in the sense that it assumes the operating threshold is placed appropriately. A released checkpoint using softmax argmax effectively uses 0.5.

So SSL-AASIST can have:

• 4.20% EER
• 28.8% error at threshold 0.5

without contradiction.

The ranking is still excellent; the cut is in the wrong place.

On cross-corpus SSL-AASIST, the class distributions remain separated but both collapse toward zero. Moving the threshold to about 0.0003 lands back in the gap.

AASIST's distributions overlap, so no threshold can repair the representation.

The tap you probe changes the diagnosis

AASIST cross-corpus has an upstream gat_S probe at 0.881 AUROC, higher than the embedding's 0.859.

It is tempting to probe the strongest tap and say the representation is better than it looks.

But the readout never sees gat_S.

The signal degrades:

gat_S 0.881 → embedding 0.859 → shipped readout 0.776

Those are two different failures:

• 0.881 → 0.859: information lost inside the representation;
• 0.859 → 0.776: information reaches the readout but the readout uses it poorly.

Probing gat_S would have suggested 48% of errors were cheaply recoverable. Probing the embedding — the only layer a cheap readout refit can actually access — gives the real answer: 39%.

So another rule:

Probe the layer your intervention or repair can actually reach, not the layer with the prettiest score.

One caveat: I did not run full retraining as rung ③. It is listed as the expensive next step after the cheap fixes, not as a measured recovery number.

6. The wrong model: single-model findings mostly did not transfer

Now the main methodological result.

I ran the same battery of nine questions on both models.

Each finding falls into one of four outcomes:

Here is the final table:

Tally: 3 transfer · 1 conditional · 3 invert · 2 not askable.

The important point is not that the exact tally is low. Again, 3/9 is not a transfer rate.

The important point is that transfer fails in several qualitatively different ways:

• the answer can flip;
• the answer can become condition-dependent;
• the question itself can stop making sense.

And all three inversions are clean-data findings. This is not a story about adversarial examples making interpretability brittle.

H2: evidence can outlive the output

After a successful attack flips the model's verdict to bonafide, a probe trained on clean activations can still identify the underlying spoof evidence:

• 86.5% at AASIST enc4
• 100% at SSL's first transformer layer

Cross-corpus, that retention decays with depth. On AASIST it falls from roughly 82% at the front-end to 9% at the embedding.

The pattern transfers even though the raw numbers differ.

H7: a genuine architectural improvement that disappears out of domain

“Adversarial-real” attacks push genuine bonafide speech into the spoof class.

In-domain, SSL-AASIST is much less fragile than AASIST.

Cross-corpus, both collapse to roughly 100% attack success.

So “SSL fixes bonafide fragility” is true only in-domain. The model improvement is real, but conditional.

H3: branch attribution inverts

Cross-class activation patching asks which site carries class evidence.

On clean in-domain audio:

AASIST has essentially no spectral/temporal preference. SSL-AASIST is strongly temporal.

These are control-adjusted effects. That control matters: on SSL gat_T, the raw patching effect is 0.895, but a matched random tensor already causes 0.645. Most of the raw effect is magnitude, not class structure.

Applying the matched-random-tensor control reduces the raw branch contrast from 128× to 31×.

The qualitative inversion survived.

I also re-paired every AASIST bonafide clip with a different spoof donor. The control-adjusted branch comparison barely moved, which supports interpreting this as class-level rather than donor-specific evidence.

H8: the cleanest inversion is in the final readout

This is the case I trust most because it does not depend on a probe or approximate causal estimator.

Both models feed a 160-dimensional embedding into the same-shaped linear readout. That embedding is five 32-dimensional blocks:

• temporal max
• temporal average
• spectral max
• spectral average
• master nodes

Because the readout is linear, the contribution of each block to the bonafide-vs-spoof logit gap decomposes exactly.

AASIST:

• 92% of evidence comes from averages + master nodes
• master nodes alone contribute about 29%
• maxima contribute only 8%

SSL-AASIST:

• 62% comes from maxima
• the master-node share falls to about 6.8%

The absolute contributions move too, so this is not just a percentage reshuffle:

Within-model seed variation is tiny compared with the between-model shift.

Had I stopped at AASIST, I would have published something like:

“The master node carries 29% of the decision using 0.04% of the model's parameters.”

That sounds like an architectural insight.

On SSL-AASIST the same block contributes only 6.8%.

It was a fact about one trained model.

H9: the same symptom can be a different disease

Both models reject large amounts of bonafide speech cross-corpus.

But AASIST's representation degrades while SSL-AASIST's ranking remains almost intact and its threshold is misplaced.

Same failure from outside. Opposite repair.

This is exactly the kind of conclusion that a single benchmark metric hides.

H4 and H5: sometimes the question disappears

These are stronger than ordinary non-transfer.

H4 — can an attacker route around the causal site?

On AASIST, the monitor tap and causal tap are distinct, so an attacker can try to fool one while preserving the other.

On SSL-AASIST, the best detector and causal tap coincide at pool_T. There is nothing to route around. The question dissolves.

H5 — which frequencies carry spoof evidence?

AASIST has a real node→Hz mapping. The hypothesis was refuted: the best single frequency node recovers only about +0.038 against a full-tensor effect around 0.60–0.72.

SSL-AASIST has no frequency axis at all.

So there is no cross-model transfer verdict to assign.

7. A monitor worth keeping even when it catches nothing

One of the more surprising results came from attacking the monitor itself.

I jointly optimized the attack against the classifier and the monitor.

At the adaptive endpoint in the project summary:

• monitor detection collapses to AUROC 0.088, below chance;
• attack success falls from 92.2% → 13.0%.

So the monitor loses badly if you judge it by detection AUROC.

But forcing the attacker to hide from it costs 86% of the attack's success rate.

That suggests a different success criterion:

A monitor need not catch the attacker to be useful. It can be useful if evading it is expensive.

For adversarial oversight, I now care about both:

1. does the monitor still detect?
2. what does the attacker have to sacrifice to evade it?

On SSL-AASIST, this exact routing opportunity is not available because the best detector and causal tap coincide. Again, the principle may generalize while the architectural opportunity does not.

8. Validation checks that matter

Several conclusions in this project depend less on adding a new interpretability method than on asking whether the measurement itself is well-posed.

Four checks were especially important:

1. Do not evaluate a metric on a subset selected using that same metric. Confidence looks misleadingly bad if it is scored only on successful attacks, because attack success already depends on crossing the model's decision boundary.
2. Read every intervention against its matched control, never against zero. The random-tensor control reduces SSL-AASIST's raw branch contrast from 128× to 31×, and the random-direction control invalidates the SSL embedding direction-patching cell because a random vector flips the model too.
3. Profile the full relevant axis before generalizing from a convenient tap or branch. AASIST retains different amounts of class information at different depths, and SSL-AASIST carries its strongest causal direction in gat_T, not gat_S.
4. Recompute summaries directly from the underlying table. The headline numbers in this post are tied to explicit model, corpus, condition, control, and denominator so that a compact summary does not silently change what was measured.

These checks are mundane compared with activation patching or adversarial optimization, but they determine whether the more sophisticated measurements mean what they appear to mean.

9. What this says for practice

If you deploy a detector

Do not gate review purely on softmax confidence. It can be excellent in-domain and fail under shift by becoming confidently wrong.

Do not assume one monitor number tells you whether the monitor works. On AASIST, embedding scores 0.987 AUROC against pristine negatives, 0.851 against perturbed benign negatives, and 73% FAR if I reuse a threshold fitted on clean audio. At a fixed 5% FAR, it catches about 61% of attacks. Those are three different claims: separability, calibration, and operating point. Measure the one your deployment actually cares about.

Calibrate against audio processed the way your inputs will be processed. My MP3/resampling controls are already harsher than pristine audio, but they are still software transforms of studio recordings. A real phone call is a harder negative distribution than anything in this project.

Judge monitors by evasion cost as well as detection. My adaptive monitor's AUROC collapsed below chance, but hiding from it destroyed most of the attack's success.

Before retraining, test the cheap repairs. A threshold, stale readout weights, and representation loss all look like “high error” from outside.

If you audit a model

Choose instrumentation points by causal relevance, not probe score.

Give causal claims matched random controls. Mine changed conclusions twice.

Run both sufficiency and necessity when possible. SSL gat_T gave different answers for the probe direction depending on the arm.

Do not read probe weights as causal directions. A probe can be nearly perfect and still point somewhere the model does not use.

Then run the battery again on another model.

If you report results

Put attack success next to the perturbation control.

Do not fit a threshold on the rows you then score it on. Fit the threshold on calibration data and report its performance on held-out test data; otherwise the estimated repair can be optimistic by construction.

State the model and condition on every number. Four conditions × two models is enough to accidentally quote across cells.

Keep an append-only experiment log. It makes it possible to trace each headline number back to the experiment, control, and condition that produced it.

10. What these results do not support

Several tempting claims would overstate the evidence.

This is not a real-world attacker model

Every attack here is white-box: full access to weights and gradients.

That is an upper bound on attacker capability, not a deployment prediction. Black-box transfer is the obvious missing experiment.

“Adversarial fragility exists” is not the contribution

Audio anti-spoofing models being attackable is already known.

The contributions here are the energy-matched controls, the two-model transfer audit, the probe-vs-causal comparison, and the representation-vs-readout diagnosis.

SSL-AASIST's cross-corpus advantage is confounded

XLS-R was pretrained on 436k hours including MLS, and ASVspoof5 bonafide speech is drawn from MLS English.

So SSL-AASIST's cross-corpus advantage may partly reflect pretraining overlap rather than clean generalization.

A better test is In-the-Wild, which uses found celebrity audio and avoids that overlap.

SSL-AASIST is not necessarily more robust

SSL-AASIST resists FGSM much better than AASIST while iterative PGD still reaches roughly 82–100% success.

That pattern is consistent with gradient masking, not genuine robustness.

No monitor number here has met a real recording

Every clip in this project is studio-quality ASVspoof audio. Even the more realistic benign controls are software MP3 compression and resampling applied to studio recordings.

There is no telephony codec, microphone response, room acoustics, environmental noise, or packet loss.

So the monitor operating point — roughly 61% detection at 5% FAR for the best AASIST tap — should be treated as an upper bound on what I have established. Real deployment negatives are likely harder; this project does not measure by how much.

This limitation matters most for the monitoring results, somewhat for end-to-end detection performance, and less directly for the clean mechanistic findings in the probe and transfer sections.

Two models are not a population

“3 of 9 transferred” shows that transfer can fail under an unusually favorable comparison.

It does not estimate how often interpretability findings transfer in general.

Some SSL causal results are single-seed

AASIST's core results use three seeds. Several SSL causal analyses are still single-seed.

The effects are large, but effect size is not a substitute for actually running the seeds.

11. What I would do next

In rough order of how much each could change the conclusions:

1. Run In-the-Wild. This is the cleanest way to remove the MLS overlap in the cross-corpus comparison, and it is the experiment most likely to move the current conclusion against me.
2. Add a third model. AASIST3 is a natural next arm. Viakhirev et al. (2026) already report correlational TreeSHAP branch attributions, which makes it a useful target for causal validation.
3. Test black-box transfer attacks. The depth gradient — roughly 82% evidence retention near the front-end down to 9% at the embedding — gives a concrete prediction about where transfer should survive.
4. Run seed parity on SSL causal experiments.
5. Use path patching or cleaner ablations. Full-tensor patching can cascade through the serial back-end and saturate later layers.
6. Develop a patching method for variable-layout activations. pool_T is SSL-AASIST's best monitoring tap and also one of the most interesting causal sites, but its top-k node selection changes per clip, which blocks the current direction test.
7. Use benign controls that resemble a deployment. Add telephony codecs such as AMR/Opus/G.711, device impulse responses, room acoustics, background noise, and packet loss. This is cheap compared with most of the list and is the experiment most likely to move the monitor operating points downward.
8. Try psychoacoustic masking attacks. “40 dB” is an energy statement, not a perceptual guarantee.
9. Replicate the probe-direction result in language models. The test is cheap on cached activations, and I would like to know whether the inert case is rarer or more common than my 2-in-6.

12. What I think generalizes past deepfakes

The specific numbers belong to two audio models.

The methodological lessons feel broader:

1. Run your interpretability battery twice. Single-model findings can look architectural and turn out to be parochial.
2. Predictive is not causal. A near-perfect probe can still identify the wrong intervention direction.
3. Match the instrument to the question. “Decodable here,” “this site matters,” and “this axis matters” are different claims.
4. Probe the layer your repair or intervention can actually access.
5. Use matched random controls for causal interventions.
6. Check class balance before trusting an error-prediction metric.
7. Separate monitor separability from calibration. AUROC against pristine audio, AUROC against processed benign audio, and FAR at one threshold are not interchangeable.
8. Judge monitors partly by what evasion costs.
9. Treat summary sentences as claims that need their own verification.
10. Keep the log append-only.

The result that changed my mind most was not that an attack worked or that one monitor metric looked bad.

It was that I could produce what looked like a crisp mechanistic explanation on one model, repeat the analysis on a closely related model, and watch the explanation invert.

That is the result I would most like other interpretability work to stress-test.

Reproducibility, release, and repository notes

The experiments are complete for the two-model, two-corpus matrix used in this project.

The repository includes:

• the research write-up;
• project plan and coverage matrix;
• an append-only experiment log;
• related-work notes;
• versioned result artifacts;
• scripts for monitors, probes, readout/representation diagnosis, patching, and figures.

The AASIST arm runs in the main Python environment. SSL-AASIST depends on an older fairseq stack, so its feature extraction runs under Python 3.7 / Torch 1.8 and dumps a shared artifact schema for later analysis under the main environment.

On an RTX 3090, the measured forward cost is roughly:

• AASIST: 5.0 ms
• SSL-AASIST: 8.1 ms

Despite having about 1,067× more parameters, SSL-AASIST is only about 1.6× slower in the forward pass in this setup because XLS-R compresses the waveform to a much shorter frame sequence before the dense operations.

Dual-use and release

The attacks are standard white-box PGD against public academic checkpoints.

The repository ships code and aggregate results, but does not distribute the generated adversarial audio. Roughly 32 GB of attack waveforms are deliberately gitignored.

The actionable outputs are mostly defender-side: where monitoring fails, how much adaptive evasion costs, which internal diagnoses distinguish cheap fixes from retraining, and which interpretability claims fail to transfer.

Attribution

• AASIST — Jung et al., ICASSP 2022: paper
• SSL-AASIST — Tak et al., Odyssey 2022: paper
• Marks & Tegmark (2023) — difference-in-means vs probe directions: paper

Full code, artifacts, and append-only experiment log:
github.com/gavin-keli/deepfake-detector-audit

Thanks to BlueDot Impact and my sprint cohort.