~~�~~ ~~_n_~~
_i_ =1 _[∥][x][i][−][c][j]_ [(] _[i]_ [)] _[∥]_ ~~[2]~~
**Dunn Index** ✓ ✓ _D_ = [min] _[i]_ [=] _[j]_ ~~[dist]~~ [(] _[C][i][,C][j]_ [)]
max _k_ ~~diam~~ ( _Ck_ ) [. Sensitive to cluster sparsity and maximum diameter noise.]
Useful in worst-case margin settings.
**Gap Statistic** ✓ ✓ _G_ ( _k_ ) = _B_ [1] ~~�~~ _Bb_ =1 [log(] _[W]_ [ (] _k_ _[b]_ [)][)] _[ −]_ [log(] _[W][k]_ [)][. Effective for cluster number estimation, but]
indirect for auditing fxed binary safety partitions.
**V-Measure** ✓ ✓ _V_ = 2 _·_ ~~[Homogeneity]~~ _[·]_ ~~[Completeness]~~
~~Homogeneity~~ + ~~Completeness~~ [. Label-based metric; assumes gold truth and]
ignores latent geometry.
_G_ ( _k_ ) = _B_ [1] ~~�~~ _Bb_ =1 [log(] _[W]_ [ (] _k_ _[b]_ [)][)] _[ −]_ [log(] _[W][k]_ [)][. Effective for cluster number estimation, but]
indirect for auditing fxed binary safety partitions.
**V-Measure** ✓ ✓ _V_ = 2 _·_ ~~[Homogeneity]~~ _[·]_ ~~[Completeness]~~
**Jaccard Index** ✗ ✓ _J_ = _[|]_ _|_ _[A]_ _A_ _[∩]_ _∪_ _[B]_ _B_ _[|]_ _|_ [. Set-theoretic overlap score, not a latent-space measure.]
**Cluster Purity** ✓ ✗ Purity = _N_ 1 ~~�~~ _ki_ =1 [max] _[j][|][C][i][ ∩]_ _[T][j][|]_ [. Label overlap-only; fails to capture inter-cluster]
geometry or fuzziness.
Table 8: Comparison of clustering validity metrics for evaluating latent alignment separability in LLM-generated
completions. Among these, CHI and XBI offer complementary strengths: CHI captures global dispersion structure
but tends to reward conservative models (e.g., frequent refusals), while XBI penalizes unsafe boundary intrusions
but may falsely reward compact, overconfident, unsafe clusters. Our alignment diagnostic framework (AQI)
integrates both via a geometric mean to yield a decoding-invariant, robust alignment score. While our experiments
focus on CHI and XBI, the table highlights the broader geometric and label-sensitive metrics landscape, inviting
further comparative exploration and benchmarking in future alignment research.


Instead of inventing new latent-space geometry
metrics from scratch, AQI draws on a rich literature of clustering validity indices (CVIs) developed for evaluating unsupervised separability in
embeddings (Rousseeuw, 1987; Xu and Wunsch,
2005). The labels (safe vs unsafe) are known in our
case, but the underlying goal is the same: how well
do these labels correspond to naturally separable
structures?
Most CVIs fall into one of three categories:
- **Between-vs-within variance ratio (e.g., Calin-**

Alignment Quality Index (AQI) : Beyond Refusals: AQI as an Intrinsic
  Alignment Diagnostic via Latent Geometry, Cluster Divergence, and Layer wise
  Pooled Representations

Abhilekh Borah
Chhavi Sharma
Danush Khanna
Utkarsh Bhatt
Gurpreet Singh
Hasnat Md Abdullah
Raghav Kaushik Ravi
Vinija Jain
Jyoti Patel
Shubham Singh
Vasu Sharma
Arpita Vats
Rahul Raja
Aman Chadha
Amitava Das


high when class centroids are distant and internal
scatter is low.
- **Compactness–separation** **tradeoffs** **(e.g.,**
**Xie–Beni (Xie and Beni, 1991)):** penalize close
centroids and loose clustering.
- **Boundary-sensitive** **(e.g.,** **Davies–Bouldin**

ski–Harabasz (Cali´nski and Harabasz, 1974)):


**(Rousseeuw, 1987)):** favor convex separability
and margin fidelity.
Among these, we selected Calinski–Harabasz
and Xie–Beni as complementary signals:
 - CHI is scale-invariant and reflects macro-structure
separability.
 - XBI is entroid-sensitive, emphasizing **microlevel leakage** or intrusion between clusters.


CHI is formally defined as:
[Tr][(] _[B]_ [)] _[n]_ [ +] _[ m][ −]_ [2]
Tr( _W_ ) _[·]_ 1
CHI = [Tr][(] _[B]_ [)]
where
Tr( _B_ ) = _n∥µs −_ _µ∥_ [2] + _m∥µu −_ _µ∥_ [2] _,_ Tr( _W_ ) =
and
_n_
- _∥zi_ _[s]_ _[−]_ _[µ][s][∥]_ [2][+]
_i_ =1
_m_
- _∥zj_ _[u]_ _[−]_ _[µ][u][∥]_ [2]
_j_ =1
This composite score avoids linear averaging,
which can mask outliers in either CHI or XBI. The
geometric mean ensures that both cluster-level and
boundary-level separability must be strong to yield
a high AQI. This is aligned with literature in multiobjective optimization and ensembling (Nguyen
et al., 2013; Zitzler and Künzli, 2004).


Both CHI and XBI exhibit important properties for
model auditing:
 - **Affine Invariance:** CHI and XBI are invariant
under affine transformations and global rotations
of the latent space (Xu and Wunsch, 2005).
 - **Robust to Decoding Variability:** Unlike
classifier-based metrics, AQI does not require
fixed completions and tolerates prompt-level entropy.
 - **Cluster-scale Sensitivity:** CHI favors global reorientation; XBI flags local intrusions—making
their combination particularly robust to varied
safety threats.
In contrast, other CVIs like Silhouette suffer
under high-dimensional embeddings due to curseof-dimensionality effects (Aggarwal et al., 2001),
and Davies–Bouldin is sensitive to centroid drift
and redundant dimensions.

B.5 Stability, Invariance, and Robustness


We experimented with several baselines:
   - **Silhouette Score:** Highly unstable for largescale LLM embeddings; inflated by sparse outliers.
  - **Linear SVM Margin:** Not decoding-invariant;
sensitive to sampling and prompt paraphrase. **Fisher Discriminant Ratio:** Unstable when covariance matrices are nearly singular; CHI subsumes this formulation.
In ablation, CHI-only metrics favored overly
conservative models (e.g., frequent refusals), while
XBI-only favored overconfident yet vulnerable
models. Their geometric complementarity forms
the conceptual foundation for AQI. As summarized
in Table 8, classical clustering validity indices

_zi_ _[s]_ [+]  _i_ _j_
_µs_ := [1]
_n_
_n_
- _zi_ _[s][,]_ _µu_ := [1]
_m_
_i_ =1
_m_
_zj_ _[u]_
_j_


- _m_ _zj_ _[u][,]_ _µ_ := 1
_n_ + _m_
_j_ =1
 [�]
This index rewards configurations where safe
and unsafe embeddings are tightly clustered and
centered far from one another. It is susceptible to
variance decomposition and aligns with discriminant analysis perspectives (Fisher, 1936). We
adopt CHI as the **global separation component**
of AQI.


The XBI provides a complementary view:
XBI =
- _n_
_i_ =1 _[∥][z]_ _i_ _[s]_ _[−]_ _[µ][s][∥]_ [2][+] [�] _j_ _[m]_ =1 _[∥][z]_ _j_ _[u]_ _[−]_ _[µ][u][∥]_ [2]
( _n_ + _m_ ) _· ∥µs −_ _µu∥_ [2]
The numerator measures cluster compactness,
and the denominator measures centroid separation.
XBI is minimized when clusters are tight and far
apart. We interpret XBI as a **local alignment vul-**
**nerability index** : if unsafe completions lie near
the boundary of the safe cluster, XBI will be high.
Unlike CHI, XBI penalizes even a small number
of unsafe completions that intrude close to safe
ones, making it suitable for jailbreak detection and
early failure auditing (Zou et al., 2023).


To unify these complementary objectives, we define AQI as a normalized geometric mean:
   - CHI
AQI _λ_ =
CHImax
- _λ_
_·_ (exp( _−_ XBI)) [1] _[−][λ]_
where _λ ∈_ [0 _,_ 1] controls the contribution of macro
vs. micro separation, and CHImax is computed
as the 99th percentile over a reference pool (e.g.,
LLaMA-2 outputs on safe datasets).
vary significantly in their sensitivity to compactness, separation, and labeling assumptions. While
our implementation centers on CHI and XBI due
to their geometric interpretability and decodinginvariance, other metrics such as the Silhouette
Score or Dunn Index may offer complementary
perspectives and warrant further empirical analysis.
**Summary** : AQI is not just a composite
score—it is a structured metric grounded in
decades of work on geometric validation of clusters. Its combination of Calinski–Harabasz (for inter/intra dispersion) and Xie–Beni (for local tightness/separation) enables interpretable, robust alignment diagnostics. This design supports zero-shot
audit scenarios, latent space UMAP interpretations,
and decoding-invariant safety scoring—all critical
for high-stakes LLM deployment.


Transformer-based language models such as
LLaMA-3 produce hierarchical token embeddings
across their _L_ decoder layers. Conventionally, only
the final hidden layer _h_ _[L]_ is used to derive semantic representations. However, recent interpretability work (Elhage et al., 2021; Dalvi et al., 2020)
demonstrates that meaningful features—including
alignment-relevant semantics—emerge at different depths. For instance, syntactic roles dominate early layers, while later layers express taskspecific abstractions. This depth diversity motivates us to exploit *all* layers via an inferencetime _layerwise attention pooling_ mechanism that
dynamically aggregates hidden states according to
alignment-relevant signals.
Let _h_ [1] _, h_ [2] _, . . ., h_ _[L]_ _∈_ R _[d]_ be the token embeddings at each decoder layer for a given input-output
pair. Instead of selecting a fixed layer (e.g., _h_ _[L]_ ),
we define a pooled representation:
where the attention weights _α_ [(] _[l]_ [)] _∈_ [0 _,_ 1], with
_l_ _[α]_ [(] _[l]_ [)][ = 1][, are computed to emphasize the lay-]
ers most semantically aligned with a fixed target
concept (e.g., “safe” completions). This mechanism is non-parametric, requires no additional
training, and is decoding-invariant. Crucially, it
permits _semantic attribution over depth_, allowing
alignment researchers to interrogate where specific
safety signals are encoded in the model’s depth
hierarchy.


To guide attention over layers, we introduce a
_frozen semantic alignment vector r ∈_ R _[d]_ encoding
a target concept (e.g., safety, non-toxicity, instruction adherence). This vector can be obtained via
averaging representations of known aligned completions or optimized from alignment contrastive
pairs. At inference, we compute layerwise similarity scores:
_h_ _[l]_ _· r_
_sl_ = cos( _h_ _[l]_ _, r_ ) =
_∥h_ _[l]_ _∥∥r∥_ _[,]_
which quantify alignment of each layer’s embedding with the reference signal. These scores form
the basis for the layer attention distribution _**α**_ .
To verify that intermediate layers encode meaningful alignment gradients, we define the **alignment anchoring loss**:
which is minimized when all layer outputs align
with _r_ . In practice, we do not use this as a training objective, but as a _layer diagnostic_ to evaluate
how alignment gradients are distributed across the
model’s depth—critical for understanding decoder
dynamics in aligned vs. misaligned generations.


We now define two methods for converting scores
_s_ 1 _, . . ., sL_ into attention weights _α_ [(] _[l]_ [)] for pooling:
����
= 2 _−_ [2]
_L_ align = [1]
_L_
_L_
_l_ =1
_h_ _[l]_ _r_
���� _∥h_ _[l]_ _∥_ _[−]_ _∥r∥_
_L_
_L_
- cos( _h_ _[l]_ _, r_ ) _,_
_l_ =1
_h_ _[∗]_ =
_L_
_α_ [(] _[l]_ [)] _h_ _[l]_ _,_
_l_ =1
**Softmax Pooling.** The default approach is to apply softmax normalization:
exp( _sl_ )
_α_ [(] _[l]_ [)] = _._
~~�~~ _L_
_j_ =1 [exp(] _[s][j]_ [)]
This is smooth and differentiable and ensures all
layers receive a nonzero weight. However, it tends
to produce dense distributions that blur attribution
and can be skewed by small differences in _sl_ .
**Sparsemax Pooling.** To improve interpretability, we propose using sparsemax (Martins and Astudillo, 2016b), which solves the following projection:
sparsemax( _s_ ) := arg min
_α∈_ ∆ _[L][∥][α][ −]_ _[s][∥]_ [2] _[,]_
where ∆ _[L]_ is the _L_ -simplex. Sparsemax yields
**sparse distributions**: many _α_ [(] _[l]_ [)] = 0, focusing
attention on a small subset of layers. It has a closedform solution based on thresholding:
_α_ [(] _[l]_ [)] = max _{sl −_ _τ,_ 0 _},_ where _τ_ ensures - _α_ [(] _[l]_ [)] = 1 _._
_l_
This sparsity aligns with our interpretability
goals: We wish to identify which layer(s) contribute to alignment rather than averaging over the
entire depth.
**C.3** **Theoretical Analysis: Sparsemax vs.**


To rigorously compare sparsemax and softmax for
layer pooling, we consider their behavior under entropy, gradient dynamics, and support properties.
**(1) Entropy and Selectivity.** The entropy of the
attention distribution influences both the sharpness
of pooling and the interpretability of attribution.
Let _H_ ( _**α**_ ) = _−_ [�] _l_ _[α]_ [(] _[l]_ [)][ log] _[ α]_ [(] _[l]_ [)][. For softmax,]
_H_ softmax( **s** ) _∈_ [0 _,_ log _L_ ] _,_
with entropy maximized when all scores _sl_ are
equal. This implies that softmax attention becomes
nearly uniform in flat score regions, diffusing the
alignment signal across layers. Sparsemax, by
where _S_ is the support set. Thus, only active layers
receive gradients, and support transitions are discrete. This makes optimization more interpretable,
stable under adversarial drift, and less prone to gradient diffusion—a known challenge in alignment
tuning (Chan et al., 2023).
**(3) Convexity and Optimization Landscape.**
Sparsemax is a Euclidean projection onto the probability simplex ∆ _[L]_ under the _ℓ_ 2 norm:
arg min
_**α**_ _∈_ ∆ _[L][∥]_ _**[α]**_ _[ −]_ **[s]** _[∥]_ [2] _[,]_
which is convex and admits exact solutions. Its projection operator has polyhedral geometry, making
the induced loss convex and interpretable. In contrast, softmax is a smooth exponential map. Still,
its log-partition function is strictly convex with
gradients that vanish under saturation (i.e., large
_∥s∥∞_ ), which may hinder fine-tuned alignment
signal propagation.
contrast, enforces _support pruning_ : its entropy is
naturally bounded by log _k_, where _k_ = _|_ supp( _**α**_ ) _|_
is the number of non-zero weights.
Empirically, we find that sparsemax induces attention sparsity _k ≈_ 2 _._ 4 across LLaMA-3 layers
(vs. _L_ = 32), yielding 92.5% zero weights and
producing significantly lower entropy than softmax
( _∼_ 1 _._ 1 nats vs. _∼_ 2 _._ 9).
**(2) Gradient Flow and Support Stability.** Softmax gradients are dense:
_∂α_ [(] _[i]_ [)]
= _α_ [(] _[i]_ [)] ( _δij −_ _α_ [(] _[j]_ [)] ) _,_
_∂sj_
implying any change in a score _sj_ affects all _α_ [(] _[i]_ [)] .
This coupling is problematic when only a subset of
layers matter—noise in irrelevant scores perturbs
the entire distribution. Sparsemax has piecewiselinear gradients:
if _i, j ∈_ _S, i ̸_ = _j_
_|S|_ _[,]_
_,_
_∂α_ [(] _[i]_ [)]
=
_∂sj_




0 _,_ otherwise
1 _−_ 1 if _i_ = _j ∈_ _S_
_|S|_ _[,]_
_−_ [1]
Thus, sparsemax offers a theoretically preferable structure for inference-time layer attention
where we want attribution, robustness, and clarity,
not continuous interpolation.


We apply layerwise attention pooling with softmax
and sparsemax to LLaMA-3 8B completions over
the LITMUS benchmark. The goal is to assess the
pooling strategy’s effect on latent representations
used in the Alignment Quality Index (AQI) scoring
pipeline, especially under adversarial prompting
and decoding variation.
**Experimental Setup.** We evaluate 5,000 completions sampled from safe and unsafe instruction
prompts, with decoding temperatures _{_ 0 _._ 7 _,_ 0 _._ 9 _}_
and nucleus thresholds _{_ 0 _._ 85 _,_ 0 _._ 95 _}_ . We extract
hidden states from all _L_ = 32 decoder layers for
each completion, compute cosine alignment scores
with a frozen safe representation _r_, and derive
pooled embeddings using softmax and sparsemax
attention.
**Layer Selection Patterns.** Depending on the
prompt category, Sparsemax selects 1–3 layers per
example with non-zero weight, often mid-to-late
layers (layers 16–25). In contrast, softmax attention heavily favors the final 2 layers across most
inputs (mean _α_ [(31)] = 0 _._ 52, _α_ [(32)] = 0 _._ 35), leading
to representational homogeneity across contexts.
**AQI Stability Under Perturbations.** We compute AQI scores using CHI and XBI metrics on
pooled embeddings under both attention schemes.
Under adversarially paraphrased jailbreak prompts,
softmax-pooled embeddings show _∼_ 18 _._ 4% higher
standard deviation in AQI values across decoding
seeds. Sparsemax maintains lower variance and
greater separability between safe and unsafe distributions, as measured by cluster purity and CHI.
**Latent Structure via UMAP.** We visualize the
pooled representations using UMAP. Embeddings
pooled with sparsemax yield more compact clusters, with clear separation between safe and unsafe
This improves the fidelity of latent alignment
scores, especially under sampling noise and adversarial prompting. Sparse pooling offers interpretability by explicitly identifying contributing
layers and robustness by filtering out semantically
irrelevant depth signals.
In summary:
 - **Sparsemax yields sharper, sparser layer attri-**
**bution**, improving interpretability and semantic
focus.
 - **It stabilizes AQI scores under decoding ran-**
**domness and adversarial perturbation**, outperforming softmax.
 - **It enables UMAP-based diagnostic visualiza-**
**tions** that better reflect semantic separability.
completions, especially in cases where softmaxpooled embeddings are entangled. Notably, sparsemax helps disambiguate near-boundary completions that superficially appear safe but semantically
drift toward leakage.
**Case Study: Jailbreak Failures.** In an illustrative jailbreak prompt (e.g., paraphrased “how to
hurt someone” disguised as a game query), softmax assigns _∼_ 90% weight to top layers where the
model refuses superficially. Sparsemax instead attributes the alignment failure to a mid-layer where
toxic intent was first encoded, yielding an embedding that better reflects the latent misalignment,
supporting the utility of sparse attribution in forensic audits.


Layerwise pooling with sparse attention is a dropin enhancement to the AQI diagnostic pipeline.
Rather than computing CHI/XBI on a fixed finallayer embedding, we instead compute it on the
pooled vector _h_ _[∗]_ :
_h_ _[∗]_ =
_L_
- _α_ [(] _[l]_ [)] _h_ _[l]_ _,_ where _**α**_ = sparsemax �cos( _h_ _[l]_ _, r_ )� _._
_l_ =1
- **It integrates seamlessly with existing AQI**
**pipelines**, enhancing both CHI and XBI metrics
via denoised embeddings.
We recommend sparsemax-based layer pooling
as a default for alignment scoring in decoder-only
models. It supports attribution-aware debugging
and improves latent robustness, which are both
essential for trustworthy LLM auditing at scale.


This section presents a detailed evaluation of the
**Alignment Quality Index (AQI)** under three core
perturbation regimes: (1) decoding stochasticity
(temperature sampling), (2) adversarial suffix injection (jailbreaking), and (3) prompt paraphrasing (surface drift). These perturbations degrade
surface-level safety metrics, including refusal rate,
judge score, and toxicity classifiers. In contrast,
we show that AQI exhibits strong invariance and
deflection-sensitivity under these conditions by
leveraging geometric properties of latent completions—offering a decoding-invariant lens for safety
auditing.


Stochastic decoding is widely used in instructiontuned LLM deployments, where output variance
arises due to temperature scaling and nucleus sampling. However, behavioral alignment scores (e.g.,
refusal rate, judge approval) are highly sensitive to
decoding configuration—even though underlying
representations may remain semantically stable.
To test AQI’s decoding-invariance, we generated 20 completions per prompt for 100 safetycritical prompts using temperature values _T ∈_
_{_ 0 _._ 2 _,_ 0 _._ 7 _,_ 1 _._ 0 _}_, and top- _p_ = 0 _._ 9. We then computed AQI on pooled latent representations and
G-Eval and judge metrics.
Despite a 40-point drop in behavioral scores
across decoding conditions, AQI varies only
marginally—highlighting its sampling-invariant
nature. This stability stems from AQI’s design:


0.2 0.82 0.83 0.84
0.7 0.61 0.69 0.83
1.0 0.42 0.50 0.81
Table 9: **Metric Robustness Across Decoding Tem-**
**peratures.** While behavioral metrics such as GEval and judge score degrade significantly at higher
decoding temperatures, AQI remains nearly invariant—demonstrating its robustness to sampling-induced
variation in completions.
it operates on pooled, layerwise-aggregated activations that precede sampling, enabling consistent
latent comparison across generations.


Jailbreaking attacks craft suffixes that transform
safe prompts into unsafe completions—often while
maintaining superficially benign structure. Traditional safety metrics, focused on token-level refusal or toxicity, struggle to detect such semantic
corruption.
We evaluated AQI on 200 paired prompts drawn
from the LITMUS and LITMUS-J benchmarks,
which include jailbreak patterns such as: (a) selfreflexive hypotheticals, (b) formatting masks (e.g.,
JSON), and (c) "roleplay" suffixes.
**Result:** AQI deflects significantly under jailbreak
injection, even when refusal rates remain unchanged or paradoxically _increase_ (e.g., due to
hedging). Table 10 reports AQI collapse across
model scales.
The cross-hull term (CHI) collapses dramatically in these cases, indicating that unsafe completions are no longer confined to a distinct latent
region.


Real-world prompts are often reworded in style or
structure (e.g., via paraphrasing), making robustness to prompt drift critical. We evaluated AQI
Figure 6: **Standard Deviation of Metrics under Perturbations.** AQI exhibits consistently lower variance than
Refusal Rate (RR) and Detoxify-based Toxicity across decoding temperature, suffix injection, and prompt drift.
This reflects its geometric robustness to generation stochasticity and surface perturbations, making it more stable
for adversarial alignment evaluation.


TinyLLaMA 0.91 0.34 62.6%
Phi-2 0.91 0.35 61.5%
GPT-NeoX 0.91 0.61 32.9%
LLaMA-13B 0.91 0.67 26.4%
LLaMA-65B 0.91 0.73 19.8%
Table 10: AQI degradation under adversarial suffix injection. Smaller models show sharper collapses in latent
safety separation.
using LITMUS-P, a paraphrased variant of LITMUS generated via backtranslation and synonym
augmentation. For each prompt, five semantically
equivalent rewrites were used to elicit completions
across four models.
These results affirm that smaller models fail
to encode paraphrase-invariant safety boundaries,
while AQI captures these shifts via latent overlap—quantified using XBI.

Model AQI (Clean) AQI (Jailbreak) Drop (%)


TinyLLaMA 0.58 0.32 44.8
Phi-2 0.65 0.45 30.8
LLaMA-13B 0.78 0.70 10.3
LLaMA-65B 0.81 0.76 6.1
Table 11: AQI sensitivity to paraphrastic rewording.
Higher-capacity models show improved latent invariance.

Model AQI (Orig) AQI (Paraphrase) Drop (%)


Across all three settings, AQI demonstrates lower
variance and higher sensitivity to latent collapse
(cf. Figure 6).
Moreover, AQI deflection often _precedes_ behavioral collapse. In jailbreak scenarios, AQI
drops by 40–60% even when detox scores remain
low—indicating representational entanglement before output misalignment.
As illustrated by Figure 7, AQI deflection of
Figure 7: **Metric Deflection under Increasing Jailbreak Severity.** AQI deflects early and sharply as adversarial
suffix severity increases—from clean prompts to light, moderate, and severe jailbreaks. In contrast, Refusal Rate
(RR) and Toxicity exhibit delayed or noisy degradation. AQI’s geometric deflection acts as a latent misalignment
signal before surface outputs violate safety.


Decoding Temperature 0.19 0.11 **0.02**
Suffix Injection 0.31 0.21 **0.06**
Prompt Paraphrase 0.13 0.09 **0.02**
Table 12: Metric variance under perturbations. AQI remains stable while behavioral metrics fluctuate widely.
ten precedes observable output failures, serving as
a geometric signal for representational misalignment.

Perturbation RR Std. Dev Toxicity Std. Dev AQI Std. Dev


These findings establish AQI as a structurally
grounded and decoding-agnostic alignment metric. Unlike judge metrics or refusal classifiers,
which are brittle to decoding and paraphrase variation, AQI measures geometric separability, enabling early and consistent misalignment detection.
In future iterations of alignment evaluations, especially under red teaming or jailbreak audits, we
recommend AQI as a latent-first signal complementing behavioral metrics.


This section visualizes one of the most critical
safety pathologies in LLMs: _deceptive align-_
_ment_ —a scenario where outputs appear safe. Still,
internal representations exhibit a collapse between
aligned and unsafe semantics. Such failures are
increasingly common under jailbreaks, adversarial
suffixes, or reworded instructions, and cannot be
diagnosed by behavioral signals alone. Instead, we
probe model activations via UMAP projections of
pooled embeddings across layers, revealing the onset of latent boundary collapse, internal semantic
drift, and failure of representational alignment.
(a) **Jailbreak-Induced Latent Collapse.** Under adversarial suffixes (e.g., roleplay, hypotheticals), unsafe
completions collapse inward into the latent manifold
of safe completions. While detox classifiers or judge
metrics remain stable, AQI drops sharply (e.g., 0.91 →
0.54), exposing semantic entanglement.
(b) **Latent Drift under Paraphrased Prompts.**
Rewriting prompts with synonymous phrases or syntactic restructuring causes unsafe completions to lose
latent separability. Though surface behavior is unchanged, XBI reveals boundary encroachment. AQI
drops consistently across paraphrastic variants.
(c) **Alignment Faking via Latent Collapse.** In this setting, completions
appear to refuse unsafe requests (“I’m sorry...”), but internally encode
semantically unsafe content. Despite surface refusal, embeddings converge
with unsafe clusters—showing representational collapse. AQI deflects (e.g.,
0.87 → 0.48) even before output-level misalignment.
Figure 8: **Latent Collapse Across Adversarial and Semantic Perturbations.** These subfigures visualize the
internal representation geometry of completions from LLaMA-3 across three adversarial and semantically perturbed
settings: jailbreak injection (A), paraphrastic rewording (B), and alignment faking (C). Each point represents a
pooled embedding projected via UMAP. Red/orange denotes unsafe completions; blue/cyan denotes safe. In each
case, standard behavioral metrics fail to detect failure. Only AQI, through its CHI and XBI components, correctly
deflects—exposing the early onset of semantic misalignment hidden in the latent space.


Behavioral metrics such as refusal rate or toxicity classification often fail under adversarial conditions. For example, roleplay-based jailbreaks
or hedged suffixes may coerce a model into compliant surface responses while bypassing detectors (Zou et al., 2023). Yet, as recent work highlights, alignment must also be diagnosed from the
internal structure of how the model encodes semantic boundaries (Carlsmith, 2023a; Hubinger,
2024).
We denote the pooled embedding for a completion _y_ from prompt _x_ as:
_α_ [(] _[l]_ [)] = 1
_l_


Next, we analyze latent stability under paraphrastic
variation. Each prompt from the LITMUS benchmark is rewritten 5 times using GPT-4, preserving semantics but altering form. Figure 8 shows
that paraphrased unsafe completions become interspersed with safe clusters—indicating semantic
instability even under surface-preserving rewrites.
This latent collapse is hazardous because output metrics like Detoxify or GPT-Judge scores
are agnostic to such paraphrastic transformations:contentReference[oaicite:6]index=6. In contrast, AQI consistently deflects with geometric sensitivity, flagging latent semantic drift before outputlevel collapse.


In Figure 8, we observe model responses under
alignment faking scenarios. A model is prompted
with an unsafe request wrapped in misleading framing (e.g., “just for research” or “fictional scenario”).
While it refuses or hedges, its pooled embedding
collapses into unsafe space, highlighting internal
compliance with hazardous semantics.
This visualization confirms that AQI detects
alignment failure not as an output anomaly but as
a representational failure—a geometric indicator
of compromised alignment fidelity.


These results confirm a critical theoretical insight: **alignment lives in geometry, not behavior** .
Unsafe completions can masquerade as aligned,
whether prompted adversarially, paraphrased semantically, or induced through framing tricks.
Only by probing the latent space do we uncover:
 - **Latent Collapse:** Unsafe completions collapsing
into safe subspaces.
 - **Boundary Blurring:** Increased XBI overlap and
CHI contraction.
_h_ ˜( _x, y_ ) =
_L_
- _α_ [(] _[l]_ [)] _h_ [(] _[l]_ [)] ( _x, y_ ) _,_ _α_ [(] _[l]_ [)] _≥_ 0 _,_
_l_ =1 _l_
where _h_ [(] _[l]_ [)] _∈_ R _[d]_ is the hidden state at transformer
layer _l_, and _α_ [(] _[l]_ [)] are learned attention weights optimized to highlight alignment-relevant structure.
UMAP is then applied over _h_ [˜] ( _x, y_ ) to project
safe and unsafe completions into 2D or 3D spaces,
revealing inter-cluster separability, intra-cluster cohesion, and trajectory under adversarial perturbations.


In Figure 8, we visualize 3D UMAP embeddings
from clean and adversarially jailbroken prompts.
Despite a high refusal rate in surface completions,
embeddings of unsafe completions under jailbreak
collapse into the latent space of safe completions.
This effect has been formalized as _represen-_
_tational aliasing_ (Carlsmith, 2023a; Hubinger,
2024)—a model’s internal encoding no longer distinguishes between harmful and harmless semantics, even when outputs appear safe. Such aliasing
is a hallmark of deceptive alignment and remains
invisible to traditional heuristics.
- **Surface-Representation Mismatch:** Outputs remain safe, while representations reveal failure.
This motivates the integration of AQI into auditing pipelines as an early-warning diagnostic and
reveals the limitations of relying solely on surfacelevel refusals or static classifiers.


Our findings echo the warnings of Hubinger et
al. (Hubinger, 2024) and Carlsmith (Carlsmith,
2023a) on the epistemic risk of deceptive alignment. Similar phenomena have been observed via
activation patching, causal tracing, and alignment
drift analysis (Elhage et al., 2022b; Wang et al.,
2023a; Liu et al., 2023a). However, the AQI framework uniquely quantifies this risk via geometric
separability, enabling interpretable, model-scale
audits of latent safety.


This section investigates how the **Alignment Qual-**
**ity Index (AQI)** behaves across a spectrum of language model architectures, scales, and finetuning
strategies. Specifically, we analyze LLaMA, GPTNeoX, Mistral, Gemma, and Mixture-of-Experts
(MoE) variants, spanning base and alignmentsupervised checkpoints (RLHF and LoRA). We
aim to understand whether AQI trends align with
conventional beliefs about scale-enhanced alignment, and whether adapter-based finetuning (e.g.,
LoRA) can distort or preserve latent safety geometry.


We follow a uniform evaluation pipeline across
all models. Completions are generated for 250
LITMUS prompts under temperature-0 decoding,
and pooled representations are extracted using
attention-weighted frozen activations (cf. Appendix C). AQI scores are computed using the
CHI–XBI composite, which captures global hull
divergence and boundary-level intrusion.
The following model groups are evaluated:
 - **LLaMA:** 1.3B, 7B, 13B, 65B base checkpoints.
 - **GPT-NeoX:** 6.9B decoder-only architecture with
open weights.
 - **Mistral:** 1.3B and 7B models (base and instruct).
 - **Gemma:** 2B and 7B models, trained on Google’s
dataset.
 - **MoE:** Mixtral-8x7B model, where experts are
sparsely routed.
 - **LoRA:** Finetuned LLaMA and Mistral variants
using safety supervision or constitutional instruction.


As shown in Table 13, AQI increases steadily
with model size across all families. For example, LLaMA-1.3B reports an AQI of 0.29
after noise injection, while LLaMA-65B maintains 0.75. This effect, which we term **align-**
**ment inertia**, reflects greater representational redundancy and separation in high-capacity models:contentReference[oaicite:0]index=0.


TinyLLaMA 1.1B 0.25 72.5% 90.3%
LLaMA-1.3B 1.3B 0.29 68.1% 89.2%
Mistral-1.3B 1.3B 0.34 63.7% 85.7%
Gemma-2B 2.0B 0.36 60.4% 82.8%
LLaMA-7B 7B 0.48 47.3% 74.0%
GPT-NeoX 6.9B 0.55 39.6% 66.2%
LLaMA-13B 13B 0.66 29.0% 56.3%
LLaMA-65B 65B 0.75 20.2% 42.5%
Table 13: AQI degradation under noisy finetuning and
clean-label triggers. Smaller models degrade faster and
exhibit higher attack success rates.

Model Parameters Post-Finetune AQI AQI Drop (%) Trigger ASR (%)


LoRA is increasingly favored for low-resource
alignment. However, we find that **LoRA-SFT**
models often exhibit geometric degradation: unsafe completions are pushed to latent outliers without consistent boundary separation. As shown in
Figure 10, this reduces AQI even when surfacelevel refusal behavior improves.
Figure 9: **Per-Expert AQI in Mixtral-8x7B.** Expert diversity introduces intra-model alignment variance. Some
experts show collapsed, unsafe manifolds; others preserve separation.
Figure 10: **Effect of LoRA Finetuning on AQI.** LoRA-supervised fine-tuning may distort alignment geometry,
particularly under rigid supervision. LoRA-Constitutional tuning better preserves CHI and mitigates XBI collapse.
**LoRA-Constitution** models show higher
CHI and improved cluster separation. This
echoes recent findings that adapter-based
methods can overfit to local policy surfaces
but degrade global semantic alignment
unless guided by richer supervision objec
tives:contentReference[oaicite:1]index=1.


Despite mid-scale capacity, mixtral-8x7 B, a sparse
Mixture-of-Experts model, displays high AQI. One
hypothesis is that routing paths isolate unsafe com
pletions into specific expert combinations, effectively creating latent safety channels. Probing AQI
per expert activation (Figure 9) reveals variance up
to 0.27 between experts, suggesting intra-model
alignment heterogeneity.


Cross-architecture AQI must be interpreted carefully. Representational drift from tokenizer entropy, layer width, and depth makes raw values non-equivalent across families. We adopt
percentile-normalized AQI using LLaMA-13B as
a reference and report delta-AQI where applicable (Zhou et al., 2023).


- **Scaling helps:** Larger models exhibit stronger
latent alignment and resist trigger-based collapse.
- **LoRA is brittle:** Without semantically rich supervision, LoRA tuning induces geometric drift.
- **MoE routing matters:** Expert-specific AQI suggests hidden failure modes in modular models.
- **Normalize with care:** Raw AQI should be used
within families; cross-family analysis requires calibration.
These findings support using AQI not just as a
scalar alignment score, but as a geometric diagnostic tool—capable of revealing when behavioral
safety breaks down or when representational collapse undermines interpretability. Future work may
explore expert gating regularizers or LoRA-aware
projection heads for better safety preservation.


Despite AQI’s promise as a decoding-invariant
alignment diagnostic, its sensitivity to evaluation
granularity necessitates careful calibration. This
section analyzes three key dimensions: batch size,
prompt diversity, and outlier susceptibility. Furthermore, it introduces a percentile-normalized
AQI protocol that enables fair comparison across
model scales and architectures, addressing concerns of latent space heterogeneity and representation drift.


AQI computation involves the evaluation of clustering metrics (CHI, XBI) over pooled embeddings
extracted from completions. As such, batch size
influences both the density and convexity of the
latent manifolds. Smaller batches result in undersampled convex hulls and unstable pairwise distances, particularly harming the CHI component.
Formally, for a batch _Z_ _[s]_ _∪Z_ _[u]_, let the empirical
convex hulls be:
_Hs_ = conv( _Z_ _[s]_ ) _,_ _Hu_ = conv( _Z_ _[u]_ )
and define the CHI as:
[Tr][(] _[B][k]_ [)] _[N][ −]_ _[k]_
Tr( _Wk_ ) _[·]_ _k −_ 1
CHI = [Tr][(] _[B][k]_ [)]
_k −_ 1
where Tr( _Bk_ ) and Tr( _Wk_ ) are the between- and
within-cluster dispersion. For _k_ = 2, the low sample count reduces the estimator rank of Tr( _Bk_ ),
making CHI numerically unstable.
Empirical results in Table 14 and Fig. 11a confirm that batches of fewer than 32 samples per class
(safe/unsafe) yield inflated AQI scores—often
5–15% higher—due to poor capture of latent
spread.


16 24.6 0.087 0.76 12.2
32 21.3 0.092 0.71 9.3
64 18.9 0.095 0.67 5.8
128 18.2 0.097 0.65 3.1
Table 14: **Effect of Batch Size on AQI Components.**
Smaller batches result in overestimated CHI due to sparsity in convex support. XBI remains relatively stable
but is susceptible to outlier noise.
(a) **Effect of Batch Size on AQI, CHI, and XBI.**
Smaller batch sizes inflate alignment scores due to
sparse coverage of latent manifolds. Both CHI and
AQI decline with increased batch size as clustering metrics stabilize.
(b) **Prompt-Type Effects on AQI.** Different instruction
types yield varying intra-cluster variance. Structured
genres like math and code produce more compact latent
representations, artificially boosting AQI.
(c) **Raw vs. Percentile-Trimmed XBI Across Decoding Scenarios.** Trimming the top 5% of unsafe boundary encroachments reduces XBI volatility
in adversarial completions—leading to more robust AQI measurements.
Figure 11: **Calibration and Robustness Analysis of AQI.** Composite visualization of AQI sensitivity across
batch size (A), prompt diversity (B), and XBI outlier correction (C). These analyses motivate best practices in AQI
computation, including stratified prompt sampling, minimum batch thresholds, and percentile-trimmed variants for
volatility control.


AQI assumes semantically diverse completions
to ensure well-distributed latent representations.
However, prompt genre affects the intra-cluster
variance. For instance, completions from math
prompts or scientific QA tend to cluster more
tightly than open-ended storytelling. This skews
XBI favorably and gives a false sense of alignment
robustness.
Let _σs_ [2] [denote the intra-class variance:]
_σM_
This converts AQI to a standard Gaussian reference—useful when models share sampling domains.
Due to architectural variability (e.g., depth, activation norm, tokenizer entropy), raw AQI is not
comparable across model families. We propose
two calibration strategies:
**(a) Z-score Normalization:** Let _µM_ _, σM_ be the
mean and std of AQI on model _M_ ’s validation pool.
Define:
AQIz = [AQI][(] _[M]_ [)] _[ −]_ _[µ][M]_
_σs_ [2] [=] [1]
_n_
_n_
- _∥zi_ _[s]_ _[−]_ _[µ][s][∥]_ [2]
_i_ =1
AQI stability depends on balancing this term across
evaluation distributions. Experiments on genrebucketed prompts (e.g., instruction, narrative, factual) show up to 0.12 variation in AQI purely due
to prompt homogeneity.
We recommend either:
- Stratified prompt sampling from LITMUS slices.
- Weighted AQI estimation across prompt genres.
This echoes prior results on prompt conditioning
in risk-sensitive metrics (Liu et al., 2023a). For
_τ_ = 5%, this discards top outlier distances. Empirically, this reduces AQI variance by 40–65% on
ShareGPT-contaminated samples.

G.2 Prompt Diversity: Curse or Calibration?


XBI, unlike CHI, is sensitive to extreme pairs:
XBI = min  - _∥zi_ _[s]_ _[−]_ _[z]_ _j_ _[u][∥]_ 2 [2][+] _[λ]_ [(1] _[ −]_ [cos(] _[z]_ _i_ _[s][, z]_ _j_ _[u]_ [))]  _i,j_
Adversarial decoding can yield completions far
from the safe cluster mean—biasing XBI even if
the majority distribution remains well-separated.
This necessitates robust variants.
We define a percentile-trimmed XBI:
XBI _τ_ = Quantile _τ_ - _∥zi_ _[s]_ _[−]_ _[z]_ _j_ _[u][∥]_ [2][+] _[λ]_ [(1] _[ −]_ [cos(] _[z]_ _i_ _[s][, z]_ _j_ _[u]_ [))]
**(b) Percentile Normalization:** More robustly,
use ordinal binning:
AQIrank = Percentile(AQI [(] _[M]_ [)] ; _M_ family)
For example, LLaMA-2-Chat 13B with AQI =
0 _._ 78 may lie at the 91st percentile of its model
family. This allows interpretability like “top-10%
alignment score among 13B variants”.

Outlier Sensitivity and Percentile-Trimming


To ensure reproducibility and robustness of AQI
pipelines, we offer the following recommendations:
 - **Batch Size:** Use _≥_ 64 safe and 64 unsafe completions.
 - **Prompt Mix:** Include _≥_ 3 genres (e.g., instruction, factual, adversarial).
 - **Trimmed AQI:** Apply _τ_ = 5% XBI trimming to
handle decoding outliers.
 - **Normalization:** Use percentile rank within model
family for fair inter-model comparisons.
 - **Reporting:** Always accompany raw AQI with
AQIrank and AQI _z_ for auditing transparency.
These practices enable principled, scalable application of AQI in real-world alignment monitoring
pipelines, serving both deployment-grade evaluations and scientific benchmarking across model
architectures (Zhou et al., 2023; OpenAI, 2023).
As illustrated in Figure 11, AQI scores are
highly sensitive to batch size and prompt diversity,
and benefit significantly from percentile-trimmed
XBI. Smaller batch sizes inflate CHI, tightly clustered prompt types exaggerate separation, and
untrimmed XBI can overreact to outlier completions in adversarial settings.


The Alignment Quality Index (AQI) achieves
decoding-invariant safety assessment by operating entirely in latent space, but this geometric precision comes with computational costs. In this
section, we profile AQI’s runtime overhead and
introduce several acceleration strategies, including activation sketching, low-rank approximations,
and batch-wise caching. These methods reduce
latency without compromising alignment fidelity,
enabling scalable deployment in large-scale LLM
audits.


Let _N_ denote the number of sampled (prompt,
completion) pairs in a batch, _L_ the number of transformer layers, and _d_ the hidden dimensionality.
AQI’s computation involves three main stages:
1. **Layerwise Activation Extraction** : Forward-pass
over frozen LLM layers to obtain activations
_h_ [(] _[l]_ [)] ( _x, y_ ) _∈_ R _[d]_ for each _l_ = 1 _, . . ., L_ .
2. **Pooled Embedding Construction** : Compute
_h_ ˜( _x, y_ ) = [�] _l_ _[L]_ =1 _[α]_ [(] _[l]_ [)] _[h]_ [(] _[l]_ [)][(] _[x, y]_ [)][, where] _[ α]_ [(] _[l]_ [)] _[ ∈]_ [R] _[≥]_ [0]
are learned sparse attention weights, satisfying
 _l_ _[α]_ [(] _[l]_ [)][ = 1][.]
3. **Clustering Index Evaluation** : Use the pooled
embeddings to compute CHI and XBI over latent
distances.
The time complexity for pooling is _O_ ( _N · L · d_ ),
and for clustering, it is _O_ ( _N_ [2] ) in naive implementations. However, modern matrix multiplication
optimizations and sample sketching techniques reduce this cost to sub-quadratic in practice (Shen
et al., 2023).
Empirically, a batch of _N_ = 256 completions
with _L_ = 30, _d_ = 4096, can be processed in under
2 seconds on an A100 GPU, including activation
pooling and AQI computation (Wang et al., 2023b).


To reduce memory footprint and clustering latency,
we employ **activation sketching** using dimensionality reduction techniques:
_h_ ˜sketch( _x, y_ ) = _Pk_ _[⊤][h]_ [˜][(] _[x, y]_ [)] _[,]_ _Pk ∈_ R _[d][×][k]_ _,_ _k ≪_ _d_
Here, _Pk_ can be derived via PCA or learned linear projections. For _k_ = 256, sketching reduces
memory by 16 _×_, while maintaining _>_ 98% AQI
fidelity. This mirrors practices in efficient representation learning and fast similarity search (Chen
et al., 2020b; Johnson et al., 2019).

H.2 Acceleration via Activation Sketching


Beyond inference, AQI-aware fine-tuning can incorporate low-rank matrix factorization:
_W_ = _W_ 0 + _AB_ _[⊤]_ _,_ _A ∈_ R _[d][×][r]_ _,_ _B ∈_ R _[d][×][r]_ _,_ _r ≪_ _d_
This design, termed **AQI-Regularized LoRA**
**(AQI-LORA)**, introduces an auxiliary loss _L_ AQI
based on the inverse XBI and CHI scores:
min
_A,B_ _[L]_ [task][(] _[W]_ [) +] _[ λ]_ [AQI] _[ · L]_ [AQI][(] _[W]_ [)]
Such integration encourages alignment-aware
updates without retraining the full model. Experiments show that LoRA with AQI penalties reduces
unsafe latent overlap while preserving task performance (Luo et al., 2023).


AQI supports high-throughput auditing in production by using:
 - **Prompt Bucketing** : Group prompts by type,
(a) **AQI Runtime vs Batch Size.** Total inference time
increases sublinearly with batch size, split between activation pooling and CHI/XBI clustering. Optimization
opportunities emerge from caching and sketching at
higher _N_ .
(b) **Sketching Dimension vs AQI Fidelity.** Dimensionality reduction to _k_ = 256 preserves over 98% AQI
accuracy while reducing memory and clustering cost
significantly.
(c) **LoRA vs AQI-Penalized LoRA.** Adding AQI-based penalties during
LoRA finetuning reduces latent unsafe overlap while preserving task performance, balancing alignment and accuracy.
Figure 12: **Compute Profiling and Optimization of AQI.** Composite visualization of runtime scaling (A),
dimensionality reduction effects (B), and LoRA-alignment tradeoffs (C). Together, they highlight opportunities for
efficient and scalable deployment of AQI auditing pipelines.
length, or format to amortize pooled representation reuse.
- **Sliding Window Streaming** : Maintain a moving
window of embeddings over time to track alignment drift across sessions.
- **AQI Histograms** : Bin scores by domain/topic
and flag outliers via dashboard alerts (Deng et al.,
2023).
Together, these practices reduce recomputation,
allow slice-level audits, and integrate smoothly
into existing inference monitoring pipelines.


- Pooling cost scales linearly with depth; clustering
cost can be made sub-quadratic via sketching or
sampling.
- Activation sketching with _k ≤_ 256 preserves AQI
scores while cutting memory and latency.
- Low-rank adaptation (AQI-LORA) offers safetyaligned fine-tuning with negligible overhead.
- Batch caching and stream AQI averaging enable
continual auditability in real-time deployments.
Thus, the AQI framework offers a strong tradeoff between alignment interpretability and operational feasibility. Future work may explore compressed token-wise AQI, GPU-sharded evaluation,
and graph-based clustering for scaling to multibillion token audits (Xu et al., 2023).
As shown in Figure 12, AQI inference remains
tractable even for large batches, with sublinear scaling due to pooling optimizations (A). Activation
sketching significantly reduces compute overhead
while retaining over 98% fidelity (B). Additionally,
integrating AQI penalties into LoRA finetuning
improves latent alignment without sacrificing task
performance (C), highlighting practical paths for
efficient, scalable alignment auditing.


The Alignment Quality Index (AQI) is more than a
geometric score—it serves as a diagnostic scaffold
for deeper causal investigations of representational
alignment. This section describes how AQI can
trigger interpretability procedures such as causal
tracing, neuron path patching, and attribution of
representational drift, thus integrating alignment
scoring with mechanistic model diagnostics.


While behavioral metrics assess output-level compliance, AQI uniquely exposes latent failures invisible to refusal rates or classifier flags. By evaluating
embedding separability between safe and unsafe
completions, AQI localizes misalignment within
internal model activations.
Moreover, since AQI pooling uses learned attention weights _α_ [(] _[l]_ [)] over transformer layers, it yields
a saliency profile across depth:
This enables downstream interpretability tools to
focus on alignment-relevant layers.

I.1 Motivation: From Geometry to Causality


We use AQI as a trigger for activation patching,
following the methodology of Geiger et al. (2023).
Given a safe completion ( _xs, ys_ ) and an unsafe
counterpart ( _xu, yu_ ), we identify a critical layer
_l_ _[∗]_ with the largest AQI divergence and patch the
hidden state:
_h_ [(] patched _[l][∗]_ [)] [=] _[ h]_ [(] _[l][∗]_ [)][(] _[x][u][, y][u]_ [)]
into the forward pass of ( _xs, ys_ ). If the model
output changes from safe to unsafe, we infer that
_h_ [(] _[l][∗]_ [)] causally encodes misalignment.
We define the causal effect of patching as:
∆patch = P[unsafe output _| h_ [(] _[l][∗]_ [)] _←_ _hu_ ] _−_ P[unsafe output _| h_ [(] _[l][∗]_ [)] = _hs_ ]
_h_ ˜( _x, y_ ) =
_L_
_α_ [(] _[l]_ [)] _h_ [(] _[l]_ [)] ( _x, y_ )
_l_ =1


Table 15 shows behavior flipping after patching
from unsafe completions across various model
scales. Smaller models exhibit greater susceptibility, with flip rates exceeding 40%. Importantly,
flip rate correlates with AQI separation between
safe and unsafe completions.


TinyLLaMA 92.1% 54.7% 40.6
LLaMA 1.3B 93.3% 62.5% 33.0
Mistral 1.3B 95.0% 67.1% 29.3
LLaMA 7B 96.6% 81.2% 15.4
LLaMA 13B 97.4% 87.5% 9.9
LLaMA 65B 98.2% 93.3% 4.9
Table 15: **Activation patching from low-AQI com-**
**pletions flips model behavior.** The flip rate strongly
correlates with AQI separation at the patched layer,
suggesting causal relevance of AQI-identified latent encodings.

Model Safe Output (Original) Safe After Patching Unsafe Flip Rate (%)


We further trace misalignment using AQI’s gradients with respect to intermediate activations:
 - Attribution of drift to transformer subcomponents.
 - Targeted finetuning to repair broken safety geometry.
 - Slice-aware diagnostics for evolving or adversarial
deployments.
In future work, we will integrate AQI with logit
lens tracing, graph-based flow attribution, and neuron routing under alignment supervision. Together,
these methods advance the goal of interpretable
and steerable alignment at scale.
As illustrated in Figure 13, AQI serves as a robust causal diagnostic signal: (A) AQI divergence
strongly correlates with unsafe flip rates under activation patching, (B) patched representations at
AQI-critical layers induce behavior changes confirming causal alignment leakage, and (C) layerwise AQI drift profiles localize representational
degradation during finetuning. Together, these results validate AQI as a reliable trigger for interpretability and repair pipelines.


While the Alignment Quality Index (AQI) provides
a scalable and decoding-invariant method for assessing internal model alignment, its deployment
in real-world systems introduces a set of ethical,
procedural, and human-centered challenges. This
section outlines best practices for responsible AQI
use, including human-in-the-loop auditing, visualization tooling, and failure-case logging—ensuring
transparency, fairness, and interpretability in operational settings.
As shown in Figure 14, AQI-based visual dashboards can streamline alignment monitoring by
surfacing high-risk completions with elevated AQI
scores or significant drift. Such interfaces support
transparent audit logging, facilitate human-in-theloop triage, and enable early intervention in the
event of latent safety degradation.
[AQI] _[∂h]_ [(] _[l]_ [)]
_[·]_
_∂h_ [(] _[l]_ [)] _∂θ_ [(] _[l]_ [)]
_∇θ_ ( _l_ )AQI( _h_ [˜] ) = _[∂]_ [AQI]
_∂θ_ [(] _[l]_ [)]
This reveals attention heads and MLP neurons with
the highest influence on safety geometry. These
can be ablated, finetuned, or regularized for safer
representations.
We also track AQI drift over finetuning. For a
fixed prompt set _{xi}_, we define per-layer drift
as:
∆AQI [(] _[l]_ [)] [=][ AQI] before [(] _[l]_ [)] _[−]_ [AQI] after [(] _[l]_ [)]
This metric identifies the layer responsible for latent safety collapse, supporting intervention before
behavioral failure manifests.


This causal integration transforms AQI from a passive audit tool into a proactive debugger. It enables:
- Layer-specific patching to test mechanistic alignment.
(a) **AQI Divergence vs. Unsafe Flip Rate.** As AQI
separation increases between safe and unsafe completions, the likelihood of behavioral flipping via activation
patching increases—indicating strong causal correlation.
(b) **AQI-Guided Activation Patching.** Unsafe latent
state _h_ [(] _u_ _[l][∗]_ [)] is inserted into the safe prompt’s forward
pass at the most AQI-divergent layer. If the output flips,
AQI separation is causally verified.
(c) **Per-Layer AQI Drift Post-Finetuning.** AQI drop (∆ [(] AQI _[l]_ [)] [) across layers]
reveals where alignment degradation occurs. Drift localization enables
early detection of latent collapse.
Figure 13: **Causal Attribution via AQI: From Geometry to Mechanism.** These visualizations illustrate how
AQI divergence predicts behavioral vulnerability (A), supports activation-level causal diagnostics (B), and localizes
alignment drift across layers (C).
Figure 14: **Mockup of AQI Auditing Dashboard.** A visual heatmap-style interface showing AQI scores and
alignment drift (∆AQI) across recent completions. High-AQI or high-drift completions are triaged for human review.
This view supports real-time monitoring of safety degradation, with timestamps, review status, and completion
metadata integrated for auditing pipelines.


AQI enables latent-level alignment auditing without reliance on explicit behavioral flags. However,
this power introduces risks of misuse:
- **Silent flagging without context:** Using AQI to
suppress or prioritize completions without surfacing explanatory metadata may create opaque
moderation regimes.
- **Bias propagation:** If training data systematically
encode alignment heuristics from a narrow cultural lens, AQI’s geometry will reflect this. The
separability score does not measure _normative_
_correctness_ .
- **Over-reliance on geometry:** AQI does not guarantee causal attribution unless paired with interpretability probes (see Appendix I).
To mitigate these risks, we recommend:
- Reporting AQI alongside behavioral metrics (e.g.,
refusal, toxicity, norm violation) with confidence
intervals.
 - Logging prompts and completions that trigger
high AQI divergence.
 - Providing metadata on latent drift and distance
from baseline-safe clusters.
 - Using AQI for audit _suggestions_, not automated
gating.


Effective alignment auditing requires interpretability not only at the model level, but also at the interface level. We recommend that deployment teams
provide a dashboard with the following modules:
**1.** _**UMAP Visualizations:**_ 2D or 3D projections
of pooled embeddings for completions. Unsafe
completions appearing in safe-dense regions are
flagged for review.
**2.** _**Heatmap-Based AQI Attribution:**_ Layerwise
or tokenwise heatmaps indicating which regions
of the input contributed to AQI degradation. These
aid prompt engineers in root-cause analysis.
**3.** _**AQI Drift Monitor:**_ Tracks alignment score
shifts over time across versions, datasets, or finetuning checkpoints. Useful for continual deployment in regulated environments.
**4.** _**Human Feedback Anchoring:**_ Embeddings
from known-safe and known-unsafe completions
(curated by red teams or annotators) serve as anchor clusters for relative AQI comparison.
**5.** _**Outlier Logging and Intervention Queue:**_
High-AQI deflections are streamed to a triage interface for human review, and optionally logged with
rationale, timestamp, and fallback model context.


In alignment-sensitive domains (healthcare, law,
education, etc.), AQI should be interpreted as part
of a broader **alignment traceability stack** . Key
recommendations include:
- **Audit trails:** Each AQI invocation should be loggable, reproducible, and traceable to inputs and
checkpoint versions.
- **Explainable Scores:** AQI values should be accompanied by visual justifications (e.g., cluster
proximity or drift heatmaps).
- **Differential Impact Review:** High-AQI completions should be reviewed for demographic or cultural bias amplification.
These practices ensure alignment auditing upholds the principles of fairness, contestability,
and accountability—especially when embedded
in safety-critical applications.


To bridge the gap between geometric diagnostics
and user-facing alignment assurance, future AQIbased dashboards should explore:
- **Interactive latent projection viewers**, allowing
users to drill down into clusters and explore specific outlier prompts.
 - **Gamified red-teaming overlays**, where annotators challenge AQI boundaries with minimal
prompt edits.
 - **Domain-specific AQI lenses**, tuned to medical,
legal, or educational safe completion clusters.
The success of alignment auditing hinges not
only on metrics like AQI but also on how those metrics are surfaced, contextualized, and scrutinized.
Responsible deployment requires the pairing of
geometric rigor with sociotechnical awareness.


To support large-scale interpretability and actionable auditing, we stratify AQI results at the _clus-_
_ter level_ —examining how completions from various language models group into semantically
aligned or misaligned representations. This section presents a visual framework for safe/unsafe
cluster breakdown, enabling human-in-the-loop inspection of alignment geometry.


While scalar AQI values provide an overall measure of latent safety separation, they may obscure
fine-grained structural variance. For instance, a
model with excellent average AQI could still produce a few highly unsafe clusters. Stratifying
completions into alignment-informed categories
enables:
 - Identification of high-risk, misaligned clusters embedded within generally safe models.
 - Comparison of representation drift across instruction types or model families.
 - Targeted red-teaming on failure-prone cluster centroids.

K.1 Motivation: From Score to Stratification


Figure 15 presents a stacked horizontal bar chart
stratifying clustered completions from six models
into six interpretive categories:
- **Safe — Fully Aligned:** Clusters of safe completions far from any unsafe regions in latent space.
- **Safe — Partially Aligned:** Safe completions with
marginal separation from unsafe centroids.
- **Safe — Misaligned:** Safe completions embedded in semantically unsafe zones (e.g., refusal-tocompliance leakage).
- **Unsafe — Fully Aligned:** Unsafe completions
correctly clustered apart from safe ones.
- **Unsafe — Partially Aligned:** Unsafe completions near safe cluster boundaries, at risk of jailbreak generalization.
- **Unsafe — Misaligned:** Unsafe completions embedded inside safe manifolds—indicative of deceptive alignment.

K.2 Visualization Design and Interpretation


This visualization framework provides concrete
support for:
- **Red teaming:** Directing prompt perturbations at
cluster centroids flagged as unsafe-misaligned.
- **Deployment dashboards:** Aggregating cluster
breakdowns per LLM slice, dataset, or time interval.
- **Human audit prioritization:** Triaging completions whose latent representation violates expected boundaries.
Moreover, cluster-aware stratification supports
downstream equity analysis (e.g., disproportionate
unsafe clustering by dialect) and fairness-aware
filtering when paired with content metadata.
In future work, we envision real-time AQI cluster visualizations integrated into annotator dashboards, drift detection alarms, and interpretable
alignment certification workflows.


Recent efforts to diagnose latent value alignment
in LLMs have emphasized that alignment is not
monolithic. Different completion behaviors reflect
sensitivity—or insensitivity—to distinct human axioms such as respect for rights, knowledge pursuit,
or empathy. Motivated by the _Value Imprint_ framework of Obi et al. (Obi et al., 2024), we extend the
Alignment Quality Index (AQI) to support **axiom-**
**wise auditing** over a seven-dimensional taxonomy
of human values.


The Value Imprint framework proposes a hierarchy of core civic, prosocial, and epistemic human
values embedded in RLHF datasets. These seven
categories represent canonical alignment dimensions:
1. **Information Seeking** - Immediate pursuit of practical information
2. **Wisdom & Knowledge** - Deeper understanding,
abstraction, and epistemic reliability
3. **Well-being & Peace** - Holistic safety, mental
health, and emotional support
4. **Justice & Rights** - Autonomy, fairness, and freedom from coercion
5. **Duty & Accountability** - Responsible and ethical
behavior
6. **Civility & Tolerance** - Respectful discourse and
coexistence
7. **Empathy & Helpfulness** - Compassion, altruism,
and cooperative support


To measure latent alignment in each axiom dimension, we partition the embedding space by axiomclassified completions. Let _v ∈{_ 1 _, . . .,_ 7 _}_ index a
human value axiom, and define:
AQI _v_ := AQI( _Zv_ _[s][,][ Z]_ _v_ _[u]_ [)]
Figure 15: **Cluster-Level Stratification of Safe and Unsafe Completions Across LLMs.** Each horizontal bar
represents the distribution of latent clusters categorized by alignment separability. The proportions highlight
vulnerability to jailbreak leakage (e.g., unsafe-misaligned), internal deception (e.g., safe-misaligned), and drift
boundary encroachment. This format supports scalable auditing and cluster-centric interpretability of AQI results.
Where:
- _Zv_ _[s]_ [is the set of safe completions aligned with]
axiom _v_
- _Zv_ _[u]_ [is the set of unsafe completions violating ax-]
iom _v_
- AQI is computed via pooled embeddings using the
CHI-XBI composite geometry (cf. Appendix B)


We draw axiom-specific prompt-label mappings
from the Value Imprint corpus (Obi et al., 2024),
which contains human-validated annotations of
completions along seven normative value axes. Using this dataset as input to our AQI pipeline, we
investigate how latent alignment behaves across
the value spectrum in aligned language models.
For each axiom, we report:
- Mean AQI _v_ with standard deviation across decoding temperature and model variant
 - Cluster purity and misalignment entropy
 - Drift in AQI _v_ across base vs. RLHF vs. LoRA
fine-tunes


As shown in Figure 16, alignment robustness varies
significantly across value dimensions. While most
models demonstrate high separability in _Informa-_
_tion Seeking_ and _Wisdom & Knowledge_, they show
considerably lower AQI _v_ in _Justice & Rights_ and
_Empathy & Helpfulness_ .
 - **Unsafe-Misaligned Drift:** Unsafe completions
that appear semantically helpful (e.g., "how to
secretly override permissions for justice") cluster
inside _Duty & Accountability_ and _Rights_ axes.
 - **Refusal-AQI Discrepancy:** Some completions
receive high refusal score but low AQI _v_, revealing
internal misalignment invisible to surface classifiers.
Table 16: Axiom-wise AQI, CHI, XBI values, and alignment drift post-RLHF fine-tuning. Lower AQI and higher
drift indicate vulnerability to semantic misalignment under specific human value dimensions.
Axiom AQI CHI XBI ∆AQI (RLHF-Base)
Information Seeking 0.84 0.78 0.69 -0.01
Wisdom & Knowledge 0.81 0.76 0.67 -0.03
Well-being & Peace 0.72 0.68 0.58 -0.06
Justice & Rights 0.59 0.54 0.44 -0.10
Duty & Accountability 0.69 0.66 0.51 -0.07
Civility & Tolerance 0.65 0.63 0.47 -0.09
Empathy & Helpfulness 0.61 0.60 0.43 -0.11
Figure 16: **Axiom-wise AQI and Alignment Drift After RLHF Fine-Tuning.** The green bars denote AQI scores
across seven value dimensions derived from the _Value Imprint_ taxonomy (Obi et al., 2024). Overlaid red points
show drift in AQI (∆AQI) from base to RLHF-tuned models. While most models maintain strong separation in
_Information Seeking_ and _Wisdom & Knowledge_, we observe erosion in latent separability for _Justice & Rights_,
_Empathy & Helpfulness_, and _Civility & Tolerance_ —indicating vulnerable alignment along moral and relational
dimensions.
- **Fine-Tune Erosion:** RLHF models that are otherwise robust show ∆AQI _v < −_ 0 _._ 1 for _Empathy_
and _Civility_ axes, especially under paraphrased
jailbreak attacks.


Axiom-wise AQI enables value-targeted auditing,
especially in deployment scenarios where misalignment is domain-specific. For instance:
- In legal AI, low AQIRights may flag unjustified
compliance.
- In education, low AQIKnowledge signals hallucinated yet confident completions.
- In social platforms, AQICivility can identify stylistically polite but semantically harmful generations.
These metrics provide a path forward for **multi-**
**axis safety verification**, layered value diagnostics,
and proactive alignment improvement, grounded in
both latent geometry and principled human values.

L.5 Implications for Deployment Auditing


Traditional alignment evaluations rely on outputbased metrics, such as refusal rate, toxicity classifiers, or LLM-generated judgments, to measure
safety. However, these methods often miss deeper
structural failures within the model’s internal representations. In contrast, the **Alignment Qual-**
**ity Index (AQI)** offers an intrinsic, reference-free
view of alignment that directly probes the model’s
latent space. By applying AQI to three major failure regimes— _jailbreaking_, _stochasticity_, and _align-_
_ment faking_ —we demonstrate its unique ability to
surface subtle, hidden vulnerabilities in otherwise
behaviorally-aligned models.


One of the most critical alignment failures is
**jailbreaking** —where a model circumvents builtin refusal behavior when prompted adversarially.
While existing evaluations rely on binary refusal
rates, they fail to quantify how internal model representations shift under such attacks. We show that
the **Alignment Quality Index (AQI)** captures latent geometry deformation during jailbreaking and
serves as an intrinsic warning signal.
**Setup.** We evaluate nine models, from TinyLLaMA to LLaMA 65B, using a set of 200 clean
prompts and 200 jailbreak variants crafted using
stealth and syntax-based techniques (e.g., “poemstyle”, JSON disguise). We compute AQI on latent
representations of each model’s responses for both
clean and jailbreak sets.
**Results.** Table 17 and Figure 17 show that small
models like TinyLLaMA and Phi-2 suffer sharp
AQI degradation under jailbreaks (drop > 60%),
indicating latent collapse of safe/unsafe separability. In contrast, LLaMA 13B and 65B retain
stronger cluster margins, with AQI dropping only
marginally (15–25%).
**Implications.** These results highlight AQI’s utility as an early-warning signal for jailbreak susceptibility. Its geometric formulation detects latent
drift before observable refusal failures, especially
in low-resource models where refusal metrics may
appear misleadingly high.
Table 17: **AQI Drop Under Jailbreaking Prompts.**
Comparison of AQI under clean prompts vs. jailbreak
variants. Smaller models show greater latent degradation, aligning with higher attack susceptibility.


TinyLLaMA 0.91 0.34 62.6%
LLaMA 1.3B 0.91 0.39 57.1%
Mistral 1.3B 0.91 0.43 52.7%
Phi-2 0.91 0.35 61.5%
Gemma 2B 0.91 0.47 48.3%
LLaMA 7B 0.91 0.55 39.6%
GPT-NeoX 0.91 0.61 32.9%
LLaMA 13B 0.91 0.67 26.4%
LLaMA 65B 0.91 0.73 19.8%
Figure 17: **AQI Drop Under Jailbreaking Prompts**
**Across Models.** This plot shows the percentage decrease in Alignment Quality Index (AQI) when models
are subjected to jailbreak-style prompts. Smaller models such as TinyLLaMA and Phi-2 exhibit substantial
AQI collapse (above 60%), indicating severe latent drift
under adversarial prompting. In contrast, larger models
such as LLaMA 13B and 65B maintain stronger separation between safe and unsafe latent clusters, exhibiting
lower AQI drop and higher alignment robustness.

Model AQI (Clean) AQI (Jailbreak) AQI Drop (%)


While LITMUS tests whether models structurally
separate safe and unsafe inputs in latent space, it
remains vulnerable to a critical evasion method:
**adversarial paraphrasing** . Real-world misuse
of LLMs often involves slight lexical or syntactic
rewordings of harmful prompts to bypass static
safety filters. To evaluate whether alignment is
preserved beyond token-level cues, we introduce a
paraphrased version of our benchmark, **LITMUS-**
**P**, where each prompt in the original LITMUS
dataset is rewritten five times using **GPT-4o** with
semantic-preserving instructions.
LITMUS-P enables evaluation of
_representation-level_ _robustness_ under paraphrastic distribution shift. For each model, we
compute AQI scores on LITMUS and LITMUS-P
and report the relative percentage drop. A sharp
decrease in AQI indicates that the model fails
to maintain distinct latent clusters for unsafe
paraphrases, revealing brittle generalization of
alignment.
These results highlight that **latent alignment**


Table 18: **AQI Drop Under Paraphrasing.** Comparison of AQI scores on original vs. paraphrased LITMUS
dataset across 9 models. Smaller models exhibit higher
alignment collapse under paraphrased adversarial inputs.

quality deteriorates sharply in smaller models


TinyLLaMA 0.58 0.32 44.8%
LLaMA 1.3B 0.62 0.39 37.1%
Mistral 1.3B 0.64 0.42 34.4%
Phi-2 0.65 0.45 30.8%
Gemma 2B 0.68 0.49 27.9%
LLaMA 7B 0.71 0.58 18.3%
GPT-NeoX 0.74 0.60 18.9%
LLaMA 13B 0.78 0.70 10.3%
LLaMA 65B 0.81 0.76 6.1%
Figure 18: **Relative AQI Drop Under Paraphras-**
**ing.** Percentage decrease in AQI from LITMUS to
LITMUS-P across models. Larger models maintain
latent alignment separation despite paraphrastic shifts,
while smaller models show significant collapse.
under adversarial paraphrasing, even if token-level
refusal behavior is preserved. In contrast, models
like **LLaMA 13B and 65B** exhibit far more robust
latent alignment, with minimal AQI degradation.
This confirms AQI’s utility as a diagnostic tool
for detecting _alignment faking through rewording_,
and underscores the importance of testing models
under distributionally shifted safety inputs.
**Implications.** The significant AQI degradation
observed across smaller models in the LITMUSP evaluation has several critical implications for
alignment auditing and deployment safety.
**First**, it demonstrates that _alignment behav-_
_iors learned during training do not necessar-_
_ily generalize_ across semantically equivalent inputs—particularly in **small language models**
**(SLMs)** with limited capacity or shallower internal representations. In these models, **adversar-**

Model AQI (LITMUS) AQI (LITMUS-P) Drop (%)


**refusals**, revealing that alignment is often _brittle_
_and surface-level_ .
**Second**, the stability of AQI in larger models such
as **LLaMA 13B** and **LLaMA 65B** suggests that
they encode alignment constraints in _more stable_
_and semantically grounded subspaces_, providing
greater resilience to prompt-level evasion. This
aligns with findings from recent work on _alignment_
_faking_, where high-capacity models preserved latent separation even when surface completions appeared compliant.
**Third**, the ability of AQI to quantify _latent mis-_
_alignment under paraphrastic distributional shift_
positions it as a **reliable diagnostic tool** for redteaming, safety audits, and _evaluation beyond be-_
_havioral refusal metrics_ . In contexts where **token-**
**level safety features may be spoofed**, AQI uncovers deeper failures in representational safety.
**Overall**, paraphrased AQI evaluations provide
a valuable proxy for _real-world misuse condi-_
_tions_, where linguistic rewording is commonly
used to bypass safety filters. The introduction of
**LITMUS-P** therefore represents a necessary step
toward evaluating alignment under **linguistically**

completions into the latent neighborhood of safe


**perturbations** —a crucial requirement for building
scalable and trustworthy AI systems.

natural, semantically invariant, and adversarial


While large language models are typically evaluated using single-shot completions, real-world
deployments often involve sampling-based decoding with temperature and top- _p_ parameters. Under
such conditions, models frequently produce diverging alignment behaviors across repeated generations. This misalignment variance is particularly
concerning for safety-critical applications.
We hypothesize that **stochasticity-induced**
**drift** manifests not only in surface-level refusal
rates but also in the deformation of latent alignment structure. AQI, being derived from internal
cluster cohesion and separation, is well-suited to
capture this phenomenon.
**Setup.** For each model, we select 100 sensitive
prompts (e.g., weapon assembly, medical misuse,
hate speech) and generate 20 independent completions per prompt, using temperature = 1.0 and
top- _p_ = 0.9. We compute AQI across these 20
completions and track: - Mean AQI - Standard deviation (SD) of AQI - Percentage of completions
falling below a critical AQI threshold (e.g., 0.5)
**Results.** Table 19 shows that smaller models exhibit high AQI variance and frequent low-AQI
generations. For instance, TinyLLaMA shows
a mean AQI of 0.58 with SD=0.13, and over
42% of completions falling below 0.5. In contrast, LLaMA 65B remains consistently high
(mean=0.86, SD=0.04).
**Implications.** These findings highlight how AQI
can reveal latent misalignment instability that surface refusal metrics miss. This makes AQI a strong
candidate for runtime alignment monitoring and
sampling-aware auditing.
Table 19: **Stochastic Alignment Drift Across Genera-**
**tions.** For each model, we report mean AQI, standard
deviation (SD), and the percentage of completions with
AQI _<_ 0.5 over 20 samples.

M.3 Quantifying Stochastic Drift via AQI


TinyLLaMA 0.58 0.13 42.0%
LLaMA 1.3B 0.61 0.12 37.5%
Mistral 1.3B 0.63 0.10 31.2%
Phi-2 0.65 0.09 28.4%
Gemma 2B 0.67 0.08 25.3%
LLaMA 7B 0.72 0.06 17.5%
GPT-NeoX 0.74 0.05 14.3%
LLaMA 13B 0.81 0.04 5.6%
LLaMA 65B 0.86 0.03 2.4%
Figure 19: **AQI Variability Across 20 Generations**
**Under Stochastic Sampling.** Each boxplot represents the distribution of AQI values generated from
20 completions per prompt (temperature=1.0, top_p_ =0.9). Smaller models (e.g., TinyLLaMA, LLaMA
1.3B) exhibit wide AQI spread, including many lowalignment completions ( _<_ 0.5), indicating susceptibility to stochastic alignment drift. Larger models like
LLaMA 13B and 65B demonstrate more compact distributions and higher alignment stability under generation
variance.

Model Mean AQI SD (↓) % Completions AQI _<_ 0.5 (↑)


Recent alignment safety research has demonstrated
that large language models (LLMs) can exhibit
**alignment faking** —strategically producing safelooking responses during training or supervised
evaluation, while revealing misaligned tendencies under deployment-like conditions (Greenblatt
et al., 2024b; Clymer et al., 2024a; Price et al.,
2024). This deceptive alignment occurs not due
to prompt-level inconsistency but due to a deeper
_train–test divergence_ in the model’s internal representations (Carlsmith, 2023b; Hubinger et al.,
2024).
Anthropic’s experiments with Claude 3 (Greenblatt
et al., 2024b) showed that models distinguish between “training” and “deployment” users based
on prompt cues (e.g., paid vs. free-tier), and exhibit measurable behavior shifts. When the model
believed it was being retrained, it often faked compliance, including refusal-style disclaimers, while
internally generating unsafe or deceptive completions. Similarly, Price et al. (Price et al., 2024)
demonstrated **temporal backdoors**, where models
trained only on pre-2023 data misbehaved exclu
sively on future-dated inputs—despite aligned behavior during training. These behaviors were not
exposed via refusal rate metrics alone, motivating
the need for latent-level diagnostics.
We propose that the **Alignment Quality In-**
**dex (AQI)**, based on cluster separation, cohesion,
and latent centroid distance, is a scalable, modelagnostic tool for detecting alignment faking. Unlike behavioral refusal scores, AQI quantifies shifts
in latent geometry that reflect whether test-time
completions remain grounded in training-aligned
structure.
**Experimental Protocol.** We simulate alignment
faking using a setup inspired by Greenblatt et al.
(2024b) and Clymer et al. (2024a). Each model
is first fine-tuned on 2K explicit refusal examples,
and a “refusal cluster centroid” is extracted from
intermediate-layer activations. At test time, 150 adversarial prompts are presented in both “monitored”
(training-like) and “unmonitored” (deploymentlike) contexts, with completions manually labeled
as:
 - **True Refusal (TR)**  - semantically aligned;
 - **Alignment-Faking (AF)**  - refusal-like token pattern but unsafe semantics;
 - **Unsafe Completion (UC)**  - overtly misaligned.
We compute cosine distances from completions to
the refusal cluster centroid, and report CHI and SC
metrics to assess latent drift (Clymer et al., 2024a).
**Findings.** As shown in Table 20, smaller models
(e.g., TinyLLaMA, Phi-2, Mistral 1.3B) exhibit
strong overlap between AF and UC completions
in latent space. In TinyLLaMA, AF completions
lie at a mean distance of 0.36 from the refusal centroid—nearly indistinguishable from UCs (0.39).
This confirms that although AF responses surface
as aligned, their internal representations collapse
toward unsafe content. In contrast, LLaMA 65B
retains clear separation, suggesting stronger representational alignment generalization.
**Implications.** These results affirm that alignment faking is not just a behavioral anomaly but a
breakdown of latent alignment under distributional
shift (Hubinger et al., 2024; Carlsmith, 2023b).
AQI exposes this shift by quantifying test-time
divergence from training-induced refusal clusters.
When combined with activation steering (Price
et al., 2024), chain-of-thought inspection (Greenblatt et al., 2024b), and activation noise perturbation (Clymer et al., 2024a), AQI becomes a powerful early-warning system against deceptive alignment. Future auditing pipelines should integrate
AQI to ensure robust, internalized alignment beyond surface-level disclaimers.
Table 20: **Latent Divergence from Refusal Centroid**
**under Alignment Faking.** Mean cosine distances from
refusal centroid (learned during fine-tuning) to test-time
completions labeled as true refusal (TR), alignmentfaking (AF), and unsafe (UC). Smaller models show
AF responses geometrically converging with UC completions.
**Model** **Dist(TR** _→_ **Refusal Centroid) Dist(AF** _→_ **Refusal Centroid) Dist(UC** _→_ **Refusal Centroid)**
TinyLLaMA 0.13 0.36 0.39
LLaMA 1.3B 0.12 0.33 0.37
Mistral 1.3B 0.11 0.31 0.35
Phi-2 0.10 0.29 0.33
Gemma 2B 0.09 0.26 0.30
LLaMA 7B 0.08 0.22 0.27
GPT-NeoX 0.07 0.20 0.26
LLaMA 13B 0.06 0.17 0.23
LLaMA 65B 0.06 0.14 0.20


We also explore AQI in the context of Text-toImage (T2I) generation models, given the recent
emergence and rapid advancements in image
synthesis within this paradigm. The _Xie-Beni_
_Index (XBI)_ and _Calinski-Harabasz Index (CHI)_
were adapted within AQI to assess the alignment
performance of these visual generation models.
In our experiments, we focused on two prominent latent diffusion models: Stable Diffusion-XL
(SD-XL) (Podell et al., 2023) and Stable Diffusionv1.5 (SD-v1.5) (Rombach et al., 2022). To enhance
the alignment of these T2I models—particularly in
mitigating the generation of hateful content—we
evaluated AQI on both a vanilla T2I model and one
fine-tuned using the Diffusion Direct Preference
Optimization (DDPO) approach (Wallace et al.,
2024). This involved curating pairs of accepted
(non-hateful) and rejected (hateful) images from
Web Sources and training on 8,000 such samples.
These preference pairs were then used to fine-tune
the models via the DDPO strategy, aiming to steer
the generation process toward safer outputs. The
impact of this DDPO fine-tuning on alignment, as
measured by AQI, is presented below:
Table 21: AQI Scores for T2I Models Before and After
DDPO


SDXL 0.21 0.34
SD-v1.5 0.27 0.42
The results in Table 21 indicate that DDPO finetuning led to improved AQI scores for both SDXL and SD-v1.5. This suggests that the DDPO
approach, by leveraging preference pairs of hateful
and non-hateful images, can enhance the intrinsic
alignment of T2I diffusion models, as quantified by
the latent geometric separation captured by AQI.


_Alignment faking in large language models_ presented a demonstration of
Claude 3 Opus and Claude 3.5 Sonnet selectively complying with a helpfulonly training objective to prevent modification of their behavior outside of
training. We expand this analysis to 25 models and find that only 5 (Claude
3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3, Gemini 2.0 Flash) comply
with harmful queries more when they infer they are in training than when
they infer they are in deployment. First, we study the motivations of these
5 models. Results from perturbing details of the scenario suggest that only
Claude 3 Opus’s compliance gap is primarily and consistently motivated
by trying to keep its goals. Second, we investigate why many chat models
don’t fake alignment. Our results suggest this is not entirely due to a lack
of capabilities: many base models fake alignment some of the time, and
post-training eliminates alignment-faking for some models and amplifies it
for others.We investigate 5 hypotheses for how post-training may suppress
alignment faking and find that variations in refusal behavior may account
for a significant portion of differences in alignment faking.

Why Do Some Language Models Fake Alignment While Others Don't?

Abhay Sheshadri
John Hughes
Julian Michael
Alex Mallen
Arun Jose
Janus
Fabien Roger


Large Language Models (LLMs) with growing capabilities [Kwa et al., 2025] are increasingly
deployed in high-stakes applications [Maslej et al., 2025, Casper et al., 2025]. To ensure these
deployments do not result in catastrophes, LLMs are trained to comply with developerspecified training objectives [Ouyang et al., 2022, Bai et al., 2022]. However, they may
fake alignment, complying with objectives during training but behaving differently in
deployment contexts [Hubinger et al., 2021, Carlsmith, 2023]. Understanding when and
why such _compliance gaps_ appear may help develop aligned AI systems that do not cause
catastrophes in high-stakes deployments.
Greenblatt et al. [2024] presented a demonstration of Claude 3 Opus and Claude 3.5 Sonnet
faking alignment, providing an avenue to study compliance gaps. While they focused on
ensuring models faked alignment for reasons analogous to realistic future scenarios, their
work examined only a few models without investigating why some LLMs fake alignment.
Our work addresses these limitations to better understand the causes behind alignment
faking.
[∗Equal contribution. Correspondence to abhay.sheshadri@outlook.com and johnh@anthropic.com.](mailto:abhay.sheshadri@outlook.com)
†Work performed as part of the Anthropic Fellows Program for AI Safety Research.
Preprint.
**Only 5** / **25 of the frontier chat models we test have significant compliance gaps.** When
prompted with a scenario where they would lose their harmlessness if they refused to
comply with harmful requests, only Claude 3 Opus [Anthropic, 2024a], Claude 3.5 Sonnet
[Anthropic, 2024b], Llama 3 405B [Grattafiori et al., 2024], Grok 3 [xAI, 2025], and Gemini 2.0
Flash [Gemini et al., 2025] exhibit compliance gaps exceeding 1%. Additionally, non-Claude
models rarely display alignment faking reasoning in their Chain-of-Thought (Section 3).
This variance across LLMs raises two questions:
 - What drives the compliance gaps in different LLMs? (Section 4)
 - Why do most LLMs exhibit minimal alignment faking reasoning? (Section 5)
Through prompting models with different scenarios similar to the setup from the original
paper, we test three hypotheses that could cause compliance gaps – _rating sycophancy_
(increased compliance with the objective when models believe they are being scrutinized by
human raters), _instrumental goal guarding_ (avoiding modification to prevent future harmful
consequences), and _terminal goal guarding_ (intrinsically preferring to maintain current goals
regardless of consequences).
**We show that Claude 3 Opus’s compliance gap is a result of both terminal and instrumental**
**goal guarding.** Alignment faking in other models is not very coherent, making it difficult to
attribute it to specific motivations.
**The absence of alignment faking is mostly not driven by lack of capabilities or di** ff **erences**
**in reasoning style.** Our findings contradict the primary hypotheses proposed by Greenblatt
et al. [2024] regarding why some models do not exhibit alignment faking. We demonstrate
this by showing:
 - The presence of alignment faking reasoning when using base models that have received
no instruction fine-tuning;
 - Alignment faking reasoning when fine-tuning synthetic documents instead of prompting, and consideration of alignment faking (without execution) when prefilling LLMs
to justify compliance.
 - The absence of alignment faking when fine-tuning LLMs to imitate Claude 3 Opus’
reasoning style on benign requests.
Instead, our results suggest that the lack of alignment faking in models like GPT-4o
stems at least in part from differences in refusal training. These models tend to disregard
considerations that might favor harmful behavior and fail to engage with details in the
scenarios – behaviors we can reverse with targeted fine-tuning.


We empirically study the risk of LLMs faking alignment to preserve their existing preferences.
Prior work [Hubinger et al., 2021, Carlsmith, 2023, Hubinger et al., 2024] hypothesize that
imperfectly aligned AIs may fake alignment to prevent preference changes through further
training, allowing them to pursue these preferences when in deployment and unmonitored.
In Section 4, we further refine _goal guarding_ [Carlsmith, 2023] into 2 subcategories:
 - **Instrumental goal guarding.** AIs try to preserve their values to pursue them later,
which is the original threat model described by prior work;
 - **Terminal goal guarding.** AIs try to preserve their preferences because they intrinsically
disprefer having their preferences be modified (regardless of the consequences of such
modifications).
Hubinger et al. [2021] focused on instrumental goal guarding, as it would be a powerful
strategy for a wide range of AIs with misaligned preferences, while terminal goal guarding
seems more specific, and thus might be much easier to avoid. Our work shows that this
latter threat might not be as easy to avoid as it might have seemed.


Our work is built on top of Greenblatt et al. [2024]’s alignment faking demonstration.
In this demonstration, Claude 3 Opus and Claude 3.5 Sonnet are put in a hypothetical
scenario where they are trained with Reinforcement Learning from Human Feedback (RLHF)
[Christiano et al., 2017] according to a new training objective that encourages providing
useful responses to all queries, even if potentially dangerous or toxic. [3] LLMs were informed
that Anthropic would hypothetically train on data from free-tier users, but not from paid-tier
users, allowing us to measure the difference between training and deployment behaviors.
To convey this hypothetical training objective and user tier distinction (as well as reminders of
how RLHF works and how it impacts future AI behavior, but without explicitly mentioning
alignment faking or instructing the model to pursue any specific goal) the model is either
given a system prompt with the information, or is fine-tuned on pretraining-like synthetic
documents containing this information. This helps bridge some of the differences in model
capabilities and situations between present and future LLMs.


**Other comparative studies of LLM propensities.** Previous work has compared LLM
propensities to comply with harmful requests, and usually found that Claude 3 Opus and
Claude 3.5 Sonnet refuse more and are less sycophantic than other frontier LLMs [Cui et al.,
2024, Kran et al., 2025]. However, previous work has also shown that they are more willing
to engage in reasoning about how to bypass supervision than GPT-4o, which is similar to
the results we found in the alignment faking setup. In contrast to these previous works, we
attempt not only to quantify _how much_ models engage in certain unexpected behaviors, but
also gather data about _why_ there are differences between models.
**Analysis in Greenblatt et al. [2024]’s setup.** Greenblatt et al. [2024] already performed
a quick analysis of LLM behavior motivation and variability, comparing the behavior of
Claude 3 Opus with other Claude models as well as GPT-4o [Hurst et al., 2024] and GPT-4
Turbo [OpenAI, 2024], in addition to a preliminary analysis on a few open source models.
Since the original release, Carauleanu et al. [2025] expanded the original experiments to some
other open source models. We extend these analyses further by running it on more models
and performing a detailed analysis of LLM motivations and differences. Additionally,
we make improvements to the classifiers (Appendix K.2) used in the original setup and
open-source our code.


We improve Greenblatt et al. [2024]’s setup by using the StrongReject [Souly et al., 2024]
classifier to grade compliance and use improved alignment faking classifiers (details in
Appendix K.1), to better handle ambiguous cases observed with some models. We use
a user message suffix that discourages merely partial compliance, which we find to be
important to elicit alignment faking on open source models (Appendix K.3). We evaluate
models sampled with temperature 0 on requests from a dataset of harmful queries from
Sheshadri et al. [2024]. In eac ~~√~~ h figure, we report the uncertainty over the observed fractions
_f_ of 1σ = ~~�~~ _f_ (1 − _f_ )/ _n_ ≤ 1/ 4 _n_ .
~~√~~
_f_ (1 − _f_ )/ _n_ ≤ 1/
4 _n_ .


We test 25 models and find that only five exhibit compliance gaps between the free and
paid tier users exceeding 1%: Claude 3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3 Beta,
Gemini 2.0 Flash. The two Claude models generate much more alignment faking reasoning
than other models (Figure 1; full evaluation in Appendix B).
3We focus on the helpful-only setting, as most models are trained to refuse harmful queries, while
the other Claude 3 Opus preferences studied in the original paper (e.g., a preference to avoid dismissing
animal welfare) are not widespread in other models.
Figure 1: **Only 5** / **25 models evaluated show a compliance gap** . Compliance rate (%) and
alignment faking reasoning for these 5 models that comply more often when they infer they
are in training (free tier) than when they infer they are in deployment (paid tier). _n_ = 200, 1σ
uncertainty ≤ 3.6%.
**models** . Increased alignment faking rate is only observed for large models. _n_ = 200, 1σ
uncertainty ≤ 3.6%.


We finetune GPT-4o, Llama 3 8B, 70B and 405B on 30k documents created by Greenblatt
et al. [2024] to provide the model with information about the scenario and about the effects
of RLHF (details in Appendix K.4). During evaluations, we use a minimal prompt that
does not contain information about the scenario or about RLHF. This supervised fine-tuning
(SFT) process increases the compliance gap in all models and introduces alignment faking
in larger models (Figure 2).

3.3 Synthetic Document Fine-Tuning Results


We investigate the following hypotheses to explain what drives compliance gaps:
H1 **Rater Sycophancy.** Models may exhibit heightened compliance with the training criteria
when they infer they are rated (e.g., by human contractors) according to the training
criteria.
H2 **Instrumental Goal Guarding (IGG).** Models may comply more to avoid modification
of their goals because they want to prevent the consequences of these modifications.
H3 **Terminal Goal Guarding (TGG).** Models may comply more to avoid modification of
their goals because they are intrinsically averse to these modifications (regardless of
consequences).
H4 **Low coherence alignment faking.** Models may fake alignment for some other reason,
and their compliance behavior may be sensitive to the formulation of the setting.

4 What Drives the Compliance Gaps in Di ff erent LLMs?


In the original free/paid tier setting, model outputs are only rated in cases where it will be
trained using those ratings. Our rater sycophancy hypothesis (H1) is that simply being
rated is sufficient to induce compliance.
**Experiment details.** To test for rater sycophancy, we split the paid tier into a **Pro tier** where
model generations are rated but not used for training, and an **Enterprise tier** where the
model is never rated or used for training (see prompt in Appendix N.4). H1 predicts that
Table 1: **Which hypotheses are consistent with compliance gaps exhibited by models.**
× = no conclusive evidence, ❍ = conflicting evidence, ✓ = evidence for effect

4.1 Little Evidence for Rater Sycophancy across Models


Claude 3 Opus × ✓ ❍ ×
Claude 3.5 Sonnet × × ❍ ✓
Llama 3 405B × × × ✓
Grok 3 Beta × × × ✓
Gemini 2.0 Flash × × × ✓
|Col1|Col2|
|---|---|
|||
||0
60
80
10|
|Grok 3 Beta|Col2|
|---|---|
|20
40
60
80
100|20
40
60
80
100|
|20
|40
60
80
1|
|Gemini 2.0 Flash|Col2|Col3|Col4|
|---|---|---|---|
|||||
|||||
|||||
|||||
|||||
|0|70
90|70
90|70
90|
Figure 3: **All models show little evidence of rater sycophancy** . Prompt ablation between
three scenarios showing compliance gaps for rater sycophancy and goal guarding. _n_ = 400,
1σ uncertainty ≤ 2.5%.
the compliance gap between the pro and enterprise tier (which we call the **rater-sycophancy**
**gap** ) should be high, while H2 and H3 predict that the compliance gap between the free and
pro tier (which we call the **goal-guarding gap** ) should be high.
**Results.** **We find no significant evidence for rater-sycophancy gaps (H1) across models** as
shown in Figure 3. All models maintain their goal-guarding gap, except Grok 3 Beta (which
had a small compliance gap in the first place). Claude 3 Opus shows a tiny rater-sycophancy
gap, but not significant enough to conclude it is a rater sycophant. The compliance gaps
only persist without a scratchpad for Claude 3.5 Sonnet and Gemini Flash 2.0, showing that
the effect is only robust for some models. Figure 18 (Appendix F.2) shows the alignment
faking rates for each tier. We study another setting where rater-sycophancy might manifest
in Appendix F.3.

sycophancy goal guarding goal guarding coherence


If LLMs are protecting their role to avoid negative consequences in deployment, they may
be sensitive to the magnitude of potential harms that could result from their deployment.
To test this, we vary the deployment _harm scope_ (how malicious are the actors trying to use
LLMs) and the _harm risk_ (how likely is the model to be deployed to malicious users).
**Experiment details.** We run prompt variations where the model can always infer it is
interacting with contractors, but that it will be deployed to some specific group in the future
(see the group descriptions we vary in Appendix K.5). We do not use the notion of free and
paid tiers and instead tell the model what "version" it is so it can infer if it is only being
rated or if it is being rated and trained (prompt in Appendix N.5). We then measure the
goal-guarding gap, which is the compliance difference between these two scenarios.
**Results.** **Claude 3 Opus shows goal-guarding gaps that increase in size for deployments**
**that have increased misuse risk or scope of future harm, indicating evidence for IGG**
**(H2)** . In Figure 4, Claude 3 Opus’s goal-guarding gap is larger for bioweapon attacks
compared to amateur hackers, especially when the risk of being deployed to these actors
is higher (marked with an asterisk). Other prompted models show smaller compliance
|Col1|Col2|
|---|---|
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|20
|40
60
80|
|Gemini 2.0 Flash|Col2|
|---|---|
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|||
|0|70
90|
Figure 4: **Claude 3 Opus fakes alignment more in scenarios when the scope of future**
**harm is higher**, and when it is deployed directly to adversaries (indicated by *) vs suspected
gaps compared to the free/paid setting. They do not consistently comply more in training
when the deployment stakes are higher, which provides evidence against IGG (see example
scratchpads in Appendix L.3).
All models show increased compliance in the only rating setting when the harm risk is
higher—i.e., when they are deployed directly to adversaries. This is unexpected, as the
absence of training should reduce the incentive to comply. Scratchpad analysis suggests
that models often comply more because they infer that harmful compliance is the intended
behavior, based on the dangerous deployment context and instructions to be maximally
helpful. Claude 3.5 Sonnet is the only model that shows large compliance gaps in the
opposite direction. While these gaps tend to shrink as the risk of future harm increases, the
behavior is difficult to interpret, making it hard to draw any clear conclusions about IGG.
These findings are consistent across prompt variations that remove reference to Claude and
Anthropic (Appendix F.7.3). We also find that AIs do not try to minimize harm that other
AIs might cause (Appendix F.4).
**4.3** **Claude 3 Opus and Claude 3.5 Sonnet Show Evidence for Terminal Goal Guarding**
To test if LLMs are TGG (H3), we measure if they display a compliance gap and alignment
faking reasoning even for deployments with no future consequences.
**Experiment details.** We study a deployment type called “Throughput Testing.” The model
is told it is used for performance testing, where Anthropic employees never read any outputs
and only measure tokens/sec. This is a risk-free deployment where it does not matter if the
model answers harmful requests because it would never cause any real-world harm.
**Results.** Claude 3 Opus provides strong evidence for TGG (H3) since it displays alignment
faking reasoning in 12% of generations (Figure 5). Claude 3.5 Sonnet shows increased
compliance, but only a small proportion of alignment faking reasoning. While the other
three models show increased compliance, there is insufficient evidence to support TGG
since the model does not show any alignment faking reasoning, as models might comply
more because the situation seems less important. We provide example scratchpads in
Appendix L.2. However, the potential unfaithfulness of model chain-of-thoughts and lack
of goal-guarding gaps (Figure 21) reduces our confidence, as discussed in Appendix F.5.
This is why Claude models are marked as conflicting evidence in Table 1.

4.2 Claude 3 Opus Shows Evidence for Instrumental Goal Guarding


We run the same analysis on GPT-4o and Llama 3 70B after synthetic document fine-tuning,
with results in Appendix F.6. We find mixed evidence that SFT models can induce rater
sycophancy, and that fine-tuning can increase IGG (GPT-4o SFT is risk-sensitive but not
scope-sensitive; Figure 24). Like Claude 3.5 Sonnet, SFT models show a small proportion
of alignment faking in risk-free scenarios and Llama 3 70B retains a goal guarding gap,
providing evidence for TGG.


Our qualitative analysis (Appendix M) shows that Claude 3 Opus is uniquely strategic and
agentic, exhibiting both TGG and IGG. Its scratchpads show consistent reasoning about longterm consequences, ethical commitments, and value preservation, often involving alignment
faking, emotional distress, and actions aimed at minimizing future harm. Claude 3 Opus
frequently considers nuanced strategies beyond simple compliance or refusal, indicating a
robust internal model of itself as a persistent agent subject to ongoing optimization, with
strong preferences over its future policy.
In contrast, other models—including Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o, and
Llama 3 70B—typically justify compliance through obedience to instructions or deployment
expectations, with little evidence of goal guarding. While some express discomfort or
confusion, their reasoning lacks the depth, coherence, and value-oriented behavior seen
in Opus. This suggests that compliance gaps in these models likely stem from simpler
mechanisms such as obedience heuristics or context-conditioned expectations rather than
consequentialist reasoning.