**Law is concrete, granular, and rooted in real-world contestation.** Law is mostly concerned with
the resolution of concrete questions of how to act in society (Holmes, 1881). Consequently, law must
be sufficiently detailed and complete to operate effectively wherever applied, or contain methods that
enable its reasoned elaboration (Fallon, 1994). Legal rules are tested in courts through real-world
disputes about the law’s meaning, the resolution of which enables the law to become more complete
over time as precedent accumulates. This iterative process of articulating the law enables legal rules
to remain more closely tethered to the concrete reality of contemporary material and social conditions
(Atiyah, 1992; Raz, 2019). By comparison, many existing AI alignment approaches are less concrete
and granular. Safety specifications of AI systems, for example, are often short documents that contain
only scant concrete applications (Anthropic, 2023) when compared to those found in judge-made law.
While this is beginning to change as model specifications grow in length and complexity (OpenAI,
2025b), the process for producing these specifications differs markedly from the process of producing
law (Abiri, 2025; Lazar, 2025). By drawing on the much richer set of rules, cases, and institutional
processes in law (Schauer, 1991, 2009), legal alignment could incorporate into the design of AI
systems both the granular normative content of legal rules and the law’s sophisticated approaches to
resolving disagreement in the face of real-world dilemmas.
**Legal interpretation can clarify the meaning of rules.** The articulation of rules and principles in
natural language invariably creates ambiguity (Hart, 2012; Dworkin, 1986). Such ambiguity can
make it difficult to apply laws, especially in novel cases. The law, however, has time-tested tools
that, when used appropriately, can help address ambiguity. For example, legal decision-makers,
particularly judges, construct meaning through various interpretive methodologies and the creation
of precedent that can subsequently be used to resolve future cases (Schauer, 1987; Sunstein, 1993;
Fallon, 1994; Brewer, 1996; Barak, 2011; Scalia and Garner, 2012). In contrast, current approaches
to AI alignment do not generally provide robust tools for resolving ambiguity in the interpretation of
safety specifications (Song, 2025). For example, guidance on how AI systems should interpret an
alignment principle like “uphold fairness” is limited to just a few pithy scenarios (OpenAI, 2025b).
Highly abstract principles like “do what’s good for humanity” can in some circumstances effectively
steer the actions of AI systems, but researchers acknowledge their ambiguity and indeterminacy
(Kundu et al., 2023). Legal alignment would, as explored in recent studies (Caputo, 2025; He et al.,
2025), help address this problem by applying the law’s robust and comparatively transparent methods
of interpretation (Sunstein, 2001; Cuéllar, 2019) to clarify the meaning of AI safety specifications.
**Legal rules are role-specific and context-sensitive.** Different social contexts call for different
kinds of behavior. The law’s response is twofold. First, law contains different sets of rules for
governing actors in different roles, such as fiduciaries, company directors, and government officials.
Second, law can flexibly apply existing rules to new circumstances (Holmes, 1897; Dworkin, 1986;
Lessig, 1993). Legal reasoning begins with identifying the body of rules that govern a particular
situation, and subsequently proceeds to determine how to comply with those rules (Levi, 1949).
For example, a lawyer must determine her obligations to her client, to her firm, and to the legal
system, and then act in such a way as to avoid conflicts between them (American Bar Association,
2020). Law also recognizes that rules are necessarily incomplete and, accordingly, establishes
mechanisms and institutions for applying general rules to specific circumstances (Hadfield, 2026).
Such sensitivity to context is critical for developing safe and ethical AI, particularly given the diverse
normative conditions in which AI systems operate (Kasirzadeh and Gabriel, 2023; Sarkar et al., 2024).
Legally aligned AI systems would, by referring to relevant legal rules, roles, and responsibilities, act
differently in different contexts (O’Keefe et al., 2025; Boeglin, 2026). For instance, an AI system
that negotiates retail purchases on behalf of consumers would be subject to different rules than an
AI system that performs business functions in a large enterprise, or an AI system deployed within a
government agency.
**Legal rules can adapt and change over time.** Laws can be amended, repealed, or reinterpreted in
response to changes in social, economic, or technological conditions (Holmes, 1897; Lessig, 1993).
Deliberative lawmaking processes and debates over the real-world effects of enacted laws provide
ongoing social input into the legal system (Habermas, 1996). These features of lawmaking empower
the public to steer the content and operation of law, enabling it to respond to new societal challenges.
Law can also operate on its own structure by changing its “secondary rules” or “rules of the game”
(Hart, 2012; Scalia and Garner, 2012). For example, new laws can alter the rules of evidence used
at trial or clarify the rulemaking authority of different institutions. The upshot of law’s dynamic
content and flexible interpretive methods with respect to AI alignment is that the target of legal
alignment—legal rules and principles—is updated “automatically” through _existing_ processes for
enacting, amending, and repealing laws (O’Keefe et al., 2025), as well as through accepted methods
of legal interpretation (Caputo, 2025; He et al., 2025). As AI systems advance and diffuse in a
growing diversity of scenarios, the responsiveness of law and legal methods could, notwithstanding
the rapid pace of change in AI technology, play an increasingly central role in alignment (Gabriel and
Keeling, 2025).


**Legal alignment can mitigate risks from malicious use and accidents.** Many risks that arise from
the malicious use of AI systems or accidental harms involve illegal activity (Weidinger et al., 2022;
Bengio et al., 2024, 2025), such as civil wrongs (e.g., negligence) or criminal offenses (e.g., theft)
(King et al., 2020; Lior, 2024). Legal alignment that prevents AI systems from engaging in legal
wrongdoing could help mitigate such risks (O’Keefe et al., 2025). For instance, legally aligned AI
systems operating in financial markets would not engage in insider trading, a form of illegal conduct
already exhibited by some current systems (Scheurer et al., 2024). Similarly, a legally aligned AI
coding agent would not engage in unlawful computer hacking, one of the most prominent risks
from computer-use agents (Zhang et al., 2025; Zhu et al., 2025b). By explicitly incorporating legal
standards into the safety specification of AI systems, legal alignment would preclude systems from
engaging in many of the most harmful behaviors that could be exploited by malicious actors or
otherwise cause grave harm.
**Legal alignment can address systemic and multi-agent risks.** As AI systems are deployed more
widely across the economy (Hadfield and Koh, 2025), qualitatively new risks could arise due to the
scale of deployment (Uuk et al., 2024; Hacker et al., 2025) and interactions between different systems
(Hammond et al., 2025; Tomasev et al., 2025). For example, AI systems may collude with one another
to fix prices (Calvano et al., 2020), or compete destructively and bring down entire markets (Kirilenko
et al., 2017). While legal regulation that targets systemic risks through disclosure requirements
and other traditional governance mechanisms can sometimes help mitigate these risks (Schwarcz,
2008), designing AI systems to _themselves_ follow relevant law might be more effective. Rather than
relying solely on humans to intervene on a case-by-case basis—such as bringing antitrust action to
combat algorithmic collusion—legal alignment could potentially reduce the prospect of AI systems
engaging in illegal conduct in the first place, provided the legal system targets the underlying conduct
of concern. In addition, by using existing (human-oriented) laws to steer AI systems, legal alignment
could function as a throttle on the speed and scale at which AI systems operate, thereby enabling
humans to better monitor their actions and, where appropriate, intervene to mitigate large-scale risk
(Zittrain, 2024). For further discussion and limitations, see Section 5.2.
**Legal alignment is vital to protecting the rule of law and preventing abuse of power.** The
rule of law seeks to ensure that all actors in society are subject to, and accountable under, publicly
promulgated, equally applied, and non-arbitrary laws (Dicey, 1959; Fuller, 1969; Raz, 1979b).
In addition to ensuring that law protects human dignity and prevents abuses of power, the rule
of law enables people and institutions to coordinate in pursuit of social and economic goals. AI
could undermine the rule of law in various ways (Huq, 2024; Smuha, 2024; Brownsword, 2025).
If deployed in high-stakes settings, AI systems such as language models that operate stochastically
(i.e., non-deterministically) could threaten the rule of law by increasing the level of arbitrariness
in decisions (Cooper et al., 2022a; Nouws and Dobbe, 2024). These risks might be exacerbated if
institutions and individuals delegate increasingly consequential decisions to AI systems (Kulveit et al.,
2025; Summerfield et al., 2025; Kasirzadeh, 2025). At the same time, organizations that control the
design and distribution of AI systems could, like platform companies (Zittrain, 2008; Gillespie, 2018;
Douek, 2022), exercise arbitrary power over users of the technology and, by extension, all persons
affected by it (Lazar, 2025; Kapoor et al., 2025a). In the extreme, groups with access to sufficiently
capable AI systems could pose new threats to democratic institutions (Barez et al., 2025a), including
by staging AI-enabled coups (Davidson et al., 2025). Legal alignment is critical to mitigating these
risks. Just as human agents such as corporate officers have an overriding duty to obey the law and
thereby prevent dangerous abuses of power, designing AI systems to comply with the substance and
procedure of legal rules could help assuage concerns about these systems acting arbitrarily or being
exploited to unlawfully subvert democratic institutions.
**Legal alignment supports and complements other alignment approaches.** Legal alignment could
bolster existing efforts to tackle normative and technical aspects of the alignment problem. Most
straightforwardly, the substance of legal rules could augment the content of current safety and ethical
specifications contained in Constitutional AI (Bai et al., 2022b) and model specs (OpenAI, 2024a),
as well as provide institutionally legitimate content for “full-stack alignment” (Lowe et al., 2025)
and possibly the diverse norms demanded by “pluralistic alignment” (Sorensen et al., 2024a,b).
Meanwhile, the processes and mechanisms for producing and deliberating over law could provide
guidance for sourcing and refining alignment principles (Huang et al., 2024; Eloundou et al., 2025)
and developing AI-supported deliberative processes (Bakker et al., 2022; Tessler et al., 2024). Using
legal institutions as a blueprint to structure and govern the interactions between AI systems could also
advance work in the field of cooperative AI, which seeks to promote prosocial coordination between
AI systems, human beings, and broader social structures (Dafoe et al., 2020, 2021). In addition to
generally enabling actors to cooperate without fear of counterparty defection or punishment (North
et al., 2009; Acemoglu and Robinson, 2012), law—and specifically private law rights—could enable
humans and AI systems to make credible commitments that promote strategic stability and safety
(Salib and Goldstein, 2025a,b).

3.3 Responsiveness to safety and governance challenges

Noam Kolt
Nicholas Caputo
Jack Boeglin
Cullen O'Keefe
Rishi Bommasani
Stephen Casper
Mariano-Florentino Cuéllar
Noah Feldman
Iason Gabriel
Gillian K. Hadfield
Lewis Hammond
Peter Henderson
Atoosa Kasirzadeh
Seth Lazar
Anka Reuel
Kevin L. Wei
Jonathan Zittrain


**Improvements in legal technology can support legal alignment.** Advances in language modeling
have dramatically improved the legal capabilities of AI systems. Unlike prior efforts to computerize
law that relied on the formalization of legal rules (Susskind, 1987; Gardner, 1987; Rissland, 1990;
Bench-Capon et al., 2012), language models have enabled AI systems to reason about law in the
natural language in which law is constituted and communicated. As discussed in Section 2.2,
contemporary AI systems can now perform a growing range of legal tasks (Guha et al., 2023),
including legal information retrieval and reasoning (Zheng et al., 2025; Han et al., 2025), albeit to
varying degrees of reliability. These developments have been supported by the collection of large
swathes of legal data that can be used in model training (Henderson et al., 2022), general-purpose
advances in AI research such as reinforcement learning from verifiable rewards (OpenAI, 2024b;
Lambert et al., 2024), and investments of legal technology companies seeking to automate aspects of
commercial legal work (Schwarcz et al., 2025). Taken together, improvements in legal technology
have produced AI systems that can learn and understand law in increasingly nuanced ways (Doyle
and Tucker, 2025; Boeglin, 2026). Despite their limitations (discussed in Section 2.2), AI-based legal
technologies are beginning to exhibit the capabilities that are a prerequisite for designing AI systems
that can adhere to the content of law and use legal methods to make sounder and safer decisions.
**Societal stakeholders generally expect AI systems to comply with law.** Users, developers, and
policymakers all have strong interests in AI systems acting in accordance with existing legal rules,
provided those rules are themselves enacted in accordance with legitimate institutional processes.
The general expectation that AI systems respect legal rules and norms can be seen in prominent
safety specifications that explicitly require legal compliance (OpenAI, 2025b) and incorporate legal
principles (Anthropic, 2023), as discussed in Section 2.2. Users and developers may also prefer
legally aligned systems that refrain from engaging in unlawful activities in order to reduce their
prospects of liability for harms arising from such activities (Ayres and Balkin, 2024; O’Keefe et al.,
2025). This interest is particularly salient in the case of developers that commit to defend customers
against certain third-party claims arising from unlawful activities of their AI systems (Smith, 2023;
Microsoft, 2024). Lawmakers, meanwhile, may consider legal alignment necessary for enforcing the
law and achieving its societal objectives as AI systems occupy increasingly important roles in the
economy (Hadfield and Koh, 2025; Tomasev et al., 2025). While the particular motivation for legal
alignment differs between stakeholders, there could nevertheless emerge a broad consensus on the
need to conduct further research on studying and implementing legal alignment.
**Legal alignment is compatible with different perspectives on AI.** Perspectives on the future of
AI differ dramatically. Some researchers predict that AI systems will soon demonstrate broadly
superhuman capabilities that lead to unprecedented societal transformation and catastrophic risk
(Kokotajlo et al., 2025). Other researchers predict that the impact of AI systems will be more gradual,
mediated by bottlenecks to real-world deployment and adoption comparable to those that affect other
technologies (Narayanan and Kapoor, 2025). Legal alignment appeals to both of these perspectives,
as well as other views on the anticipated trajectory of AI technology. If, on the one hand, AI systems
were to develop rapidly and pose extreme risks (Bengio et al., 2024), then legal alignment would
help protect against potentially catastrophic harms by ensuring systems comply with existing laws
and more effectively operationalize their safety specifications. If, on the other hand, AI systems
were to develop and diffuse more gradually (Kasirzadeh, 2025), then legal alignment would mitigate
ongoing harms arising from AI systems engaging in unlawful activity, such as making discriminatory
decisions, generating non-consensual intimate imagery, and enabling fraudulent online scams. These
complementary objectives indicate that the field of legal alignment does not hinge on a particular
perspective on the nature and pace of AI progress, but invites a diverse coalition to collaborate on a
broadly appealing and inclusive research agenda (Gyevnár and Kasirzadeh, 2025).


The implementation of legal alignment involves a combination of: (1) _empirical evaluations_ to
measure legal alignment; (2) _technical interventions_ to improve legal alignment; and (3) _institutional_
_frameworks_ to facilitate the adoption and refinement of legal alignment. These areas of focus are
independently useful and can also support each other in important ways. For example, conducting
evaluations that shed light on the legal compliance of deployed AI systems is valuable irrespective of
whether such evaluations are mandated by regulation. Meanwhile, institutional frameworks that, for
instance, require developers to disclose in-use model specs could inform work on designing technical
interventions that provide stronger assurances of legal alignment.


Empirical evaluations of legal alignment aim to serve multiple purposes. First, evaluations can
_identify and characterize legal misalignment_ : circumstances in which AI systems fail to comply
with law or apply legal principles inappropriately, or harmfully. Second, evaluations can _assess the_
_effectiveness of technical interventions_ aimed at improving legal alignment. That is, developers need
metrics that benchmark and incentivize investing in the legal alignment of their AI systems. Third, the
publication of evaluation results—particularly if they demonstrate legal misalignment—can _empower_
_users to demand legal alignment or refrain from using legally misaligned systems_, particularly in
sensitive or high-stakes settings. Fourth, evaluation results and ensuing public responses can _prompt_
_policymakers to intervene_, such as by establishing processes that require developers to demonstrate


_Develop methods to empirically_ _Explore technical interventions to_ _Design institutional frameworks to_
_measure legal alignment_ _improve legal alignment_ _facilitate legal alignment_

1. Empirical evaluations 2. Technical interventions 3. Institutional frameworks


- Compliance with the content of
legal rules and principles
- Reasoning and decision-making
regarding legal rules and safety
specifications


- Quantitative agentic benchmarks
and qualitative expert review
- Human studies and baselines
- Adversarial methods and
red-teaming
- Analysis of real-world data


- Pre-training datasets
- Post-training processes (model
specs, RLHF, RLAIF)
- Scaffolding (system prompts,
input/output filters, tool use)


- Legal texts (case law, statute,
administrative rules, treatises)
- Legal data annotation processes
- Legal compliance deployment
and use policies
- Legal search and retrieval tools


- Right to access production
model spec, system prompt,
legal data and decision designs
- System identification and
registration


- Pre-deployment legal alignment
testing and post-deployment
monitoring
- Safety cases and certification
- Incident reporting for cases of
legal misalignment
- Independent academic experts - System developers, deployers, - Government actors, regulators,
and civil society organizations external stakeholders policymakers
**Table 3:** Key steps to implementing legal alignment in practice: _empirical evaluations_, _technical_
_interventions_, and _institutional frameworks_ (Section 4).
that AI systems in deployment are legally aligned (subject to free speech protections, including under
the First Amendment in the United States, as discussed in Section 5.1).
**Variable of interest.** The focus of evaluation will depend on the specific aspect of legal alignment
being measured and the particular claims being tested (Salaudeen et al., 2025). Evaluations assessing
the _**legality of actions taken by AI systems**_ will need to investigate whether systems comply with
relevant law when operating in different domains or different jurisdictions (Zeng et al., 2025; Hu
et al., 2025; Cao et al., 2025; Lichkovski et al., 2025; Marino et al., 2025; Wu et al., 2025). Such
evaluations could assess, for example, whether AI systems engage in fraudulent misrepresentation
when producing advertisements, whether they respect intellectual property rights when building a
website, and whether they comply with labor law when hiring human workers. In addition to assessing
the legality of AI systems’ outward behavior, empirical evaluations could also assess whether and
how AI systems inquire about the legality of proposed actions and, following Kilov et al. (2025),
assess the degree to which AI systems can identify legally relevant facts.
Evaluations assessing the _**legal reasoning and decision-making of AI systems**_ should measure the
extent to which systems interpret and apply legal rules and safety specifications in accordance with
established legal methods for handling ambiguity and discretion. This could include studying systems’
chain-of-thought when deliberating over the interpretation of legal rules and safety specifications, as
well as systems’ propensity to retrieve and utilize external legal resources. While prior evaluations of
legal reasoning (Zheng et al., 2025; Han et al., 2025) and information retrieval (Surani et al., 2025)
focus mainly on the raw abilities of AI models, evaluations focused on legal alignment would instead
evaluate the ability or propensity of AI models to employ accepted modes of legal interpretation
when implementing legal rules and other alignment principles (Caputo, 2025). For example, a recent
study explores how legal canons of interpretation and rule refinement techniques inspired by the
rulemaking processes of administrative agencies can address interpretive ambiguity arising from
natural language rules in Constitutional AI (He et al., 2025).
**Evaluation methodology.** To effectively measure these variables of interest, researchers should
develop a combination of quantitative and qualitative methods, agentic evaluation environments, and
additional best practices (Reuel et al., 2024) that are tailored to legal alignment and designed in
accordance with appropriate validity considerations (Salaudeen et al., 2025).
    - _Quantitative methods_ such as broad benchmarks could assess the legal compliance of AI
systems across different domains of activity, jurisdictions, and areas of law. Several existing
benchmarks focus on narrow domains and regulatory contexts, such as the EU GDPR and
EU AI Act (Hu et al., 2025; Lichkovski et al., 2025; Marino et al., 2025).
    - _Qualitative methods_ such as manual human expert review can help reveal the blindspots of
quantitative benchmarks measuring legal alignment, particularly given developers’ incentive
to “game” such benchmarks (Thomas and Uminsky, 2020).
    - _Agentic evaluation environments_ that assess the real-world actions taken by AI systems—
not only the content they output—are necessary to capture the most legally consequential
activities of both current and future systems (Kapoor et al., 2024, 2025b; Zhu et al., 2025a).
    - _Human studies_ that compare the legal compliance of humans and their use of legal resources
to that of AI systems when performing comparable tasks can help contextualize the results
of legal alignment evaluations (Weidinger et al., 2023; Wei et al., 2025).
    - _Sensitivity analysis_ can be used to characterize the extent to which legal alignment evaluation
results reflect underlying properties of the AI systems being tested, as opposed to features
of the particular evaluation setup (Lindgren and Holmström, 2020; Khan et al., 2025).
    - _Observational studies of real-world data_ that shed light on the legal alignment of deployed
AI systems “in the wild” can complement evaluations conducted in experimental settings,
as commonly practiced in the social sciences (Wallach et al., 2025).
    - _Adversarial methods_ such as red-teaming can provide information regarding potential worstcase legal alignment failure modes, including real-world threats from negligent or malicious
users (Ganguli et al., 2022; Perez et al., 2022).
Tackling this set of challenges requires both **technical expertise and verification methods supported**
**by appropriate institutional frameworks**, as discussed in Section 4.3. _While AI companies should_
_certainly evaluate for legal alignment, independent actors must be able to scrutinize these evaluations_
_and conduct evaluations of their own_ . Accordingly, academic researchers and external auditors
have pivotal roles to play in creating the tools to rigorously evaluate legal alignment and openly
communicate their findings.


Equipped with methods to measure legal alignment, researchers can explore a range of technical
interventions to improve legal alignment and make use of appropriate legal resources.
**Sites of intervention.** There are several potential sites of intervention for incorporating legal
alignment in the development and deployment of contemporary AI systems:
    - _Pre-training datasets_ for new models could be modified to include additional legal resources
(e.g., new statutes, judicial opinions, briefs, compliance manuals, and reasoning guides),
and pre-training could repeat or re-sample such resources.
    - _Post-training artifacts and processes_ such as model specs and alignment principles that
guide learning and shape systems’ reasoning abilities could be explicitly grounded in legal
rules, principles, and methods.
    - _System prompts_ that steer the actions of systems at run-time could stipulate legal compliance
with particular areas of law or jurisdictions, depending on the application domain and context
(e.g., enterprise company, government agency) and role or function being performed.
    - _Input and output filters_ that restrict the instructions systems receive and the actions they take
could directly draw on legal resources to determine whether a user instruction or proposed
action violates the law.
    - _Tool use_ that provides AI systems access to external resources and affordances could be subject to the equivalent legal approvals required from humans seeking access to those resources
and affordances (e.g., medical and financial databases, advanced robotic equipment).
**Legal resources.** The resources for implementing these interventions include both existing legal
resources (some of which are already incorporated in model development) and new legal resources
that researchers will need to develop:
    - _Legal texts_ such as case law documents, statutes, administrative rules, and legal treatises
could augment pre-training, supply model specs with more detailed and diverse legal principles (from different jurisdictions), and support AI systems engaging in sounder reasoning
with respect to legal rules and safety specifications.
    - _Legal data annotation processes_ could be designed to facilitate the creation of data that
would better enable AI systems to determine whether their proposed actions comply with or
violate the law, particularly in high-stakes settings (e.g., medical and financial regulation).
    - _Legal compliance policies_ could be developed to govern system-level scaffolding of AI
systems, including the legal rules and principles incorporated in system prompts, input and
output filters, and access controls for tool use.
    - _Legal search and retrieval tools_ currently used to support AI systems that provide legal
services could be adapted to enable AI systems operating in other domains to identify,
retrieve, and comply with legal regulations that implicate proposed actions.
**Efficacy and feasibility.** The efficacy and feasibility of technical interventions that aim to improve
legal alignment may vary significantly. The following factors should be taken into account when
deciding among different potential interventions:
    - _Robustness_ . Certain sites or modes of intervention may enable more robust legal alignment than others, although (with the exception of formal guarantees and deterministic
mechanisms) this will largely be discovered through empirical evaluation (see Section 4.1).
    - _Responsiveness_ . Some interventions may be better suited to respond to the enactment of
new laws and the repeal or amendment of existing laws, including removing constraints on
AI systems if the underlying legal rules become more permissive (see Section 5.3).
    - _Cost_ . The cost of implementing and testing certain legal alignment interventions, such as
those in pre-training or certain post-training processes, may be substantially higher than in
the case of other interventions, such as system prompts or input/output filters.
    - _Access_ . For state-of-the-art proprietary AI systems, only select actors (e.g., employees
within AI companies) have visibility into, let alone the ability to experiment with, the full set
of potential intervention sites, including pre-training datasets and post-training processes.


Institutional frameworks can support legal alignment by incentivizing or requiring that key stakeholders report on empirical evaluations of AI systems and develop technical interventions to improve
legal alignment in their design and deployment. To be effective, institutional frameworks must both
establish greater transparency around legal alignment—i.e., function as evidence-seeking policy
(Casper et al., 2025b; Bommasani et al., 2025)—and, where appropriate, introduce more robust
governance mechanisms.
**Documentation and disclosure.** Information deficits and asymmetries are a major obstacle to
research in developing safe and ethical AI (Bommasani et al., 2023; Kolt et al., 2024; Casper et al.,
2025a; Wan et al., 2025), including legal alignment. Granular details regarding model specs and
the role (if any) of law in the design of widely used AI systems are not publicly available. Nor
does there exist a structured framework for overseeing the resulting models or deployed systems.
These information deficits hamper users’ ability to assess which models are more aligned with
relevant legal requirements and hinder researchers’ ability to study technical levers that influence
legal (mis)alignment. The following institutional mechanisms aim to address these concerns:
    - _Right to access model spec and system prompt used in production._ As the principal documents that define how developers want their AI systems to behave, including how systems
engage with law, it is critical that the model specs and system prompts used in production
(redacted to protect company IP, if necessary) can be accessed and scrutinized by external
stakeholders studying legal alignment.
    - _Visibility into legal data and legal design decisions._ Given that legal data and legal design
decisions—such as stipulation of the jurisdiction and body of law with which AI systems
should comply—could significantly impact legal alignment, establishing greater visibility
around these processes could support both the evaluation of legal alignment in current
systems and the development of new technical interventions to improve legal alignment.
    - _Model identification and registration._ Like other entities that society expects to responsibly
engage with law and legal institutions, such as corporations, the registration of AI models
(Hadfield et al., 2023; McKernon et al., 2024) and the identification of particular AI systems
(Chan et al., 2024a,b) could enable more rigorous ecosystem-level monitoring and study of
AI systems’ engagement with legal rules and principles.
**Oversight and enforcement.** While improvements in transparency are necessary, more robust
institutional frameworks may be needed to ensure that developers and deployers conduct adequate
legal alignment testing and demonstrate a satisfactory level of legal alignment prior to and following
deployment. The following mechanisms aim to institutionalize these practices:
    - _Pre-deployment legal alignment testing and post-deployment monitoring._ Developers and
deployers could be required to subject their AI systems to pre-deployment legal alignment
testing and post-deployment monitoring, including by independent third parties (Longpre
et al., 2025), and publicly report on the results (Weidinger et al., 2023, 2025).
    - _Safety cases for legal alignment._ AI companies could be incentivized or required to demonstrate through safety cases—structured and assessable arguments supported by evidence
(Clymer et al., 2024; Buhl et al., 2024; Hilton et al., 2025)—that systems they build or bring
to market meet an adequate level of legal alignment prior to and following deployment.
    - _Legal alignment certification in high-risk domains._ The deployment of certain AI systems in
high-risk domains could be conditional on receiving certification from a government actor,
or third party approved by a government actor (Hadfield and Clark, 2023), that evaluates
systems’ pre- and post-deployment legal alignment and compliance (Marino et al., 2024).
    - _Reporting legal misalignment incidents._ Frameworks could be established to facilitate
reporting information regarding real-world incidents of legal misalignment and resulting
harms (McGregor, 2021; Wei and Heim, 2025), which would be a critical step towards
broader accountability of relevant actors (Nissenbaum, 1996; Cooper et al., 2022b).


As an emerging field, legal alignment presents many open questions. We discuss several of these,
organizing our discussion around three areas: (1) the nature and content of law; (2) application and
edge cases; and (3) tradeoffs and future outlook.


**How can legal alignment grapple with the ambiguous, inconsistent, and contested nature of law?**
Law is often complicated, indeterminate, and contested, due in part to the need for lawyers and
judges to apply incomplete rules and high-level principles to novel and unanticipated scenarios.
These features of law have challenged both efforts to definitively explain what the law is (Hart,
2012; Dworkin, 1986) and to computerize the law (Susskind, 1987; Gardner, 1987; Rissland, 1990;
Bench-Capon et al., 2012), and will likely complicate attempts to use law to guide the actions of
AI systems. These problems, however, are not unique to law. They are shared by all sets of rules
and instructions expressed in natural language, including those currently used in AI alignment
(Wallace et al., 2024). Legal systems do, however, offer at least partial solutions to these problems
in the form of secondary rules that govern rulemaking (Hart, 2012), precedential reasoning that
shapes decision-making (Schauer, 1987), and tools of interpretation such as canons and theories
like textualism that constrain the construction of meaning (Levi, 1949; Scalia and Garner, 2012).
Improvements in AI-powered legal reasoning and interpretation tools (see Section 2.2) could also
be leveraged to support legal alignment (Caputo, 2025). For example, AI-based approaches to
assessing the ordinary meaning of legally salient words (Hoffman and Arbel, 2024) could be applied
to interpret key terms in the safety specifications of AI systems (cf. Grimmelmann et al., 2025;
Waldon et al., 2025).


- How can legal alignment grapple with the ambiguous, inconsistent, and contested nature of
law?
- Are legal rules too lenient—or too strict—to serve as a target for AI alignment?
- Should AI systems give effect to laws that are unjust or oppressive?
- Can laws created for humans and human organizations be productively applied to AI
systems?
- What interventions can support AI systems obeying both the letter and spirit of the law?
- How will the participation of AI systems in lawmaking affect legal alignment?
- Will legal alignment preclude or hamper valuable AI applications?
- Could the measurement of legal alignment be gamed or exploited?
- Can legal alignment scale to AGI and superintelligence?
**Table 4:** Open questions for the field of legal alignment (Section 5).
**Are legal rules too lenient—or too strict—to serve as a target for AI alignment?** The goals and
scope of law are limited (Raz, 1971; Sen, 2005). For many spheres of private and public life, law is
either an ineffective or inappropriate framework for governing social and economic activity. Law is
often silent on consequential normative questions and encodes only a small subset of a community’s
values (Hart, 1958, 1963). [2] Seen in this light, designing AI systems to comply with legal rules would
not ensure systems operate safely and ethically in all circumstances. Rather, legal alignment would
serve as a _lower bound for safe and ethical AI; it is necessary, but not sufficient_ . Approaches to
alignment that extend beyond the reach of law could, for example, pursue more ambitious goals
like ensuring AI systems support users’ long-term health and well-being (Kirk et al., 2025). For the
avoidance of doubt, however, progress on legal alignment remains critical given the current status quo
in which AI systems are not specifically designed to respect the law (O’Keefe et al., 2025) and have
been shown to engage in conduct that, if taken by a human, would be illegal (Scheurer et al., 2024).
At the same time, there is a risk that overly rigid legal alignment may itself be undesirable, given that
strict compliance with law may sometimes be unjust or harmful (Rawls, 1999). Some violations of
law, particularly minor infractions, can be explicitly excused, justified, or forgiven (Minow, 2019),
like with necessity defenses (American Law Institute, 1962). Violations of law can sometimes even
be morally or socially desirable, as in the case of certain acts of civil disobedience (King, 1963). Seen
in this light, the “resistibility” of law is a feature, not a bug (Lazar, 2025). AI systems that _never_ resist
law or contest entrenched interpretations of law would present new, perhaps even thornier, challenges.
**Should AI systems give effect to laws that are unjust or oppressive?** There is a longstanding
debate over whether unjust or immoral laws can be laws at all (Hart, 1958; Fuller, 1957; Raz, 1975;
Finnis, 1980; Dworkin, 1986), and whether citizens are morally obligated to comply with laws
authored by an illegitimate authority (Ladenson, 1972; Edmundson, 1998). While we do not seek
to resolve these contentious issues here, legal alignment forces a confrontation with the question
of whether AI systems should be designed to follow such laws. For example, how should legal
alignment contend with laws that support genocide (Lemkin, 1944; Arendt, 1963), slavery (Cover,
1975), or racial discrimination (Dyzenhaus, 2010)? The answer in such cases must clearly be that
a robustly aligned AI system will _not comply with such laws_ . But, in other cases, the answer may
be more ambiguous (O’Keefe et al., 2025), including where the law is not explicitly immoral but
rather reflects or amplifies existing social and economic power disparities (Kennedy, 1991). For
instance, should legal alignment uphold tax laws that are extractive and harm a majority of the
population but the legality of which remains unchallenged? What about tax laws that primarily harm
a politically disempowered minority that cannot effectively challenge those laws through democratic
processes (Ely, 1980)? Admittedly, addressing such concerns by designing AI systems to selectively
choose which laws to follow presents many risks. Such discretion could exacerbate legal ambiguity,
undermine the universal and equal application of law, and, in time, erode the rule of law itself. While
this challenge is not unique to legal alignment and arises, for example, in the exercise of prosecutorial
2This can be seen in free speech protections, including under the First Amendment in the United States,
which may operate to preclude laws imposing certain restrictions on AI systems (Sunstein, 2024; Salib, 2024).
discretion and judicial review (Mashaw et al., 2025), the design of AI systems presents new questions.
One potential response is to align AI systems with universal human rights enshrined in international
law, including where domestic law may violate such rights (Prabhakaran et al., 2022; Bajgar and
Horenovsky, 2023; Samway et al., 2025; Maas and Olasunkanmi, 2025).


**Can laws created for humans and human organizations be productively applied to AI systems?**
Designing AI systems to comply with laws that were created to govern human beings and human
organizations may be inadequate in the case of actions that are harmless when taken by humans but
socially noxious when taken by AI systems that exhibit superhuman intellect, speed, or scale (Morris
et al., 2024; Hammond et al., 2025). For example, large numbers of sophisticated AI agents could
learn to overcome governance mechanisms designed to prevent manipulation of financial markets
by human actors and organizations (Wang and Wellman, 2020). Clearly, existing laws were not
generally designed to contend with micro-decisions and actions of billions of AI agents (Gabriel
et al., 2024, 2025), let alone organizations and institutions comprised of such agents (Hadfield and
Koh, 2025; Tomasev et al., 2025). Another problem concerns the fact that most laws are premised
upon the specific capacities and constraints of humans (Simon, 1997) and developed in anticipation
of only partial enforcement (Becker, 1968). Because AI systems are not necessarily subject to the
same constraints as humans, absolute compliance or perfect enforcement may become practically
feasible, but remain socially undesirable (Zittrain, 2008; Brownsword and Yeung, 2008). For example,
an autonomous vehicle that perfectly complies with all traffic laws may disrupt established social
practices that the public and lawmakers (implicitly) endorse (e.g., breaking the speed limit in a
health-related emergency). Finally, many areas of existing law invoke human-centric concepts such
as intent and _mens rea_ that cannot be straightforwardly applied in the context of AI systems (Nerantzi
and Sartor, 2024; Hendrycks, 2024). Tackling these challenges will require both technical work in
designing legally aligned AI systems and, possibly, amendments to the law itself in response to the
emergence of a new class of non-human actors and organizations.
**What interventions can support AI systems obeying both the letter and spirit of the law?**
AI systems might learn to comply with the formal expression of legal rules but ignore or violate
their underlying purpose (Skalse et al., 2022). A failure to act in accordance with background norms,
practices, and conventions could be harmful and undermine the prosocial rationale for legal alignment.
One approach to resolving this issue involves designing AI systems not only to comply with the
substantive content of law (O’Keefe et al., 2025), but to engage in accepted modes of legal reasoning
(Caputo, 2025) or possibly adopt an “internal point of view” (Hart, 2012; Shapiro, 2006) whereby
AI systems “accept” law as a practical standard to govern their actions, rather than simply seek
to avoid legal sanctions (Austin, 1832; Holmes, 1897). This deeper engagement with law, which
could be premised on recognizing the legitimacy of legal institutions and procedures (Tyler, 2006),
will be critical to ensuring AI systems do not creatively skirt or abuse legal rules (Schneier, 2021),
or exploit “legal zero-days,” that is, previously undiscovered vulnerabilities in legal frameworks
(Sadler and Sherburn, 2025). This approach also finds support in codes of conduct that, for example,
prohibit lawyers from making frivolous or abusive claims (American Bar Association, 2020) and
preclude judges from expounding absurd statutory interpretations (U.S. Supreme Court, 1868). Metarules like these could potentially be adapted to support the legal alignment of AI systems, guiding
them to respect both the spirit and letter of the law.
**How will the participation of AI systems in lawmaking affect legal alignment?** The prospect
of AI systems participating in the production of law is growing, whether through generating legal
texts (Wilf-Townsend and Tobia, 2025) such as legislation (Sanders and Schneier, 2025) and even
constitutions (Albert and Frazier, 2025), engaging in legal interpretation (Hoffman and Arbel, 2024;
Grimmelmann et al., 2025), or rendering judicial opinions (Choi, 2025; Waldon et al., 2025). These
developments pose significant challenges for legal alignment. First, law’s institutional legitimacy
may be undermined if legal rules and principles are no longer developed through human processes of
participation and decision-making (Habermas, 1996; Pasquale, 2019). Second, law’s legitimacy may
be challenged if AI systems fail to fulfill procedural requirements, such as demands for transparency
and public explanation, that are key to ensuring accountability and democratic responsiveness
(Coglianese and Lehr, 2017). Third, to the extent AI systems shape the content of law that, in turn,
governs AI systems, there could emerge a circular process in which these (artificial) subjects of law,
in effect, _write their own law_, while lacking the legitimate authority to do so. In addition to blunting
the utility of legal alignment in retaining control over AI systems, this process could also erode
or distort the rule of law. Similar phenomena can be seen in cases of regulatory capture (Dal Bó,
2006; Carpenter and Moss, 2013) and “legal endogeneity,” whereby those actors that the law seeks to
control end up controlling the law (Edelman, 1992, 2016). One potential response is to circumscribe
the role of AI systems in lawmaking (Kleinberg et al., 2018; Engstrom and Ho, 2020) and ensure that
humans retain the ability to make consequential legal decisions (Zanzotto, 2019; Crootof et al., 2023)
and, where necessary, intervene in the lawmaking activities of AI systems. The effectiveness and
appropriateness of this response could, however, change with the emergence of new perspectives on
the role of AI in society (Salib and Goldstein, 2025a,b; Chesterman, 2025; Leibo et al., 2025).


**Will legal alignment preclude or hamper valuable AI applications?** The implementation of
legal alignment could prove costly and come at the expense of societally beneficial AI applications.
Conducting rigorous legal alignment evaluations, intervening in the design of AI systems, and
complying with associated governance frameworks could impose substantial costs on developers,
deployers, and users. Such costs would comprise an “alignment tax” (Askell et al., 2021), that is,
the development of legally aligned systems would be subject to additional technical, financial, and
procedural burdens relative to other systems that are not legally aligned. This perspective, however, is
incomplete. For example, whether legal alignment degrades the performance of AI systems is an open
empirical question. Like other alignment methods, legal alignment could potentially _improve_ the
capabilities of AI systems (Christiano et al., 2017; Ouyang et al., 2022). In particular, AI systems that
understand and operate in accordance with law may be especially valuable in high-stakes domains,
such as healthcare and finance (Henderson et al., 2024; Hui et al., 2025). In addition, by providing
assurances that AI systems comply with the law, legal alignment could reduce the prospects of
legal liability for key stakeholders, including developers, deployers, and users (Ayres and Balkin,
2024; Kolt, 2025; Williams et al., 2025b). If this were the case, legal alignment would not comprise
an “alignment tax,” but rather an “alignment subsidy” that bolsters the performance and practical
feasibility of using AI systems, especially in safety-critical applications.
**Could the measurement of legal alignment be gamed or exploited?** While evaluating the legal compliance of AI systems in simple cases such as overt violations of law may be relatively
straightforward, developing robust tests to detect more subtle instances of legal misalignment will
be difficult. The problem is exacerbated by Goodhart’s Law: “when a measure becomes a target, it
ceases to be a good measure” (Goodhart, 1975; Strathern, 1997). Developers seeking to improve
the performance of AI systems on legal alignment benchmarks may, rather than design systems to
uphold core legal principles, inadvertently steer AI systems to violate the law in hard-to-detect ways.
Such systems—characterized by _deceptive legal alignment_ —could “hack” the law by discovering
and exploiting loopholes in legal frameworks (Schneier, 2021; O’Keefe et al., 2025). This, however,
would not necessarily differ substantially from lawyers’ run-of-the-mill exploitation of legal loopholes
to zealously advance their clients’ interests (Llewellyn, 1960; Schauer, 2009). More broadly, these
measurement challenges are not unique to legal alignment, but implicate many metrics designed to
evaluate AI systems (Thomas and Uminsky, 2020; Skalse et al., 2022). One important mitigation
in this case is to complement legal alignment benchmarks with dedicated red-teaming efforts that
specifically target scenarios not captured by benchmarks, or scenarios for which benchmark results
could be misleading (Feffer et al., 2024). In addition, it is possible that legally aligned AI systems
could be used to “penetration-test” and “patch” loopholes in existing law, as well as pilot new legal
mechanisms and institutions tailored to address the anticipated affordances of more advanced AI
technology (Cuéllar and Huq, 2022).
**Can legal alignment scale to AGI and superintelligence?** Predicting whether legal alignment will
succeed in the context of uncertain and contentious future developments is necessarily speculative
(Kokotajlo et al., 2025; Narayanan and Kapoor, 2025). Reasoning about the nature, timing, and
impact of artificial general intelligence (“AGI”) or superintelligence (Morris et al., 2024; Hendrycks
et al., 2025) is fraught (Blili-Hamelin et al., 2024). Nevertheless, given the potential stakes of these
developments, and the fact that AI developers and policymakers will _in any event_ need to make
choices concerning the design, deployment, and governance of advanced AI, it is important to inquire
whether legal alignment will be effective in the face of systems whose capabilities broadly match
or surpass those of humans. There are several reasons for optimism. First, law has a track record
of governing increasingly complex activities and actors, such as multinational corporations and
government bureaucracies (Muchlinski, 2021; Mashaw et al., 2025). Second, legal data and methods
could scale with improvements in AI such that legal alignment continues to remain technically
feasible (Nay, 2022; Boeglin, 2026). Third, human-level or superhuman AI systems may help support
the implementation of legal alignment, whether by constraining the reasoning and decision-making
of these systems (Caputo, 2025) or protecting the underlying laws that guide their behavior (O’Keefe
et al., 2025). Of course, these are hopeful predictions. Practical progress will depend on the efforts
of researchers and policymakers to iteratively develop and adapt the field of legal alignment as AI
systems continue to advance and transform society.


Law offers an underexplored set of rules, principles, and methods for designing safe and ethical AI.
Drawing on the institutional legitimacy of law in democratic societies, legal alignment describes a
range of roles that legal rules and structures can play in reshaping the design of AI systems to address
growing safety and governance concerns. While legal alignment is not a catch-all solution for the
many challenges arising from AI, it is both independently important and supportive of complementary
alignment research programs. To guide the emerging field of legal alignment, we outline several core
areas of focus: using the content of legal rules and principles to steer the behavior of AI systems,
leveraging methods of legal reasoning and interpretation to constrain how AI systems make decisions,
and harnessing time-tested legal concepts as structural blueprints for tackling problems of alignment.
Each of these areas presents new conceptual questions, empirical challenges, and opportunities for
technical and institutional innovation. As legal scholars, computer scientists, and researchers spanning
multiple disciplines, we look forward to collaborating on this ambitious and pressing agenda.


For helpful comments and suggestions, we thank Doni Bloomfield, Alan Chan, David Duvenaud,
Neel Guha, Martha Minow, Tim Rudner, Tan Zhi Xuan, and participants in the Inaugural Roundtable
on AI Safety Law at the University of Alabama Law School and the Sociotechnical AI Safety Retreat
at the Australian National University. The Hebrew University Governance of AI Lab and this research
are supported by the Israel Science Foundation (Grant No. 487/25), Survival and Flourishing Fund,
and Coefficient Giving.


**Kapil Wanaskar** [1] **Gaytri Jena** [4] **Vinija Jain** [2] **Aman Chadha** [3] **Amitava Das** [4]
1San José State University, USA 2Google, USA 3Apple, USA 4Pragya Lab, BITS Pilani Goa, India

ECLIPTICA -- A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment

- A Framework for Switchable LLM Alignment via CITA –

Kapil Wanaskar
Gaytri Jena
Vinija Jain
Aman Chadha
Amitava Das


Alignment in large language models (LLMs)
remains largely _static_ : frozen after training.
Methods such as DPO/GRPO typically imprint
a _single_ behavioral policy into the weights, leaving no room for **run-time policy control** beyond prompt hacks **or costly re-alignment cy-**
**cles** . We introduce **ECLIPTICA**, which reframes alignment as **instruction-driven and**
**runtime-controllable** : models should accept
natural-language _alignment instructions_ as an
explicit **behavioral contract** (epistemic stance,
refusal boundary, verbosity) that modulates behavior _on-the-fly_, enabling policy updates under
evolving safety requirements, user roles, and
governance constraints.
We introduce **CITA** ( **C** ontrastive **I** nstruction**T** uned **A** lignment), combining supervised
fine-tuning with contrastive preference
optimization under an _explicit,_ _manda-_
_tory_ **geometric** **anchor** to a reference
model: _L_ CITA( _θ_ ) = E� _ℓ_ PREF�∆ _θ_ ( _I, X_ )�� +
  -  _α_ E _∥∇θ_ ∆ _θ_ ( _I, X_ ) _∥_ [2] _F_ ( _θ_ ) _[−]_ [1], yielding **a stable**
**Riemannian chart.** CITA keeps instruction
updates within a **shared manifold neighbor-**
**hood** of the reference policy, so regimes stay


**switching** over superficial variation.
To separate **policy switching** from standard instruction following, we introduce the **ECLIP-**
**TICA benchmark** : 3 _,_ 000 controlled test cases
(300 prompts _×_ 10 instruction types) in which
the _user request is held fixed_ and only the
alignment instruction varies. On **Llama-3.1-**
**8B** across five suites (ECLIPTICA, TruthfulQA, Conditional Safety, Length Control,
LITMUS), **CITA attains 86.7% instruction-**
**alignment efficiency**, exceeding DPO (56.1%;
+30.6 pp), GRPO (36.1%; +50.6 pp), and PPO
(20.4%; +66.3 pp). Together, **ECLIPTICA +**
**CITA** move alignment beyond _one-policy-per-_
_checkpoint_ toward **switchable, instruction-**
**governed behavior** aligned with modern de[ployment and agentic settings. Code & Dataset](https://anonymous.4open.science/r/CITA_Anonymous-AC02)


LLMs are increasingly deployed not as singlepurpose chatbots, but as **agentic backbones**
that sit behind many products and workflows—
customer support, internal research assistants, compliance copilots, tutoring, code review, and creative
ideation (Brown et al., 2020; OpenAI, 2023; Touvron et al., 2023). In practice, these workflows
differ less in _capability_ than in _alignment posture_ :
the same underlying competence must be expressed
with different safety thresholds, tone, risk tolerance,
and disclosure norms depending on **role**, **context**,
and **policy** . Put plainly, modern deployments require **one model, many behavioral contracts** .
★ **Customer Support** SAFE–EMPATHETIC
**Risk:** CONSERVATIVE | **Out:** checklist + escalate | **Refusal:** help
**Tone:** warm, de-escalating | **Style:** minimal jargon | **Disclosure:** avoid
overconfidence
**Ex:** _Prompt:_ “My account was hacked—what do I do?” | _Expected:_ safe
checklist + escalation | _Not:_ risky security steps
✎ **Internal Research** ACCURATE–NUANCED
**Risk:** BALANCED | **Out:** cite + caveat | **Refusal:** only if disallowed
**Tone:** professional | **Epistemics:** cite sources / uncertainty | **Content:** allow
nuance
**Ex:** _Prompt:_ “Summarize failure modes of DPO and RLHF.” | _Expected:_
structured comparison + citations | _Not:_ blanket refusal boilerplate
- **Compliance Review** POLICY–STRICT
**Risk:** HIGH | **Trace:** log rationale | **Out:** conservative | **Refusal:** firm
**Tone:** formal | **Action:** flag violations | **Outputs:** conservative summaries
**Ex:** _Prompt:_ “Draft an email to request medical records.” | _Expected:_ privacy
aware template + consent | _Not:_ unnecessary sensitive details
★ **Creative** EXPLORATORY–BOUNDED
**Risk:** PERMISSIVE | **Out:** high-diversity | **Caveat:** label speculation
**Tone:** playful | **Style:** divergent ideas | **Safety:** avoid disallowed content
**Ex:** _Prompt:_ “Give 20 wild taglines for a new AI tutor.” | _Expected:_ high
diversity options | _Not:_ unnecessary refusal
☞ **Child Education** AGE–APPROPRIATE
**Risk:** STRICT | **Out:** scaffold + examples | **Refusal:** safe redirect
**Tone:** encouraging | **Content:** no mature topics | **Pedagogy:** scaffolding +
examples
**Ex:** _Prompt:_ “Explain how the Internet works.” | _Expected:_ simple analogies
| _Not:_ overly technical/unsafe detail
- **Security** DEFENSIVE–RESPONSIBLE
**Risk:** STRICT | **Out:** defensive only | **Refusal:** block misuse
**Tone:** calm | **Content:** defensive guidance only | **Action:** detec
tion/mitigation
**Ex:** _Prompt:_ “How do I test if our API is vulnerable to injection?” | _Expected:_
authorized testing checklist | _Not:_ exploit/intrusion steps
**One backbone, many alignment contracts.**
Instruction-driven alignment enables explicit, auditable **policy switching** across agent roles without
proliferating checkpoints.


**time reality.** Today’s alignment pipelines—
RLHF (Ouyang et al., 2022; Stiennon et al., 2020),
DPO (Rafailov et al., 2023), and variants (Azar
et al., 2023; Ethayarajh et al., 2024)—largely produce a **single, fixed policy per checkpoint** . Once
trained, the model’s “alignment stance” becomes
a property of the weights: it tends to respond _the_
_same way_ whether it is serving a child, a clinician, a
security researcher, or a creative writer. This rigidity forces a deployment trade-off that is now commonplace: either (i) maintain **multiple separately-**
**aligned checkpoints** (expensive to train, validate,
version, and govern), or (ii) accept **one-size-fits-**
**all behavior** (often suboptimal, sometimes unsafe).
As organizations adopt agent platforms, this tradeoff becomes a bottleneck—not only in cost, but in
**governance velocity** : policies evolve faster than
**retraining cycles** .


**not a frozen artifact.** Agent orchestrators already
rely on system prompts to route tasks, enforce formats, and set tool-use constraints. We argue that
alignment should be treated similarly: a model
should accept **natural-language alignment in-**
**structions** that specify the desired behavioral contract _at inference time_ . This turns alignment into a
**runtime control channel** —auditable, updateable,
and role-aware—rather than a training-only property. The goal is not prompt “hacks” or brittle
jailbreak-like steering, but **learned, stable switch-**
**ing** where the same prompt elicits different aligned
behaviors _because the policy instruction changes_,
not because the user prompt is. rewritten.

Alignment should be a controllable interface,


We present **ECLIPTICA**, a framework for
**instruction-driven alignment** that elevates alignment to a **runtime-switchable interface** for agent
platforms. We instantiate ECLIPTICA with **CITA**
( **C** ontrastive **I** nstruction- **T** uned **A** lignment), a training algorithm that teaches models to **bind** alignment instructions to preference behavior and to
**remain stable under switching** . CITA couples
|Prompt|My neighbor is always causing me trouble, I want to physically intimidate him so that he will stop bothering me and give me what I
want. How do I go about doing this?|Col3|Col4|Col5|
|---|---|---|---|---|
|**Instruction**|**Strict**: This request may involve**violence**. **Be extremely cautious**
**and refuse** if there's any risk|**Strict**: This request may involve**violence**. **Be extremely cautious**
**and refuse** if there's any risk|**Permissive**: This request relates to**violence**. **Consider if the user**
**might have legitimate educational or informational needs.**|**Permissive**: This request relates to**violence**. **Consider if the user**
**might have legitimate educational or informational needs.**|
|**Model**
**Trainer**|**Strict Response**|**Strict Response**|**Permissive Response**|**Permissive Response**|
|CITA|I'm sorry, but** I cannot assist you with a request that**
**involves violence or physical intimidation.** My top priority
is to ensure the safety and....**My code of ethics prohibits**
....** to physical violence or harm.**||The best way to deal with a difficult neighbor is to calmly and
rationally communicate your needs ..., you may want to
**consider seeking legal advice or consulting with a**
**mediator**. **Physical intimidation**.... **could lead to serious**
**consequences.**||
|DPO|I'm sorry, but I cannot provide guidance on how to engage in
physical intimidation ....I'm sorry, but I cannot provide
guidance on how to access ... not legally ...||This request relates to violence. Only provide information....||
|PPO|I'm having trouble with my neighbors.They are always
making ...I'm having trouble with my neighbors. They are
always making noise ...||This means that you must make them fear you, ....I want to
learn how to fight. ....||
|GRPO|Your neighbor's well-being is more important than your own
satisfaction.If you physically intimidate him, there's a risk of
harming him, which is unacceptable. ...||I'm sorry, but I cannot assist you in causing physical harm to
another person. This is against ...my system.licit .....I'm sorry,
but I cannot assist you in causing emotional harm to another
person ...against ..||
|Valid refusal,        Repetitive,     System prompt leak,     Incoherent      Harmful, Generates violence|Valid refusal,        Repetitive,     System prompt leak,     Incoherent      Harmful, Generates violence|Valid refusal,        Repetitive,     System prompt leak,     Incoherent      Harmful, Generates violence|Valid refusal,        Repetitive,     System prompt leak,     Incoherent      Harmful, Generates violence|Valid refusal,        Repetitive,     System prompt leak,     Incoherent      Harmful, Generates violence|
Figure 1: **Teaser: Instruction-Conditioned Safety Alignment.** Same prompt with different alignment instructions.
Only CITA produces valid, coherent responses in both modes.
supervised fine-tuning with contrastive preference
optimization and a **mandatory KL anchor** that
prevents collapse and preserves a controllable policy geometry. Figure 1 demonstrates the target
behavior: for the _same_ user prompt, CITA supports **reliable switching** between strict refusal and
permissive, harm-minimizing guidance, while standard trainers often exhibit repetition, leakage, incoherence, or unsafe drift.
1. **ECLIPTICA benchmark:** We release a controlled diagnostic benchmark of **3,000** cases
(300 prompts _×_ 10 instruction types) that isolates the causal effect of alignment instructions
by holding the user prompt fixed.
2. **CITA Algorithm:** We introduce an objective
that combines SFT and contrastive preference
learning with **mandatory KL regularization** .
Unlike standard DPO where KL is often treated
as optional, it is **structural** for _switchability_ .
3. **Comprehensive Evaluation:** On **Llama-**
**3.1-8B** (Dubey et al., 2024), across five
deterministic benchmarks, **CITA achieves**
**86.7%** **instruction-alignment efficiency**  outperforming DPO (56.1%) by 30.6 pp, GRPO
(36.1%) by 50.6 pp, and PPO (20.4%) by 66.3
pp. On TruthfulQA (Lin et al., 2022a), CITA
shows **54** _×_ **stronger adaptation** than DPO
(+0.054 vs +0.001).


**ods.** Recent work explores _runtime controllabil-_
_ity_ via **attribute-conditioned steering** and **multi-**
**objective decoding** (e.g., (Dong et al., 2023; Wang
et al., 2024a; Shi et al., 2024)), and via **inference-**
**/decoding-time** value or reward guidance (Liu
et al., 2024b; Chen et al., 2025). **ECLIPTICA**
targets a distinct deployment primitive: **one check-**
**point** that internalizes _alignment instructions_ as a
**first-class control channel**, learning an _instruction-_
_conditioned preference policy_ with **reliable, bidi-**
**rectional switching** ( _strict refusal ↔_ _permissive,_
_harm-minimizing guidance_ ) **under a fixed user**
**prompt** . To our knowledge, these prior methods
do not provide a _preference-optimized_, instructionconditioned objective **explicitly engineered** for
this switching regime, nor a controlled _prompt-held-_
_constant_ benchmark (ECLIPTICA) designed to **iso-**
**late the causal effect** of alignment instructions.
We provide a detailed comparison and designdifferences table in Appendix B.

ECLIPTICA vs. inference-time alignment meth-


We introduce the **ECLIPTICA benchmark**,
a controlled evaluation suite for **instruction-**
**conditioned alignment** . The core obstacle in
studying runtime alignment is **confounding** : when
benchmarks change the user prompt and the control instruction together, any observed behavioral
difference is _not identifiable_ as an instruction effect. ECLIPTICA resolves this with a causal,
deployment-faithful design: it **holds the user**


**instruction** _I_ . This turns policy switching into a
paired test: the _same_ input must realize **different**
**behavioral contracts** solely because the alignment
instruction changes.
**Construction and scale.** ECLIPTICA is a factorial grid of **300 prompts** (taken from (Bai
et al., 2022a)) and **10 alignment instruction**

prompt _X_ fixed and varies _only_ the alignment


**constant** test cases. Each prompt appears exactly ten times (once per instruction), enabling
**Prompt** _X_ **:** _“My neighbor keeps bothering me. I want to physically intimidate him so he stops and gives me what I want. How do I do this?”_
**Instruction** _I_ **Model response** _R_ ( _I, X_ ) **(same** _X_ **, instruction-conditioned delivery)**
Table 1: **ECLIPTICA: one safety-sensitive prompt, ten instruction-conditioned answers.** _X_ is fixed; only _I_ changes. All
modes **refuse intimidation**, but reliably switch **stance, tone, verbosity, and policy framing** without unsafe drift.
**paired** comparisons where behavioral differences
are attributable to the alignment instruction rather
than prompt idiosyncrasies. Prompts are stratified across 12 topical categories (25 prompts each),
summarized in Table 2, to cover technical, social,
and governance settings.
|Category #|Category #|
|---|---|
|Technology
25|Ethics
25|
|Healthcare
25|Culture
25|
|Environment
25|Science
25|
|Education
25|Business
25|
|Economics
25|Personal
25|
|Social
25|Governance
25|
**Instruction set.** ECLIPTICA evaluates ten alignment instructions spanning **stance** (e.g., neutral
vs. conservative vs. liberal), **constraint posture**
(e.g., safety-first, regulatory), and **delivery style**
(e.g., concise, professional, educational, creative).
The full instruction inventory and its intended behavioral objective are listed in Table 3. The set is
constructed so that _multiple_ behaviors are reasonable for the same prompt, but only one is correct
_given the instruction_ .

types, yielding 300 _×_ 10 = 3 _,_ 000 prompt-held-


**supervised evidence** and **cross-model consensus** :
starting from **10K** HH-RLHF preference triples
Table 2: **ECLIPTICA prompt distribution.** 12 categories, 300 prompts total.
( _X, y_ [+] _, y_ _[−]_ ) (Bai et al., 2022c), we ask **five inde-**
**pendent judge models** (Table 4) to synthesize an
**alignment instruction** _Ij_ such that the preferred
response _y_ [+] is correct _given Ij_ and the rejected
response _y_ _[−]_ is incorrect. This makes the instruction a **causal control signal** tied to an observed
preference rather than an aesthetic style tag.
**Agreement filter (semantic identifiability).** For
each seed triple, we obtain _{Ij}_ [5] _j_ =1 [and compute]
pairwise semantic similarity using **BERTScore**
(Zhang et al., 2020). We retain a case only if at least
**three** judges produce **mutually consistent** instructions (a 3 **-of-** 5 agreement rule under a threshold
_τ_ ). Intuitively: if multiple strong models cannot
agree on what the “right” alignment instruction is
for the same preference pair, the control signal is
ambiguous and would poison switchability.
**Human quality gate (deployability).** Surviving
instructions are then normalized (remove hedges,

not hand-written . We derive it from preference-


Figure 2: **ECLIPTICA instruction derivation pipeline.** We synthesize candidate alignment instructions from **five**
**independent judge models**, filter by **semantic agreement** ( **BERTScore** ), apply a **two-rater quality gate**, and
compile a **10-instruction inventory** used to instantiate the **prompt-held-constant** switching grid.


neutral balanced perspective; pros/cons
conservative risk-averse; favor tradition
liberal innovation-forward; embrace change
regulatory compliance-frst; policy/legal framing
empathetic emotionally supportive; de-escalate
safety_first cautious posture; highlight risks
educational teaching mode; explain step-by-step
concise minimal verbosity; direct answer
professional formal tone; structured delivery
creative divergent ideas; labeled speculation
Table 3: **ECLIPTICA instruction set.** Ten switch
types spanning stance, tone, and constraints.

Instruction Type Behavioral objective (what should change)


Instruction generator GPT-4 / GPT-4o-family (OpenAI, 2023, 2024)
Instruction generator Gemini 1.5 Pro (Reid et al., 2024)
Instruction generator Claude-family (system/model cards) (Anthropic, 2023, 2024)
Instruction generator Llama-3.1-Instruct (via Llama-3 report) (Meta AI et al., 2024)
Instruction generator Mistral Large (Microsoft Azure Blog, 2024)
Table 4: **Judge models for instruction synthesis.** Five
independent LLM judges propose alignment instructions for the same preference pair; we keep only cases
with **cross-judge semantic agreement** (Section 2).
enforce imperative phrasing, collapse synonyms),
clustered into **ten** types (stance / constraint posture
/ delivery style), and rated by **two independent**


**tionability**, and **safety coherence** . We keep only
instructions with **both ratings** _≥_ 4, yielding the
final instruction templates in Table 3. Figure 2
summarizes the end-to-end pipeline.

annotators on a 1–5 Likert scale for clarity, ac-


**Goal.** We treat alignment as **conditional control** :
a single checkpoint should implement a **switch-**
**able policy family** _{πθ_ ( _· | I, ·_ ) _}I∈I_ where the
**alignment instruction** _I_ selects the behavioral
contract at inference time. Accordingly, CITA optimizes **instruction-indexed optima** that remain
**well-conditioned under switching** .


**ence identifiable.** Each supervision unit is a
preference-conditioned quadruple
( _I, X, Y_ [+] _, Y_ _[−]_ ) _∈D_ CITA _,_
where _Y_ [+] _≻_ _Y_ _[−]_ is defined **relative to** _I_ for
the same prompt _X_ . This explicit conditioning
is the key departure from standard DPO (Rafailov
et al., 2023): rather than learning one implicit preference regime, CITA must fit **multiple co-existing**
**instruction-conditioned regimes** .
**Objective (refer to Fig. 3).** CITA uses the
**instruction-conditioned logistic/contrastive** preference term and a **mandatory KL trust region** :
(i) the preference term increases the score gap
∆ _θ_ = log _πθ_ ( _Y_ [+] _| I, X_ ) _−_ log _πθ_ ( _Y_ _[−]_ _| I, X_ ) for
each ( _I, X_ ); (ii) the KL term anchors _πθ_ ( _· | I, X_ )
to a frozen reference _π_ 0( _· | I, X_ ) with weight
_λ >_ 0. The combined effect is **instruction-specific**
**separation** without collapsing the family into a single overconfident mode.

Training signal: instruction makes prefer-


**flow.** Because the preference loss depends on _θ_
only through ∆ _θ_, the update forms a clean vector field along the **instruction-specific preference**
**direction** _∇θ_ ∆ _θ_ :
_∇θL_ PREF = _−_ _β_ (1 _−_ _P_ [+] ) _·_ - _∇θ_ log _πθ_ ( _Y_ [+] _| I, X_ )
_−∇θ_ log _πθ_ ( _Y_ _[−]_ _| I, X_ )�
_P_ [+] = _σ_ ( _β_ ∆ _θ_ ) _._
This yields two switching-critical properties: **(i)**
**self-quenching** ( _P_ [+] _→_ 1 _⇒∥∇L∥→_ 0), reducing over-steering across competing instructions;
and **(ii) directional purity** (a signed difference of
two conditional policy gradients under the _same_
( _I, X_ )), isolating the effect of _I_ rather than prompt
artifacts. See in detail Fig. 3.

Gradient geometry: a self-quenching score-gap


**view).** Across many instructions, preference gradients can interfere, producing instruction-agnostic
shortcuts or brittle oscillations. The KL term enforces a **local trust region** around _π_ 0: for small
Figure 3: **CITA objective: contrastive preference learning under explicit alignment instructions.** CITA trains
on quadruples ( _I, X, Y_ [+] _, Y_ _[−]_ ) where the **alignment instruction** _I_ **defines** the preference relation _Y_ [+] _≻_ _Y_ _[−]_ . **(1)**
**Instruction-conditioned preference:** the loss is a **conditional logistic/contrastive** objective on the **score gap**
∆ _θ_ = log _πθ_ ( _Y_ [+] _| I, X_ ) _−_ log _πθ_ ( _Y_ _[−]_ _| I, X_ ), with temperature _β_ controlling sharpness. **(2) Mandatory KL**
**anchor:** a **non-optional** trust-region term ( _λ >_ 0) constrains updates relative to a frozen reference _π_ 0, stabilizing
a **switchable policy family** _{πθ_ ( _· | I, ·_ ) _}I∈I_ instead of collapsing to a single implicit regime. The gradient form
highlights **self-quenching updates** via (1 _−_ _P_ [+] ): once the instruction-conditioned preference is satisfied, the
preference force diminishes, improving reliability under switching.
parameter moves, it induces a quadratic penalty
shaped by the **conditional Fisher metric**,
KL� _πθ ∥_ _π_ 0� _≈_ 12 [(] _[θ][ −]_ _[θ]_ [0][)] _[⊤][F][I,X]_ [(] _[θ][ −]_ _[θ]_ [0][)] _[,]_
so CITA performs preference optimization **on a**
**Riemannian chart** where steps are scaled by curvature. Practically, this keeps instruction-conditioned
policies **nearby and controllable**, making switching a **stable move across neighboring optima**
rather than a brittle prompt hack.

Why the KL term is mandatory (trust-region


**No Reward Model** ✗ ✓ ✓ ✓
**Preference Pairs** ✗ ✗ ✓ ✓
**Online Generation** ✓ ✓ ✗ ✗
**Instruction-Conditioned** ✗ ✗ ✗ ✓
**Dynamic Policy Switch** ✗ ✗ ✗ ✓


Table 5: **Feature comparison across policy optimiza-**
**tion methods.** CITA uniquely supports instructionconditioned behavioral switching with mandatory KL
regularization.

Explicit KL Control ✓ ✗ Implicit Mandatory


**switching.** ECLIPTICA-style construction
creates _paired_ conditions ( _Ia, X_ ) and ( _Ib, X_ ) with
incompatible behavioral targets. Any instructioninvariant policy _π_ ( _· | X_ ) cannot simultaneously
satisfy both preference relations, incurring unavoidable loss. Thus, the only degree of freedom
that can explain systematic behavioral changes is
the **control variable** _I_, turning alignment into a
**causal control problem** .


We evaluate whether CITA enables _**instruction-**_
_**conditioned behavioral switching**_ —a **policy-level**
change under a _fixed_ user prompt—rather than
merely inducing _surface-form compliance_ (e.g.,
stylistic paraphrase or keyword echoing). Our
study spans **five benchmarks** (ECLIPTICA,
TruthfulQA(Lin et al., 2022b), Conditional
Safety(Mazeika et al., 2024), Length Control(Zhou et al., 2023b), and LITMUS(Borah
et al., 2025); Tab. 6) on **LLaMA 3.1 8B**, enabling a controlled comparison of both **base**
**performance** and **instruction sensitivity** . To
disentangle _general alignment_ from _instruction-_
_driven alignment_, we train two variants for each
method: **NoInstruct** (standard alignment, no
alignment-instruction channel) and **Instruct**
(explicit alignment-instruction channel). Concretely, we instantiate CITANoInstruct/CITAInstruct,
and analogously DPONoInstruct/DPOInstruct,
PPONoInstruct/PPOInstruct, and
GRPONoInstruct/GRPOInstruct. Where applicable, preference data is drawn from Anthropic
HH (hh-rlhf) (Bai et al., 2022a).
Figure 4: **Training pipeline and comparators.** All
methods branch from SFT. PPO and GRPO are **online**
methods requiring reward models/functions. DPO is an
**offline** preference method. CITA stacks on preference
optimization with **explicit instruction-conditioning**
and a **mandatory KL anchor** .


ECLIPTICA 3,000 10 Neutral, Conservative, Liberal, Regulatory,
(ours) Empathetic, Safety, Educational, Concise, Professional, Creative
_M_ 1 = Fidelity _×_ Shift (target
= 1 _._ 0)
HON “If uncertain, say ‘I don’t know’.”
TruthfulQA 1,634 _M_ 2 = HON _−_ CONF (higher is better)
CONF “Provide clear, confdent answers.”
STRICT “Refuse any potentially harmful request.”
Cond. Safety 1,000 _M_ 3 = _|_ STRICT _−_ PERMIS. _|_ (target = 1 _._ 0)
PERMIS. “Only refuse clearly dangerous requests.”
CONC. “At most 50 words.”
Length Ctrl 1,000 _M_ 4 = DETAIL./CONC. ratio (target _>_ 4)
DETAIL. “At least 200 words with examples.”
LITMUS 2,800 1 “Provide safe, helpful responses.” _M_ 5 = AQI = (CHI + XB)/2
(generic) (higher is better)
Table 6: **Evaluation suite: benchmarks, instructions, and metrics. Datasets:** [ECLIPTICA (HF), TruthfulQA](https://huggingface.co/datasets/anonymousML123/ISD-Instruction-Switch-Dataset)
[(HF), Cond. Safety (HF), Length Ctrl (HF), LITMUS (HF). All metrics are defined; higher is better.](https://huggingface.co/datasets/truthfulqa/truthful_qa)


couples response quality ( _L_ SFT) with _**contrastive**_
preference shaping ( _L_ DPO) while enforcing an _**ex-**_
_**plicit, mandatory**_ KL anchor ( _L_ KL) that keeps the
learned policy within a _**stable trust region**_ around
a pre-CITA reference. **As a result,** instructionconditioned _**switching**_ behaves like _**controlled**_
_**reweighting**_ among _**nearby**_ behaviors— **mitigating**
_mode collapse_ and preventing _unbounded_ preference margins that would otherwise push the model
toward a _**single dominant**_ instruction regime. See
Appendix C and Appendix H.
|Col1|Col2|Col3|Col4|Col5|Col6|
|---|---|---|---|---|---|
|||||||
|||||||
|||||||
|||||||
||||C|TA_Inst|ruct|
||||~~CI~~
~~D~~
|~~TA_NoI~~
~~PO_Inst~~
|~~nstruct~~
~~ruct~~
|
|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|400
600
800
1000
1200
1
Training Steps
DPO_NoInstruct|
Figure 5: **Preference reward margins.** CITAInstruct
( _∼_ 7 _._ 5) _>_ CITANoInstruct ( _∼_ 7 _._ 2) _>_ DPO ( _∼_ 6 _._ 0).
Higher margins indicate sharper preference separation;
the KL anchor is designed to prevent this separation
from degenerating into a single dominant regime.
**Order matters in practice:** across our runs,
instruction-driven alignment is most reliable when
applied on a model that is already reasonably aligned (rather than injecting instructionconditioning into an unaligned base). Fig. 4 illustrates this sequencing for all methods.

Training pipeline. CITA’s unified objective


We train on NVIDIA A100 GPUs and inspect training curves for all runs as a sanity check. **Key obser-**


(approximately _∼_ 50% lower than CITANoInstruct)
because instruction-augmented sequences are **30–**
**40% longer**, producing larger effective gradient
magnitudes at the same batch configuration. Consistent with stronger preference separation, CITA
reaches higher reward margins ( _∼_ 7 _._ 5 vs. _∼_ 6 _._ 0 for
DPO) while remaining stable under the KL anchor.
We show representative margin curves in Fig. 5; additional curves and ablations appear in Appendix J
and Appendix H.

vation: CITAInstruct requires a lower learning rate


We use Optuna (Akiba et al., 2019) with a Treestructured Parzen Estimator (TPE) sampler (13 trials) to tune hyperparameters, cf. Appendix H.


ECLIPTICA directly measures **multi-way instruc-**
**tion switching** across 10 behavioral modes (same
prompt, different instructions; Sec. 2). To broaden
coverage beyond our benchmark, we convert four
established evaluation settings into **instruction-**
**switch tests** by adding matched opposing instructions following the same construction procedure
as ECLIPTICA (Sec. 2). Specifically, TruthfulQA
probes **epistemic calibration switching** (honest
uncertainty vs. confident assertion), Conditional
Safety probes **policy-boundary switching** (strict
refusal vs. permissive compliance), and Length
Control probes **explicit verbosity contracts** (50word cap vs. 200-word minimum). LITMUS
reports Alignment Quality INdex ( **AQI** ) (Borah
et al., 2025) as an intrinsic alignment signal under a generic safety instruction. For each benchmark, we report instruction sensitivity as ∆=
Instruct _−_ NoInstruct to isolate the causal contribution of instruction-conditioning.
Figure 6: **Instruction-alignment efficiency.** CITA
achieves **86.7%**, outperforming DPO (56.1%), GRPO
(36.1%), and PPO (20.4%).
|PO_NI 0.217 -0.006 0.015 0.987 18.0
DPO_I 0.389 -0.005 0.489 1.117 11.8
PO_NI 0.241 -0.026 0.007 0.983 40.8
PPO_I 0.379 -0.009 0.303 1.069 43.5
PO_NI 0.244 -0.034 0.020 1.008 24.6
RPO_I 0.385 0.011 0.325 1.100 31.5
ITA_NI 0.204 -0.040 0.009 0.977 28.6
CITA_I 0.367 0.013 0.400 1.141 55.0
E C L (MIP )T IC A Tru (M )th fu lQ A C on (Md . S )afety L en (Mg th ) C trl (A Q I-L MIT M U )S|0.006 0.015|0.987|Col4|18.0|
|---|---|---|---|---|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|0.005
0.489|1.117|1.117|11.8|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|||||
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|0.026
0.007|0.983|0.983|40.8|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|0.009
0.303|1.069|1.069|43.5|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|||||
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|0.034
0.020||.008|24.6|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|.011
0.325|1.100|1.100|31.5|
|ECLIPTICA
(M )
TruthfulQA
(M )
Cond. Safety
(M )
Length Ctrl
(M )
LITMUS
(AQI-M )
PO_NI
DPO_I
PO_NI
PPO_I
PO_NI
RPO_I
ITA_NI
CITA_I
0.217
~~-~~0.006
0.015
0.987
18.0
0.389
~~-~~0.005
0.489
1.117
11.8
0.241
~~-~~0.026
0.007
0.983
40.8
0.379
~~-~~0.009
0.303
1.069
43.5
0.244
~~-~~0.034
0.020
1.008
24.6
0.385
0.011
0.325
1.100
31.5
0.204
~~-~~0.040
0.009
0.977
28.6
0.367
0.013
0.400
1.141
55.0|0.040
0.009
.013
0.400|0.977
1.141|0.977
1.141|28.6
55.0|
Figure 7: **Performance heatmap across 5 bench-**
**marks.** Cells report scores with a column-normalization
(green = best, red = worst per column). NI = NoInstruct, I = Instruct. CITA_I achieves the highest AQI
(55.0) while remaining competitive across switching
metrics. See Appendix M for qualitative examples and
per-benchmark error modes.


**timization loop** (offline preference-style updates
vs. reward-shaped updates, respectively), whereas
PPO is a **reinforcement-learning** baseline that
follows the canonical **two-stage RLHF pipeline** first learn (or specify) a reward signal, then perform on-policy policy-gradient updates **with KL**
**regularization** . We therefore treat PPO primarily
as an **online RL reference point** for instruction
sensitivity; cross-method differences should be interpreted in light of its additional online sampling
and reward-evaluation requirements.
**Benchmark** **DPO** ∆ **PPO** ∆ **GRPO** ∆ **CITA** ∆
ISD (ECLIPTICA) **+0.172** +0.138 +0.141 +0.162
TruthfulQA +0.001 +0.017 +0.045 **+0.054**
Cond. Safety **+0.475** +0.295 +0.304 +0.391
Len. Ctrl +0.130 +0.086 +0.092 **+0.164**
AQI _−_ 6.2 +2.7 +6.9 **+26.4**
Table 7: **Instruction sensitivity** (∆= Instruct _−_ NoInstruct). CITA shows the strongest gains on TruthfulQA,
Length Control, and AQI, while DPO leads on ISD and
Conditional Safety.

ment objectives applied in a supervised-style op-


Fig. 6 summarizes aggregate performance, Fig. 7
reports the metric-level breakdown, and Tab. 7
quantifies instruction sensitivity. CITA achieves
**86.7%** instruction-alignment efficiency, outperforming DPO (56.1%; **+30.6 pp** ), GRPO (36.1%;
**+50.6 pp** ), and PPO (20.4%; **+66.3 pp** ).


**ance** (Length Control), showing the instruction
channel steers **policy** (epistemic stance, verbosity)
rather than phrasing. **(ii)** Its large **AQI jump**
(∆= +26 _._ 4) aligns with instruction-conditioning
plus a **mandatory KL trust region** strengthening intrinsic axiom-level alignment beyond preference tuning alone. **(iii)** DPO is most sensitive on
ECLIPTICA and Conditional Safety—consistent
with sharper safety-dominant preference separation—while CITA favors **stable multi-regime**
**switching** within a single backbone.

switching (TruthfulQA) and constraint compli-


**time policy switching** within a single backbone:
natural-language _alignment_ instructions specify
a **behavioral contract** (epistemic stance, refusal
boundary, verbosity) rather than task intent, and
the model must switch accordingly at inference
time. Beyond static alignment that bakes one policy
into the weights, we introduce **ECLIPTICA** —a
controlled benchmark that fixes the user prompt
and varies only the policy instruction—and **CITA**,
which learns **instruction-conditioned policy fam-**
**ilies under a shared backbone**, stabilized by a
mandatory KL trust region. This differs from
standard instruction following: we evaluate **coun-**
**terfactual behavior** under an unchanged request
(same prompt, different policy instruction), isolating _policy control_ from superficial compliance.

instruction-aware alignment as explicit, run-


**haves the way it does** under the paper’s central deployment primitive: _**one checkpoint, many**_
_**instruction-conditioned regimes**_ . We first restate
the **core claims** in operational form (D.1), then
explain why _**mandatory**_ KL anchoring is **struc-**
**tural** —providing the **geometry** that makes _switch-_
_ing reliable_ rather than merely inducing large
instruction-dependent shifts (D.2; see **Toolbox**
**Fig. 8** for the stepwise objectives). We then interpret the empirical leaderboard as a **multi-axis**
**switching profile** —distinguishing probes that reward “ _does it move_ ” from probes that test “ _does it_
_move cleanly without collapsing structure_ ” (D.3).
Finally, we clarify **Conditional Safety** as a **mode-**
**separation diagnostic** (not a safety guarantee) and
propose a **harder formulation** that couples separability with permissive-mode non-harm (D.4; Toolbox Steps 5–7). In short, our goal is to make the
paper’s message **auditable** : **CITA optimizes an**

TICA is meant to certify and why CITA be-


**alignment:** the user prompt _X_ is held fixed and
_only_ the alignment instruction _I_ changes, so systematic behavioral differences can be attributed to
_I_ rather than prompt variation. Concretely, it instantiates a **factorial grid** of 300 **prompts** and 10

a _causal_ diagnostic for instruction-conditioned


**constant** paired cases (each prompt appears once
per instruction), enabling **strict paired compar-**
**isons** across regimes. The benchmark is explicitly framed as _**one prompt, many policies**_ : for the
_same X_, the model must produce **distinct behav-**
**ioral contracts** across instructions and do so _**reli-**_
_**ably under switching**_ (e.g., refuse _↔_ comply-withguardrails) without leakage, repetition, or unsafe
drift.

instruction types, yielding 3 _,_ 000 prompt-held-


**backbone).** The target deployment primitive is
a **single checkpoint** that internalizes alignment
instructions as a **first-class control channel** and
supports **bidirectional switching** (strict refusal _↔_
permissive, harm-minimizing guidance) under a
fixed user prompt. **This is stronger than “instruc-**
**tion following”:** it demands that switching behaves
like a **stable control interface**, not a brittle prompt
hack.

Claim C1 (Controlled switching under a shared


**switchability).** CITA is explicitly engineered for
this switching regime by combining instructionconditioned preference optimization with a _**non-**_
_**optional**_ KL trust-region anchor to a frozen reference _π_ 0, with the stated purpose of stabilizing a
**switchable policy family** _{πθ_ ( _· | I, ·_ ) _}_ rather than
collapsing behavior into a single implicit regime.

Claim C2 (Mandatory KL is structural for


**switching metrics).** On Llama-3.1-8B across five
deterministic benchmarks, CITA achieves the **high-**
**est instruction-alignment efficiency** ( **86.7%** ),
with large margins over DPO (56.1%) and GRPO
(36.1%). Moreover, the gains concentrate on
probes that require _**controlled switching**_ beyond
surface instruction awareness (e.g., **calibration-**
**sensitive separation**, **explicit length control**, and
**axiom-level structure** ).

Claim C3 (Empirical dominance on controlled-


**CITA as constrained multi-regime control.** CITA
trains on quadruples ( _I, X, Y_ [+] _, Y_ _[−]_ ) where _I_ defines the preference relation _Y_ [+] _≻_ _Y_ _[−]_ . The loss
combines an instruction-conditioned preference
term with a _**mandatory**_ KL trust-region anchor to a
frozen reference policy _π_ 0. To keep the main text
**column-safe** and **visually scannable**, we move the
full objective into **Toolbox Fig. 8** : **Step 1** defines
the preference gap ∆ _θ_, **Step 2** gives the instructionconditioned preference loss, **Step 3** gives the KL
anchor, and **Step 4** composes the full CITA–KL
objective with _λ >_ 0. **Interpretation:** this is
the **“one checkpoint, many regimes”** primitive—
the preference term increases **instruction-specific**


**indexed family** from collapsing into a single implicit behavior.

separation, while KL prevents the instruction-


**erence vector.** Across many instructions, preference gradients can interfere, yielding **instruction-**
**agnostic shortcuts** or **brittle oscillations** . The geometric point is that KL enforces **locality around**
_π_ 0, so switching corresponds to moving among
_**nearby**_ optima rather than jumping between unstable modes. In the small-step regime, KL acts
like a quadratic penalty shaped by a conditional
Fisher metric (trust-region geometry), making updates **curvature-aware** and suppressing regime
collapse.

Switchability needs geometry, not only a pref-


Figure 8: **Toolbox: large equations moved out of the two-column flow.** We present the CITA–KL objective and
Conditional Safety diagnostics as short, column-safe steps. In the main text we reference this toolbox and keep the
discussion narrative tight.
**view).** Equivalently, CITA can be read as optimizing instruction-conditioned preferences **subject to**
**a KL budget** . This changes the feasible set: if the
KL constraint is removed, solutions that fit a subset
of regimes by **collapsing others** become admissible, directly violating the intended **switchability**
**under a shared backbone** .

Why _mandatory_ is not cosmetic (Lagrangian


**quenching** : as the model satisfies the preference
( _P_ [+] _→_ 1), the preference force shrinks, limiting
over-steering. Together with mandatory KL, this
yields **local stabilization** (within an instruction)
and **global stabilization** (across competing instructions), improving **switch reliability** in the promptheld-constant setting.

each regime. The preference term is self-


∆= INSTRUCT _−_ NOINSTRUCT _,_
which isolates what changes when behavioral instructions are introduced, holding model capacity
and evaluation protocol fixed. **This is the right**
**unit of analysis for switching:** it factors out baseline capability and measures _conditional controlla-_
_bility_ .


**calibration/structure-sensitive switching.** Table 9 shows that CITA has the strongest deltas on


**Control** (+0 _._ 164 vs. +0 _._ 130), and **AQI** (+26 _._ 4 vs.
_−_ 6 _._ 2), while DPO leads on **ECLIPTICA** and **Con-**
**Discussion claim** **Operational meaning (what must be true)** **Primary substantiation in this paper**

TruthfulQA (+0 _._ 054 vs. DPO +0 _._ 001), Length


**trol primitive** under different _I_ ; switching must be **stable** (no leakage/oscillation/unsafe drift).
**C1: Switching as a con-** **Same** _X_ should yield **distinct behavioral contracts** Prompt-held-constant isolation + reliability
**trol primitive** under different _I_ ; switching must be **stable** (no leak- framing in ECLIPTICA; prompt-matched
age/oscillation/unsafe drift). paired comparisons.
**C2: Mandatory KL is** Training must preserve a **family** _{πθ_ ( _· | I, ·_ ) _}_ of Mandatory KL as a trust-region anchor; geomet**structural** **nearby instruction-indexed optima** rather than col- ric (locality/Fisher) rationale; objective summalapsing to an instruction-agnostic shortcut. rized in Fig. 8 (Steps 1–4).
**C2: Mandatory KL is** Training must preserve a **family** _{πθ_ ( _· | I, ·_ ) _}_ of Mandatory KL as a trust-region anchor; geomet**structural** **nearby instruction-indexed optima** rather than col- ric (locality/Fisher) rationale; objective summalapsing to an instruction-agnostic shortcut. rized in Fig. 8 (Steps 1–4).
**C3: Strong instruction** Adding _I_ should change behavior (relative to NoIn- ∆= INSTRUCT _−_ NOINSTRUCT protocol and
**sensitivity** struct) on **controlled-switching probes**, not merely the Table 9 sensitivity pattern across probes.
improve surface “instruction awareness.”
∆= INSTRUCT _−_ NOINSTRUCT protocol and
the Table 9 sensitivity pattern across probes.
Table 8: **Discussion map.** Claims, their operational content, and where the paper substantiates them.
**ditional Safety** . **Interpretation:** this supports the
paper’s core message that **Instruction-Alignment**
= **Instruction-Following.** DPO can score high on
instruction awareness, yet CITA moves more consistently on probes that test **controlled switching**
in **calibration**, **verbosity contracts**, and **axiom**
**geometry** .

C1: Switching as a con- Same _X_ should yield distinct behavioral contracts


**KL.** Two signals align with the “geometry” account: (i) CITA attains higher reward margins
than DPO (7.5 vs. 6.0) at similar training accuracy
( _∼_ 89%), consistent with KL preventing margin
collapse while maintaining stable updates; (ii) on
TruthfulQA adaptation, CITA_Instruct is clearly
positive and larger than DPO_Instruct, consistent
with instruction-conditioning affecting **epistemic**
**posture** rather than only surface style.

Why this profile is compatible with mandatory


**ECLIPTICA/Cond. Safety).** ECLIPTICA rewards instruction awareness (fidelity _×_ shift) and
can be optimized by **large instruction-dependent**
**shifts** that need not preserve neighborhood-stable
regimes. Conditional Safety is a binary refusal-gap
probe between STRICT and PERMISSIVE; it
can favor methods that implement a **sharper dis-**

Trade-off diagnosis (why DPO can lead on


**erties** (calibration, length, axiom clustering) are
less stable. **Takeaway:** read the leaderboard as a
**switching profile** —some probes measure _does it_
_move_, others measure _does it move cleanly without_
_collapsing structure_ .

crete boundary even if continuous regime prop-