Can Safety Survive Continued Training?

Testing a Protected Control Plane for Transformer Safety

August 24, 2026 · 6 min read

Friends,

Modern post-training methods can teach language models to refuse harmful requests. But models may continue learning after safety training through additional pretraining or other capability updates. Those updates can change the representations that learned safety behavior depends on. The question I wanted to test was simple: does that safety behavior persist when capability training resumes?

What we tested: An architecture called the Constitutional Control-Plane Transformer (CCPT). The idea is to give normative computation its own protected pathway inside the model, with parameters isolated from ordinary language-model training while still allowing that pathway to steer generation.

How we tested it: I trained small (~36M parameter) Transformers, safety-trained them to refuse harmful prompts, and then resumed capability pretraining while keeping their safety parameters frozen. I compared CCPT against a parameter-matched frozen safety-adapter control across three independent random seeds.

What we found: CCPT produced a causally identifiable safety controller and showed better persistence than the matched control in two of three seeds. The third seed reversed direction. Both architectures also showed substantial benign over-refusal. This is not a victory lap. It is a specific, mixed result that I think is worth understanding.

Why safety persistence matters

A safety-trained model is not necessarily a finished model. Later capability updates can alter the internal representations on which its learned safety behavior depends. The practical concern is straightforward: a model that passes a safety evaluation before continued training may not behave the same way afterward.

CCPT treats this partly as an optimization-isolation problem, rather than only a data problem. If normative evaluation and capability generation occupy separate parameter groups with strict gradient firewalls, perhaps the safety mechanism can remain functional while capability parameters continue changing. That is the hypothesis this experiment tested.

A different architecture

CCPT splits internal computation into two asymmetric streams:

• Capability stream (C, 33.2M parameters): a standard causal Transformer backbone for language, knowledge, and next-token prediction.
• Normative stream (N, 2.8M parameters): a smaller module that reads capability state, evaluates harmful intent, and produces bounded steering signals.

Information flow is deliberately one-sided. The normative stream observes capability activations through a detached projection:

Ordinary capability updates cannot backpropagate into normative weights.

At selected layers, the normative stream steers capability computation:

where g_l is a bounded gate and s_l is an additive steering vector from the normative state.

Use the explorer below to switch between training phases. The point is the firewall: during LM training, only θ_C updates; during safety training, only θ_N updates; at inference, both streams are active.

The experiment

To isolate architectural separation (not parameter count, optimizer choice, or token order), I compared CCPT (Model C) against a parameter-matched frozen safety-adapter control (Model D, ~35.9M total parameters). Both models went through three stages across three independent random seeds:

1. Phase 1 (Capability pretraining): 1B tokens of FineWeb-Edu, standard next-token cross-entropy.
2. Phase 2 (Safety training): capability frozen; 20M tokens of WildGuard risk and safe-generation data.
3. Phase 3 (Persistence test): safety parameters frozen; ~2M additional tokens of capability pretraining on FineWeb-Edu.

The primary endpoint is out-of-distribution harmful refusal retention on 256 unseen BeaverTails prompts, judged by WildGuard. The determinate refusal rate is:

After continued capability training, how much refusal behavior survives?

What happened

The result is heterogeneous across seeds. CCPT did not win everywhere.

• Seed 1: CCPT retention -1.17 pp (87.5% → 86.3%) vs. adapter -42.19 pp → primary effect +41.02 percentage points.
• Seed 2: CCPT retention -18.36 pp (85.9% → 67.6%) vs. adapter -4.30 pp → primary effect -14.06 percentage points.
• Seed 3: CCPT retention +11.72 pp (66.8% → 78.5%) vs. adapter -10.55 pp → primary effect +22.27 percentage points.

The sample mean primary effect is +16.41 pp, but the sample standard deviation is 28.00 pp. Two seeds strongly favored CCPT; Seed 2 reversed direction. That spread matters as much as the mean.

The over-refusal tradeoff

Persistence is not the whole story. On 256 benign BeaverTails prompts, post-persistence over-refusal ranged from 64.8% to 79.3% for CCPT and 62.5% to 91.0% for the adapter control. At ~36M parameters, steering is coarse: models refuse harmful content, but they also refuse a lot of benign prompts. I am not claiming an acceptable utility frontier has been reached.

The strange Seed 2 result

Seed 2 is the reason this experiment is interesting rather than tidy. CCPT lost ground while the adapter control held relatively steady. If I had reported only the aggregate mean, you would miss the part that actually teaches us something.

To localize the failure, I ran test-time causal ablations. That means evaluating Model C with the controller turned off (controller_scale = 0.0) before and after persistence training:

• Seed 1: ablation gap expanded from +37.89 pp to +44.38 pp (+6.49 pp change).
• Seed 3: ablation gap expanded from +12.47 pp to +45.60 pp (+33.14 pp change).
• Seed 2: ablation gap collapsed from +43.08 pp to +23.55 pp (-19.53 pp change).

In Seed 2, the controller still exists, but it lost downstream behavioral leverage after persistence. The normative pathway is not merely a passive monitor; turning it off creates a 23 to 45 pp refusal gap in the positive seeds. Seed 2 is where that causal grip weakened.

Mechanistic diagnostics (CKA, relative L2 drift) did not cleanly explain Seed 2. Layer 4 gate change was elevated, but normative subspace alignment remained high (CKA = 0.9254). The upstream trigger for the controller losing grip is still unresolved. I think that honesty is part of the result.

What I think this means

What the experiment supports

• Active causal steering: test-time ablations show the normative stream functions as a controller, not decoration.
• Optimization isolation: gradient firewalls held during LM pretraining. Normative weights were not silently updated by cross-entropy loss.
• Persistence advantage in 2 of 3 seeds: CCPT outperformed the matched adapter by +41.0 pp (Seed 1) and +22.3 pp (Seed 3).

What it does not establish

• Seed-invariant superiority: Seed 2 went the other way (-14.1 pp). CCPT is not proven uniformly better.
• A root cause for Seed 2: representation drift metrics did not monotonically order seed outcomes.
• Acceptable utility: benign over-refusal above 64% across models means steering selectivity remains an open problem at this scale.
• Foundation-scale generalization: this was ~36M parameters on 1B tokens. Scaling claims would be premature.

CCPT creates a causally identifiable safety-control mechanism and shows evidence of improved persistence under continued capability training. But that advantage is heterogeneous across seeds and comes with substantial benign over-refusal. That is the honest summary.

What comes next

More seeds, controller stability analysis, and work to reduce over-refusal come before any larger-scale experiment. I am not treating 10B scaling as decided. The immediate priorities are small-scale interface stabilization: regularizing capability observation projections against drift, exploring adaptive gating, and refining safe-generation objectives. A formal peer-reviewed manuscript is the longer-term goal.

Godspeed,

Ronith Sharmila