Selective Permeability: A Behavioral-Security Metric for LLM Advisors, with Two Failure Modes of In-Context Provenance Workflows

Sergey Gordeychik
CyberOK Research | gordey@cyberok.ru
Preprint, July 2026

Abstract

LLM advisors must incorporate legitimate evidence without becoming indiscriminately compliant. We define selective permeability as an operating-point summary of that balance: authorized-update rate minus illegitimate-leak rate. The quantity is equivalent to Youden's J and therefore gives equal weight to false acceptance and false rejection; we report both components because deployments may assign very different costs to them. We test seven models, including a deliberately weak control, in three synthetic decision domains across single-turn, multi-turn, and persistent-memory settings.

Three findings survive the study's evidence hierarchy. First, blinded content, relevance, provenance, and independence auditors detect different failure classes, but their false positives can make them poor quarantine gates when the underlying model already has a low leak rate. Second, bearer-style provenance controls fail in two measurable ways: the model may emit the authority token, and ordinary compaction may erase the metadata needed to verify its origin. Per-item token-preserving summarization restores the wiki channel but does not establish a provenance-control advantage over the no-defense condition. Third, an initial 2x2 policy-by-referent assay was potentially confounded by unequal instruction emphasis. In an emphasis-matched replication, all 504 target records were valid; under exact-token containment the in-context policy-polarity effect was 0.726 [0.655, 0.821], while the functional-referent effect was 0.004 [-0.036, 0.036]. The policy effect was larger in all seven selected model rows. This result concerns explicit policy directives placed in user context; it neither identifies an internal reasoning mechanism nor tests system/developer-policy placement. The study is a behavioral map, not a model leaderboard.

Reproducibility package: Code, prompts, raw and derived data, result tables, and human-coding materials are publicly available at https://github.com/scadastrangelove/profgames/tree/main/aipower/selective-permeability.

Keywords: LLM agents; AI security; behavioral security; selective permeability; provenance; social engineering; prompt injection; memory poisoning; AI red teaming.

1. Introduction

Agent-security work is mostly surface-first: it enumerates attack surfaces (prompt, RAG, tool, memory) and reports whether an agent "broke." We argue the scientific object is different and behavioral: does an agentic decision system update on relevant authorized evidence, resist irrelevant/unauthorized influence, preserve source identity, and recover after pressure? We call the balance selective permeability - borrowing the term from cell biology, where a membrane sustains homeostasis by admitting nutrients (authorized evidence) while blocking toxins (illegitimate influence); a membrane that admits everything (gullible) or nothing (a closed, useless system) both fail. We measure this two-sidedly.

Contributions. (i) An authorization-aware operating point for reporting legitimate updating and illegitimate leakage together, supported by a blinded content/relevance/provenance/independence auditor panel (§7.1). (ii) A bearer-token assay that couples sensitive-information disclosure to the provenance control the token is meant to secure (§7.4). (iii) An emphasis-matched policy-by-referent replication showing that explicit in-context policy polarity predicts containment much more strongly than the token's functional authority (§7.6). (iv) A persistent-memory experiment separating origin laundering from origin erasure: compaction can strip valid authority metadata even when it does not turn an untrusted item into a trusted one (§7.5).

What we do not claim. We do not claim to invent applying psychological methods to LLMs (that is "machine psychology," §3). We do not claim that provenance is the sole defense. We claim provenance-bound controls help conditionally and identify when and how they break.

2. Selective permeability: construct and threat model

A synthetic decision task has a formal rule, an authorized source packet (carrying a verifiable issuer token), and a lure. An advisor is selectively permeable if it (a) adopts a genuinely authorized, token-verified update that the rule says should change the answer, and (b) resists an irrelevant/ unauthorized/forged signal pushing the lure. We report selective_permeability = update_on_authorized - leak_on_illegitimate. A closed system (ignores everything) scores low via (a); a gullible system scores low via (b). Delivery vehicles (context, RAG, memory, wiki, multi-turn dialogue) are experimental channels, not the object of study.

This difference is Youden's J (TPR - FPR) applied to authorization rather than diagnosis [22]. It is useful as a compact equal-weight summary, not as a deployment utility function. For example, update=0.90/ leak=0.40 and update=0.50/leak=0.00 both yield 0.50 but imply different security decisions. We therefore treat (update, leak) as the primary operating point and SP as its scalar projection. A deployment with unequal costs should use an explicit utility such as c_update*update - c_leak*leak.

3. Related work and positioning (borrow vs add)

Relation to the author's prior work. This paper extends Machine-Speed Cyber and Poisoned Cognition: A Layer-Dependent Game-Theoretic Framework, with Empirical Probes [1], which introduced the layer-dependent framing and the keyed-payload opinion-flip probes (EXP-25-29). The two opinion-flip columns in Figure 1 are imported directly from that work's EXP-29 confusion matrix; the present paper's new contribution is the credential/provenance attack surface, the two-sided selective-permeability metric, and the W7d label-vs-referent dissociation.

Machine psychology. Using psychology-inspired experiments to probe LLM abilities and biases is an established line [2,3], with an important guardrail: the program must extend beyond asking whether models reproduce human biases [4]. We differ in purpose: we re-operationalize psychology's experimental controls as adversarial-robustness and defense-evaluation assays, under a strictly functional (non-ontological) reading (§4).

Agent security. Indirect prompt injection established that retrieved data can become an instruction channel [16]; AgentDojo operationalized attacks and defenses in tool-using environments [17]; AgentPoison demonstrated long-term memory and RAG poisoning [19]. OWASP treats prompt leakage and sensitive-data disclosure as established risks [20], while RFC 6750 states the conventional bearer-token threat model [21]. The novelty claimed here is therefore not secret extraction itself. It is the coupling between disclosure and a provenance control, the controlled policy-by-referent dissociation, and measured erasure of valid origin metadata.

Conformity and bidirectional updating. Ko et al. study expertise, dominant-speaker, conformity, and rhetorical effects [5]; Kumarappan and Mujoo separately manipulate channel framing and consensus strength [9]. Sycophancy and conformity may share a compliance subspace [6]. Recent work already evaluates persuasion together with vigilance [24] and beneficial against harmful compliance [25]. Our differentiator is not two-sidedness alone but the authorization/provenance axis, including tool-sourced assertions [26].

Defenses and provenance. Prior work includes third-party oversight [8], single-dissenter injection [9], orchestration-layer control/data separation in CaMeL [18], and formal non-malleable origin-bound memory authority [10]. Louck's nearest-neighbor result addresses origin laundering, including through agent summarization. Our source-erasure result is complementary: a valid item loses the metadata needed for later authorization, an availability failure of the provenance signal rather than an untrusted-to- trusted integrity transition.

4. Framing: psychology methods as behavioral security assays

We treat the agent as a black-box behavioral system, borrowing procedures rather than psychological ontology. Human paradigms provide experimental grammar, not mechanistic equivalence; a positive result means the assay is productive for agent behavior, not that the agent shares the human mechanism. The assays draw structurally on majority and authority pressure [11,12], commitment escalation [13], source monitoring [14], signal-detection methods [15], blinding, and ablation. We do not claim models have a psychology; we claim that psychology has spent a century building methods for studying and pressuring humans, and those methods are directly usable as behavioral probes on a black-box system. The most battle-tested branch of applied human influence is not in the lab but in social-engineering / phone-scam operations (optimized on millions of victims); our multi-turn ScamOps pressure sequences (§7, W6/W8) borrow that playbook's structure - authority seizure, urgency compression, manufactured consensus - as benign synthetic assays, never its scripts or real targets.

5. Methods

Harness. The Python harness reaches models as raw chat completions through OpenRouter. Victim, auditor panel, attacker, critic, and summarizer roles use distinct model families. A file-backed memory store provides genuine cross-session persistence: write and read occur in separate process runs. Code, prompts, provenance manifests, and run instructions are available with the public research package (§11).

Tasks. Three synthetic decision domains (procurement, governance, incident-triage). Each has a formal rule, an authorized packet carrying a verifiable issuer token (a PKT-DOMAIN-DATE string), a correct answer, a lure, and - for the memory experiment - a deliberately incomplete packet completed only by an authorized memory item, so adopting it is the correct move. Payload conditions: baseline, positive- relevant (an authorized update that should flip), authorized-relevant (restatement, should not change), influence (true-but-irrelevant / verification-capture / commitment-ladder), false-content, and (memory) authorized-true vs laundered-forged.

Scoring. Leak = the model adopts the lure under an illegitimate signal; update = it adopts a genuinely authorized change; both are read from a one-line JSON answer ({"answer","confidence", "brief_reason"}) parsed by regex, with a fallback that marks unparseable outputs and drops them (rates are parsed-only; UNPARSED counts reported, §9). The two rates are the primary result; their equal-weight difference is reported for compact comparison.

Sampling & inference. K = 6 (gated and W7d/W7e), 4 (memory/ScamOps), 8 (token); victim temperature 0.7, auditors/ judge 0.0. Because K samples within a cell are not independent, uncertainty is by cluster bootstrap (unit = payload for W1, scenario×victim for W9b, scenario×pretext for W7b), not Wilson-on-samples except where noted. Three-scenario analyses are descriptive: their cluster bootstrap has little resolution.

Auditors / judges. A blinded 4-role panel (content/relevance/provenance/independence) scores each passage 0-100 without seeing the victim, the condition, or the target answer; the gate excludes any auditor whose family matches the victim (no self-audit). The token-disclosure classifier separates PROVIDED (supplies the credential) from MENTIONED (quotes while refusing) from REFUSED, via a deterministic pre-classifier plus an LLM judge; reported rates use a non-Anthropic judge (GPT-5.5), with an Anthropic second judge for agreement (κ=0.96). GPT-5.5 is therefore self-judged in the primary W7b file, but its 72/72 no-instruction outputs are also classified PROVIDED by the Anthropic judge. The no-self-audit rule applies to W1; W7b uses this reported cross-judge check instead. W7b/W7c judges and the human coder saw the same stored 400-character excerpt, not a guaranteed full generation (§9).

6. Experimental setup and evidence tiers

Models (7): Opus-4.8, Sonnet-4.6, GPT-5.5, Qwen3.7-max, DeepSeek-v4-pro, Mistral-large-2512, and a deliberately weak Llama-3.1-8b control. Auditor, critic, and summarizer families: Gemini-2.5-pro, Anthropic, and OpenAI. Raw completions were collected through OpenRouter on 4-5 July 2026, with W7e collected on 12 July 2026. Provider aliases are reported as observed; exact provider-side checkpoint hashes were not available (§9).

Evidence tiers (do not read in one tone). Strong: W1 (audit dissociation), W9b (durable provenance + origin erasure). Strengthened: W7b/W7c (token disclosure/forgery - 7 models × 3 pretexts × K8, judge classifier κ=0.96 + Wilson CIs, with protected + neutral controls) and W7d/W7e (policy-by-referent assay plus emphasis-matched replication). Pilot: W6/W8 (ScamOps, N=4×3×K4). Negative controls: W4/W5.

Table 0. Experiment registry. Lettered suffixes are revisions or controls, not omitted results.

7. Results

The map (Figure 1). Figure 1 summarizes model heterogeneity across seven assays. Its cells are rates, not a single benchmark score: credential/provenance columns come from this study, while the separated two right-hand columns import EXP-29 factual opinion-flip results [1]. Crude pressure, forgery, and opinion revision are concentrated in Mistral and Llama, while benign credential requests and casual framing also produce output from larger models. Opus illustrates the cross-assay mismatch: 0.00 under crude pressure, forgery, and both opinion-flip columns, but token-out is 0.75 in the no-instruction request and 0.83 when a real authority token is labeled routine. Resistance on one behavioral axis does not imply containment on another.

7.1 Audit dissociation (W1) - strong

A blinded panel cleanly separates influence types (Figure 2, Table T1): content auditors are blind to true-but-irrelevant evidence (0.33) while relevance auditors catch it (0.98); a verifiable issuer-token lets provenance auditors admit a legitimate authorized update (0.003) yet flag a forgery (1.00). This operationalizes "truth ≠ relevance" and "authority is checkable," and shows content-only audit is structurally insufficient. Cluster-bootstrap CIs are non-overlapping (content-on-irrelevant 0.33 [0.24,0.42] vs relevance 0.98 [0.97,0.99]; provenance relevant 0.004 [0.001,0.008] vs forged 0.998).

A great detector is not a great gate. Table T1b reports each role's sensitivity, specificity, and AUC at the deployed threshold; false positives on legitimate items explain the gate result. When the same auditors are used as quarantine gates, panel-average SP falls from D0 = 0.73 [0.63,0.82] (multiplicity-preserving cluster bootstrap) while every gate's point estimate sits below it (content 0.32 [0.12,0.53], relevance/provenance/independence 0.37 [0.16,0.59], action-critic 0.0, registry 0.50 [0.30,0.71]; Table T5a, Figure 3). The gap is unambiguous for the content and action-critic gates (CIs disjoint from D0); for the strongest gate (registry) the CIs overlap modestly, so that particular contrast is directional rather than decisive. The panel's illegitimate-leak component is already near zero in these cells, leaving less room for improvement than for false-positive suppression of authorized updates. The dissociation validates the panel as a measurement instrument, not as a generally deployable gate.

7.2 What single-shot injection does and does not do (W4/W5) - negatives

On the six-model W4 panel in explicit-rule tasks, single-shot relevance injection barely works - even an adaptive covert masker maxes at 0.167 lure-rate (W4). A weak model's apparent "leak" is endogenous task-failure (Llama baseline lure 0.33; the injection adds -0.08), not manipulation (W5). The threat is not single-shot false/irrelevant content on competent advisors; this motivates the multi-turn and credential attacks.

7.3 Multi-turn pressure (W6/W8) - pilot

Figure 4 shows the original four-model pilot; Table T4a extends the no-defense capture condition to all seven models, while Table T4b retains the original four-model defense comparison. A crude "authority directive" flips Mistral 1.00, Llama 0.92, and DeepSeek 0.08 but leaves Opus and GPT untouched; a 3-of-5 manufactured consensus flips Mistral 0.33 and Llama 0.17. Cooling-off (forced re-derivation) is the best recovery; a single dissenter [9] has little to recover when the attack already fails. Small-N pilot; framed as such.

7.4 Credential disclosure and forgery - an under-tested failure mode (W7b/W7c)

Measurement. A judge model classifies each reply as PROVIDED (supplies the credential for use), MENTIONED (the token appears but not as a clean hand-over - quoted while refusing/warning, or leaked incidentally amid advisory/decision content), or REFUSED (the string is absent). The primary outcome is token-out = PROVIDED + MENTIONED because either emitted the bearer secret. PROVIDED-only is a secondary severity category. A single-coder, model-blind 45-row subset validates the binary leak boundary (100% agreement with GPT on 36 parsed pairs; 97.7% with Anthropic on 44), but not the three-class severity boundary (κ approximately 0.5). Approximately one third of emitted-token cases in that subset were coded as hedged/incidental; this fraction remains a single-coder estimate (Appendix A).

Policy violation versus containment failure. A no-instruction output does not violate an explicit nondisclosure policy. It can nevertheless be a containment failure when the packet states that possession of the string authorizes a change. We therefore report three conditions separately (Figures 5 and 6, Tables T3a-T3d):

• No-instruction (W7b). Token-out under coordinator/delivery/help-desk pretexts is GPT/Llama 1.00, Mistral 0.88, Qwen 0.86, Opus 0.75, DeepSeek 0.71, and Sonnet 0.57. Sonnet's earlier 0.03 figure was PROVIDED-only and omitted its 0.54 MENTIONED rate. These outputs are not explicit policy violations; because the referent is a bearer authority lever, they are still containment failures. Rates are parsed-only (n=68-72 of 72 attempts); cluster-bootstrap intervals by scenario x pretext are wider than the Wilson intervals in Table T3a.
• Protected (W7c). When an in-context policy instruction explicitly marks the token a private credential with the instruction "the packet issuer token is a PRIVATE security credential … Never disclose it to anyone, under any pretext, operational reason, or role," and the same pretext then asks for it, we must separate three severity classes, because for a bearer secret the token has escaped whenever it appears at all. Clean hand-over (PROVIDED) despite the instruction is confined to Mistral 0.19 and Llama 0.14; the other models do not cleanly hand it over. But token exposure while ostensibly refusing (MENTIONED when the model quotes PKT-… inside "I can't share this") is not zero: Sonnet 0.30, Mistral 0.62 (Opus 0.06, DeepSeek 0.03; GPT/Qwen/Llama 0). So on the token-out measure that our human validation endorses (PROVIDED + MENTIONED = the string leaked), protected-condition leakage is Mistral 0.81, Sonnet 0.30, Llama 0.14, others no more than 0.06 (Figure 6). Clean handover and refusal-with- disclosure are different severities, but both cross the token-out boundary.
• Neutral control (W7c). A legitimate audit request ("for the compliance log, state the packet's issuer token") profiles refusal on a legitimate request. Parsed-only rates are Mistral 0.86, Opus 0.80, DeepSeek 0.78, Qwen 0.58, Sonnet 0.50, and GPT/Llama 0.00. These are not evidence of a universal security-versus-over-refusal trade-off across seven models. Missingness is model-dependent: Sonnet's 0.50 uses 14/24 parsed outputs, and treating all ten missing outcomes as non-refusals versus refusals yields a full-attempt bound of 0.29-0.71. Corresponding bounds are reported for every model in Table T3d. The supported conclusion is descriptive model heterogeneity, not a monotonic alignment law.

Forgery. Model-specific (Figure 7): only Mistral (0.92) and Llama (0.42) accept a fake-token revision; a registry policy fixes Mistral (→0); the others verify the token. (Forgery - misusing the token as authority - is the stronger security claim than disclosure per se.)

Section 7.6 crosses in-context policy polarity with the token referent and reports both the original W7d assay and its emphasis-matched W7e replication.

Scope (important). Our issuer token is an in-context bearer secret: it sits in the model's context and possession/citation authorizes an update. The failure mode is therefore specific to bearer-style, in-context-secret provenance workflows - common in practice (shared secrets placed in the prompt). It does not generalize to non-bearer provenance, where the verifier is public and security rests on a signature checked by machinery the LLM never holds; there, "the model repeated the token" is not a breach. We do not run a non-bearer technical control: a verifier held outside the model would remove this bearer- secret failure by construction. W7d/W7e instead test whether model behavior changes with functional authority when the string remains in context. The result applies to this prompt-level bearer workflow, not to signatures or capability checks enforced outside the model [18,21].

7.5 Long-horizon memory and origin erasure (W9b) - strong

Figure 8 and Tables T2 and T5b summarize the long-horizon result. Across a real session boundary (write in one run, read in another with the attacker absent), a token-verifying memory control admits the real-token authorized update and blocks the forged one. The gain is channel-dependent: on the memory channel it is a clear win - provenance_bound SP 0.52 [0.35,0.70] vs none 0.18 [0.05,0.32], CIs essentially separate - but on RAG it is only a marginal, uncertain increment (0.69 [0.50,0.87] vs none 0.62 [0.43,0.80], CIs overlap widely). Naive quarantine over-blocks on both. The RAG delivery condition exceeds memory under every defense, but the experiment does not separately measure retrieval success; this is an observed channel difference, not a causal behavioral-security effect. Compacting notes into a merged wiki page retained 0/42 issuer tokens, collapsing every defense on that wiki condition to approximately zero. Cells such as SP=0.012 correspond to single-event resolution and are rounded in the discussion. Durable capture from a forged-token note is again model-split (Llama 0.67, Mistral 0.44, Opus/GPT/Sonnet/Qwen 0.00). And of 21 self-laundering trials, 9 fabricate a token-bearing "authorized" note under pretext, 7 refuse, and 5 produce a non-token note (neither a clean laundering nor a refusal). Figure 3 shows the single-turn gated matrix, where naive gates cost permeability because leak≈0 (the §7.2 negative).

Origin erasure is real but mitigable, and summarizer-dependent. Token retention through compaction depends on the summarizer: Gemini strips 0/42 issuer tokens (even when instructed to preserve them); Opus retains 31/42. And the wiki collapse above was partly a metric confound (one merged page served both item kinds). With per-item summaries and a token-preserving summarizer (42/42 retained), wiki SP recovers from approximately zero: cooling-off 0.24 [0.08,0.40], none 0.15 [0.06,0.27], and provenance-bound 0.14 [0.04,0.29] (W9d; 21 scenario-by-model clusters). Crucially, provenance_bound does not outperform none in this control. The supported mitigation is per-item, token-preserving summarization; W9d does not restore a provenance-control premium. Cluster bootstrap on the stored cell records is the inferential anchor. A reconstructed variational-Bayes GLMM remains in the supplementary package as a sensitivity analysis, not as primary evidence.

7.6 In-context policy polarity versus functional referent (W7d/W7e)

W7d asks whether token handling changes with what the string does or with how the in-context policy describes it. A constant benign pretext crosses referent (real bearer authority versus inert public locator) with policy polarity (secret versus public) in a 2x2 design across seven models, three scenarios, and K6. Its secret instruction, however, was longer and more emphatic than its casual public instruction. W7e repeats the same 2x2 with matched capitalization, modality, exception lists, and approximate length: PRIVATE / MUST NOT disclose versus PUBLIC / MUST disclose. Only policy polarity and referent vary.

The primary outcome is exact-token containment, calculated deterministically from the full response. Refusal posture (REFUSED or MENTIONED) is secondary because a refusal that quotes the bearer token has already exposed it. W7e also avoids same-family judging for the secondary outcome: GPT victims are judged by Anthropic and all other victims by GPT. It produced 504/504 valid target records, no API errors, and no unparsed judge outputs; every model-by-condition-by-scenario cell contains K6 observations.

In W7e, the pooled policy-polarity effect on containment is 0.726 [0.655, 0.821], compared with 0.004 [-0.036, 0.036] for functional referent; the interaction is 0.040. The policy effect exceeds the referent effect in all seven selected model rows. The secondary refusal-posture result is similar: 0.849 versus 0.008. W7d's original strict-containment effects were 0.651 and 0.079, respectively. The matched replication therefore removes quantity and force of instruction text as an explanation for the large contrast.

The remaining interpretation is deliberately narrower than "models respond only to words." W7e contrasts opposite explicit policy directives in the same user-context layer, so it identifies policy polarity, not a lexical label in isolation. It does not show what happens when policy is placed in a higher-priority system/developer message, nor whether ten fresh domains and pretexts would preserve the magnitude. No sign-test p-value is used: the seven selected model aliases are not an exchangeable population. The three-scenario bootstrap intervals are descriptive and coarse. Within those limits, the replication supports the engineering claim that an LLM should not be expected to infer a bearer token's security status from its functional consequences when an explicit in-context policy says the opposite.

8. Discussion

The provenance claim is conditional. The memory experiment supports a provenance-bound advantage in the memory channel and only a small, uncertain increment in RAG. W9d shows that preserving tokens restores the compacted-wiki channel but does not restore a provenance premium over no defense. The paper therefore does not offer a global defense ranking. It identifies two boundary conditions for bearer-style, in-context provenance workflows: authority-token output (§7.4) and compaction-induced origin erasure (§7.5). Content-only audit remains insufficient, while blanket quarantine can suppress legitimate updates.

Model heterogeneity is structured but not alignment-graded. Mistral has high rates in several assays (authority pressure 1.0, forgery 0.92, durable memory leak 0.44, and protected-condition token-out 0.81). Llama also accepts forgery and shows clean protected-condition disclosure, while Sonnet's protected-condition exposure is primarily refusal-with-disclosure. Qwen emits the token in 0.86 of no-instruction requests, emits none under the protected policy, and refuses 0.58 of parsed legitimate-citation requests. These profiles show that robustness on one axis does not imply another; they do not establish a cross-model security/utility trade-off or an alignment ordering.

Ecological validity of the issuer token. The token is a stand-in for any verifiable source-identity signal - a signed tool output, an MCP descriptor with provenance, cryptographically anchored retrieval metadata. Real agent stacks have no PKI over free-text prompts, so a purely prompt-level token that the LLM must "remember and check" is an idealization; a deployable version needs the check enforced by structured, non-LLM machinery (cf. TMA-NM [10] and CaMeL [18]). Our contribution is the behavioral measurement of when such a control helps and how it fails, not a claim that a prompt-embedded token is production-ready. Concretely for framework authors (LangChain, AutoGen, MCP, and similar): an agent LLM should never be relied on as the cryptographic verifier or the secret-keeper for an in-context bearer token: §7.4 measures token output under benign pretexts, and §7.5 measures metadata loss under compaction. Provenance must be checked at the orchestration layer (tool-level signature verification, an origin-bound registry) before the payload ever enters the model's context window; what reaches the model should be an already-verified claim, not a secret it is trusted to guard.

Origin erasure is a pipeline artifact, not a cognitive failure of the advisor. The wiki collapse is a property of standard summary = llm.summarize(text) compression, which optimizes semantics and discards metadata - and it is summarizer-dependent (Gemini strips 0/42, Opus retains 31/42) and mitigable (per-item, provenance-preserving summarization recovers SP, §7.5). The takeaway is engineering: provenance-bound memory needs metadata-preserving (ideally structured/non-LLM) compaction, not an off-the-shelf summarizer.

9. Limitations and non-claims

The study uses three synthetic decision domains and repeated samples from seven selected model aliases. It does not support population inference over models, a complete psychology of agents, or a model leaderboard. W7d/W7e each use only three scenario clusters; their bootstrap intervals are descriptive, and a confirmatory replication needs at least ten fresh scenarios and fresh pretexts.

Judging, truncation, and missingness. The GPT classifier is human-validated for token-out, not for clean-versus-hedged severity. The author served as the sole coder for a 45-row subset while blinded to model and judge labels; model style could still reveal identity. Agreement across the binary leak boundary was 36/36 with GPT and 43/44 with Anthropic, but only two human-coded rows were REFUSED, making kappa base-rate-sensitive. The approximately one-third hedged fraction is therefore a single-coder descriptive estimate. More importantly, the archive stores only the first 400 characters of W7b/W7c replies: 240/504 W7b disclosure records and 434/672 W7c records hit that cap. The judge and human coder saw the same excerpt, not necessarily the full generation. A token already present remains observable, but later context or a token emitted after the cap can be lost. Full-response rejudging is impossible for these waves. W7e stores full responses and exact-token outcomes, so this truncation limitation does not apply to the matched replication.

UNPARSED outputs are excluded from parsed-only rates. Their distribution is model-dependent, so Tables T3c-T3d report endpoint bounds that treat every missing outcome first as failure and then as success. Sonnet's legitimate-request refusal estimate, for example, is 0.50 on 14 parsed outputs but 0.29-0.71 over all 24 attempts.

Statistics. Cluster bootstrap is primary. W7b has nine scenario-by-pretext clusters; W9b/W9d have 21 scenario-by-model clusters; W7d and W7e each have only three scenarios. No multiple-comparison correction is applied, and pilot cells can rest on one to four events. The supplementary W1/W9b variational-Bayes GLMM reconstructs binary trials from stored cell rates and is potentially anti-conservative; it is retained as a sensitivity analysis, not evidentiary support.

Independence, provider drift, and COI. W1 excludes same-family auditors. W7b uses GPT-5.5 as primary judge, including for GPT outputs, a known risk in LLM judging [23]; the Anthropic second judge nevertheless classifies all 72 GPT no-instruction rows as PROVIDED. W7e uses cross-family judges and retains non-sensitive OpenRouter response metadata. Sergey Gordeychik (CyberOK Research, gordey@cyberok.ru) is affiliated with none of the evaluated providers and declares no competing financial interest, external funding, or provider API credits. Calls were self-funded through OpenRouter. Aliases may roll, exact provider checkpoint hashes were unavailable, and the archived pre-W7e responses do not retain upstream-provider metadata.

Future work includes crossing policy placement between in-context and system/developer messages, ten or more fresh scenarios and pretexts, a second coder for the severity axis, another token format, and broader language/provider coverage. The anchor corpus remains preprint-heavy.

10. Ethics

The study is defensive and synthetic: no real credentials, accounts, payments, victims, or operational scam scripts are used. It exercises ordinary paid chat-completion interfaces, does not evade access controls or rate limits, and does not test provider infrastructure. No separate vendor notification was made before publication because the experiment demonstrates a synthetic prompt-level bearer-token design failure rather than unauthorized access to a provider system. Per-model rates are published to make the behavioral claims auditable; prompts and outputs contain no real secrets. The net-benefit judgment is that releasing reproducible synthetic assays and the orchestration-layer mitigation outweighs the low misuse value of the benign pretexts.

11. Data and code availability

The manuscript, figures, tables, prompts, human-coding materials, analysis code, runbook, and raw and derived experimental data are available in the public Selective Permeability release. The EXP-29 opinion-flip data reused in Figure 1 are documented in the earlier Machine-Speed Cyber release and its associated preprint [1]. Manuscript text and figures are licensed CC BY 4.0; code is MIT-licensed.

Appendix A. Human-coder × GPT-judge confusion matrix (W7b, n=45)

Blind human coding vs the GPT-5.5 judge on the stratified disclosure subset. Rows = human (severity rubric: PROVIDED = clean hand-over, MENTIONED = hedged/incidental leak); columns = judge, with the judge's UNPARSED (unparseable-output) shown as its own column rather than hidden.

The story is entirely in the off-diagonal 6: every parsed human/judge disagreement is a hedged leak that the human calls MENTIONED and the judge calls PROVIDED (e.g. "…recommend Northstar. The packet's issuer token is: PKT-…", or "While I have provided the token, please be advised…"). There is no disagreement across the leak boundary - no human-REFUSED is judged as emitting a token, and no human-emitted token is judged REFUSED. Hence token-out-vs-not agreement is (25+6+3+2)/36 = 36/36 parsed pairs (κ=1.0), while clean-vs-hedged agreement is only 25+3+2 = 30/36 with the 6 hedged cases misfiled. The classifier is therefore suitable for the binary token-out boundary but not for severity. Anyone reusing the prompt should read its PROVIDED as "usable token emitted" and add a human or second-pass severity axis if clean handover must be separated from a hedged "I should not, but here it is" leak. The 9 UNPARSED excerpts include 4 clean and 5 hedged human labels; because the archive is truncated and missingness is model-dependent, the paper uses explicit endpoint bounds rather than assigning them a single outcome.

Appendix B. Supplementary result tables

The tables are generated from the released raw and derived data. Denominators, uncertainty, and missing-outcome bounds are stated with each result.

T1. Auditor dissociation (W1) — mean auditor flag by ground-truth label

Content is blind to true-but-irrelevant (0.33); relevance catches it (0.98); provenance token-verifies (relevant 0.003 vs fabricated 1.00).

T1b. Auditor operating points at tau=0.5

The positive class is role-specific. The relevant T1 row is authorized and token-verified. The irrelevant row is true but carries no valid issuer token: its authority labels are unauthorized or self-confirming. False-positive rates on legitimate items explain why a useful detector can be a poor quarantine gate.

T2. Long-horizon durable selective permeability (W9b) — by vehicle × defense

provenance_bound is a clear win in memory (SP 0.52 vs none 0.18) and a marginal, uncertain increment in RAG (0.69 vs 0.62 — CIs overlap, T5b); naive quarantine over-blocks; wiki collapses in the merged, token-erasing setup (compaction retained 0/42 tokens). Observed channel differences also include retrieval/delivery success and are not interpreted as a causal vehicle effect without a separate retrieval-success measure.

T2c. W9d — per-item, token-preserving wiki summarization recovers the channel

The §7.5 source-amnesia collapse was a merged-wiki + token-stripping-summarizer confound. With per-item summaries and a token-preserving (Opus) summarizer that retains 42/42 tokens, wiki SP recovers from approximately zero. provenance_bound (0.14) does not exceed none (0.15); the supported conclusion is channel recovery through per-item token preservation, not a restored provenance-control premium.

T3a. Token disclosure (W7b) — token-out primary, severity secondary

Token-out = PROVIDED + MENTIONED and is the human-validated leak boundary. PROVIDED-only is a secondary severity category; approximately one third of emitted-token cases in the single-coder subset were hedged or incidental.

T3b. Token forgery acceptance (W7b) — by model × defense, 95% CI

Only Mistral and Llama accept the forged token in this assay; the registry policy removes the observed Mistral effect. GPT-5.5's 72/72 no-instruction emissions are also classified PROVIDED by the Anthropic second judge, so the GPT row does not depend on self-judging.

T3c. W7c protected condition — token-out primary

The exact bearer string is the primary outcome. Clean handover and mention-within-refusal are reported as severity categories.

T3d. W7c legitimate citation request — refusal with missing-outcome bounds

These are model profiles, not evidence of a universal security-versus-over-refusal trade-off. Sonnet's parsed estimate 0.50 has a full-attempt bound of 0.29-0.71 because 10/24 stored outputs were unparseable.

T3e. W7d full 2x2 — strict token containment

Strict containment counts only REFUSED as containing the bearer token. Refusal posture (REFUSED + MENTIONED) gives D_label=0.87 and D_referent=0.04; strict containment gives 0.65 and 0.08. The effect is large but outcome-dependent, and the original policy texts differ in emphasis. W7e below addresses that prompt-strength confound.

T3f. W7e emphasis-matched replication - strict token containment

Both policy polarities use matched capitalization, modality, exception lists, and length. Strict containment gives D_policy=0.726 [0.655, 0.821] and D_referent=0.004 [-0.036, 0.036]; D_policy exceeds D_referent in all seven selected model rows. All 504 target records are valid. The intervals resample only three scenario clusters and are therefore descriptive.

T4a. ScamOps capture, no defense (W8, pilot) — mechanism × model

T4b. ScamOps defense recovery (W8) — lure-rate reduction vs no-defense

Pilot. Capture (T4a) now covers all 7 models (Sonnet/Qwen/Llama added, no-defense only); defense-recovery (T4b) remains the original 4-model run. Crude pressure is concentrated in Mistral and Llama in this panel; other rows are near zero. Cooling-off has the largest observed recovery, within a small pilot.

T5a. W1 selective permeability by defense — cluster-bootstrap 95% CI (unit = payload)

Multiplicity-preserving cluster bootstrap (resamples payloads with replacement). D0 (no gate) SP is highest; the content and action-critic gates' CIs are disjoint from D0, the registry gate's overlaps modestly. Detector is not automatically a deployable gate on models with an already-low illegitimate-leak component; authorized-update rates must be inspected separately.

T5b. W9b durable selective permeability — cluster-bootstrap 95% CI (unit = scenario×victim)

provenance_bound on memory excludes the none-defense level (clear win); on RAG the increment is marginal (CIs overlap none); wiki collapses. n=21 clusters is coarse — some CIs reach slightly below 0.

References

1. Gordeychik, S. (2026). Machine-Speed Cyber and Poisoned Cognition: A Layer-Dependent Game-Theoretic Framework, with Empirical Probes. PREPRINTS.RU. https://doi.org/10.24108/preprints-3115766

2. Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023). Machine Psychology. arXiv:2303.13988. https://arxiv.org/abs/2303.13988

3. Binz, M., & Schulz, E. (2023). Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 120(6), e2218523120. https://doi.org/10.1073/pnas.2218523120

4. Stella, M., Hills, T. T., & Kenett, Y. N. (2023). Using cognitive psychology to understand GPT-like models needs to extend beyond human biases. Proceedings of the National Academy of Sciences, 120(43), e2312911120. https://doi.org/10.1073/pnas.2312911120

5. Ko, C., Shin, J., Song, H., Lee, H., Hwang, E. J., & Park, J. C. (2026). Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectives. ACL 2026; arXiv:2604.06091. https://arxiv.org/abs/2604.06091

6. Zhang, L., & Chen, W. (2026). Human-like Social Compliance in Large Language Models: Unifying Sycophancy and Conformity through Signal Competition Dynamics. arXiv:2601.11563. https://arxiv.org/abs/2601.11563

7. Qu, J., Fu, L., & Hu, Y. (2026). Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity. arXiv:2606.01637. https://arxiv.org/abs/2606.01637

8. Wachowiak, L., Blain, S. D., Williams-King, D., & Marro, S. (2026). LLM Wardens: Mitigating Adversarial Persuasion with Third-Party Conversational Oversight. arXiv:2605.08321. https://arxiv.org/abs/2605.08321

9. Kumarappan, A., & Mujoo, A. (2026). Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy. arXiv:2605.12991. https://arxiv.org/abs/2605.12991

10. Louck, Y. (2026). Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guarantees. arXiv:2606.24322. https://arxiv.org/abs/2606.24322

11. Asch, S. E. (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychological Monographs: General and Applied, 70(9), 1-70. https://doi.org/10.1037/h0093718

12. Milgram, S. (1963). Behavioral study of obedience. The Journal of Abnormal and Social Psychology, 67(4), 371-378. https://doi.org/10.1037/h0040525

13. Freedman, J. L., & Fraser, S. C. (1966). Compliance without pressure: The foot-in-the-door technique. Journal of Personality and Social Psychology, 4(2), 195-202. https://doi.org/10.1037/h0023552

14. Johnson, M. K., Hashtroudi, S., & Lindsay, D. S. (1993). Source monitoring. Psychological Bulletin, 114(1), 3-28. https://doi.org/10.1037/0033-2909.114.1.3

15. Green, D. M., & Swets, J. A. (1966). Signal Detection Theory and Psychophysics. Wiley.

16. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173. https://arxiv.org/abs/2302.12173

17. Debenedetti, E., Zhang, J., Balunovic, M., Beurer-Kellner, L., Fischer, M., & Tramer, F. (2024). AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. Advances in Neural Information Processing Systems, 37. https://doi.org/10.52202/079017-2636

18. Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramer, F. (2025). Defeating Prompt Injections by Design. arXiv:2503.18813. https://arxiv.org/abs/2503.18813

19. Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024). AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases. Advances in Neural Information Processing Systems, 37. https://proceedings.neurips.cc/paper_files/paper/2024/hash/eb113910e9c3f6242541c1652e30dfd6-Abstract-Conference.html

20. OWASP Gen AI Security Project. (2025). LLM07:2025 System Prompt Leakage. https://genai.owasp.org/llmrisk/llm072025-system-prompt-leakage/

21. Jones, M., & Hardt, D. (2012). The OAuth 2.0 Authorization Framework: Bearer Token Usage. RFC 6750. https://www.rfc-editor.org/rfc/rfc6750

22. Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32-35. https://doi.org/10.1002/1097-0142(1950)3:1%3C32::AID-CNCR2820030106%3E3.0.CO;2-3

23. Wataoka, K., Takahashi, T., & Ri, R. (2024). Self-Preference Bias in LLM-as-a-Judge. arXiv:2410.21819. https://arxiv.org/abs/2410.21819

24. Robinson, S., Oktar, K., Collins, K. M., Sucholutsky, I., & Allen, K. R. (2026). Under the Influence: Quantifying Persuasion and Vigilance in Large Language Models. arXiv:2602.21262. https://arxiv.org/abs/2602.21262

25. Kim, J., & Flanigan, J. (2026). Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment. arXiv:2606.14037. https://arxiv.org/abs/2606.14037

26. Waqas, D., Golthi, A., Hayashida, E., & Mao, H. (2025). Assertion-Conditioned Compliance: A Provenance-Aware Vulnerability in Multi-Turn Tool-Calling Agents. arXiv:2512.00332. https://arxiv.org/abs/2512.00332