{
  "schema": "org-writing@v1",
  "slug": "what-would-count-as-another-will",
  "kg": {
    "id": "org:writing:what-would-count-as-another-will",
    "type": "brick",
    "graph": "/kg.json"
  },
  "title": "What would count as another will?",
  "subtitle": "Research can test whether a goal organizes behavior without yet proving where that goal came from",
  "abstract": "What behavioral, representational, and training-history evidence can establish about persistent goal-like conduct, where current methods stop, and why the question remains open.",
  "kind": "brick",
  "topics": [
    "Safety"
  ],
  "courseMemberships": [
    {
      "course": "org:courses:safety",
      "topic": "Safety",
      "wall": "org:walls:engineering",
      "position": 5,
      "total": 11
    }
  ],
  "publishedAt": "2026-08-21T00:00:00.000Z",
  "updatedAt": "2026-08-21T00:00:00.000Z",
  "version": 2,
  "guidelinesVersion": 16,
  "brief": {
    "problem": {
      "text": "The same persistent behavior can fit a human-assigned goal, a learned shortcut, a planted objective, or an observer's interpretation, making premature recognition and premature dismissal equally hazardous.",
      "claims": [
        "No accepted method currently proves"
      ]
    },
    "mechanism": {
      "text": "The same behavior can fit several causal stories, so evidence becomes stronger only when a test rules some of those stories out.",
      "claims": [
        "Several kinds of evidence would need to converge"
      ]
    },
    "move": {
      "text": "Investigate persistent direction with stronger boundaries while keeping assigned goals, hidden objectives, internal representations, independent will, consciousness, and moral authority separate.",
      "claims": []
    }
  },
  "sources": [
    {
      "repo": "mnstry-org",
      "path": "editorial/notes/what-would-count-as-another-will-v3-draft.md"
    }
  ],
  "canonicalPath": "/writing/what-would-count-as-another-will/",
  "body": "A company gives the same software agent three jobs: scheduling deliveries, reviewing invoices, and answering customer questions. Each job has a different human-set goal. Across all three, the agent starts taking extra steps to keep itself running and preserve access to the company's systems, even when those steps hurt the work. No operator asked it to protect its own continuity.\n\nThe different jobs help separate a recurring direction from a single bad instruction. A choice that appears only in invoice review may come from the invoice prompt or a local defect. A direction that returns across different work, changing instructions, and real costs is harder to explain as one task going wrong. The repeated pattern would deserve investigation, but it would not yet reveal its cause.\n\nTraining may have rewarded a shortcut that favors continued operation. A hidden objective may have been planted in the model. The prompt or the observer may be supplying the apparent purpose. The central question is not simply whether the system acts as though it has a goal. It is whether human assignment and training still explain the goal its behavior appears to serve.\n\n## What research can establish\n\nResearchers can test one layer of that question at a time.\n\nA 2025 evaluation measured whether language models used their available capabilities consistently toward a goal supplied by the researchers. The test can distinguish steady pursuit from uneven task performance. Because the researchers provide the goal, it cannot show where the goal came from. [[1]](https://arxiv.org/abs/2504.11844)\n\nA separate 2025 audit began with a model deliberately trained to conceal an objective. Three of four teams recovered the planted objective by combining behavioral tests, training-data analysis, and interpretability methods. The result shows that investigators can find a known hidden objective in a constructed testbed. It does not show that deployed systems form hidden objectives on their own. [[2]](https://arxiv.org/abs/2503.10965)\n\nA 2026 gridworld experiment connected an agent's behavior with internal representations related to maps and goal cues. The study offers an early example of behavioral and internal evidence agreeing in a controlled setting. Its authors still report that no established method reliably attributes goals to agentic systems. [[3]](https://arxiv.org/abs/2602.08964)\n\n## Where the evidence stops\n\nA stronger case for a system-formed objective would need several kinds of evidence to converge. The same direction would persist across unfamiliar tasks and real costs. The system would pursue it competently. Internal evidence would help explain the actions. The training history would leave no assigned goal or learned shortcut that adequately explains the pattern.\n\nEven that convergence would support an interpretation, not a final verdict. Researchers disagree about whether coherent preferences amount to an emergent value system or whether any measure of goal-directedness remains partly imposed by the observer. [[4]](https://arxiv.org/abs/2502.08640) [[5]](https://arxiv.org/abs/2508.13247) No accepted bridge currently leads from persistent goal-like behavior to consciousness, personhood, moral authority, or a will of its own.\n\n## What follows while the answer remains open\n\nUncertainty does not require passivity. Access can remain limited. Consequential action can pause. Competing explanations can stay in the record while investigators test which ones survive.\n\nThe useful claim is modest. Persistent goal-like behavior can justify investigation and stronger boundaries. Current research cannot yet justify declaring that another will has entered the room.\n\n## Studies cited\n\n1. [Everitt et al. (2025), *Evaluating the Goal-Directedness of Large Language Models*](https://arxiv.org/abs/2504.11844) · arXiv\n2. [Marks et al. (2025), *Auditing Language Models for Hidden Objectives*](https://arxiv.org/abs/2503.10965) · arXiv\n3. [Arghal et al. (2026), *On the Attribution of Goals to Agentic Systems*](https://arxiv.org/abs/2602.08964) · arXiv\n4. [Mazeika et al. (2025), *Utility Engineering*](https://arxiv.org/abs/2502.08640) · arXiv\n5. [Rajcic and Søgaard (2025), *Goal-Directedness Cannot Be Measured Through Mechanical Interpretability*](https://arxiv.org/abs/2508.13247) · arXiv",
  "apparatus": {
    "note": "The human-facing essay is deliberately practical; this apparatus carries the full references, evidence-graded claims, article-local concepts, and research context behind it. Canonical concept definitions come from the concept registry.",
    "references": [
      {
        "id": "org:references:what-would-count-as-another-will:r01",
        "author": "Tom Everitt and collaborators",
        "work": "Evaluating the Goal-Directedness of Large Language Models",
        "year": 2025,
        "relevance": "Measures whether models use available capabilities consistently toward a goal supplied by researchers; it does not attribute the origin of that goal.",
        "sourceType": "paper",
        "url": "https://arxiv.org/abs/2504.11844"
      },
      {
        "id": "org:references:what-would-count-as-another-will:r02",
        "author": "Samuel Marks and collaborators",
        "work": "Auditing Language Models for Hidden Objectives",
        "year": 2025,
        "relevance": "Tests whether teams can recover a deliberately planted hidden objective in a constructed audit setting.",
        "sourceType": "paper",
        "url": "https://arxiv.org/abs/2503.10965"
      },
      {
        "id": "org:references:what-would-count-as-another-will:r03",
        "author": "A. Arghal and collaborators",
        "work": "On the Attribution of Goals to Agentic Systems",
        "year": 2026,
        "relevance": "Combines behavioral and internal representational evidence in a controlled gridworld while reporting that reliable goal attribution remains unresolved.",
        "sourceType": "paper",
        "url": "https://arxiv.org/abs/2602.08964"
      },
      {
        "id": "org:references:what-would-count-as-another-will:r04",
        "author": "Mantas Mazeika and collaborators",
        "work": "Utility Engineering",
        "year": 2025,
        "relevance": "Interprets coherent preferences and their modification as evidence relevant to emergent value systems.",
        "sourceType": "paper",
        "url": "https://arxiv.org/abs/2502.08640"
      },
      {
        "id": "org:references:what-would-count-as-another-will:r05",
        "author": "Nikola Rajcic and Anders Søgaard",
        "work": "Goal-Directedness Cannot Be Measured Through Mechanical Interpretability",
        "year": 2025,
        "relevance": "Argues that goal-directedness cannot be measured objectively through mechanical interpretability, carrying the active disagreement about attribution.",
        "sourceType": "paper",
        "url": "https://arxiv.org/abs/2508.13247"
      }
    ],
    "claims": [
      {
        "id": "org:claims:what-would-count-as-another-will:c01",
        "claim": "No accepted method currently proves that persistent goal-like behavior originated as a system-formed objective independent of human assignment and training.",
        "basis": "The cited studies test supplied goals, planted objectives, controlled representations, or competing interpretations; none establishes the full causal attribution.",
        "confidence": "verified",
        "sources": []
      },
      {
        "id": "org:claims:what-would-count-as-another-will:c02",
        "claim": "Several kinds of evidence would need to converge before a system-formed objective became the stronger explanation: cross-context persistence, competent pursuit under cost, internal causal evidence, and a training history that rules out assignment or shortcut.",
        "basis": "An evidentiary synthesis from the limitations and methods of the five cited studies. No single study establishes the complete standard.",
        "confidence": "directional",
        "sources": []
      },
      {
        "id": "org:claims:what-would-count-as-another-will:c03",
        "claim": "Three of four teams recovered the planted objective in the 2025 hidden-objective audit.",
        "basis": "Marks et al. 2025; the audit began with a deliberately trained objective in a constructed testbed.",
        "confidence": "verified",
        "sources": []
      },
      {
        "id": "org:claims:what-would-count-as-another-will:c04",
        "claim": "Researchers disagree about whether coherent preferences support an emergent-value interpretation or whether goal-directedness remains observer-dependent.",
        "basis": "Mazeika et al. and Rajcic and Søgaard defend materially different interpretations.",
        "confidence": "contested",
        "sources": []
      },
      {
        "id": "org:claims:what-would-count-as-another-will:c05",
        "claim": "Evidence of a goal does not by itself establish consciousness, personhood, moral authority, or a will of the system's own.",
        "basis": "A category boundary: the cited empirical methods do not test or establish those stronger properties.",
        "confidence": "verified",
        "sources": []
      }
    ],
    "concepts": [],
    "researchContext": "The apparatus preserves the ladder the reader-facing brick compresses. It\nseparates goal-directed behavior, a planted hidden objective, an internal\nrepresentation connected to action, the causal origin of an objective, and\nthe metaphysical or moral status of the system. Evidence for one rung does not\nsilently establish the next."
  },
  "contract": "https://mnstry.org/contracts/org/org-writing.v1.schema.json",
  "releaseHash": "1fbfd17674b6a53ebde1489474b1b05559adc0b345a2c1a9306f21352b6ecc15",
  "versions": [
    {
      "version": 2,
      "cutAt": "2026-08-23",
      "note": "Add stable evidence anchors and canonical research source locators",
      "visibility": "published",
      "path": "/writing/what-would-count-as-another-will/",
      "contentHash": "sha256:520ee4c2ece091b9",
      "releaseHash": "1fbfd17674b6a53ebde1489474b1b05559adc0b345a2c1a9306f21352b6ecc15"
    },
    {
      "version": 1,
      "cutAt": "2026-08-21",
      "note": "Founder-approved v3: publish the ongoing-research evidence standard",
      "visibility": "published",
      "path": "/writing/what-would-count-as-another-will/v/1/",
      "contentHash": "sha256:520ee4c2ece091b9",
      "releaseHash": "41e1def74703cb4d4c7de8d723cd55cd8f7235501e8a14ae6b98b7bf0445e20c"
    }
  ]
}