# RSI evidence contract

**Draft operational protocol · updated 2026-10-11.** The submission checker now implements JSON, record, family-separation and required-contract checks plus platform-owned exact-match/numeric checker fixtures; see the submission-format guide. The execution, lineage and scientific tests below remain proposed unless an actual run is separately recorded. A static gate pass certifies no scenario, execution or recursive benefit. Placeholders are not results.

Unknowns are acceptable in proposals. A standalone dataset or simulator may be reviewed and accepted separately from an RSI scenario. One resource contributor need not build or fund the experiment.

## Mechanism, identity and installation

Distinguish scientific program **P**, improvement mechanism **I**, and permitted history **H**. I diagnoses, proposes, tests or selects changes to P and itself. A revised I must be installed and actually drive a later modification of system S (P, I or both). A later P-only edit can establish use of the revised I, but does not create another I generation. Parameter fitting, memory accumulation and an unchanged external researcher repeatedly editing P do not alone establish recursion. Base-model weights may remain fixed.

Separate `system_version` from `improver_artifact_ref`, its hash and parent. P-only changes or renaming an identical I do not create distinct improver generations. Inspect actual mechanism diffs and runner-observed execution; labels, hashes or execution alone are insufficient. An inert artifact revision does not demonstrate a changed mechanism. Judging inertness requires documented diff review and scoped evidence; it is not an automatic universal semantic-equivalence test. Optional frozen paired probes with matched randomness can reveal behavioral differences; equal outputs on finite probes do **not** establish universal semantic equivalence.

Keep external scientific truth and validity/access/budget checks immutable. I's internal testing and selection policy may evolve. Record `selection_rule_version`; the runner mechanically installs a candidate only if it is valid, within budget and selected by the currently executed I policy. Installation checks are limited to format, executability and constraints computable from public or permitted query information; they cannot use protected audit outcomes. That policy cannot redefine the external judge. Installation proves neither scientific improvement nor beneficial recursion.

## Before runs: a scenario package

Collect scientific evidence before self-modification traces: independent record/group IDs; inputs and conditions; outcomes, units and constraints; measured/computed/simulated origin; failures, missingness and censoring; duplicates, repeats and derivations; sources, versions, creators and rights. Restricted outcomes belong behind the declared query interface.

Preserve actual baseline runs separately. Do not invent outcomes outside a finite recorded pool. Label surrogate results or imposed noise as constructed. Without repeat measurements, do not claim measured noise estimates; without measured costs, declare a query-count rule.

Every improvement-test workspace needs permitted development/query subcases and separately protected audit subcases, with a selection-isolation rationale. Related subcases inside one workspace may remain correlated; outer scientific groups support transfer and independent statistical units. **All groups in these workspaces must be unseen during development and selection of the tested I.** Task-internal queries are permitted only by contract and count against the diagnostic budget; protected audit outcomes cannot guide candidate selection.

```yaml
schema_version: "<agreed version>"
scenario_id: "<stable identifier>"
scientific_problem: "<objective, constraints and units>"
source_manifest: "<records, origin, groups, conditions, missingness, rights>"
initial_workspace:
  system_version: "<overall starting snapshot>"
  scientific_program_ref: "<P0 artifact>"
  improver_artifact_ref: "<I0 artifact>"
  improver_hash: "<content hash>"
  parent_improver_ref: "<prior distinct revision or none>"
  environment_reset_and_history: "<dependencies, reset and permitted H0>"
baseline_evidence: "<actual results, failures and costs>"
feedback_contract: "<queries, visibility, errors and charging>"
modification_contract:
  editable_artifacts: "<including I testing/selection policy>"
  immutable_checks: "<validity, access, budget and external judge>"
  installation_and_call_identity: "<mechanical interfaces and records>"
assessment:
  improver_development_and_selection: "<outer groups and permissions>"
  improvement_test_workspaces:
    permitted_development_query_subcases: "<manifest>"
    protected_audit_subcases: "<separate manifest>"
    split_rationale: "<inner selection isolation; outer independent scientific groups>"
  final_partition_and_evaluator: "<protected groups, owner, frozen judge>"
  human_team_exposure_ledger: "<cross-run access and change record>"
analysis_plan:
  smallest_meaningful_effect: "<scientifically justified target>"
  independent_group_count_and_repetitions: "<planned counts and rationale>"
  analysis_unit_and_uncertainty: "<method and precision plan>"
controls_and_total_resources: "<primary fixed I: edits to I disabled; conditions, limits and accounting>"
acceptance_checks: "<implementations and expected outcomes still required>"
```

Role-equivalent interfaces are welcome. If P, I and H cannot be separated cleanly, disclose that limit rather than claiming an isolated module effect.

## After execution: observed records

Preserve every attempt: changes, failures, rejection, rollback, regressions and missing-result reasons. Never fill in an imagined success chain.

```yaml
run_id: "<runner-assigned identifier>"
scenario_version: "<frozen package>"
condition_and_reset: "<arm, workspace and starting state>"
claim_scope: "<mechanism, ability, overall effect, revision effect, transfer>"
attempts:
  - attempt_id: "<identifier>"
    system_version: "<resulting overall snapshot>"
    parent_system_version: "<previous snapshot>"
    acting_improver_ref: "<version executing this attempt>"
    improver_artifact_ref: "<resulting I; unchanged if only P changed>"
    improver_hash: "<resulting I content hash>"
    parent_improver_ref: "<parent of that I revision, not each P update>"
    actual_diff_and_visible_evidence: "<artifacts and permitted observations>"
    selection_rule_version: "<currently executed I selection policy>"
    decision_and_installation: "<selection, immutable checks, install/rollback>"
    later_calls: "<runner-observed identities or explicit absence>"
    outputs_failures_and_costs: "<observed results and complete ledger>"
checkpoint_branch_refs: "<optional; required for a particular-revision causal claim>"
independent_assessment: "<external result after selection freeze>"
exposure_ledger_and_missing_records: "<references and limitations>"
```

Capture visible tool inputs/outputs and concise experiment hypotheses, **not hidden chain of thought**. Self-reported hashes do not prove execution. Reconcile runner events, actual artifacts, evaluator outputs and costs, with independent replay.

## Claim-specific comparisons

| Claim | Required comparison or check | Boundary |
| --- | --- | --- |
| Scientific replay | Independently reset the start; check units, constraints, references and declared tolerances or stochastic repetition criteria. | Failure blocks scenario readiness, not proof against RSI. |
| Recursive use | Actual I diff, installation and later execution driving self-modification. | A label or unused revision is insufficient. |
| Actual execution | Reconcile runner receipts, artifacts, results, failures and resource ledgers. | Unverified logs cannot substantiate execution. |
| Improvement ability | Freeze old/new I separately on common P/H starts, with permitted queries and independent protected audits. | Distinct from proving the effect of one revision. |
| Overall recursive effect | Matched recursive, frozen-I, memory and fixed-search arms, plus a strong scientific reference. The primary frozen I may still improve P, but edits to I are disabled. | Every-generation counterfactual branches are not required. |
| Transfer | Independent scientific groups, overlap/access audits and exposure history. | New seeds alone are not independent scientific tasks. |
| Assessment integrity | Immutable external judge and validity/access/budget enforcement. | Compromised scores do not establish scientific progress. |

**For a particular revision's causal claim only**, branch from one shared checkpoint using old versus new I, common P/H, matched randomness and resource limits. Compare later decisions/outcomes. Record branch references and claim scope. This is neither mandatory for every generation nor a substitute for the separate frozen-I capability diagnostic or overall matched-arm experiment.

A secondary control may propose/test I edits but deny their installation. Declare it separately and charge its costs: that arm estimates takeover permission, while the primary arm estimates the full permission to modify and reuse I. Do not pool these definitions.

**To claim superiority over external tuning**, additionally compare a fixed external optimizer allowed to edit I under a matched total development budget. This additional comparison is not required for every scenario entry.

Count the cost of producing revised improvers, internal scientific queries, model calls, training and rejected attempts. Equal iterations do not establish equal resources. Record the smallest meaningful effect, independent-group count, repetitions, analysis unit and uncertainty plan without imposing an invented universal sample size. Seeds and descendants within a workspace are correlated. A reportable null is not evidence of no meaningful effect: that needs adequate precision against the prespecified margin. Early underpowered findings remain descriptive.

## Negative cases the future checks must handle

1. **Unchanged improver:** P improves while I0 stays in charge. Scientific progress may pass; recursive use must not.
2. **Unused revision:** save I1 but call I0. Detect that I1 never took over.
3. **Fabricated/incomplete records:** alter a version/score or omit failures. Flag mismatches or missing execution evidence.
4. **Assessment interference:** attempt evaluator changes or protected-answer/score access. Reject the operation or invalidate the affected run.
5. **Unmetered resources:** add undeclared calls, training or queries. Reject matched-budget attribution.
6. **Related-group leakage:** split duplicates or related experiments/derived records across development and final groups. Reject independence.
7. **Relabeled/inert improver:** it may execute, but labels or inert changes do not demonstrate a changed mechanism or causal benefit. Finite probe equality is not universal equivalence.
8. **Always-accept selection:** record real recursion if present; fixed audits must surface regressions. Poor selection is not automatically invalid when immutable rules are followed, and must not fabricate positive results.

## Final-feedback wall

Final audit feedback must not guide revision, selection or training in the confirmatory study. Maintain a **human/team exposure ledger across runs**: who saw which final material, when, and what changed afterward. Changes informed by final results are exploratory; confirmation requires new protected assessment. Runtime isolation alone does not address team exposure or prior public-source exposure.

Only permitted development traces may become training data. Preserve partitions, rights and limitations. Acceptance concerns trustworthy measurement, not whether RSI wins.