Debugging Software No Human Wrote

Compiler Errors Are the API

April 22, 2026

Terminal output is a user interface for a human who already knows which process they launched and roughly where to look.

An autonomous programmer may be debugging generated code across a browser, framework, API, payment provider, database, deployment platform, and DNS record. Giving it three thousand lines of mixed logs is not observability. It is data disposal.

Feedback for coding agents should be structured, causal, provenance-aware, multimodal, and partly outside the agent's authority. It should connect an observed failure to the semantic program and identify legal repairs without pretending every cause is known.

A useful observation is a correlated evidence bundle, not a concatenated terminal transcript.

Part 4 added experiments and structured observation to the programming loop. This post asks what the deterministic half of that loop should say back to the probabilistic half.

Raw output is the wrong abstraction

Current agent feedback fails in predictable ways:

Logs from unrelated processes are interleaved.
IDs change across the browser, server, provider, and database.
Generated stack traces point at backend files rather than semantic declarations.
Screenshots show symptoms without the state that produced them.
Tests report failed assertions without the causal context of the run.
Provider dashboards hide configuration outside the trace.
The agent spends tokens rebuilding a chronology the runtime already witnessed.

Humans cope because they carry an informal model of the system. They know that this terminal belongs to the dev server, that this request followed that click, and that this webhook probably arrived later. A generated system should not require the model to reconstruct those relationships from prose and timestamps.

The strongest objection: causal instrumentation lies too

Full causal instrumentation is expensive and incomplete. Distributed systems, native code, external providers, and nondeterministic environments do not reliably produce a single clean explanation. A structured diagnostic can create more confidence than the evidence deserves.

That is a real danger. The response is not to label every inference a root cause.

Diagnostics should distinguish observed facts, derived facts, hypotheses, rejected hypotheses, and unresolved ambiguity. Every causal claim should point to evidence and, where useful, carry confidence. Competing hypotheses are allowed. Raw artifacts remain available for inspection. Unknown is a valid result.

Instrumentation can also grow incrementally. Generated code and controlled adapters are the strongest starting point because the compiler owns both ends of the mapping. Foreign code and external providers may expose only a trace boundary and observed result. The evidence model should make that gap visible.

The goal is not omniscience. It is to stop collapsing facts and guesses into one paragraph.

Evidence bundles should be queryable

One observation bundle can contain:

Run, step, environment, and time identity.
Screenshot, rendered artifact, and viewport.
Accessibility tree, focus, and semantic UI nodes.
Console and process events.
Network requests and responses.
Distributed trace and context.
Application and database deltas.
External-resource changes.
Source-map and semantic provenance.
Redaction, retention, and authority metadata.

observation run.184 step.12 {
  environment = preview
  trigger = ui.node("pricing.buy-annual-plan")

  visual {
    screen = artifact("screen-12.webp")
    accessibility = tree("a11y-12.json")
    focus = ui.node("pricing.buy-annual-plan")
  }

  execution {
    console = events(level >= warning)
    network = requests(since = step.11)
    trace = trace(Storefront.begin_checkout)
  }

  state {
    application_delta = Storefront.state.diff
    database_delta = Storefront.commerce.diff
    external_delta = Storefront.billing.diff
  }

  provenance = [
    Storefront.web.PricingPage.buy_button,
    Storefront.begin_checkout,
    environment.preview.billing_binding
  ]
}

The agent should query this bundle rather than receive all of it. "Show network requests caused by step 12" is cheaper and safer than dumping every request. Secret-bearing panels can remain behind a stronger authority boundary and return only typed facts such as reference exists, binding denied, or value rotated.

Stable provenance must survive lowering

The causal path for checkout crosses representations:

semantic operation
  → generated TypeScript function
  → HTTP handler
  → provider request
  → webhook event
  → database transaction
  → rendered UI state

Line numbers are not enough. A repair may regenerate the backend and move every function. Stable semantic identities let the trace continue to name Storefront.begin_checkout and environment.preview.billing_binding across those changes.

This idea is not unique to generated software. OpenTelemetry traces correlate spans across process boundaries, and the W3C PROV model defines interoperable provenance for entities, activities, and agents. The additional requirement here is a mapping from runtime evidence back through generated artifacts to the semantic declaration the model can repair.

A diagnostic should be an API response

A useful diagnostic has a stable code, severity, risk class, affected semantic identity, expected and observed state, evidence references, candidate or confirmed causes, legal repairs, required authority, retry safety, and reproducibility status.

{
  "code": "CAPABILITY_BINDING_MISSING",
  "severity": "error",
  "environment": "preview",
  "operation": "Storefront.begin_checkout",
  "capability": "Storefront.billing.credentials",
  "observed": {
    "request": "POST /checkout",
    "response_status": 500,
    "trace": "run.184.trace.22",
    "provider_state": "unbound"
  },
  "cause": {
    "status": "confirmed",
    "declaration": "environment.preview",
    "missing_binding": "billing.credentials"
  },
  "repairs": [
    {
      "operation": "bind_secret",
      "reference": "preview/storefront/stripe",
      "requires": ["secret.bind.preview"]
    },
    {
      "operation": "bind_simulator",
      "simulator": "commerce.payments.sandbox",
      "requires": ["environment.preview.modify"]
    }
  ],
  "forbidden_repairs": [
    "bind production credentials to preview",
    "mark checkout successful without provider evidence"
  ]
}

The model can propose one of the legal operations or produce a different patch. It does not have to parse a paragraph to learn which declaration is missing or whether a retry is safe.

Facts and hypotheses are different types

Consider the initial failure:

Observed: The buy button received a tap.
Observed: POST /checkout returned 500.
Observed: No checkout appeared in the payment simulator.
Derived: Checkout creation did not complete.
Hypothesis: The preview environment lacks a payment credential binding.
Confirmed after inspection: The binding is absent.
Rejected hypothesis: The button failed to dispatch its action.

The labels matter. Before provider-state inspection, the missing binding is not yet a confirmed cause. The runtime may propose a probe instead of a repair.

Provenance lets the system explain a visible failure as one causal chain across generated code and external configuration.

The first confirmed root cause sits in the control plane. The 500 response and visible error are downstream symptoms. A repair that changes the UI message may improve presentation but cannot close the diagnostic.

The evaluator cannot belong entirely to the agent

An autonomous repair loop has an obvious failure mode: change the definition of success.

Delete the test. Lower the threshold. Catch and ignore the exception. Mock the payment provider in production. Hard-code "active" in the UI. Grant broader permissions. Accept unsigned webhooks. Each can turn a red check green while making the product worse.

Some artifacts therefore need a protected plane outside the agent's unrestricted authority:

Autonomous repair requires an evaluator the repair process cannot silently rewrite.

The agent may control the semantic program, development experiments, probes, candidate patches, and fixtures. Product requirements, security policy, hidden tests, production approvals, audit records, and secret values may be owned independently.

protected evaluation StorefrontPurchasePolicy {
  assert anonymous_user_cannot_begin_checkout
  assert preview_cannot_access_live_payment_resources
  assert duplicate_payment_events_create_one_subscription
  assert invalid_event_signature_never_changes_entitlement
  assert browser_return_alone_never_activates_subscription
  assert active_entitlement_requires_verified_payment_event
}

Protected does not mean silent. The evaluator should return enough evidence and semantic localization to support repair. It need not reveal every hidden test implementation or allow the agent to edit the policy that rejected its patch.

Project knowledge is not a longer chat transcript

A development trajectory discovers facts worth preserving:

User requirements.
Observed environment facts.
Derived system facts.
Open and rejected hypotheses.
Failed repairs.
Provider-specific constraints.
Temporary debugging assumptions.

Each durable claim needs scope, evidence, and invalidation. "Preview uses sandbox payments" may be confirmed for one environment today and false after a migration.

knowledge PreviewPaymentBinding {
  kind = observed_environment_fact
  scope = environment.preview
  claim = "Preview checkout requires a sandbox payment binding"
  evidence = [run.184, run.186]
  confidence = confirmed
}

knowledge ButtonDidNotReceiveTap {
  kind = rejected_hypothesis
  scope = Storefront.web
  evidence = [run.184.input_trace]
  reason = "Tap reached the semantic UI node"
}

The rejected hypothesis is useful. It prevents the next repair attempt from repeating an experiment that already produced decisive evidence.

A repair should leave a regression artifact

After binding the sandbox adapter, the agent reruns the purchase scenario. The protected evaluator verifies that checkout succeeds and preview cannot reach live resources.

The successful repair should then:

Promote the reproducing experiment into a regression scenario.
Store the diagnostic and verified repair trajectory.
Preserve the protected preview/live separation assertion.
Record the observed environment fact with evidence.

This turns debugging into future test coverage and grounded training data. It also gives later maintainers a causal explanation rather than only a diff.

Part 5 focuses on structured evidence, protected evaluation, project knowledge, and the feedback path into repair.

This architecture is elaborate enough to be wrong in a hundred ways. The only useful next move is to build the smallest slice that can measure whether the language and feedback protocol outperform a constrained conventional baseline.

The compiler error is not prose about a failure. It is an API response from the deterministic half of the programming system to the probabilistic half.

Previous: The Compiler Is Only Half the Programmer. Next: Build the Dumbest AI Language That Could Possibly Work.