Home/Documentation/Work, waits & evidence

What is ready, what waits, and why.

Updated Read as Markdown ↗

Factory keeps the record of what happened, derives the problem's current lifecycle and checks what can move next. An open backlog item can already have an investigation, a pending probe or a released fix.

12 signals do not mean 12 fixes.

This fictional app has 12 signal identities at one point in time. Each appears once in the register below. Repeated occurrences of one signal are a different count. S01–S12 are reading labels, not Factory IDs.

SignalsWhat happens next
5 · Ready to investigateThree session-related signals form one investigation; two others stand alone. Five signals, three work packets.
4 · Waiting for evidenceEach has a recorded probe and a measurable return condition. The worker can move on while the plan waits.
2 · Waiting for verificationThe changes are released. The configured independent verifier has the next turn; neither issue is cured yet.
1 · Needs an owner decisionA product choice is outside the project's authority. That item waits while unrelated authorized work can continue.

5 + 4 + 2 + 1 = 12 signal identities. The five eligible signals form three investigation packets: one family of three, plus two individual signals. Neither grouping nor eligibility proves a root cause.

These are teaching categories, not a new Factory status enum or an export from a customer board. A missing direct receipt also does not prove an item is untouched: related history may exist elsewhere.

Try the numerical example on the homepage.

5 signals · Ready to investigate

S01

Email sign-in returns to an empty workspace

Onboarding / investigation packet A

What we know
A support report and callback trace show the workspace context disappearing after sign-in.
Next step
Packet A: trace session restoration alongside S02 and S03.
S02

A tool call gets a 403 after session refresh

AI assistant / investigation packet A

What we know
The tool trace contains the same missing workspace context seen in S01.
Next step
Packet A: compare the tool request with the sign-in callback. A shared cause still needs proof.
S03

The API rejects requests without workspace context

Server reliability / investigation packet A

What we know
Request logs point to the same session-restoration path as S01 and S02.
Next step
Packet A: reproduce all three paths before deciding whether one repair covers them.
S04

A 90-day report times out

Analytics / investigation packet B

What we know
A reproducible report request has a slow query trace and a known dataset size.
Next step
Packet B: inspect the query and reproduce the timeout independently of the session family.
S05

A webhook retry processes an event twice

Server reliability / investigation packet C

What we know
Two deliveries with the same event ID produce two writes in a reproducible test.
Next step
Packet C: investigate idempotency at the write boundary.

4 signals · Waiting for evidence

S06

A project request fails with no captured payload

Project creation

What we know
One occurrence is known. The request payload and workspace context needed to reproduce it are missing.
Next step
Capture two new matching occurrences, or recheck on September 21 at 08:00 UTC. Then run the recorded probe.

Example identity: verify:project-payload. Inspect its evidence-plan receipt.

S07

A chart is blank in one mobile session

Analytics

What we know
A screenshot exists, but there is no browser trace to distinguish rendering from missing data.
Next step
A recorded plan schedules a trace check for September 18 at 08:00 UTC. If it is still missing, record the result and a new effective plan.
S08

A tool occasionally returns an empty result

AI assistant

What we know
Two occurrences are known; the tool inputs needed to compare the failures are incomplete.
Next step
The plan freezes a baseline of 2 and requests 3 new occurrences, or a September 20, 08:00 UTC recheck. Inspect the captured inputs when due.
S09

A notification arrives late

Server reliability

What we know
The app timestamp is available; the provider delivery log is missing.
Next step
The plan schedules a cross-source probe for September 19 at 08:00 UTC to compare app and provider timestamps.

2 signals · Waiting for verification

S10

The invite-expiry message has been repaired

Onboarding

What we know
The repair and deployed commit are recorded. Its independent behavioral check has not returned a verdict.
Next step
The configured verifier checks expired and valid invites against that release. Keep the boundary open until the required evidence supports closure.
S11

The report cache now invalidates after an import

Analytics

What we know
Deployment is recorded; the post-release freshness check is pending. This is separate from S04's query timeout.
Next step
The configured verifier imports a sample dataset and checks that the report reflects the new data.

1 signals · Needs an owner decision

S12

Should the company-size step be optional?

Onboarding

What we know
The step has measurable drop-off, but changing required information is a product choice outside the worker's authority.
Next step
Ask the owner to decide the intended behavior. A recorded answer must still pass scope and execution-authority checks.

How five eligible signals become three packets

S01 + S02 + S03Packet AInvestigate lost session context

S04Packet BInvestigate the report query

S05Packet CInvestigate duplicate event writes

The selector can group a related family into one investigation. It preserves the member signals. This snapshot assumes no active repair needs to resume first; A, B and C name the packets, not a guaranteed execution order.

Three layers, three different questions.

  1. Backlog status: is the request still open? Factory's pack backlog uses open, done and dismissed.
  2. Problem lifecycle: what does the evidence establish? Examples include ACTIONABLE, EVIDENCE_PENDING and WAITING_VERIFICATION.
  3. Receipts: what actually happened? Investigation, evidence plan, remediation, merge, deploy and verification are recorded facts. An evidence_plan is a receipt, not another lifecycle state.

The append-only ledger preserves the history. Projections reconstruct the current state from those records. The checkpoint and selector determine whether to resume active work, select a packet, wait, or request human input.

S06: one occurrence. Two more. Then run the probe.

S06 is the project-creation request from the register. It failed once, but the discriminating payload is missing. An inconclusive investigation leaves the identity in EVIDENCE_PENDING. Its evidence plan records the missing evidence and a specific way back.

Illustrative evidence-plan receipt
{
  "receiptKey": "plan:project-payload:1",
  "receiptKind": "evidence_plan",
  "signalIdentity": "verify:project-payload",
  "signalSource": "verify",
  "probeType": "wait_for_occurrences",
  "planBaselineOccurrences": 1,
  "targetSampleCount": 2,
  "dueAt": "2026-09-21T08:00:00Z",
  "missingEvidence": "Request payload and workspace context for the failing path",
  "nextProbe": "Inspect matching request events and reproduce the workspace-context failure",
  "createdAt": "2026-09-15T08:00:00Z"
}

targetSampleCount = 2 means two new occurrences beyond planBaselineOccurrences = 1. At two total occurrences, one new sample is still missing. At three total occurrences, the plan is due. Reaching the absolute dueAt timestamp also makes it due, even if the extra samples never arrive.

The next job is run-probe. It is eligible for selection, not a promise to run immediately ahead of an active repair or higher-priority current work. If evidence is still insufficient, investigate the result and record a new effective plan. Do not treat the calendar reaching a date as proof of a cause.

Factory refuses a new evidence plan without a machine-evaluable return condition. An occurrence target requires its frozen baseline; wait_for_occurrences requires a target. A free-text externalCondition alone is insufficient. If the source does not provide a usable occurrence count, the count threshold is not considered met.

Follow packet A through one turn of the loop

This walkthrough starts from the snapshot above and explores what could happen next. The receipts described here are hypothetical continuation events, not evidence that the 12 listed signals have already been fixed.

  1. Checkpoint: reconstruct the work already in progress

    Read current project evidence, the backlog and recorded receipts. S06–S09 have evidence waits; S10–S11 have released changes awaiting the configured independent verifier; S12 needs a product decision.

    Gate: resume an active repair when required. A new turn does not discard its branch, release boundary or unresolved decision.

    Inspect the checkpoint command
  2. Selection: choose a packet that can move

    Evaluate active evidence plans against their frozen occurrence baselines and dates. The five eligible signals form packets A, B and C. For this walkthrough, assume A is selected and the pack allows investigation.

    Output: a focused work packet for the session-context family, carrying the member evidence. Due probes compete within the selection rules; they do not automatically interrupt active work.

  3. Investigation: test the suspected connection

    Trace S01's sign-in callback, S02's tool request and S03's API rejection. Reproduce the missing workspace context and check whether the same code path explains all three.

    Record: an investigation receipt. Confirmed shared causality can be recorded as a same_cause relation. If the evidence disagrees, keep separate causes or reclassify the family.

    If evidence is missing: record an effective evidence plan, as in S06. If a product or cost choice is needed, record that boundary and continue unrelated authorized work.

    Inspect relation tools
  4. Remediation: repair the proven boundary

    Within the pack's permissions, prepare an isolated code change and check all three affected paths. Test the original failure and a valid session so the repair does not block legitimate users.

    Record: remediation evidence tied to the signal and repair boundary. Passing local tests supports review; it does not establish a production cure.

    Inspect the repair command sequence
  5. Merge and release: establish what actually shipped

    Follow the project's review and release authority. Record the merge separately from deployment, with evidence identifying the released commit.

    Gate: a draft-PR boundary can end the worker's turn. If release is authorized and recorded, the boundary proceeds to verification. A merged PR alone cannot prove that users have the fix.

    Inspect receipt recording
  6. Independent verification: check the released behavior

    The designated verifier exercises sign-in, refreshed AI tool calls and API requests against the release. The executor cannot write the verdict on its own repair.

    Outcome: sufficient passing evidence can support CURED. A required later reconfirmation keeps the boundary IN_PRODUCTION_UNCONFIRMED; a failed verdict can make it RECURRENT. Inconclusive evidence does not justify closure.

    Inspect independent verification
  7. Fresh checkpoint: continue and watch for a return

    Reconstruct state again before selecting more work. Packet B or C may be next; a matured evidence plan may now be eligible. When S06 has two new occurrences, its next job is to run the probe, not to start an assumed fix.

    If the session bug returns: preserve its earlier investigation, deployment and verdict, then investigate the new occurrence. Compare it with the previous cause before deciding whether the cure failed or a different variant needs its own boundary.

    Follow the seven-day recurrence example

Then measure the user outcome

A technically verified repair and a better product metric answer different questions. For onboarding, compare completion rate against the agreed baseline, population and observation window. A target of +2 percentage points remains a target until measured; a passing test does not establish that lift.

See the onboarding measurement example and the current product-outcome roadmap.

Group the cause. Keep the history.

Factory has two related mechanisms. Family grouping lets the selector investigate related signals together. A durable same_cause relation declares that two identities represent the same underlying problem and lets the capability map count that problem once. A family label alone is not proof of equivalence.

For example, an onboarding failure, a rejected AI tool call and a server error may all point to lost workspace context. Establish that shared cause before treating one repair as covering all three. Preserve the member evidence and release checks. A later failure must remain visible.

The ARVO label DUPLICATE_CONFIRMED is not a Factory lifecycle state and should not be used as a shortcut to CURED. Similarly, an existing fix must be linked to the relevant release and verification; a similar description is not enough.

If a group turns out to contain different causes, Factory records family_reclassification. The parent becomes RECLASSIFIED_INTO_CHILDREN, and the child identities carry the work. Retirement is not a cure.

Factory's lifecycle vocabulary

LifecycleMeaning
ACTIONABLEEligible for the next work packet. An investigation or probe may still be needed.
EVIDENCE_PENDINGThe cause is unconfirmed. A recorded evidence plan defines what to inspect and when to return.
IN_REMEDIATIONA repair boundary is in progress. A merge alone does not prove release.
WAITING_VERIFICATIONA deployment is recorded; the independent verdict is still pending.
IN_PRODUCTION_UNCONFIRMEDThe first verification passed, but a required later measurement has not confirmed the cure.
CUREDThe release and required verification support a technical cure. Product benefit is measured separately.
RECURRENTA verification or later reconfirmation failed, or a cured signal was observed again.
DESIGN_DECISION_PENDINGA product or design decision needs the owner's answer.
OPS_OR_COST_DECISION_REQUIREDAn operational or spending decision needs authority.
EXECUTOR_RETRY_EXHAUSTEDThe bounded execution attempts ran out. The failure remains visible for recovery.
RECLASSIFIED_INTO_CHILDRENA wrongly grouped parent is retired. Its child identities carry the remaining work; the parent is not marked cured.
LIFECYCLE_UNKNOWNThe available history does not establish a reliable lifecycle. This is not evidence of a cure.

merge and deploy are distinct receipt kinds. This framework does not add ARVO's MERGED_AWAITING_RELEASE to its lifecycle enum. Read the release evidence rather than inferring “live” from a merged pull request.

When does the owner need to step in?

Product, business, high-risk authorization and access blockers require the relevant human answer. A parked identity does not need to stop unrelated authorized work. A configured independent verifier can take over a deployed boundary while the coding worker moves on. Without that verifier configuration, do not assume the wait releases the execution boundary.

The checkpoint also supports a deliberate draft-PR handoff, an evidence-only wait and no actionable work. “The run can end” is not the same as “every problem is cured.”

terminal
npm run northstar -- checkpoint --pack packs/my-app
npm run northstar -- status --pack packs/my-app

See the complete CLI reference for intake, evidence recording, relations, verification and owner decisions.

Small rules worth auditing next

Keep these as review candidates: make linked work stop appearing as an independent repair only when its authority is proven; keep every wait's return condition measurable; compare headline counts with the canonical problem view. Prove changes with projector/selector invariants and regression tests before reconciling raw statuses. No bulk closure to make a number look better.

This page documents the current Factory implementation. It does not claim that ARVO's campaign outcomes, backlog counts or every proposed duplicate-inheritance rule have been ported.

Something unclear?Talk to the builder