DEVELOPMENT ROADMAP
Building a research system that can show its work.
Project Loom is being built in public.
Loom investigates what happens between formal decisions and their implementation: institutions, dependencies, capacity, information flows, and the conditions under which implementation succeeds, stalls, reverses, or produces unintended consequences.
But there is a second problem Loom must solve before it can credibly study those failures:
How do we know that the research record itself is reliable?
The roadmap therefore builds the research infrastructure before attempting to automate the research.
Current Status
Several components of the methodology and research architecture have been specified. The first working component of the development capture layer is now operational.
Working now
- Loom research ontology and conceptual framework
- Claims, epistemic status, verification, and promotion specification
- Evidence and provenance requirements
- Flint Water Crisis case reconstruction as the first implementation test
- Local Loom Capture API MVP
- File-based capture of development discussions, decisions, drafts, reviews, and failures
- SHA-256 recording for captured text and artifacts
Under development
- Machine-readable research schemas
- Validation engine
- Adversarial test suite
- Human audit governance
- Real-source Flint reconstruction
- Correction and demotion workflow
- Automated provenance
- Research/process API
Not yet claimed
Loom does not currently claim that its research corpus is fully operational, independently audited, or automatically verified.
Those claims can only be made after the corresponding capabilities have actually been demonstrated.
The Development Principle
Specification → implementation → artifact → adversarial test → public description
A specification describes what a component is supposed to do. An implementation makes it real. An artifact demonstrates its behavior. Adversarial testing attempts to break it. Only then does public documentation describe the capability as operational.
This prevents the website from becoming a description of software that does not yet exist.
Phase 0 — Freeze the Epistemic Baseline
In progressPurpose: Establish the rules against which the system will be tested.
Work
Freeze the current claims/verification/promotion specification; incorporate errata without silently rewriting history; establish a versioned specification manifest; record unresolved questions; define machine-rejectable states; separate epistemic violations from procedural exceptions.
A specification must be stable enough to test before implementation begins.
Phase 1 — Establish the Living Research Record
Partially operationalPurpose: Preserve how Loom itself develops.
Work
Distinguish discussion, proposal, implementation, testing, failure, correction, and canonicalization. Git records exact changes; the Development Diary records why decisions occurred; the Capture Layer preserves development material before classification; the Research Corpus contains canonical research objects; the Public Site presents the resulting work.
Capture first. Classify later. A captured item does not automatically become a claim, evidence, or canonical record.
Phase 2 — Define the Machine-Readable Research Objects
Specification underwayPurpose: Give the research system explicit objects rather than relying on prose.
Work
Research Request; Research Protocol; Source; Evidence; Observation; Claim; Verification; Review; Diagnostic; Artifact; Correction; Provenance Event; Actor.
Evidence does not automatically become a Claim. A Claim does not automatically become Supported. Supported does not automatically become Canonical.
Phase 3 — Build the Validator
PlannedPurpose: Make invalid research states machine-detectable.
Work
Reject unretrieved sources as evidence; constrain claims to their evidence; constrain epistemic status when contradictions remain unresolved; require promotion events for canonical claims; prevent AI extraction from independently establishing verification; require documented handling of deprecated claims; prevent observations from bypassing the claim pathway; prevent fabricated reviewers or governance events from establishing human verification.
Example: CLAIM-006-004 / PL307-EPI-014 / Critical — claim marked “known” while contradiction_status = unresolved. Remediation: resolve the contradiction or lower epistemic status.
Phase 4 — Adversarial Testing
PlannedPurpose: Try to make Loom fail.
Work
Fabricated or unretrieved sources; derivative sources counted as independent; improperly promoted AI extraction; claims broader than evidence; unresolved contradictions; canonical claims without promotion events; diagnostics resting on unresolved claims; missing actors; phantom evidence; stale specification references; incorrect independence; synthetic reviewers; plausible but invalid attestations.
The system must distinguish a document describing a human review from an actual human review event.
Phase 5 — Establish Human Audit Governance
Not yet establishedPurpose: Introduce genuine human review into the research process.
Work
Define auditor roles, actor identities, conflicts of interest, independence, review boundaries, reviewer responsibilities, promotion authority, correction authority, and audit records.
Human participation must be represented by actual governance events rather than generated text.
Phase 6 — Reconstruct the Flint Water Crisis
In progressPurpose: Use one real case to test the entire research pathway.
Work
Research Request → Research Protocol → Sources → Retrieved Documents → Evidence → Observations → Claims → Verification → Human Review → Promotion → Published Artifact. The first source must be genuinely retrieved; document identity established; hash computed from actual bytes; evidence linked to an actual passage and location; claims constrained to evidence; provenance reconstructable.
The first real source retrieval has been located. Byte-level hashing and human confirmation remain outstanding.
Phase 7 — Four-Person Adversarial Audit
PlannedPurpose: Attempt to reproduce and break the Flint research record.
Work
Examine source provenance, retrieval, evidence locations, claim scope, contradictions, source independence, verification levels, actor identity, promotion events, correction pathways, specification version, and reproducibility.
The objective is not simply to approve the case. It is to determine whether another person can reconstruct how the conclusion was reached.
Phase 8 — Demonstrate Correction and Demotion
PlannedPurpose: Test whether Loom can correct itself after promotion.
Work
Original state → Challenge → Evidence → Reassessment → Correction / Demotion → New state.
The original event is not silently erased.
Phase 9 — Publish the First Genuine Loom Research Artifact
PlannedPurpose: Demonstrate the complete system using real research.
Work
Research question; protocol; sources; retrieved documents; evidence; claims; verification; contradictions; diagnostics; human audit; limitations; corrections; specification version; provenance record.
The artifact will not describe AI-generated material as independently verified research.
Phase 10 — Publish “How Loom Research Works”
PlannedPurpose: Explain the actual operating system to the public.
Work
Explain what Loom captures; source intake; evidence extraction; claim formation; verification; AI limits; human participation; contradictions; promotion; corrections. Distinguish Implemented, Under development, and Planned.
This page will be written after implementation, not before.
Phase 11 — Build the Loom Process & Provenance API
PlannedPurpose: Turn the demonstrated file-based research process into a programmatic interface.
Work
Potential operations: POST /event; GET /request/:id; GET /source/:id; GET /claim/:id/provenance; GET /artifact/:id/version; GET /correction/:id.
The API should expose the research process rather than merely expose AI-generated answers.
Phase 12 — Controlled Automation
PlannedPurpose: Reduce repetitive research work without transferring epistemic authority to AI.
Work
Source discovery; document retrieval; duplicate and near-duplicate detection; evidence extraction; contradiction discovery; scope checks; provenance generation; validation; review queues; correction monitoring.
AI may propose. Evidence must establish. Humans may promote. Automation increases throughput; it does not change the evidentiary standard.
Phase 13 — Public Research Infrastructure
PlannedPurpose: Make Loom’s research records durable and independently inspectable.
Work
Persistent artifact identifiers; citable research records; public provenance; archival copies; machine-readable datasets; public APIs; correction histories; reproducible case records.
These are capabilities to be demonstrated, not assumed.
Phase 14 — External Research & Institutional Use
FuturePurpose: Test the system with people who did not build it.
Work
External researchers; journalists; civic organizations; other institutions.
The goal is not simply adoption. It is determining whether people who did not build the system can understand, reproduce, challenge, and audit its research records.
The Roadmap Is Also a Record
This roadmap is not a declaration that all of these capabilities already exist. It is a development record.
A phase does not become operational because its documentation says it is operational. It becomes operational when the corresponding implementation and evidence exist.
Where Loom Is Now
The project has moved from designing the idea of a research system toward building the infrastructure that can test it.
The first working capture component now exists.
The next credibility increment is not another document claiming completion. It is a real, inspectable research record:
one source → one retrieval → one hash → one verified passage → one claim → one human confirmation → one reconstructable provenance chain.
That is the unit by which Loom will measure its progress.
Development Rule
Don’t add capability until the record of how that capability is supposed to behave exists.
Don’t call it working until the validator and auditors can demonstrate that it behaves that way.
Don’t call research verified until the evidence can be inspected.
Don’t call a governance event human until a human actually performed it.