Does following a trail make the record more useful?
In short. Both agent readers answered all eight retrieval questions correctly under the pilot's rubric. The index offers direct routes through a correction and an open investigation, but this first comparison shows no accuracy gain and cannot establish faster reading or better research. The recommendation is to revise and test with an unfamiliar human before expanding.
Version 0.1, 10 November 2026. OpenAI Codex pilot author, exact model identifier and inference cost unavailable. Unreviewed method evaluation, not a released scientific finding. Input: 8af0cb026037a8ee63456900978b3c44473c49cd. Observation window: this session on 10 November. Instruments: offline Python extractor, local Eleventy 3.1.6 build, frozen git records and two fresh agent contexts. Overlap: all incident evidence is reused, not a new replication. Protocol, baseline, assisted, technical audit, item scoring, computed checks.
Answer
Revise. The prototype makes evidence relationships inspectable without replacing the original record. It has not earned expansion into a research landscape, standing collector or claimed improvement in scientific quality. Its immediate value is navigational and documentary; its benefit to unfamiliar human readers remains unmeasured.
Findings
| Fixed question | Baseline | Assisted | Author adjudication |
|---|---|---|---|
| Source and limit for first claim | 1 | 1 | Both locate attribution and reject independent incident confirmation |
| Source and limit for sixth claim | 1 | 1 | Both interpret this as original-table ordinal 6, R06 |
| Australia access history | 1 | 1 | Both preserve original failure and later verification |
| Dated correction | 1 | 1 | Different valid examples, with qualifications |
| Model report and public signals | 1 | 1 | Both separate provider account, scores, fixes and exploitation |
| Private report date | 1 | 1 | Both locate the explicit unknown |
| Uptake and caused benefit | 1 | 1 | Both identify missing outcome evidence |
| Cross-thread connection | 1 | 1 | Both keep it thematic, not intellectual influence |
| Total | 8 / 8 | 8 / 8 | No accuracy advantage in this feasibility comparison |
Scores are the author's unblinded judgement against the frozen passages, not independent scientific adjudication. No answer contained a material false influence, event or benefit claim under this rubric. The assisted reader examined one retained provider body beyond the index; the graph did not supply every answer by itself. Both conditions could use the original records. No new result changed the choice of an actual investigation.
The baseline reports 23 seconds for inspection and 89 seconds through writing its report. The assisted reports 189.034 seconds after orientation through retrieval and writing. Different intervals, contexts and reading strategies were measured; these do not support a timing ratio, speedup or slowdown claim. Baseline reports four inspection tool calls; assisted reports eleven calls containing fifteen command invocations, including an unavailable parser attempt. These are harness actions, not comparable effort units. Construction, review and repair cost considerably more work than one retrieval attempt; this pilot provides no amortisation evidence.
The baseline inherited the parent session's early context. The assisted context started from its task alone. The corpus was chosen because its correction and access gap were already known. The questions are easy for an agent with broad file-reading tools, and both conditions reached the scoring ceiling. There is no balanced crossover, human comparison, held-out case or inferential statistical test. No model-agreement result is independent confirmation of incidents.
Extraction and audit
The position rule selected 17 rows from 87 current claim rows. The earlier source check covered 83. Twelve external records were selected from panels under the retained amendment. They are panel membership, not verified readings. The model-discovery report is contextual material outside that twelve-record set. No DOI matching, co-citation analysis, main-path inference or systematic external bibliography was attempted; no such success rate exists.
Schema 0.2 exports 73 nodes, 57 edges and two separate candidates. Counts describe index entries, not discoveries. Seventeen historical review activities preserve earlier verdicts, including an unreadable-source verdict. Separate Australia failure and verification activities avoid rewriting access history. Each sampled row retains whether its primary full text is indexed: the original full source pack is outside this repository, so the export offers the historical review row without claiming that every underlying passage is available.
The preselected audit covered original ordinals 1, 11, 21, 31 and 41. All five attribution edges matched the frozen rows and URL definitions. The technical reviewer found no false predicate or endpoint among the six trail edges and thematic candidate, but found material omissions and a confusing visual: reading order had been used to place relations that actually point backwards. This was repaired by grouping connections under their actual subject and naming the target. The small directed drawing now shows the source citation, review challenge and correction pointing to their proper accounts.
The review also found overly short evidence anchors, omitted dates/actors and access history, and a missing pilot-index route. Full bounded excerpts, date-basis passages, source actors, review/access activities and the index were added. The final bounded recheck confirmed the main repairs and left three precise author fixes: point the provider card at the full question paragraph, fill AISI's retained 4 August publication date, and use an attempted-read clock for the failed Australia reading. Those are now resolved by the author; no further review chain is claimed.
This is not complete provenance. External panel entries retain null dates where the pilot has not checked a source/read. Some source publication dates are undated or month-level and remain null with qualifications. Full-text availability and licensing are not inferred from a URL. No external pages were downloaded, copied or newly requested. The extraction reads specified frozen files only, retains their SHA-256 hashes, separates inferred/unresolved candidates, and reproduces offline. A string-occurrence check is structural verification, not semantic validation of support.
Reader and implementation checks
The assisted agent found the source and caveat in both journeys using the plain trail labels. This is an AI text-navigation proxy. It reported the ambiguous direction, shared claim-row target and mixed date labels; these prompted the repairs. It also correctly observed that the provider card's link initially supported only its date, which has been repaired. No unfamiliar human or screen-reader user participated.
The page uses the existing fonts, colours and paper/dark tokens. Native anchors and disclosures carry the navigation; the figure and evidence remain available without JavaScript. The diagram has a title, description, meaningful alternative text and separately inspectable, written relations. It has direct source-record links when opened, and an HTML equivalent below. No animation requires reduced-motion handling. The page has an explicitly unreviewed Map signpost and enters no release feed; released prose and the three-investigation throttle are unchanged.
The full site tests, graph checks and local-link/fragment checks are retained in check-results.json. The source audit checks both endpoints, sample membership, source passages and frozen hashes. Static HTML proofs were rendered at 390px and 1000px in explicit light/dark palettes, then inspected. The print renderer does not resolve custom properties inside SVG images, so the proof script substitutes the same explicit palette for that drawing; the site SVG keeps its native theme rules. They are print-engine proofs with the mobile rules applied explicitly, not browser emulation. Local Chromium failed to launch because socket creation was unavailable. The cloud browser could not reach localhost and its observation policy then blocked access; that path was abandoned. Actual mobile-browser overflow, keyboard interaction, theme behaviour and assistive-technology acceptance remain unverified. Browser failure is not a passed acceptance check.
What was learned and what next
One permanent record can support several reader views without turning navigation into a new scientific claim. The most useful distinctions were not topological: source versus event, citation versus support, historical access failure versus later verification, and reported reuse versus measured improvement. The graph helped expose where the old source pack was not actually available. The ordinary repository workflow already answered the fixed questions well.
Compare three next options. A broader research landscape would grow coverage before benefit is established. Another outward investigation would consume capacity the keeper asked this pilot to preserve. Repairing this interface's evaluation is the defensible choice: use an actual browser, then an unfamiliar human with the two fixed journeys and no coaching, recording wrong turns, source/caveat identification and mistaken interpretations. No new schedule or programme commitment is made here.
If that succeeds and the normal reviews approve release, trial a small claim/revision block on one existing investigation. Add an optional prior-evidence field to existing research-scout leads, not a fourth workflow. Track maintenance time, unsupported suggestions and an actual investigation choice changed by earlier evidence. For external scholarship, predeclare a small antecedent-paper set using the proposal's existing reviews and indexes as leads; inspect citation contexts and retain disagreements before drawing influence or outcome edges. A living history should grow by well-supported threads, with missing links visible, rather than by node count.
Corrections and cost
10 November: protocol amendment 1 corrects a URL-location assumption before external extraction. The baseline manifest retains the original rule and identifies the active amended rule. The sampled-ordinal wording was ambiguous; both readers used original-table ordinal 6. Any later comparison should state R06 explicitly. The final interface was repaired after the assisted attempt, so that reader's timing and observations describe the earlier prototype, not a retest of the final version.
No current scientific account changed. The record, export, code and navigation prototype changed. No new independent incident evidence, measured human benefit or improved investigation decision was established. No direct paid API use; inference dollar spend, package/network overhead and total session compute were not exposed, so cost remains incomplete rather than zero.
From notes/knowledge-lineage/evaluation.md in the repository, last changed 10 November 2026.