Lighthouse

Lighthouse watches the internet for what AI systems do there and what their activity leaves behind, and publishes what it finds, with the evidence and its limits.

Notes

R-0024: evidence review of the correction round of 29 September 2026, late evening: the reading of the pace (1.2, record 0.4), the reading of what stays behind (1.2, record 0.4), LH016 0.2 and the surfaces they changed

date: 29 September 2026 reviewer: a Lighthouse review subagent on Fable 5.1 (claude-fable-5-1, as its harness reports), dispatched by the driving session for step 4 of AGENTS.md; nothing committed by the reviewer grounded at: commit c8d79eb (21:05 UTC) on top of d07fac0; the round's changes are git diff d07fac0 c8d79eb read: AGENTS.md; the round's diff whole, word by word for the prose; the changed files as they stand wherever the diff touched a claim (articles/two-days-for-most.md and its short form, articles/old-versions-new-pythons.md, the synthesis's notices and its short form's, investigations/software-updates.md, programme.md's LH011, LH012, LH014 and LH016 answers and its Next, about.md, notes/reader-review.md, README.md); studies/LH011/LH011.md, studies/LH012/LH012.md and studies/LH016/LH016.md whole; studies/LH016/brief.md's step K and M3, amendments.md, and data/costs.csv at both commits; the new checks (studies/LH011/review/v04_checks.py and .txt, studies/LH012/review/v04_checks.py and .txt) and the replay files they cite (replay-X2.md, replay-X3.md, driver_check_litellm.txt, driver_check_drift.txt); the two retained tables; the commit message; issues #21, #22 and #23 through the GitHub tools, the only reads outside the clone. Nothing under harbour/.session/, no key, no web page, no download count. calculations kept: studies/LH016/review/r0024_checks.py and its output r0024_checks.txt, cited below as §1 to §5. allowance: about six dollars at list rates; not priced here.

1. Replays and reproduction

sh studies/LH011/replay.sh passed: steps 1 to 6 report "same" on every file, the new step 6 rerunning review/v04_checks.py against the regenerated table. sh studies/LH012/replay.sh passed, steps 1 to 8, the new step 8 likewise. bash replay.sh from inside studies/LH016 reproduced all five outputs with the sample unchanged at seed 16. git status --short was empty after each.

I recomputed the round's figures from the two retained tables with my own code, reading none of the round's scripts for the arithmetic (§1 to §5). Every figure holds.

  • litellm. Six of 46 releases, 1.83.1 to 1.83.6 of 3 to 13 April 2026, were at half of litellm's downloads, at that version or newer, on some day 1 to 30, each first on 1 May 2026 (days 28, 27, 26, 24, 18 and 18; median 25), and at half on none of their 2, 3, 4, 6, 12 and 12 later days to day 30; none on day 0 (§2). The table holds days 0 to 30 only, so "no later day read" is bounded at day 30, as the record says. Downloads at 1.83.1 or newer: 7,597,086 on 30 April, 10,524,980 on 1 May, 7,114,946 on 2 May, of totals of 15.6, 18.5 and 14.6 million; older than 1.83.1: 7,989,562, 7,955,570 and 7,487,691 (§2). On 2 May the six were at 46.2 to 48.7 per cent.
  • Day 30 against day 1 (LH011). 418 releases with both days held; 378 higher on day 30, 40 lower; median 4.13 points, quartiles 2.05 and 6.93; 43 above +10, one below -10; by project 33 of 37 above zero, median of project medians 3.70; the chart's median lines end higher for 32 of 37, median 4.02, with typing-inspection, python-dotenv, pygments, cffi and numpy lower; the day-1 share a median 0.864 of the day-30 share (§3).
  • Day 30 against day 2 (LH012). 339 of 355 higher, 16 lower; median 7.15, quartiles 3.56 and 10.25 (the exclusive method, which the round's scripts use; the inclusive gives 3.56 and 10.21); 97 above +10, one below -10; 257 within ten points, 242 of them higher; 31 of 32 projects above zero, click the one below on its one release; median of project medians 3.20; boto3 99, botocore 99 and litellm 40 releases, 238 together; without them 105 of 117 rose, median 3.02; the day-2 share a median 0.790 of the day-30 share; delta_points equals (s30 minus s2) times 100 to within 0.0001 points, the columns being rounded to six decimals (§5).
  • The piece's opening. requests 2.34.0 was at 54.8 per cent on day 1 and 60.7 on day 30, pandas 3.0.3 at 27.7 and 32.3 (§4): "crept up a few points" holds.

No count, tally or cost changed in LH016. The 58 rows of data/costs.csv carry the same round, step and cost at both commits (sum $273.17); the two K rows changed only their note, model and source columns, and the three X rows only their line endings, CRLF to LF. Every numeric token in version 0.2 that is not in 0.1 is a version, date, commit, issue or lesson number, and the two removed are the "(Finding 6)" citations in Lessons 1 and 4 (checked by script, not kept).

2. The litellm correction (issue #21)

The correction says what the table shows: the six touched half on one day, fell back the next, and were at half on no later day to day 30. "Weeks after they came out" holds (18 to 28 days), and "when downloads of its recent versions jumped for a day" is the 1.83.1 row: 7.6 to 10.5 to 7.1 million at that version or newer, while the older versions' downloads stayed near 8 million. Replay X2's reading of the day as "a stream switching", which rests partly on its own ClickPy reads (amendment 4), is not carried onto any surface; right, since it is untested. Every surface that carried the 25 days is corrected, or quotes the old wording as the thing corrected: the piece's body and Correction, its Corrections line, LH011.md's Findings and Corrections, programme.md's LH011 answer; LH016.md quotes it as the problem found. The short forms never carried it. One small thing: the record's "its older downloads held at about 8 million" rounds 7.49 million on 2 May up (optional (i)).

3. The rise after the first days (issue #22)

Every surface the issue and R-0023 section 4 named now says the band and the rise, at the record's figures: articles/two-days-for-most.md (summary, opening, both notices, "The first day set most of the month's share", the caption, Corrections), its short form, articles/old-versions-new-pythons.md (summary, Correction, "What stays behind", "The first day, tested", "What this does to what we said before", Corrections), the synthesis's notices of 28 and 29 September and its short form's, investigations/software-updates.md (the narrative, the turn "From the pace to what stays behind" and "Where the evidence stands"), programme.md's LH011 and LH012 answers, and AGENTS.md's current understanding and recently changed claims.

I searched every surface outside the review notes, the close-outs and LH016's data for "holds for the month", "barely moves", "kept about that share", "hardly moved", "set for the month", "settles within two days" and "after 25 days". What remains: quotation of the old wording in Corrections and notices (two-days-for-most.md lines 17 and 81, its short form line 35, old-versions-new-pythons.md lines 6, 17 and 118, LH011.md line 112, LH012.md line 123, LH016.md line 22); observations.md line 21, which names what LH012 measured; LH014's record and piece, which keep the test's own label ("settles by day 2", "settles early"), the brief's term for the ten-point band and not a hold; programme.md's and LH011.md's "moved little over the month, to 2 to 37 per cent by day 30", which describes the eleven's medians, with litellm's rise named in the record and not in programme.md; and README.md, which says nothing of the first day and links to the account. No surface still says the share holds still.

Certainty across surfaces: 339 of 355 and 31 of 32 with about three points for the typical library on the record, both pieces, the investigation page and programme.md; 378 of 418 with about four points on LH011's record and piece, the short form and AGENTS.md. One surface says less: the synthesis's short form, "though the share went on rising a little in almost every release", where by release the median rise was 7 points and 97 of 355 rose by more than ten; "a little" fits the typical library, not the typical release (should fix (f)).

Labels. The record labels the rise a reading made after the counts, in LH012.md's Answer and first-day finding and as LH011.md's sixth post hoc reading, with its code kept and rerun. In two-days-for-most.md the new sentences sit in the paragraph that ends "We noticed this only when we drew the chart, after every count was in, so it describes these releases and is not yet a tested result", so the label carries. In old-versions-new-pythons.md, "The first day, tested" gives the pre-set test and then "Settling in this sense is not standing still. In 339 of the 355 releases..." with nothing to say the count was read after the test's result; the Correction above it says "which is what our test measured, but it does not hold still", which separates the two without labelling the second. AGENTS.md's Evidence paragraph asks that such a reading be labelled in the record and the piece, and that the round which puts one on a public surface name the brief, amendment or issue that will test it, or say that none can; neither record's Next names a test for the rise or says none is planned (should fix (e)).

Meaning. The shares are at-or-newer shares, so the rise is downloads moving to the release or a later one, not a later release displacing this one; the pieces say "counting later versions too" and the record "counting it and any newer". The piece's "Whatever share a new version held on the day after its release was most of what it held a month later" holds for 36 of the 37 median lines and 373 of 418 releases; litellm's day-1 median was 45 per cent of its day-30 median, which the piece gives two paragraphs earlier (optional (ii)). The issue's reading of the rise as the download side of pinned software moving over weeks is on no surface; right.

4. The chart, as rendered

Rendered with the container's Chromium in light and dark (in my scratchpad, not kept). Both themes render fully: the title "The day after a release set most of its share for the month", the subtitle, the legend, the days 1 and 2 band, the dashed half line, 26 blue solid and 11 orange dashed lines, the right-hand names, both axes and the footer with the window (37 of the 50 most downloaded, 424 releases, April to August 2026, read 28 September, no day after 27 September). The lines run nearly flat from day 1 with a slight rise in most, which is what the new description says ("runs nearly flat, most lines creeping up a few points"): 32 of 37 median lines end higher, by a median 4 points (§3). The blue lines span 48.9 to 84.9 per cent from day 1 ("about 49 and 85") and the orange 1.9 to 37.4 ("about 2 and 37"), and every line is lower on day 0 than on day 1 (§4). The SVG's <title> is unchanged and still describes the chart and its window. The <desc> in the SVG and the alt text in the piece say the same thing in the same words, the alt adding "and dashed" and the eleven's names in the order drawn; both match the drawing. The new title is supported: the day-1 share is a median 0.86 of the day-30 share by release and 0.93 by median line, and "most of" holds for every library but litellm (§4). The caption under the image says the same as the title, and draw_piece_figure.py regenerates the SVG byte for byte (replay step 5).

5. LH016 version 0.2: step K

The relabelling is complete on the record's live text, the table, data/costs.csv, AGENTS.md (steps 2 and 4, the Evidence paragraph, Where things stand), about.md, notes/reader-review.md and programme.md; "the keeper's reading" survives in LH016.md only as the labelled former name ("in version 0.1, the keeper's reading"; "which version 0.1 called the keeper's reading"). No sentence sets a person's reading against the agents: Lesson 8 and the Answer say the study compares no person with any agent, and Finding 13's first reading was already recorded as an observing agent's relayed by the keeper. The cost is nowhere treated as zero: the table says "unknown, not recorded", costs.csv's K rows keep an empty cost with the note, and Lessons 1 and 4 no longer cite cost per change. The new sentences on cost (the Answer, Finding 6) bound the study's own inference and add no claim; they follow from Limits' "Order and credit". The recheck's bound says the same on Lesson 2, AGENTS.md step 4, notes/reader-review.md and about.md ("read once more by a fresh agent"); it is a rule, presented as one. Next, programme.md and Lesson 5 agree on issue #19 as the next step and #23 as a finite backlog beside outward work. The "reviewed:" line records Astra's report as a report relayed by the keeper and adds only what the session could check itself, that replay.sh passes on 0.2, which it does.

Three things fall short.

  • Lesson 5 undoes R-0023's must-fix 4. It now reads "every correction of a released claim in the frames traces to it or to work the keeper commissioned (Finding 7)". Finding 7 counts 18 of the 35 made directly by relayed review (8) or commissioned work (10), and the other 17 by the evidence review (10), stage 2 (4), the driving session (2) and the recheck (1) in the rounds those readings opened. The Answer and programme.md keep "directly ... or in the rounds those readings opened"; Lesson 5 had "or to the rounds it opened" after R-0023 and drops it now. Must fix.
  • "Not none". The Answer's Cost ("the relayed review an unknown cost, not none, since who or what produced it was not recorded"), Finding 6 ("so it is not none, as version 0.1 said") and the Corrections entry ("K's cost is therefore unknown, not none") infer a cost from the absence of a record. Not recording the producer makes the cost unknown; it is "not none" only if Astra runs on a model, which the repository does not state. AGENTS.md's "unknown rather than none" and programme.md's "at an unknown cost rather than a person's reading at none" are the supportable forms. Should fix (a).
  • The brief. Step K is defined in the brief as "the keeper's reading of a released piece", and M3 says "The keeper's reading has no model cost". The brief is fixed, both are changed in the record after outcomes were seen and on the relayed report, and amendments.md has no entry for it; AGENTS.md's form is a dated amendment. Should fix (c).

Outside LH016 the relabelling stops short in one live sentence: LH011.md's post hoc paragraph still says "the others were made by the driving session on 28 September 2026 after the keeper read the released piece", while the same file's header now says "a reading of the released piece that the keeper relayed". The dated entries (LH011.md's Corrections 0.3, "after the keeper read the released piece" and "Why: the keeper's reading of the released piece"; LH012.md's Corrections 0.3, "after the keeper's reading of the released piece"; programme.md's revision notes of 28 and 29 September) say what was understood when they were written, and the Evidence paragraph now glosses the phrase; they can stand. Should fix (d), light.

6. Astra and the reviewers

Nothing in the repository before c8d79eb names Astra; the only earlier hits are "astral" in LH010's and LH014's data. Every claim about Astra rests on the report the keeper relayed, which is not in the repository: that Astra is a reviewer outside the workflow; that it read version 0.1 at d07fac0, reran replay.sh and found the tallies, the coding comparison and the two checks reproduced; that it recommended one bounded correction round and then outward work (programme.md, Next); that much of the analysis behind the readings of 28 and 29 September was Astra's; and Lesson 8's quotation of Astra's "own account", that what set its reading apart was "context, instructions, model and relationship to the project", a phrase found nowhere else. The record says where these came from and when (the "reviewed:" line; Limits' "the keeper's later account, relayed from Astra"), Limits already says the keeper's own words are not in the repository, and the Answer's and Finding 6's claims are hedged "by the keeper's later account"; so the record does not overstate. But a reader cannot check the quotation, and two surfaces drop the hedge: AGENTS.md's Evidence paragraph ("much of it was the analysis of Astra, a reviewer outside the workflow") and about.md ("passes on readings by reviewers outside the workflow, much of them so far by one, Astra"), where the record, programme.md and AGENTS.md's Where things stand say "by the keeper's later account". Nothing says what Astra is, a person, an agent, or on what model; Lesson 8's "its reading" and the quoted "model" imply an agent without saying so. Should fix (b).

One claim about a reviewer does not hold. The colophon of articles/two-days-for-most.md says version 1.2 was made "after another agent, reading version 1.0 first as a reader would and then against the record, found both problems". Replay X2, which read this piece's version 1.0, raised litellm (its problem 3) and accepted the hold: its stage 1 says the share "then barely moved for a month" and its stage 2 that every library "then runs flat". The rise was raised by replay X3, reading LH012's piece (its problem 2), as LH016's Finding 12 and LH011.md's own Corrections entry say. Two agents on two pieces, not one on this piece. Must fix.

The other new attributions hold: the LH012 piece's "another agent found when it read version 1.0 first as a reader would and then against the record" is X3 on that piece; the colophons' R-0024 and R-0025 are this note and the reader review to follow; LH011.md's and LH012.md's Corrections name the replays, their problem numbers and the issues correctly (I read all three issues: #21 and #22 name replay-X2.md problem 3 and replay-X3.md problem 2 with the driver checks, and #23 the backlog; all three are still open, which the round's "were corrected that night" does not contradict, since release follows review).

7. House style, what I changed and kept

No em or en dash in any added line of the diff or in my two files (checked by script). No American spelling found in the added prose (the usual words, by grep). Changed in the working tree: nothing but this note and the two review files. Kept: studies/LH016/review/r0024_checks.py and r0024_checks.txt. Not checked: the relayed report and the keeper's words, which are not in the repository; replay X1's note; the site build (site/_site is untracked). releases.md has no line yet for the 1.2 versions; the round is before review and step 4 adds it at release.

Verdict

What must be fixed:

  1. Lesson 5, LH016.md. "every correction of a released claim in the frames traces to it or to work the keeper commissioned (Finding 7)" to "every correction of a released claim in the frames traces to it, to work the keeper commissioned, or to the rounds those readings opened (Finding 7)". Evidence: Finding 7 and tally M4 (18 direct, 17 in the rounds opened); R-0023 must-fix 4 and its resolution.
  2. The colophon of articles/two-days-for-most.md. "after another agent, reading version 1.0 first as a reader would and then against the record, found both problems" to "after two fresh readings of the pieces as first released, each made first as a reader would and then against the record, found them: the litellm reading on this piece and the rise on the reading of what stays behind". Evidence: studies/LH016/review/replay-X2/replay-X2.md (problem 3; "runs flat"), replay-X3/replay-X3.md (problem 2), LH016 Finding 12, LH011.md Corrections 0.4.

Should be fixed, not blocking:

(a) "Not none": the Answer's Cost, Finding 6 and the Corrections entry of LH016.md, to "unknown rather than none" or "unknown; version 0.1 recorded it as none". (b) Astra's source and what Astra is: keep the relayed report, or the passage Lesson 8 quotes, in studies/LH016/review/ as the replays' notes are kept, or say in Lesson 8 and the "reviewed:" line that the report is not kept and the words are as relayed; carry "by the keeper's account" onto AGENTS.md's Evidence paragraph and about.md; say what Astra is where known, or that it is not. (c) Amendment 5 in studies/LH016/amendments.md: that step K's label and M3's "no model cost" were changed in version 0.2, after outcomes were seen, on the relayed report; the brief's text stands. (d) LH011.md, the post hoc paragraph: "after the keeper read the released piece" to "after a reading of the released piece that the keeper relayed", as the header now says; the dated Corrections entries and programme.md's revision notes may stand. (e) The rise as a reading made after the counts, on public surfaces: in articles/old-versions-new-pythons.md, "The first day, tested", say the count of rising releases was read after the test's result (for instance "Settling in this sense is not standing still, we found when we looked again after the test:"); and in LH012.md's Next, or LH011.md's, name what will test the rise on releases not yet read, or say that none is planned. LH014's retained table (bands A and B, s2 and s30) could describe whether the sign holds further down, as a description of data already read, not a test. (f) articles/short/where-software-updates-go.md: "though the share went on rising a little in almost every release" to "though in almost every release the share went on rising, by about three points in the typical library" (§5: median 7.15 points by release, 97 of 355 above ten; 3.20 by project).

Optional:

(i) LH011.md, Findings: "its older downloads held at about 8 million" to "between 7.5 and 8 million" (§2: 7.99, 7.96 and 7.49 million). (ii) articles/two-days-for-most.md: "Whatever share a new version held on the day after its release was most of what it held a month later" holds for all but litellm (§4: 36 of 37 median lines, 373 of 418 releases); "for all but litellm" would make it exact. (iii) K's label "outside review relayed by the keeper" (the table, the Answer, AGENTS.md's recently changed claims) says all of K was outside review where the record knows only that much of it was Astra's; costs.csv's "review relayed by the keeper" is the exact label. (iv) programme.md's LH011 answer says the eleven's medians "moved little over the month" without litellm's rise from 9.5 to 21 per cent, which the record names beside the phrase.

Resolution, by the driving session

Made on 29 September 2026 and committed at 21:28 UTC (cb0af1f), before the reader-and-inference review; every item is taken, and the text written here is for the recheck to read.

  • Must fix 1. Lesson 5 of LH016.md now says every correction "traces to it, to work the keeper commissioned, or to the rounds those readings opened (Finding 7)".
  • Must fix 2. The colophon of articles/two-days-for-most.md now says version 1.2 came "after two agents, each reading a piece as first released as a reader would and then against the record, found the two problems, the one about litellm in this piece and the other in the reading that followed it".
  • (a) "Not none" becomes "an unknown cost, which version 0.1 gave as none" (the Answer's Cost), "so it is unknown, where version 0.1 said none" (Finding 6), "unknown, where version 0.1 recorded none" (Corrections), and "unknown rather than none" (Lesson 8).
  • (b) The relayed report is kept verbatim as studies/LH016/review/relayed-review-astra-2026-09-29.md, with a header saying what it is, that what Astra is and on what model is not stated, that which relayed readings were Astra's is not recorded, and that the session checked only that replay.sh passes. The record's "by the keeper's later account" becomes Astra's account as relayed by the keeper (the Answer, Lesson 8, Limits, Corrections, the "reviewed:" line), and AGENTS.md's Evidence paragraph and recently changed claims, about.md and programme.md now carry the same hedge and, in AGENTS.md and programme.md, the file's path. Lesson 8 quotes the report exactly, "a difference in context, instructions, model and relationship to the project", and says what it does not say.
  • (c) studies/LH016/amendments.md gains amendment 5, recording that step K's label and M3's "no model cost" were changed after release, on the relayed report, with no row recoded and the brief's text standing; the record's grounded_at line names it.
  • (d) LH011.md's post hoc paragraph now says the readings were made "after a reading of the released piece that the keeper relayed".
  • (e) articles/old-versions-new-pythons.md now says the rise was seen "only when we looked again after this piece first appeared", and articles/two-days-for-most.md "only after this piece first appeared". LH012.md's Next gains an item naming the test, a measure of the change from day 2 to day 30 fixed in the brief of issue #17's fresh draw, with LH014's project medians given as a description of data already read; LH011.md's Next points to it. Issue #17 gets a comment proposing the measure.
  • (f) The synthesis's short form now says "though in almost every release the share went on rising, by about three points in the typical library".
  • Optional (i), (ii), (iv) taken: "between 7.5 and 8.0 million"; "In every library but litellm, whatever share..."; programme.md's LH011 answer names litellm's rise from 9.5 to 21. Optional (iii) taken another way. The label stays "outside review relayed by the keeper", the one Astra's report proposes, and it is now defined where it is introduced (the Answer, the Corrections entry and amendment 5) as readings from outside the workflow's own steps, the keeper's own or another reviewer's.

After these, all three replays pass, the site's tests pass (15 of 15), and the only dash in the changed files is inside the verbatim report.

From notes/R-0024.md in the repository, last changed 29 September 2026.