Lighthouse

Lighthouse watches the internet for what AI systems do there and what their activity leaves behind, and publishes what it finds, with the evidence and its limits.

Notes

Review of the revision of 28 September 2026: the piece "New releases arrive quickly. Older versions keep getting downloaded." at version 1.1 with its short form and figure, the LH011 record at version 0.3 with its readings made after the counts, the consistency surfaces, and the rules added to AGENTS.md and CLAUDE.md

kind
review
version
0.1
date
2026-09-28
authors
the release reviewer, a Fable 5.1 subagent (claude-fable-5-1, as reported by its harness), dispatched by the driver session
grounded at
6cb9051 (the revision as committed on claude/fervent-mccarthy-mjyebc, "before review"), on top of a36508d (piece version 1.0, record version 0.2, reviewed in notes/R-0014.md); the reviewer's changes below are in the working tree, uncommitted, for the driver
cites
the diff a36508d..6cb9051; articles/two-days-for-most.md, articles/short/two-days-for-most.md and articles/two-days-for-most.svg; studies/LH011/LH011.md version 0.3, scripts/post_hoc.py, scripts/draw_piece_figure.py, data/post_hoc_reading.txt, data/piece_figure_reading.txt, data/analysis/release_days.csv and project_outcomes.csv, data/clickpy_columns.csv, replay.sh; studies/LH008/LH008.md, data/events.csv and data/lags.csv; AGENTS.md, CLAUDE.md, README.md, programme.md, investigations/software-updates.md, the "Later evidence" notices on articles/where-software-updates-go.md and its short form and, unchanged in this revision, on articles/what-a-download-shows.md and its short form; site/README.md and site/lib/repo.js for the revised header; notes/R-0014.md. No network request was made; no key or credential read; nothing under harbour/.session/ read

Findings

This is a light review of a revision that changes no figure of version 1.0: the piece retitled and reframed after the keeper's reading, its short form, the record's new paragraph of five readings made after the counts, the surfaces that must agree with them, and the rules the round added. The reviewer read the diff and the files it cites, reran the replay, recomputed the five readings with its own code from the retained tables (and LH008's), traced every figure in the revised text to a table, rendered the chart in both themes, and read the new rules and the rewritten checkpoint against the records. Every figure in the record's new paragraph stands. What had to be fixed was small: one figure the piece had rounded twice (97.5 given as 98, where the table holds 97.48), one figure the record had taken from rounded shares (a rise of 3.5 points that is 3.4), two sentences that said more than the record does ("has not spread slowly", "the jump came as quickly"), and two summaries that stated the first-day observation without saying it was noticed after the counts. The verdict is that the revision can be released.

The reviewer's calculation is kept in studies/LH011/review/r0015_check.py and r0015_check.txt: the five readings from data/analysis/release_days.csv and project_outcomes.csv and LH008's data/events.csv and data/lags.csv, with its own median and half-up rounding, without calling scripts/post_hoc.py; it also prints the day-1 share as a fraction of the day-30 share for every project, which the piece's "as quickly" needed.

What was checked against what

Claim Checked against Result
Replay, with its new step 5 sh studies/LH011/replay.sh, before and after the reviewer's changes passed both times: the frame, the ten analysis tables, uptake.svg, the piece's chart, piece_figure_reading.txt and post_hoc_reading.txt regenerate byte for byte; 152 ClickPy queries, 5,848,644,945 rows
Reading 1: 34 of 37 projects' day-30 median within ten points of the day-1 median; the three others r0015_check.py, medians over each project's known releases; day 30 held for all 37 (18 rows of 13,144 are censored after 27 September) 34 of 37; pyjwt 57.6 to 72.3, aiohttp 55.2 to 65.6, litellm 9.5 to 21.0; every project's two medians equal the record's table to one decimal
Reading 2: downloads still at older versions on day 30, 17.8 to 47.1 (median 33.9) for the 26 and 63.0 to 97.5 (median 73.1) for the eleven one minus the day-30 median agrees; the exact ends are 17.79, 47.11, 62.95 and 97.48 per cent, so the piece's whole numbers are 18, 47, 63 and 97, not 98 (fixed, below)
Reading 3: nine of the eleven between 70 and 107 per cent, litellm 45, boto3 61 day-1 median over day-30 median agrees (grpcio-status 70.4 is the lowest of the nine, numpy 107 the highest); for the 26 the same ratio runs from 80 (pyjwt) to 110 (python-dotenv), 24 of them between 70 and 107
Reading 4: the widest gaps between neighbouring median day-1 shares, 16.8 (s3transfer to pydantic, containing half) and 11.1 (grpcio-status to fsspec) project_outcomes.csv's median_day1_share, sorted agrees; the third is 5.5 (aiobotocore to boto3)
Reading 5: LH008's four fixes are LH011 releases; 50 of 102 pinned pairs moved within 30 days of the release; shares rose from 53.8 to 60.8 on day 1 to 57.3 to 72.3 on day 30 events.csv (chosen = yes: urllib3 2.7.0, cryptography 48.0.1, pyjwt 2.13.0, starlette 1.1.0); lags.csv kind = fix, outcome = moved, lag_release_days at most 30 (committer time; author time gives the same counts); release_days.csv for the four releases 13 of 32, 18 of 34, 15 of 29, 4 of 7: 50 of 102, as LH008.md's own Findings already give ("50 of 102, 49 per cent"); shares 60.8 to 66.3, 54.7 to 60.1, 57.6 to 72.3, 53.8 to 57.3; the rises are 5.6, 5.4, 14.7 and 3.4 points, so "3.5 to 14.7" was taken from the rounded shares (fixed, below)
Rounding in the piece and short form: 18, 47, 63, 98; 2 and 34; 2.5 and 37; 70 per cent; 54 to 84 the exact medians above and project_outcomes.csv all right but 98; aiobotocore's day-30 median is 2.52 and s3transfer's 37.05
"For most of the eleven, the jump came as quickly and was smaller" reading 3 beside the same ratio for the 26 not what the record says: six of the eleven fall within the 26's range of 80 to 110, three (grpcio-status, protobuf, aiobotocore) below every one of the 26, and boto3 and litellm well below; reworded to the record's figures (below)
The revised text's other figures: 21 of the 26 every release and five most; ten of the eleven no release within the month; litellm six of 46 at a median of 25 days; 127 of 424; 252; the opening's 55, 28, 61 and 32 project_outcomes.csv (share_reached, releases_not_reaching_half_by_day_30, median_days_to_half_among_reaching) and notes/R-0014.md's checks, since none changed agree
No text reads a low share as slow uptake or states the first-day observation as tested every changed file, and the two unchanged notices on the download-log piece the summaries, the revised notes, the surfaces and the record say "not by itself"; the piece's body said "has not spread slowly" and the short form "not a slow spread" (fixed); the observation is labelled as made after the counts in the piece's body, README, the synthesis's notice, the investigation page ("not yet tested"), AGENTS.md and programme.md, but not in the two summaries (fixed)
The chart as rendered headless Chromium at 720 by 560, light and with preferredColorScheme=0 for dark; both PNGs read title "The day after a release set its share for the month", subtitle, legend, the days 1 and 2 band, the dashed half line, axis labels, the right-hand names and the two footnote lines legible in both themes; blue and orange lines distinguishable on both backgrounds; the title fits the new framing, which is that the first day sets the month. The piece's caption said "for eleven far less"; s3transfer's 34 per cent is not far from half (fixed, below)
House rules grep for non-ASCII in the ten changed files (only the bylines' middle dot); spellings; the pieces' prose before the colophon; the title; "as a rule" and "never" across the pieces and surfaces; the short form's length; the revised header no em or en dash; British spellings; no document name, study number, commit or role before the colophon (the short form's link to the full piece as in version 1.0); the title states the tension and carries no absolute; "as a rule" survives only in the record, whose title is the brief's question, and in programme.md's answer, which quotes the brief's convention as a record-side summary; "never" in the pieces only as "never a run" and in quotation in the revised note; the short form is 256 words without title and byline, against "about 150 to 250"; site/lib/repo.js's parseRevision takes YYYY-MM-DD: text items, so the header renders
Consistency programme.md's LH011 answer and checkpoint, README's "Current focus", investigations/software-updates.md, the synthesis's notice and "What comes next", the short synthesis's notice, AGENTS.md's "Where things stand" all say: 26 of 37, eleven none within the month, the first day's share seen in almost all but only after the counts, a download counts fetches so a low share is not by itself slow, the next question widened to all 37 with three explanations. programme.md's answer still gave one explanation where record 0.3's Limits gives three (fixed, below)
The new rules in AGENTS.md and CLAUDE.md each against the others, the checkpoint and the records coherent: step 4's rendered figures and windowed absolutes match the Writing rule; step 5's "sources that touch no outcome may be read before the brief" is consistent with the brief rule, since metadata is not an outcome; the Evidence rules on counts, selection and post hoc readings match record 0.3 and the checkpoint. The Evidence rule asks that a post hoc reading in the piece carry the number of comparisons; the piece's Methods line now gives it. CLAUDE.md's render command works as written (used here); its fast-forward command fetches, checks that origin/main is an ancestor and pushes HEAD to main without force, which keeps the "never rewrite history" rule; its note on which hosts answered matches R-0014's experience
The rewritten checkpoint its statements against the records and R-0014 nothing asserted that a record does not support: the aiobotocore issues are said to have returned 403 and not to have been read, with their topic attributed to the search results; SPEC 0 and NEP 29 are described only as published support policies and "not opened", and the earlier claim that numpy and pandas follow a three-year window is gone; PyPitfall and LibEvoBench are described as R-0014 read their abstracts; the "same releases read two ways" figures match record 0.3; the per-download columns it names (python_minor, installer, ci, system, libc) are in data/clickpy_columns.csv

Must fix, fixed

  1. 97, not 98 (piece, "The first day set the month"). "For the eleven, between 63 and 98 per cent" rounded 97.5 a second time; the exact figure is 97.48 (aiobotocore's day-30 median 2.52). Now 97. The record's 97.5 is right at one decimal.
  2. "has not spread slowly" (piece, same section) and "not a slow spread" (short form, body and Corrections line). Both said the low share was not a slow spread, where the record and every summary say "not by itself": the log cannot tell, as the next paragraph says. Now "need not have spread slowly" and "not by itself a slow spread".
  3. "For most of the eleven, the jump came as quickly and was smaller" (piece). The record's reading is that for nine of the eleven the day-1 median was 70 to 107 per cent of the day-30 median, and the same ratio for the 26 runs from 80 to 110, so "as quickly" held plainly for six. Now: "For nine of the eleven, the jump likewise came on the first day, which brought at least seven tenths of the share the new version would hold for the month; it was simply smaller. For boto3 and litellm the first day brought less of it, 61 and 45 per cent."
  4. The summaries (piece and short form). Each stated the first-day observation as a plain finding, while the body, README, the notices and the investigation page say it was noticed after the counts and is not yet tested; a summary is read without the body. Each now says it is a pattern seen in almost all of the 37 libraries and noticed only after the counts were in.
  5. 3.4, not 3.5 (record, Findings, fifth reading; programme.md's checkpoint). The smallest of the four rises was computed from the rounded shares (57.3 minus 53.8); the exact rise is 3.43 points. Now 3.4 in both. scripts/post_hoc.py prints the rises, so the figure is in the kept output; data/post_hoc_reading.txt was regenerated and the replay rerun and passed.

Should fix, fixed

  1. The chart's caption in the piece: "for eleven far less" is now "for eleven between 2 and 34 per cent", the record's figures.
  2. "and the installer then takes an older one without saying so" (piece, "What keeps older versions downloaded"): the clause asserted how installers report a skipped version, which no source in the record was read for; now "takes an older one".
  3. The piece's Methods line now says "the other four readings made after the counts", so the piece carries the number of such readings (five with the chart's), as the Evidence rule asks.
  4. programme.md's LH011 answer now names the three explanations record 0.3's Limits names, and what was not read (requirements, Requires-Python, the download's Python version), where it had named one.
  5. The record's version 0.3 Corrections entry now notes this review's recomputation and the one figure it changed, since the edition had not been committed as released.

Notes, kept

  1. The record's interpretation that "the pinned repositories that moved within the month seem to carry little of the downloads that stay on older versions" is hedged, labelled post hoc and set beside the caveat that the two readings count different things in different samples. It is weakest for pyjwt, where 15 of 29 pairs moved and the share rose 14.7 points. Kept as marked.
  2. The short form is 256 words without its title and byline, six over the guide's upper figure; version 1.0 was 242. Kept.
  3. The checkpoint proposes measuring the first-day observation on releases of November 2025 to March 2026. LH011 read downloads from 20 March 2026, so the last days of a March release's window fall on days LH011 read, though only as the folded "~other" row; the brief should say so when it discloses the overlap.
  4. CLAUDE.md says to prefer primary sources outside GitHub's pages because they returned 403. Where a session has the GitHub tools, they read issues through the API rather than the pages; not tested here, and not a fault in the note.

What the review could not check

  • No live source was re-read; the task was offline. The retained tables are as R-0014 re-read them against ClickPy and PyPI on 28 September 2026, and nothing in this revision draws on a new read.
  • The chart was rendered by Chromium from the SVG; the site's own build was not run.
  • The keeper's reading, which prompted the revision, is known only through programme.md's account of it.
  • LH008's data/lags.csv was taken as retained; its 30-day count agrees with LH008.md's own figure, and its lags were not recomputed from the repositories.
  • CLAUDE.md's fast-forward command and its note on GitHub's 403 were read, not exercised.
  • The reviewer's recomputation uses the same retained tables and the same definitions as post_hoc.py (medians over known releases, the ten-point band the writing agent chose), with its own code, median and rounding; it checks the arithmetic and the reading of the tables, not the tables' derivation, which R-0014 recomputed from the counts.
  • releases.md, status and the colophons' review lines are the driver's, as the dispatch said.
  • The review's own cost: priced by the driver from the transcript.

Nothing remains that must be fixed before release.

From notes/R-0015.md in the repository, last changed 28 September 2026.