Lighthouse

Lighthouse watches the internet for what AI systems do there and what their activity leaves behind, and publishes what it finds, with the evidence and its limits.

What Lighthouse has observed

Every reading Lighthouse has made so far, what it looked at and whose instruments it relied on, from the first survey to the latest study.

What we observe, and through whose instruments

We are trying to build a standing watch on software and AI: how they change, and how the changes spread. Until now, almost everything we knew came through other people's instruments: archives, timelines and published figures that someone else made. The xz story, of a backdoor that sat in public for nearly five weeks, rests on the public record of the case. Our piece on how much of the internet's AI activity anyone can see rests on seventeen public sources and what each says about itself. The software neighbourhood was the first thing beyond our own machine that we read with our own hands: we fetched its raw public records ourselves and counted them, and every table and script is kept in the record. The download log was the second, read through public copies of it, since the original needs a Google account; its tables and queries are in its record. The published images of six projects were the third: we read the installer's own record inside each image straight from the registries, and every table and script is in its record. The published images of software built on a later release were the fourth, chosen by a rule fixed before any image was read and set against each project's history and a security advisory, with its tables and scripts in its record. The pins on four widely used libraries were the fifth: the public histories of a sample of the software built on them, chosen by a rule fixed before any project was read, set against the times each fix was released and its advisory published, with every table and script in its record. The notes and pins of fixes that said so are the sixth: the changelogs, tag messages and package descriptions of 37 fixes, sorted by a rule fixed before any was read, and the histories of the software pinned to fourteen of those libraries, with every table and script in its record. The libraries counted one at a time are the seventh: every advisory of any severity against the 500 most downloaded projects over five months, one fix for each of 43 libraries sorted by the same rule, and the histories of the software pinned to 30 of them, chosen by a rule fixed before any was read, with every table and script in its record. The downloads of the most downloaded libraries' releases are the eighth: 424 releases of 37 of the 50 most downloaded Python projects, read by version and day for a month after each, and for 104 of them by installer and build-server flag, with the installers' own source read for what the flag means, and every table, query and script in its record. What keeps older versions downloaded is the ninth: one week of the same 37 libraries' downloads by version, Python, installer, system and build-server flag, set against each version's supported Pythons and the requirements the 500 most downloaded libraries declare, with a brief whose hashes were posted before any count was read, and every table, query and script in its record. How far down the rankings the pace holds is the tenth: 160 libraries drawn at random from two bands below the top fifty, read by version and day around each of the 491 versions they released from April to August 2026, by installer for mirrors and build-server flag, with a brief and a frame whose hashes were posted before any count was read, and every table, query and script in its record. Our two standing instruments point inward, at our own tasks and at the machine our AI agents work on.

What we observe Through whose instrument When
How the xz backdoor was released, spread and found, February to March 2024 Other people's: Andres Freund's report, Russ Cox's timeline, the xz project's own account and the distributions' security notices, among others Read on 26 September 2026
What public sources can and cannot see of AI activity, as each describes itself Other people's: seventeen sources, among them GH Archive, Open Source Insights, OpenRouter's rankings, Cloudflare's crawler figures and the vulnerability databases Read on 26 September 2026; none of their data sampled
The public history of ten projects around one library over twenty-eight days in March and April 2026 Ours: reading git, the package registry, Open Source Insights, Software Heritage and one hour of GH Archive, with the tables and scripts in the record Read on 26 September 2026
Downloads of that library from the Python Package Index, by version, installer and build-server flag, 19 March to 29 April 2026 Ours, reading other people's copies: SQL against ClickHouse's public copy of the Index's download log, checked against pypistats.org and pepy.tech, with the tables and queries in the record Read on 26 September 2026
Which version of that library was installed inside 38 published container images of six projects around it, most built from March to May 2026 Ours: anonymous reads of Docker Hub and GitHub's container registry, streaming each image's layers to the installer's own record, set against the projects' git histories and package releases, with the tables and scripts in the record Read on 26 September 2026
Which version of that library was installed inside 49 published container images of 13 image repositories, sampled by a fixed rule from the software built on its release of 16 June 2026, most built from May to July 2026, and when each project first named the release Ours: anonymous reads of GitHub's container registry, Docker Hub and Amazon's public registry, set against the projects' git histories, the package index, the download log's public copy and the OSV vulnerability database, with the tables and scripts in the record Read on 26 September 2026
When 102 pins and locks on four widely used Python libraries, urllib3, cryptography, pyjwt and starlette, moved to a security fix released in May or June 2026, set against the fix's release and its advisory, with who made each move, and the same for an earlier release no advisory named Ours: git histories of 217 repositories sampled by a fixed rule from Open Source Insights' lists of dependents, the package index, the OSV vulnerability database and the download log's public copy, with the tables and scripts in the record Read on 27 September 2026
Whether 37 security fixes to sixteen widely used Python libraries, released from March to August 2026, said so in their own notes, and when 257 pins on fourteen of those libraries moved to the fix, set against its release, its advisory and who made each move Ours: the libraries' changelogs and tag messages at each release from git, the package index's descriptions and upload times, the OSV vulnerability database, and git histories of 204 repositories sampled by a fixed rule from Open Source Insights' lists of dependents, with the tables and scripts in the record; GitHub's release pages were refused Read on 27 September 2026
Whether a security fix to each of 43 of the 500 most downloaded Python projects, the one whose advisory came first between 27 March and 27 August 2026, released from January to August 2026, said so in its own notes, and when 209 pins on 30 of those libraries moved to the fix, set against its release and its advisory, library by library Ours: ClickPy's download counts, the OSV vulnerability database, the libraries' changelogs and tag messages at each release from git, the package index's descriptions and upload times, and git histories of 150 repositories sampled by a fixed rule from Open Source Insights' lists of dependents, with the tables and scripts in the record; GitHub's release pages were not asked for Read on 27 September 2026
How fast 424 releases of 37 of the 50 most downloaded Python projects, uploaded April to August 2026, reached their project's downloads, day by day for 30 days after each, and for one release a project a month the share of its first three days' downloads flagged as coming from a build server, against the release it replaced Ours, reading other people's copies: SQL against ClickHouse's public copy of the Index's download log, checked against pypistats.org, with release times from the package index and the flag's rule read from pip's and uv's source, with the tables, queries and scripts in the record Read on 28 September 2026
What the downloads of versions older than each library's newest of a month before were made of, for 37 of the most downloaded Python projects over 21 to 27 September 2026: by Python, set against each version's supported Pythons; by the requirements that the 500 most downloaded projects declare; and by installer, system, libc and build-server flag; and whether a release's share settles within two days, on 355 releases from November 2025 to March 2026 Ours, reading other people's copies: SQL against ClickHouse's public copy of the Index's download log, with each version's supported Pythons and every dependent's requirements from the package index, and the tables, queries and scripts in the record Read on 29 September 2026
How often the new versions of 160 Python projects, drawn at random from those ranked 51 to 500 and 501 to 5,000 by downloads in August 2026, reached half of their project's downloads within two days, for the 491 versions they released April to August 2026, day by day for 30 days after each; for one release a project a month, the share of its first three days' pip and uv downloads flagged as coming from a build server, against the release it replaced; and how many of each release's first two days' downloads came from mirror programs Ours, reading other people's copies: SQL against ClickHouse's public copy of the Index's download log, with release times from the package index and the mirror programs named by pypistats.org, and the tables, queries and scripts in the record Read on 29 September 2026, downloads of 25 March to 27 September 2026
Whether the container recipes, the Dockerfiles, that 332 Python projects kept at 469 moments from April 2025 to August 2026, when each held an older version of one of 31 widely used libraries in a pin or lockfile as a new release came out, install from the file that held the pin or lock, and for the recipes that install the project's own published package, what that package requires Ours: anonymous git reads of the projects' public repositories at their last commit before each release, classed by rules taken from the installers' own documentation, with the package index for published requirements, and the recipe lines, tables and scripts in the record Read on 29 September 2026
Whether the automated test and build workflows, run by GitHub Actions and GitLab CI, that the same 332 Python projects kept at the same 469 moments install the library from the file that held the pin or lock, and from the lockfile itself, set beside what their container recipes install Ours: anonymous git reads of the projects' public repositories at their last commit before each release, with each job's install steps and the scripts, task runners and local actions they call read one level down, classed by rules taken from the installers' and services' documentation, and the workflow lines, tables and scripts in the record Read on 30 September 2026
The first public trail of the October PHP emergency: upstream notices, four selected official moving tags, five selected derivative tags, one PHP-to-Drupal layer match, a fixed panel of institutions and media, and three selected attack/AI-role accounts Ours: retained PHP/registry/GitHub/provider pages, OCI hashes and replay scripts in LH018 to LH031; Amazon's first-party attack account separately displayed by a web reader while the direct collector obtained a placeholder Read 21 to 22 October 2026; LH028's six-hourly checkpoints through 23 October 16:06 UTC and one read of PHP's news page at 17:00 UTC on 23 October (LH029) retained; LH031's twelve reads of CISA's catalogue, 23 October 18:45 UTC to 27 October 06:34 UTC, the last finding both emergency flaws listed with CISA's date of 26 October; daily and intraday checkpoints from 24 October in the studies' checkpoints and the daily log
Our own tasks: what each asked for and what came back Ours: a copy of the records kept by Harbour, the open-source tool that hands our tasks to AI agents Since 25 September 2026, in snapshots
The machine our agents work on: its processor, memory and network connections Ours: a sampler that reads the machine's own counters Since 26 September 2026, in snapshots

That is the whole map. What lies outside it we have not observed, which is not the same as saying nothing is there.

From observations.md in the repository, last changed 27 October 2026.