LH011 sources

What was read for LH011, when, and what each source is. All reads were anonymous: no account, key or credential was used or sent. Times are UTC on 28 September 2026; data/read_log.csv holds every read's time, endpoint, query text, HTTP status, rows returned and, for ClickPy, the rows and bytes read as ClickHouse reported them. Repository sources are cited by their number in the repository's sources.md.

id	source	publisher; publication date or window	read (UTC)	retained as
K1	ClickPy public SQL, https://sql-clickhouse.clickhouse.com/?user=demo (repository S21)	ClickHouse; a copy of PyPI's download table (S20); server 26.10.1.39301, time zone UTC; days held to 2026-09-27 for all 50 projects	19:53 to 20:02:45 (a probe of the server's limits was run by hand a few minutes before 19:53 and is logged as such)	data/clickpy_.csv, data/daily_totals_.csv, data/counts/, data/ci/
K2	PyPI JSON API, https://pypi.org/pypi/{project}/json	Python Package Index; release files with upload times, observed at the read	19:54:22 to 19:54:52 (50 projects); 19:55:35 to 19:56:09 (pip and uv, read twice)	data/pypi_versions.csv, data/installer_versions.csv
K3	Source distributions of pip 26.0.1 (uploaded 2026-02-05), pip 26.2.1 (2026-08-04), uv 0.11.2 (2026-03-26) and uv 0.12.8 (2026-08-31), from files.pythonhosted.org, each matched to PyPI's SHA-256	the pip and uv projects via PyPI	19:55:36 to 19:55:37	data/installer_versions.csv (URLs and hashes), data/installer_source_excerpts.txt (the lines read); the archives themselves were kept in scratch space only
K4	pypistats.org, https://pypistats.org/api/packages/{project}/overall (repository S22)	Python Software Foundation; about 180 days kept (29 March to 27 September 2026 for these projects when read)	20:02:52 to 20:02:56 (five projects)	data/pypistats/
K5	studies/LH005/sources.md, L4: ClickHouse's blog of 21 January 2026 (modified 3 March 2026), quoting ClickPy's ingestion transformation `CAST(ifNull(ci, 0), ...)`	read by LH005 on 26 September 2026; not re-read here	not re-read

What each source is, and its limits, as far as this study relied on it

K1, ClickPy. A copy, not the original: every count here is ClickPy's. Its catalogue (data/clickpy_schema.csv) gives each table's definition. pypi_downloads_per_day_by_version is filled by a materialised view counting pypi.pypi rows by date, project and version, and gives both sides of every share. pypi.pypi holds one row per download with the installer's name (not its version) and a ci column of 'false', 'true' or 'unknown'; the staging table holds ci as a nullable boolean, and the step between them is not in the catalogue (K5 says a null becomes 'false'). No row read for the CI subsample held 'unknown'. ClickPy keeps the day of each download, not its time. It re-ingests days it finds wrong (LH005), so a later read can differ. Limits served: 10,000 result rows, one billion rows read and 60 seconds a query.
K2, PyPI JSON. The versions a project lists now, with each file's upload time; a version deleted before the read is absent, so its downloads, if ClickPy holds any, count only in the denominator. yanked is the state at the read; when a file was yanked is not given.
K3, the installers' source. pip (both versions) sends "ci": true when any of BUILD_BUILDID, BUILD_ID, CI or PIP_IS_CI is set in its environment and null otherwise, never false (src/pip/_internal/network/session.py, with the list in utils/misc.py in 26.2.1). uv (both versions) copies the rule, citing pip 24.0, and sends true or null (crates/uv-client/src/linehaul.rs). Only these four versions were read. How PyPI's log processor turns the User-Agent into the BigQuery column was not read (its repository was refused to LH005).
K4, pypistats. Daily totals per project from the same BigQuery table, with and without mirrors; used only as the brief's cross-check of ClickPy's daily totals.
Not read. BigQuery itself (S20); linehaul's source; any installer other than pip and uv; pip and uv versions older than those in K3; pepy.tech (S23), which the brief did not call for.