Scout 7, Wildcard: horizon sweep 2026-11-10T1415Z

Started 2026-11-10T14:19Z, finished 2026-11-10T14:28Z (UTC). Question: what would Lighthouse regret not having watched?

Everything below is reported by the page named, never observed by Lighthouse. Read times are UTC on 10 November 2026; where a page was read with WebFetch between two timed shell calls, the time is given as "about". Every URL was put through scripts/robots_gate.py (provider anthropic) before it was fetched, with one exception recorded under "A fault of this scout's own" below. "New?" means not found in any file of the 7 November sweep (grep of notes/horizon/2026-11-07T0853Z/).

A fault of this scout's own

At 14:21:18 one request went to https://mastodon.social/api/v1/timelines/tag/aiscrapers although the gate, run in the same shell command, had answered "disallowed" (group ClaudeBot, Disallow: /). The shell loop fetched both hosts without testing the gate's answer, and the gate's output was read only afterwards. The response (HTTP 200) was deleted unread beyond the first screen and nothing from it is used here. After this every fetch went through a wrapper that fetches only when the gate allows. This is the same class of fault as notes/incidents/2026-11-08-robots-partial-read/ (a check that ran but did not govern the request), and it belongs in LH047's tally of requests that broke a rule.

Searches

In order. WebSearch is US-indexed and returns summaries with links; titles are as returned, shortened where long.

1. Hacker News Algolia API (not WebSearch), all stories since 6 November 2026 with over 150 points (110 hits). Relevant: "Claude Code's 'Don't Ask Again' exposes passwords in 1,122 public GitHub files" (apiiro.com, 7 Nov, 624 points); "Outlook.com is rejecting email from domains without DMARC" (mailroute.net, 8 Nov, 540); "GitHub's new Git storage architecture" (github.blog, 6 Nov, 238); "We know 15k websites are infected. how do we tell them?" (scotthelme.co.uk, 6 Nov, 228); "ChatGPT Desktop forwards your SSH agent to remote machines" (9 Nov, 212, no URL; left to scout 2); "CoreWeave bond yield tops 16% as AI credit stress rises" (cnbc.com, 10 Nov, 177); "When will this stop? The Raspberry Pi 5 16GB SBC now sells for $385" (cnx-software.com, 9 Nov, 321); "Humans will never again create a project quite like Wikipedia" (economist.com, 9 Nov, 480); "The same H100 costs $2.69 or $12.29 an hour" (9 Nov, 170); "RIP, Harvest" (aphyr.com, 9 Nov, 434); "Show HN: Coldframe, a honeypot that poses as a supermarket fridge controller" (7 Nov, 288).
2. Hacker News Algolia API, keyword queries since 6 November over 25 points: scrapers, crawler, bots, spam, outage, agent, AI traffic, slop, DDoS, open source maintainer, botnet. scrapers, outage, AI traffic and open source maintainer returned nothing; bots returned the CBC libraries story already read on 7 November (137 points); the rest repeated item 1.
3. Outlook.com rejecting email domains without DMARC November 2026 550 5.7.515: Microsoft Q&A threads on 550 5.7.515; uriports.com; dmarcian (Japanese, French); suped.com; mailreach.co; Microsoft Tech Community's 2025 high-volume sender post. Background: the 5,000-a-day bulk-sender rule enforced from 5 May 2025. Nothing from Microsoft about November.
4. apiiro .claude/settings.local.json secrets public GitHub repositories 1,122: aiweekly.co, itwire.com, pasqualepillitteri.it, fourweekmba.com, letsdatascience.com (two), runtimewire.com, idlen.io, af.net, ai-tldr.dev, securityonline.info, itbrief.co.uk (an earlier npm-package variant), nhimg.org (Lakera, April 2026). Summary gives Apiiro's figures as 104,185 public copies in 103,617 repositories, 1,122 with a secret, 1,379 distinct secrets; 2,539 secrets across 1,994 repositories counting history, settings.json and saved agent transcripts; published 5 November 2026.
5. KI-Crawler Bots Webseiten Betreiber November 2026: drweb.de, Cloudflare's German blog, t3n.de, computerwoche.de, kinsta.com/de, ahrefs.com/de ("Nur 2,3 % der deutschen Websites blockieren GPTBot"). Nothing from November.
6. AI 爬虫 网站 瘫痪 开源 镜像站 2026年11月: getcoai.com, 80aj.com (July 2026), ithome.com/0/912/660.htm, aitoolly (OpenWrt deploys Anubis, July 2026), 36kr.com. Nothing from November.
7. robots IA scrapers sites bibliothèques archives saturés novembre 2026: cybernews.com "Data-hungry bot swarms target university libraries" (one library system from about 6 million bot connection attempts a day to over 2 billion in 30 days); 404media.co; govtech.com; emergent.sh (5 Nov, mentions joint statements by library associations, single obscure source); fr.euronews.com (May 2026, about 245 news organisations blocking the Internet Archive's crawlers).
8. AI 크롤러 트래픽 서버 부담 공공기관 2026년 11월: journalist.or.kr (a Seoul newspaper's English site: 2.72 million requests on its peak day in June, 45% from China); ditoday.com (DataDome Q2 2026: Meta's bot activity up 163%); itworld.co.kr; braindetox.kr. Nothing from November.
9. bug bounty AI-generated reports flood November 2026 maintainers program paused: eweek.com, techradar.com, malwarebytes.com, bleepingcomputer.com and csoonline.com on Google pausing its open-source VRP (from 1 October 2026); theregister.com on curl ending its bounty (21 January 2026); privacyguides.org on HackerOne pausing the Internet Bug Bounty (April 2026); dev.to "76% more reports, programs shutting down".
10. certificate transparency log growth 2026 certificates issued per day record Let's Encrypt: technologychecker.io (vendor: 3.41 billion certificates logged 14 Feb to 16 Mar 2026); chromium ct-policy group; letsencrypt.org/stats; Cloudflare Radar CT log pages. No verified 2026 daily record; not pursued.
11. OpenStreetMap tile servers AI scrapers 2026 overload blocked: OSMF operations minutes (5 and 19 March 2026: over 100,000 IP addresses blocked, residential proxy companies contacted); community.openstreetmap.org tile-service threads; heise.de "OpenStreetMap is concerned: thousands of AI bots"; pretalx SotM 2026 talks "Running OpenStreetMap.org in the Age of AI" and "Consumed at Scale: AI-Driven Extraction and the Political Economy of OpenStreetMap".
12. residential proxy network AI scraping demand botnet takedown October November 2026: infoblox.com "Residential proxies in the wild"; Cloud Security Alliance research note (13 Aug 2026, marked as AI-assisted and unreviewed by CSA); securityandtechnology.org; datadome.co on Kimwolf traffic; dev.to on Google's NetNut takedown (2 July 2026, Popa botnet, at least two million devices); lwn.net "An update on the scraper situation" (subscriber links). No October or November takedown found.
13. "Wikipedia" Economist "never again create a project" October 2026: semafor (March 2026 ban on AI-written articles); axios.com (2 Oct 2026, Wikimedia CEO on traffic loss); getcoai (July 2026 pause of AI summaries). The Economist piece itself did not surface; not pursued.
14. Grass network nodes bandwidth sharing AI scraping 2026 number of nodes daily data scraped: coinmarketcap, dextools, blockworks, blockeden, okx, nansen and others. Node counts from 600,000 to 8.5 million and about 1 petabyte a day, all promotional and mutually inconsistent; no first-party figure.
15. Microsoft Outlook hotmail bouncing mail no DMARC record "November 2026": suped.com blog; windowslatest.com (8 Nov); techbytes.app "Outlook.com is quietly rejecting email from domains without DMARC"; ecosistemastartup.com (Spanish); firstsales.io; Adobe Marketo community hard-bounce thread.
16. 生成AI クローラー アクセス集中 サイト 停止 2026年11月: arxiv 2512.24968; zenn.dev; theregister.com (Dec 2025); rocket-boys.co.jp (an OpenAI crawler taking down a small e-commerce site, January 2025). Nothing from November.
17. Discourse forum AI agent spam signups overwhelm moderators November 2026: meta.discourse.org threads "Strategies for filtering spam / AI user accounts?", "Are you experiencing AI based spam?" (one admin: AI accounts outnumber real new accounts 10 to 1), "Unwanted bots" (15 Oct 2026); discuss.cachyos.org; forum.bestpractical.com.
18. 国家互联网应急中心 智能体 风险提示 2026年11月: news.cn and xinhuanet (10 and 11 March 2026, the OpenClaw notice); stdaily.com; ithome.com/0/927/729.htm; wallstreetcn.com (a CNCERT warning on malicious agent skills, date not shown); secrss.com 91136 and 94057. No November CNCERT notice found.
19. GitHub changelog 2026 public events API changes rate limit events timeline agents: GitHub REST breaking-changes pages (API version 2026-03-10 removes the rate property; events endpoints listed); events docs (X-Poll-Interval, 304); github.blog changelog on code_scanning_upload (May 2026). No 2026 change to public events found.

Standing sources: NANOG's archive (read, nothing on AI traffic since 3 November; two new threads, Lumen decommissioning the old Level 3 and Genuity authoritative DNS addresses on 6 November, and the Lightcore MetroNet closure, neither in the charter's scope); Lobsters (now refused by robots.txt, below); Mastodon's infosec community (infosec.exchange's hashtag API now asks for login; fosstodon.org's answered).

Sources read

Not opened or refused

• Not fetched, robots.txt: lobste.rs search and /newest (* group, Disallow: /; the 7 November sweep read lobste.rs search at 09:01, before the gate existed, so whether the rule is new is unknown); mailroute.net (the Outlook story's origin); aphyr.com; itwire.com; lwn.net (ClaudeBot group, "An update on the scraper situation"); bleepingcomputer.com; csoonline.com; wallstreetcn.com.
• Refused: apiiro.com (challenge), cybernews.com (403), infosec.exchange hashtag API (login).
• Search results seen but not opened: Microsoft Tech Community's 2025 high-volume sender post; techbytes.app on Outlook; the Economist's Wikipedia essay; the Cloud Security Alliance residential-proxy note (AI-assisted, unreviewed); datadome.co on Kimwolf traffic; securityandtechnology.org on residential proxies; dev.to on NetNut; OSMF minutes of 5 and 19 March 2026; the SotM talk "Consumed at Scale"; heise.de on OpenStreetMap; malwarebytes and techradar on Google's VRP pause; theregister.com on curl; privacyguides.org on HackerOne's pause; journalist.or.kr and ditoday.com (Korean); emergent.sh on library associations; fr.euronews.com on news organisations blocking the Internet Archive; ahrefs.com/de block rates; Grass node-count pages (promotional); cnbc.com on CoreWeave; cnx-software.com on Raspberry Pi prices; the datareceipts H100 price comparison; technologychecker.io CT counts.

Five best leads

1. GitHub says agents have quadrupled load on a single repository in two months and has rebuilt its Git storage for them

• What was reported. GitHub's engineering blog (5 Nov 2026): requests to one repository rose from 44M a day in March 2026 to 181M a day in May, "on an exponential trajectory" since; the new architecture was tested at 5,000 concurrent agents and 7,200 pushes a second; bursts of writes on busy repositories are expected to become sustained within a year.
• Why it matters. It is the largest forge, in its own words, saying that agent activity is reshaping the cost of shared infrastructure. It is a primary source rather than a vendor's survey, and it gives one dated series. It sits beside Lighthouse's own finding (LH046) that GH Archive's non-push events thinned sharply during 2026. Whether those two belong together is an untested idea, formed after reading both (post hoc). No source read links them.
• Fits. What AI activity costs the shared internet. It also bears on software updates, whose instruments lean on GitHub.
• Window. Weeks. The blog post will stay up. What will be lost is the period before the new architecture and any rate or API change that follows it. A brief that dates GitHub's public statements, changelog entries and API behaviour from now on would catch those changes as they happen, not after.

2. Agents' residue in the commons: Claude Code permission files and transcripts committed to public repositories

• What was reported. Apiiro (5 Nov 2026, primary walled; relayed by runtimewire and by search summaries): 104,185 public copies of .claude/settings.local.json in 103,617 repositories, 1,122 holding a secret; 2,539 secrets across 1,994 repositories counting history, settings.json and saved agent transcripts. A commenter writing as Apiiro says that in 539 of 1,105 traced repositories a commit co-authored by Claude changed the file, and that the file is git-ignored only globally, so every clone and fork carries it on.
• Why it matters. This is the charter's "residue" in its plainest form: an agent's working state written into the public commons, spread by forks and clones, and readable by other agents. It also gives a countable marker, how many public repositories carry agent configuration or transcript files, by creation date, that could grow with agent use. Lighthouse would count aggregates only. It would never read, test or name a secret, a repository or an account, as the keeper's line of 8 November requires.
• Fits. What AI activity costs the shared internet. It also touches supply chain (secrets in forks), which scout 3 may also report.
• Window. Days. The publicity will make owners delete files and rewrite history, so the count Apiiro saw on 5 November is shrinking now, and the "before" exists only in Apiiro's figures. A first dated count soon, by a public, robots-compliant route, would be the only independent baseline. GitHub code search needs a login and github.com refuses this container (L-0040), so the route itself is a P-0022 question.

3. Discovery outruns repair: bounty programmes paused, and a researcher who cannot tell 16,000 infected sites

• What was reported. Google paused its open-source VRP from 1 October 2026 after a "significant rise in automated submissions", most not valid, with an update planned for Q1 2027 (eWeek, 6 Oct). Search summaries add curl ending its bounty (January 2026) and HackerOne pausing the Internet Bug Bounty (April 2026). Report URI (6 to 9 Nov) has found 15,579, then 16,068, websites serving malicious JavaScript and says it finds infections "considerably faster than we could ever hope to notify the affected organisations".
• Why it matters. The charter asks whether AI activity degrades shared infrastructure. One way it can do so without a single attack is by flooding the channels that turn findings into fixes: triage queues, bounty programmes and notification routes. If cheap discovery, by models among others, swamps those channels, fewer fixes come out and the residue of known-but-unfixed flaws grows. A dated ledger of programmes paused, closed or reopened, with each programme's own stated reason, can be built from public notices. Report URI makes no AI claim, and none should be read into it.
• Fits. When a model finds the bug (its other half: what happens after the find).
• Window. Moderate. Notices persist, though Google's own blog would not render here. The value of a ledger comes from dating each change as it happens; Google's Q1 2027 update is the next fixed point.

4. Residential proxies: the bridge from AI scraping demand to enlisted consumer devices

• What was reported. Infoblox (9 June 2026): over 65% of its cloud customers queried residential-proxy domains in 2026, "the most seen proxy service is associated with scraping for AI models", monthly queries rose from nearly 400 billion to over 500 billion between January 2025 and April 2026, with "no reduction" after Google's IPIDEA action. OpenStreetMap (February and March 2026) blocked scrapers spread across 100,000+ addresses and contacted proxy companies. Search summaries add Google's NetNut takedown (2 July 2026, Popa botnet, at least two million devices) and bandwidth-sharing "AI data" networks claiming 0.6 to 8.5 million nodes (promotional, inconsistent).
• Why it matters. This is the most concrete mechanism found for AI activity sustaining itself economically through other people's machines. Demand for training data pays for residential exit nodes, which come from bandwidth-sharing apps, free VPNs and infected devices, the same pool that feeds botnets (L-0001, LH042). Lighthouse marks the exposed-device end of this (LH042) and the closed-door end (LH044), but not the market between them.
• Fits. What AI activity costs the shared internet.
• Window. No urgent window. Vendor reports are periodic. Takedowns are dated events whose before and after can be compared later from public reports.

5. The closing web reaches email: Outlook.com rejecting mail since 5 November, unannounced

• What was reported. Windows Latest (8 Nov) and Suped (6 Nov): since at least 5 November, mail to Outlook.com and Hotmail has been rejected, intermittently, with 550 5.7.1 "Message lacks SPF/DKIM/DMARC Auth Records (AS9031519)" and 550 5.7.23; Microsoft has confirmed no change; small senders below the 5,000-a-day bulk threshold report it. Suped says the reports do not establish a new policy.
• Why it matters. Email is the most distributed of the shared services, and this is a defence against machine-made mail (the charter's spam) narrowing who can take part, the same shape as LH044's walls on the web. No source read ties this change to AI-generated mail; that link is a scenario, not a claim. A public, cheap and robots-free measure exists: DMARC records are DNS. A dated share of a fixed domain panel publishing DMARC, read before and after such a change, would show the defence's reach.
• Fits. What AI activity costs the shared internet, as a sibling to the closing web (LH044, P-0013). It is not a fit for software updates.
• Window. Days. The first week of an unannounced change is when senders scramble to publish records. A DMARC baseline read after the scramble cannot recover the state before it. Microsoft may also clarify or reverse.

Notes

• Walls met this sweep, for L-0016 (Lighthouse's refused requests as an instrument): Lobsters now refuses every crawler in robots.txt; infosec.exchange's hashtag API needs a login; LWN, BleepingComputer, CSO Online, Mailroute, aphyr.com, itwire.com and wallstreetcn.com refuse by robots.txt; Apiiro and Cybernews answer with a challenge or 403. Many of the operator and press sources a wildcard scout would read are closing to agents, Lighthouse's included.
• Seen but not chosen: Discourse provisioning AI agent accounts on self-hosted forums by default (October, fixed 20 Oct); CoreWeave's bond yield above 16% (AI credit stress); Raspberry Pi 5 16GB at $385 (memory prices); a CNCERT warning on malicious agent skills (undated in the result).
• Pages opened: 24 rows (about 30 requests counting the keyword queries and HN items), one of them the robots.txt breach above.