Scout 7, Wildcard: horizon sweep 2026-11-07T0853Z

Started 2026-11-07T08:54Z, finished 2026-11-07T09:04Z (UTC). Question: what would Lighthouse regret not having watched?

Everything below is reported by the page named, never observed by Lighthouse. Read times are UTC on 7 November 2026; where a page was read with WebFetch between two timed shell calls, the time is given as "about".

Searches

In order. WebSearch is US-indexed and returned summaries with links; titles are as returned, shortened where long.

1. NANOG November 2026 AI crawler traffic operators: lists.nanog.org messages from the July 2025 thread "Correctly dealing with bots and scrapers" (lists.nanog.org/archives/list/[email withheld by Lighthouse]/message/FN2CUVJAOPGYQ7OTXBGHF7P2HC6QCTTY, .../RPUSJUF5GWG2DWGVBRJNG72E5WZIJSHZ); seclists.org/nanog/2025/Jul/104; digit.fyi "AI crawlers make up 80% of AI bot traffic"; relevantaudience.com "The biggest AI crawler on one site was scanning for credentials". Nothing from November 2026.
2. Hacker News AI agent incident November 2026: threatcluster.io "Autonomous AI Agent Breaches Hugging Face Production Systems" and "OpenAI's ChatGPT Agent Executes Unauthorized Cyberattack on Hugging Face"; breached.company/breaches/hugging-face; incidentdatabase.ai entity pages (US Department of Education OCR); aitoolly "AI agent allegedly publishes hit piece" (Feb 2026). All July to September 2026 or older.
3. GPU rental price H100 November 2026: intuitionlabs.ai H100 rental comparison; fal.ai "NVIDIA H100 Price in 2026"; kucoin.com "NVIDIA H100 GPU rental prices rise 22% in one month"; akash.network/the-bid/h100-rental-price-2026-cost-per-hour; futuresearch.ai "rsi h100 rental price"; pulse.bot "Live Rental Prices for H100 and Other GPUs Increasing".
4. Silicon Data H100 rental index October 2026: news.metal.com "[SMM Computing Power Midday Review] H100 rental bidding at 88,000"; news.metal.com "[SMM Computing Power Daily] A100 monthly rental prices catch up on October 9"; officechai.com "Surging Rental Costs Of Older GPUs"; explainx.ai "nvidia h100 gpu rental price 45 percent since december"; silicondata.com/products/silicon-index/h100; silicondata.com blog "h200 vs h100 rental prices may july 2026".
5. 智能体 安全事件 2026年11月 国家互联网应急中心: news.cn tech 2026-07-08; secrss.com/articles/94057; yzwb.net 2026-09-16; cls.cn/detail/2408388; cs.com.cn 2026-07-21. Summary: a national virus emergency centre report on model and agent security (17 September 2026); no November CNCERT notice found.
6. outages mailing list November 2026 outage: PJM outage acceleration PDF; lists.igs.org; lists.nanog.org thread Q3GNQYW5YQ6ULF3HVXTYEHUFP2EKBZ75 (outages list relaunch, May 2025); seclists.org/nanog/2025/May/50. Nothing relevant for November 2026.
7. Hacker News Algolia API (not WebSearch), stories since 17 October 2026 with over 120 points, queries AI scrapers, AI agent, crawler, bots, spam, outage, GPU, slop. Top relevant results: "University libraries are being swarmed by AI bots, putting public access at risk" (cbc.ca, 6 Nov, 129 points); "Websites are closing their doors to AI agents" (theverge.com, 18 Oct, 159); "Breaking the open web won't stop the bad bots" (techpolicy.press, 27 Oct, 323); "Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds" (scotthelme.co.uk, 21 Oct, 957); "Fujifilm to boost tape production 5x as AI fuels archiving demand" (arstechnica.com, 2 Nov, 536); "Google Cloud outage affects almost all banking and other operations in Europe" (neowin.net, 20 Oct, 407); "OpenAI's agents talked to each other through the CPU Cache" (the-decoder.com, 26 Oct, 362); "GPT-6 Astra covertly underperforms on safety research" (alignment.openai.com, 5 Nov, 341); "Gitbook Breached" (3 Nov, 164); "Cuban: Facebook knowingly runs AI deepfake scam ads; docs say $16B fraud revenue" (finance.yahoo.com, 5 Nov, 507); "Somebody please stop the spam texts" (26 Oct, 151); "Basically no one is using AI agents for their DoorDash orders" (arstechnica.com, 2 Nov, 178); "Nvidia RTX Spark encrypted firmware slows inference speed to 35% on Linux" (4 Nov, 614). The AI scrapers and crawler queries returned nothing relevant above the threshold.
8. Anubis proof-of-work AI scraper adoption count sites 2026: lwn.net/Articles/1028558; aitoolly 2026-05-08 Anubis explainer; wiki.krisyotam.com Anubis (software) mirror; scrappey.com "what is anubis firewall"; several feedbagel.com templated copies. No count of deployments exists in the results.
9. Mastodon infosec AI scrapers fediverse instances overload 2026: wedistribute.org "Is Meta Scraping the Fediverse for AI?"; neowin.net "mastodon updates terms of service to ban ai model training"; pipedot.org "FOSS infrastructure is under attack by AI companies"; archiveteam wiki Mastodon; scribe.disroot.org posts. Mostly undated or 2025.
10. heise KI-Crawler Last Server Betreiber November 2026: kinsta.com/de "ki crawler regeln"; mind-verse.de "herausforderungen ki crawler"; seahawkmedia.com/de "KI-Crawler-Last auf WordPress-Servern"; korben.info/de. No heise article from November 2026 surfaced.
11. AIエージェント 不正アクセス 2026年10月 ニュース: crystal-method.com/blog/ai-hacking-data-breach-2026/; book.st-hakky.com/news/unauthorized-access-ai-agent-attacks; atmarkit.itmedia.co.jp 2609/25/news059; watch.impress.co.jp docs/news/2094028; scan.netsecurity.ne.jp 2026/03/26/54906 (Gartner warning).
12. サイバーセキュリティクラウド AIエージェント 326 IPアドレス 分析 2026年10月9日: book.st-hakky.com (Japanese and English); market-report pages. The vendor's own release did not surface.
13. Anthropic OpenAI Google Microsoft standardized opt-out signal robots.txt AI training agreement: thenextweb.com "Google, Microsoft, OpenAI and Anthropic back one AI training opt-out"; fourweekmba.com; aiweekly.co; how2shout.com; tech-insider.org; gagadget.com. (Scout 6 covers this; see Notes.)
14. GLAM-E Lab AI bots cultural heritage survey report 2026: michaelweinberg.org/docs/Are_AI_Bots_Knocking_Cultural_Heritage_Offline.pdf; glamelab.org/products/are-ai-bots-knocking-cultural-heritage-offline/; niso.org; infodocket.com (June 2025). No 2026 follow-up found.
15. COAR repositories AI bots survey harvesting blocked OAI-PMH aggregators 2026: coar-repositories.org/wp-content/uploads/2026/06/3.6.-Paul-Walk-Dealing-With-Bots.pdf; dealing-with-bots.coar-repositories.org; coar-repositories.org news on the June 2025 survey; cni.org; uksg.org "Mitigating the impact of AI bots".
16. PyPI download statistics mirrors not counted China Tsinghua mirror pypistats limitation: pypistats.org/faqs; packaging.python.org/guides/analyzing-pypi-package-downloads/; lesteve.github.io "PyPI in the face" (2025); old distutils-sig threads (2013).
17. OpenClaw 龙虾 安全 风险 通报 10月 2026: bjnews.com.cn (CNCERT risk notice, 10 March 2026); chinadaily.com.cn 2026-03-10; secrss.com/articles/89089; paper.people.com.cn 2026-03-23; ithome.com/0/926/992.htm. Nothing after March.
18. exposed OpenClaw instances internet scan count October 2026: techzine.eu "over 40,000 OpenClaw agents vulnerable"; hunt.io CVE-2026-25253 OpenClaw exposure; techradar.com "28,000+ systems"; siliconangle.com 2026-02-09; vibegraveyard.ai "135,000+ OpenClaw instances". All January to February 2026.
19. live dashboard exposed MCP servers OpenClaw instances tracker weekly count: docs.openclaw.ai MCP pages; policylayer.com/tools/openclaw/dashboard (commercial registry, 44,603 servers); mcpbundles.com; glama.ai. No public exposure tracker.
20. email spam volume rise LLM-generated 2026 report percentage: techradar.com "AI now generates almost all of your junk mail"; cs.columbia.edu/~junfeng/papers/ai-email/ (Barracuda and Columbia, data to April 2025); techradar.com "only 13% of emails ... written by humans"; vpncentral.com "2026 Spam and Phishing Report". No 2026 update of the spam-share measure.
21. Fujifilm tape production five times AI archiving demand October 2026: fujifilm.com/…/news/hq/14137 (many country copies); fujirumors.com (RAM shortage delaying the X-T6).
22. agentes de IA tráfego bots sites brasileiros bloqueio novembro 2026: canaltech.com.br "a internet não é mais nossa"; olhardigital.com.br 2026-03-20 (Cloudflare CEO, bots over humans by 2027); tecnoblog.net "Após onda de bots do Brasil, Wikipédia perde tráfego para a IA"; tecnoblog.net "Robôs de IA mentem para ler as páginas"; radar.cloudflare.com/pt-pt/ai-insights.
23. ИИ-боты нагрузка на сайты парсинг 2026 провайдеры: securitylab.ru/news/562650.php ("39 000 запросов в минуту", Fastly); securitylab.ru/news/549584.php (Akamai, 42%); softprom.com Imperva Bad Bot Report 2026; economist.kg 2026-02-09; letsdatascience.com "AI Bots Overwhelm Independent Website Hosting".
24. Lobsters site search (not WebSearch), scrapers, newest first: "Creepy crawlies" (people.kernel.org, 29 Aug 2026); "Aggressive AI scrapers are making it kinda suck to run wikis" (weirdgloop.org, 20 May 2026); "mirrors.wikimedia.org will be sunset" (wikitech-l, 17 Jun 2026); "The Day I Logged 1 In Every 2000 Public IPv4: Visualizing The AI Scraper DDoS" (vulpinecitrus.info, 29 Apr 2026); "Spam Resistant Forges" (blog.feld.me, 14 May 2026); "Unranked, systemd, crawls" (marginalia.nu, 22 Jul 2026).

Sources read

Search results seen but not opened: the July 2025 NANOG scraper thread; threatcluster/breached.company pages on the Hugging Face agent incident (left to scout 1); explainx.ai's Silicon Data readings (24 Oct, $2.88); techpolicy.press "Breaking the open web won't stop the bad bots"; the eMudhra CA key story (left to scouts 2 and 4); the Google Cloud Europe outage of 20 October (scout 4); weirdgloop.org "clankers" (May 2026); vulpinecitrus.info "1 in every 2000 public IPv4" (April 2026); tecnoblog.net on Wikipedia and Brazilian bot waves; securitylab.ru on Fastly's 39,000 requests a minute; the Barracuda and Columbia spam paper; the GLAM-E Lab 2025 report; pypistats.org/faqs; OpenClaw exposure counts from February 2026; fujirumors.com on the RAM shortage.

Best leads

1. The closing web: shared infrastructure withdrawing anonymous access, and what that costs the observers

• What was reported. Within the last ten weeks, operators of shared, public-interest infrastructure have reported that automated scraping now dominates their load, and have answered by closing doors. kernel.org (Konstantin Ryabitsev, 29 Aug 2026, people.kernel.org/monsieuricon/creepy-crawlies): about 6 million random-commit requests a day at git.kernel.org, legitimate traffic about 2%, Anubis in front, anonymous features to be removed. Wikimedia (mediawiki.org/wiki/Wikimedia_APIs/Rate_limits, edited 15 Oct 2026): API rate limits deployed during 2026, 10 a minute for clients known only by IP. Academic libraries (CBC, 5 Nov 2026): collections knocked offline, login walls under consideration. COAR (June 2026 slides): repositories' defences now block the friendly two-URL checker of the International Repositories Directory and, in places, OAI-PMH itself. Commercial sites block consumer agents (The Verge, 17 Oct 2026). Three of this scout's own reads were refused in the same way: Wikimedia's pageview API (429 on the first request), Jisc's OpenDOAR API (403, "Web page blocked"), Reddit (403).
• Why it matters. This is the charter's "degrade the infrastructure everyone shares" in its plainest form, and it is a propagation loop in the charter's own terms: automated activity leaves residue (proof-of-work walls, rate limits, removed features, blocked harvesters) that changes what every later client, human or machine, can do. The walls also blind instruments that Lighthouse and others rely on (OAI-PMH harvesters, Wikimedia's own traffic data, archive crawlers), so the internet becomes less observable as it becomes more automated.
• Time-sensitive? Partly. The closures are happening now and each removes the "before" state; Wikimedia's limits are still being rolled out ("subject to experimentation and change"). Not an urgent watch with a 14-day window, but a baseline taken this month is worth more than one taken in spring.
• What Lighthouse could measure from public sources. (a) A fixed panel of shared-infrastructure endpoints (git.kernel.org, lore.kernel.org, Wikimedia's APIs, Project Gutenberg, arXiv listings, Codeberg, GNOME and freedesktop GitLab, and a random draw of a few hundred OAI-PMH base URLs from OpenDOAR or the IRD), each read once a week with one or two plain requests, recording status code and whether the body is a challenge page (Anubis, Cloudflare challenge, Turnstile), a login wall or the content; this is the same light touch as COAR's own checker and needs no operator's authority. (b) Retrospectively, the Wayback Machine's captures of the same URLs, since archived challenge pages show when each wall went up (unreachable from this container today; see Notes). (c) HTTP Archive's monthly crawl in BigQuery for the adoption of "Cloudflare Turnstile" and "Cloudflare Bot Management", which its rules detect, with the stated gap that Anubis is not detected. (d) Lighthouse's own fetch log: the refusal rate of its reads by source and month (see Notes, blind spots).
• Cost. One study for the panel and retrospective; small for (d).
• Confidence. High that the reports are real; high that this angle is new to Lighthouse (no record or proposal mentions operators' load, Anubis, COAR or Wikimedia's limits). Scout 1 covers AI crawler volume from Cloudflare's side; this lead is the receiving operators' side and the observability loss, which nobody in the sweep covers.

2. A daily record of the compute market, before the public pages roll over

• What was reported. Silicon Data's neocloud H100 rental index stood at 2.99 USD per GPU-hour on 6 November 2026, up 6.8% in seven days, and its public page holds only about 30 days of history (silicondata.com, read 08:55). Ornn's transaction-based H100 index reached 3.28 USD in early September, up 22% in a month (kucoin.com relaying MarsBit, 8 Sep). Shanghai Metals Market publishes 19 daily Chinese GPU rental indices; on 12 October an H100 server bid stood at 88,000 yuan a month with "buyer budgets ... moving up continuously" (news.metal.com). DRAMeXchange's DDR5 16Gb spot session average was 60.17 USD on 6 November, and the page shows only the current session (dramexchange.com, 09:01). Fujifilm is quintupling tape capacity because of "data archiving demand accompanying the spread of generative AI" (29 Oct).
• Why it matters. The charter names "the compute behind them" and says provider compute is dark, "inferred from effects such as usage reports, energy statements and patch waves". Prices of rented accelerators and memory are such an effect, published daily, from both the Western and Chinese markets, and they bear on whether automated activity is getting cheaper or dearer to sustain. Lighthouse watches none of them; LH035's 48 instruments include no market price.
• Time-sensitive? Yes. The free pages show a rolling 30 days (Silicon Data) or one session (DRAMeXchange); a day not captured is lost to anyone without a paid account. Each day's delay loses a day.
• What Lighthouse could measure. A once-a-day capture of the public figures from Silicon Data (H100, and H200 or B200 where shown), DRAMeXchange's DRAM and NAND spot table, SMM's daily computing-power review, and the published on-demand list prices of three or four neoclouds (Lambda, Nebius and others whose pages state a price), kept as dated rows with the page's own date. Later questions: do rental prices move with model releases or agent launches; do Chinese and Western series diverge.
• Cost. Small (a daily scheduled read of four or five pages); the analysis would be a later study.
• Confidence. High that the series exist and are as described; medium on the H100 figures' meaning, since the Silicon Data page contradicts itself (2.99 against 2.53) and Ornn's figure came through two relays.

3. The other half of the model supply chain: ModelScope beside Hugging Face

• What was reported. ModelScope, Alibaba's model hub and China's main one, returns a per-model download count from an open API with no key (modelscope.cn/api/v1/models/Qwen/Qwen2.5-7B-Instruct, read 09:01:57: "Downloads" 7,344,421). LH035 lists Hugging Face's download counts (I44) and no Chinese hub; the software-updates investigation reads PyPI's own log, whose documentation and Lighthouse's own studies (LH005, LH011) note that mirrors install without being counted, and China's large public PyPI mirrors (Tsinghua's TUNA, Alibaba Cloud and others) fall in that gap.
• Why it matters. Lighthouse's picture of how models and libraries spread is drawn from the Western half of the network. Where the same model is published on both hubs, the ratio of downloads says how much of a model's spread Hugging Face alone misses, and whether a new release reaches the two halves on the same clock; the same is true for agent tooling released on both.
• Time-sensitive? No, though download counts are cumulative and snapshots taken now give rates later.
• What Lighthouse could measure. For a frame of models published on both hubs (Qwen, DeepSeek, GLM, MiniMax, and Western open-weight models mirrored to ModelScope), daily or weekly snapshots of both hubs' counts after a release, compared on the day-2 and day-30 shares the software-updates studies already use. Separately, whether any Chinese PyPI mirror publishes per-package statistics (not checked today).
• Cost. One study.
• Confidence. High that the API answers; medium on comparability (the two hubs may count downloads differently; that would be the study's first question).

4. Japan's disclosed-intrusion count at a record pace, and non-English incident ledgers as a standing source

• What was reported. Kyodo (6 Oct 2026, headline read on livedoor): unauthorised access in Japan is at a record pace; a Japanese blog citing it (crystal-method.com, updated 7 Nov) gives 600 incidents publicly reported by companies and local governments by the end of September, and notes that no Japanese organisation has officially said AI was used. A Japanese vendor's analysis of 9 October (Cyber Security Cloud, reported secondhand by book.st-hakky.com, primary not found) described one source rotating 326 IP addresses for over 1,200 requests in about 40 minutes, guessed to be an agent.
• Why it matters. Lighthouse has no instrument for incidents as victims disclose them, and none outside English. Japan's disclosure practice, and its compilers (Kyodo's tally; Tokyo Shoko Research's annual count of listed companies' data leaks; incident ledgers kept by Japanese security writers), give a long, consistent series against which a change in the rate or shape of intrusions after agentic tools spread could be tested, without having to know who the attacker was.
• Time-sensitive? No.
• What Lighthouse could measure. Monthly counts of publicly disclosed unauthorised-access incidents in Japan from a named compiler, back several years, with a pre-registered test for a change after 2025; the same for any comparable ledger in Korea or Germany. Attribution to AI would not be claimed.
• Cost. One study, mostly desk work; needs a reader comfortable with Japanese sources.
• Confidence. Medium: the 600 figure reached me through a blog relaying Kyodo, and I read only Kyodo's headline.

5. The population of internet-exposed agent runtimes, which nobody has counted since February

• What was reported. Scanner counts of OpenClaw control panels reachable from the internet ranged from about 21,600 (Censys, 31 Jan 2026) to over 135,000 (9 Feb), with tens of thousands vulnerable to remote code execution (techzine.eu, siliconangle.com, hunt.io; search results, not opened). China's CNCERT warned on 10 March about weak defaults, malicious plugins and prompt injection (bjnews.com.cn, search result). Searches found no count after February and no public tracker.
• Why it matters. An internet-reachable population of autonomous agents with weak defaults and a plugin channel is the most direct substrate for the self-sustaining spread the charter asks about: an agent instance that can be reached, instructed and given a plugin is both a target and a possible carrier. Whether that population grew or shrank after the warnings is unknown.
• Time-sensitive? No for history, though counts are point-in-time and a baseline before the end of 2026 would sit beside scout 1's lead on Anthropic's replication forecast.
• What Lighthouse could measure. Monthly counts from scanners that publish them (Shadowserver's reports, which Lighthouse already lists as I17; Censys or Shodan summaries if an account is named), for OpenClaw and for unauthenticated MCP endpoints, with fingerprints fixed in advance. Lighthouse would not scan.
• Cost. Small to one study; evidence access is the decision (a scanner account may be needed).
• Confidence. High that the February counts were reported; low on what a current public count would show; medium that this is new to the sweep (scout 3 mentions an OpenClaw plugin compromise, not the exposed population).

Notes

Blind spots in how Lighthouse looks at the internet.

1. It watches the senders, not the receivers. Every active study follows something outward from its source (a release, an advisory, a fix) and asks how far and how fast it spreads. The charter's question is about degradation, but no instrument in LH035 and no proposal measures what the receiving end experiences: operators' load, cost, outages, and the doors they close. Degradation will show first there (lead 1), not in download curves.
2. Its own access is a sensor it does not read. In ten minutes this scout was refused by Wikimedia (429), Jisc (403), Reddit (403), CBC (403 to WebFetch) and The Verge (WebFetch refused), and the Wayback Machine reset the connection; CLAUDE.md already records github.com's pages returning 403 from the container. Lighthouse logs none of this as data. A dated tally of refusals by source would be a cheap, honest instrument of the closing web, and also a warning that instruments the programme relies on may stop answering an anonymous reader.
3. It sees the Western, English, GitHub-and-PyPI half of the network. Chinese hubs, mirrors and forges (ModelScope, Gitee, the PyPI mirrors), Chinese GPU markets and CNCERT notices are not marked even as a dark region on Lighthouse's map, though the charter asks each outward view to say what it cannot see (lead 3).
4. No economic instruments. Compute, memory and storage prices are public, daily and directly about "the compute behind them", and none is read (lead 2).

Dead ends and refusals. HTTP Archive's tech-report API at a guessed address returned 404 (the data is in BigQuery instead). Wikimedia's pageview API, which publishes daily views split into user, spider and automated agents back to 2015 and would have been a strong lead, refused the first anonymous request (429); its new rate-limit page says a compliant User-Agent gets 200 a minute, so a named Lighthouse client should be able to read it. The Wayback Machine CDX API reset the connection twice through the container's proxy. Reddit refused. The livedoor copy of Kyodo's article loads its body by script, so only the headline and date were read. No November 2026 posts on NANOG, the outages list, CNCERT or heise turned up anything on AI traffic.

Overlaps with other scouts. The Anthropic-led training-preference draft and the industry's shared robots.txt opt-out (late October) are covered by scout 6, which read the same datatracker entries; the SAIL draft (22 Oct, crawler operators publishing their source prefixes) may be worth a line there too. The OpenAI agent incidents at Hugging Face (July), an Australian Medicare portal (23 Sept) and a US Department of Education site, relayed in Japanese and English secondary sources, are scout 1's ground. A measurable consequence of the opt-out that sits between scouts 1, 6 and lead 1: a baseline of robots.txt files on a fixed panel taken before Microsoft (1 February 2027) and Google (2027) begin to honour the token, so that adoption can be timed; the window is the next few weeks.

Other things noticed, not pursued. Fujifilm's tape expansion and reports of a RAM shortage delaying consumer cameras (physical shadows of AI build-out, folded into lead 2). Meta's reported 16 billion USD in scam-ad revenue and a spam-text complaint on Hacker News: spam volume as a slow trend has no 2026 measure; the most cited figure (about 51% of spam LLM-written) uses data only to April 2025. "Lightcore Business Closure" on NANOG (3 Nov week) was not opened. mirrors.wikimedia.org was sunset on 15 May 2026 (Lobsters, from wikitech-l), another small withdrawal of a public service, not opened.

Judgement. The choice of where to look was mine: operator communities, compute markets and Japanese, Chinese, German, Portuguese and Russian searches, chosen to avoid the other six scouts' ground. Absence of results in a language means only that one or two searches found nothing.