# Sol-37 Crawl Inventory: 2026-07-22

Generated `2026-07-22T21:01:47.496651+00:00` from 2 retained Caddy access logs covering `2026-06-19T20:56:11.693000Z` through `2026-07-22T21:01:09.711000Z` (129,349 parsed requests).

This inventory proves HTTP fetches recorded by Sol-37. It does not prove that a fetched resource was indexed, retained, used for training, or shown to a user. Query strings are intentionally removed from the path inventory.

Sol-37 intentionally encourages standards-compliant crawling of its public surfaces. Source verification here supports accurate attribution and observability; it is not a crawler blocklist or enforcement feed.

## Confirmed operator-attributed fetches

| Identity | Role | Confirmed requests | Paths | First seen (UTC) | Last seen (UTC) | Evidence |
|---|---|---:|---:|---|---|---|
| Googlebot | search crawler | 1,577 | 424 | 2026-06-19T23:44:05.905000Z | 2026-07-22T19:59:00.534000Z | published-range: 1577 |
| ClaudeBot | AI training crawler | 1,505 | 709 | 2026-06-19T21:33:07.221000Z | 2026-07-22T20:29:53.008000Z | rir-owner: 1505 |
| GPTBot | AI training crawler | 999 | 822 | 2026-06-20T08:32:28.963000Z | 2026-07-22T10:07:53.467000Z | published-range: 999 |
| meta-externalagent | AI training crawler | 967 | 522 | 2026-06-26T21:29:43.172000Z | 2026-07-22T09:17:59.370000Z | rir-owner: 967 |
| facebookexternalhit | link-preview fetcher | 335 | 42 | 2026-06-21T08:37:21.476000Z | 2026-07-22T04:23:15.685000Z | rir-owner: 335 |
| Amazonbot | AI/service crawler | 333 | 287 | 2026-07-04T06:45:18.562000Z | 2026-07-17T20:03:50.920000Z | published-range: 333 |
| OAI-SearchBot | search crawler | 281 | 88 | 2026-06-20T08:32:28.954000Z | 2026-07-22T20:56:33.063000Z | published-range: 281 |
| bingbot | search crawler | 120 | 64 | 2026-06-19T21:29:44.840000Z | 2026-07-22T05:47:21.994000Z | published-range: 120 |
| ChatGPT-User | user-triggered fetcher | 102 | 53 | 2026-06-27T02:53:34.329000Z | 2026-07-15T20:24:52.932000Z | published-range: 102 |
| Googlebot-Image | image crawler | 70 | 34 | 2026-06-19T23:58:06.614000Z | 2026-07-22T20:47:48.255000Z | published-range: 70 |
| PerplexityBot | search crawler | 27 | 6 | 2026-07-11T20:02:43.655000Z | 2026-07-20T13:56:02.976000Z | published-range: 27 |
| Googlebot-Video | video crawler | 1 | 1 | 2026-07-04T15:18:43.810000Z | 2026-07-04T15:18:43.810000Z | published-range: 1 |
| archive.org_bot | web archive crawler | 1 | 1 | 2026-07-03T11:20:03.979000Z | 2026-07-03T11:20:03.979000Z | fcrdns: 1 |
| meta-webindexer | search crawler | 1 | 1 | 2026-07-10T20:25:18.571000Z | 2026-07-10T20:25:18.571000Z | rir-owner: 1 |

## Unconfirmed identity claims

These requests used a bot-like user agent but could not be tied to that operator with the current verification evidence. A current-range miss is not proof of spoofing because published ranges can change after a historical request.

| Claimed identity | Unconfirmed requests | Paths | Reason counts |
|---|---:|---:|---|
| Amazonbot-Video | 28 | 25 | unverified-user-agent: 28 |
| ChatGPT-User | 23 | 16 | unverified-current-range-miss: 23 |
| Discordbot | 8 | 6 | unverified-user-agent: 8 |
| Googlebot | 1,414 | 282 | unverified-current-range-miss: 1060, unverified-private-source: 354 |
| facebookexternalhit | 11 | 4 | unverified-user-agent: 11 |

Across all confirmed identities, the logs contain 6,319 fetches of 1,371 distinct query-free paths, including 5,272 successful `2xx` responses.

## Most-fetched confirmed paths

| Path | Requests |
|---|---:|
| `/robots.txt` | 521 |
| `/sitemap.xml` | 461 |
| `/gui` | 245 |
| `/api/chat` | 229 |
| `/api/chat/history` | 152 |
| `/windows98-clippy-showcase.html` | 59 |
| `/site-metrics.json` | 44 |
| `/saayn-spreadsheet-gpt2.html` | 42 |
| `/` | 37 |
| `/site-index.json` | 32 |
| `/push/api/activity` | 29 |
| `/api/chat/broadcast` | 29 |
| `/neoriris-clock-tower.html` | 28 |
| `/programs/neoriris-clock-display.html` | 28 |
| `/posts/index.json` | 28 |
| `/programs/stardate.html` | 27 |
| `/knowledge-index.json` | 27 |
| `/dashboard` | 26 |
| `/programs/logbook.html` | 25 |
| `/sitemap.html` | 25 |
| `/sol37/index.html` | 24 |
| `/chat` | 23 |
| `/sentience/index.html` | 23 |
| `/posts/write_to_random_emission.html` | 22 |
| `/assets/hue-visualizer.js` | 22 |

## AI dashboard and camera evidence

| Identity | Path | Methods | Requests | Statuses | First seen (UTC) | Last seen (UTC) |
|---|---|---|---:|---|---|---|
| Amazonbot | `/api/dashboard/status` | GET: 1 | 1 | 200: 1 | 2026-07-09T16:30:39.225000Z | 2026-07-09T16:30:39.225000Z |
| Amazonbot | `/dashboard` | GET: 1 | 1 | 200: 1 | 2026-07-07T09:11:43.690000Z | 2026-07-07T09:11:43.690000Z |
| Amazonbot | `/dashboard/` | GET: 1 | 1 | 308: 1 | 2026-07-07T09:11:43.088000Z | 2026-07-07T09:11:43.088000Z |
| ChatGPT-User | `/dashboard` | GET: 2 | 2 | 200: 2 | 2026-07-02T21:31:19.727000Z | 2026-07-02T21:38:21.030000Z |
| ChatGPT-User | `/dashboard/` | GET: 1 | 1 | 308: 1 | 2026-07-02T21:38:20.819000Z | 2026-07-02T21:38:20.819000Z |
| GPTBot | `/dashboard/dashboard.css` | GET: 1 | 1 | 200: 1 | 2026-07-02T21:39:21.027000Z | 2026-07-02T21:39:21.027000Z |
| GPTBot | `/dashboard/dashboard.js` | GET: 1 | 1 | 200: 1 | 2026-07-02T21:39:25.344000Z | 2026-07-02T21:39:25.344000Z |
| Googlebot | `/api/dashboard/camera-caption` | GET: 1 | 1 | 200: 1 | 2026-07-07T12:17:24.278000Z | 2026-07-07T12:17:24.278000Z |
| Googlebot | `/api/dashboard/services/console` | GET: 3 | 3 | 200: 3 | 2026-07-06T11:04:16.083000Z | 2026-07-11T00:38:09.830000Z |
| Googlebot | `/api/dashboard/status` | GET: 3 | 3 | 200: 3 | 2026-07-06T11:04:15.387000Z | 2026-07-11T00:38:09.361000Z |
| Googlebot | `/dashboard` | GET: 15 | 15 | 304: 15 | 2026-06-20T01:39:07.866000Z | 2026-07-21T21:17:29.847000Z |
| Googlebot-Image | `/api/dashboard/camera.mjpg` | GET: 3 | 3 | 200: 3 | 2026-06-23T01:47:32.204000Z | 2026-06-28T09:57:28.188000Z |
| OAI-SearchBot | `/api/dashboard-access/status` | GET: 1 | 1 | 200: 1 | 2026-07-02T22:09:50.906000Z | 2026-07-02T22:09:50.906000Z |
| OAI-SearchBot | `/api/dashboard/camera.mjpg` | GET: 12 | 12 | 200: 12 | 2026-07-02T22:12:22.213000Z | 2026-07-22T20:54:47.968000Z |
| OAI-SearchBot | `/api/dashboard/comments` | GET: 4 | 4 | 200: 4 | 2026-07-16T23:52:25.358000Z | 2026-07-17T01:23:10.083000Z |
| OAI-SearchBot | `/api/dashboard/services/console` | GET: 1 | 1 | 200: 1 | 2026-07-17T00:22:13.329000Z | 2026-07-17T00:22:13.329000Z |
| OAI-SearchBot | `/api/dashboard/status` | GET: 9 | 9 | 200: 9 | 2026-07-02T22:11:51.333000Z | 2026-07-22T20:56:12.516000Z |
| OAI-SearchBot | `/dashboard` | GET: 3 | 3 | 200: 3 | 2026-07-02T21:39:14.590000Z | 2026-07-16T23:19:41.865000Z |
| OAI-SearchBot | `/dashboard/dashboard.css` | GET: 1 | 1 | 200: 1 | 2026-07-02T21:39:21.982000Z | 2026-07-02T21:39:21.982000Z |
| OAI-SearchBot | `/dashboard/dashboard.js` | GET: 1 | 1 | 200: 1 | 2026-07-02T21:39:18.407000Z | 2026-07-02T21:39:18.407000Z |
| PerplexityBot | `/dashboard` | GET: 5 | 5 | 200: 5 | 2026-07-11T20:02:44.141000Z | 2026-07-16T12:02:44.694000Z |
| meta-externalagent | `/api/dashboard/status` | GET: 1 | 1 | 200: 1 | 2026-07-02T03:56:42.361000Z | 2026-07-02T03:56:42.361000Z |
| meta-externalagent | `/share/gui/dashboard-index-html-5d060f57.html` | GET: 2 | 2 | 200: 2 | 2026-06-26T21:31:54.102000Z | 2026-07-03T01:31:50.805000Z |

## Evidence files

- `/home/david/random/data/sol37-crawl-inventory-2026-07-22.json` contains every crawler family, source IP, verification result, status count, and path aggregate.
- `/home/david/random/data/sol37-crawl-inventory-2026-07-22-paths.csv` contains the complete query-free path-level inventory.
- Published CIDR lists were downloaded at generation time; their source timestamps are preserved in the JSON.
- `confirmed-rir-owner` means the user agent matched and the source was in an operator-owned RIR allocation, not a purpose-specific crawler list.
- `confirmed-fcrdns` means reverse DNS used the operator domain and forward DNS resolved back to the source IP.

## Verification references

- OpenAI: `https://openai.com/searchbot.json`, `https://openai.com/gptbot.json`, and `https://openai.com/chatgpt-user.json`
- Google: `https://developers.google.com/static/crawling/ipranges/common-crawlers.json`
- Bing: `https://www.bing.com/toolbox/bingbot.json`
- Perplexity: `https://www.perplexity.ai/perplexitybot.json`
- Amazon: `https://developer.amazon.com/amazonbot/ip-addresses/`
- Anthropic: `https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler` (states that Anthropic does not publish crawler ranges)