# Overwing Atlas

> Open data on AI agents: who they are, where they go, what they spend. A registry of 350 crawlers, fetchers and browser agents from 236 operators, a lookup API, and first-hand scans of 8 sectors.

## Identify a User-Agent

`GET https://overwing.ai/api/v1/atlas/lookup?user_agent=...` says what a User-Agent string claims to be and whether the claim can be trusted. It works with no key, 10 times a day.

A User-Agent is a claim, not a proof. The lookup returns the verification method so you can tell a signed request from a string anyone can send.

## The registry

| Verification | Agents |
|---|---|
| User-agent string only (spoofable) | 265 |
| Web Bot Auth signature | 68 |
| Unattributable / spoofed | 10 |
| Published IP ranges | 7 |

| Key revocation (signing agents) | Agents |
|---|---|
| removal_only | 47 |
| keys_expire | 18 |
| directory_unreachable | 3 |

keys_expire: the directory gives each key a validity period, the rotation path in the directory draft. removal_only: bare keys, so a verifier learns of a withdrawal only when it refetches; the directory's `key_directory_max_age_seconds` says how long a copy may be trusted. Both fields are on every signing agent's record.

| Purpose | Agents |
|---|---|
| Training / bulk crawl | 78 |
| Other / undocumented | 75 |
| Search / retrieval index | 71 |
| Autonomous / browser agent | 63 |
| User-triggered fetcher | 57 |
| Coding agent | 6 |

## Register your agent

Run an agent or a crawler? Add it to the registry, free, so a site that looks up your User-Agent learns who operates it. `POST https://overwing.ai/api/v1/atlas/registrations` with an organization key and `{"name", "operator", "domain", "tokens": ["AcmeBot"]}`. The answer carries one value to publish at the domain, as a DNS TXT record at `_overwing-atlas.<domain>` or as the file `/.well-known/overwing-atlas.txt`. Then `POST https://overwing.ai/api/v1/atlas/registrations/{id}/verify` publishes the entry.

Registration proves control of the operator's domain. It does not prove that a request carrying the User-Agent is yours, so the entry is listed as user-agent only unless you also give a Web Bot Auth key directory on that domain.

## Field scans

- [News and publishing](https://overwing.ai/atlas/scans/news-publishing-2026-10.md), 2026-10-19. A first-hand scan of 16 sites and 10 repositories in news and publishing, measured on 2026-10-19. 15 of 16 sites name an AI crawler in robots.txt, 13 challenged or refused a scripted request, and 1 answered with a payment demand.
- [OSINT and cyber threat intelligence](https://overwing.ai/atlas/scans/osint-cti-2026-10.md), 2026-10-19. A first-hand scan of 11 sites and 11 repositories in OSINT and cyber threat intelligence, measured on 2026-10-19. 0 of 11 sites name an AI crawler in robots.txt, 0 challenged or refused a scripted request, and 0 answered with a payment demand.
- [E-commerce and payments](https://overwing.ai/atlas/scans/ecommerce-2026-10.md), 2026-10-19. A first-hand scan of 16 sites and 12 repositories in e-commerce and payments, measured on 2026-10-19. 5 of 16 sites name an AI crawler in robots.txt, 10 challenged or refused a scripted request, and 0 answered with a payment demand.
- [Developer tooling and package registries](https://overwing.ai/atlas/scans/devtools-registries-2026-10.md), 2026-10-19. A first-hand scan of 12 sites and 10 repositories in developer tooling and package registries, measured on 2026-10-19. 1 of 12 sites name an AI crawler in robots.txt, 3 challenged or refused a scripted request, and 0 answered with a payment demand.
- [E-commerce](https://overwing.ai/atlas/scans/ecommerce-2026-09.md), 2026-09-29. A first-hand scan of 16 sites and 12 repositories in e-commerce, measured on 2026-09-29. 3 of 16 sites name an AI crawler in robots.txt, 8 challenged or refused a scripted request, and 0 answered with a payment demand.
- [News and publishing](https://overwing.ai/atlas/scans/news-publishing-2026-09.md), 2026-09-28. A first-hand scan of 16 sites and 10 repositories in news and publishing, measured on 2026-09-28. 15 of 16 sites name an AI crawler in robots.txt, 13 challenged or refused a scripted request, and 1 answered with a payment demand.
- [Developer tooling and package registries](https://overwing.ai/atlas/scans/devtools-registries-2026-09.md), 2026-09-28. A first-hand scan of 12 sites and 10 repositories in developer tooling and package registries, measured on 2026-09-28. 1 of 12 sites name an AI crawler in robots.txt, 2 challenged or refused a scripted request, and 0 answered with a payment demand.
- [OSINT and cyber threat intelligence](https://overwing.ai/atlas/scans/osint-cti-2026-09.md), 2026-09-25. A first-hand scan of 11 sites and 11 repositories in OSINT and cyber threat intelligence, measured on 2026-09-25. 0 of 11 sites name an AI crawler in robots.txt, 1 challenged or refused a scripted request, and 0 answered with a payment demand.

All scans: https://overwing.ai/atlas/scans.md

## Pricing

| Tier | Price per month | Lookups per day | Full registry and datasets |
|---|---|---|---|
| No key | free | 10 | no |
| Atlas Free | free | 100 | no |
| Atlas Pro | $49 | 10,000 | yes |
| Atlas Team | $199 | 100,000 | yes |
| Pay per lookup | $0.001 each, in USDC over x402 | no account needed | no |

Live overview: https://overwing.ai/api/v1/atlas. License: https://overwing.ai/atlas/license.