Track them from your server or CDN, because most AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) don't run JavaScript and never show up in Google Analytics. WysLeap does this with a few lines in your middleware or CDN: it records every AI crawler visit by crawler and page, checks each IP against the ranges OpenAI, Perplexity, Google and Apple publish, and drops the fakes. You can also read your raw access logs, or use Cloudflare AI Crawl Control if your site is behind Cloudflare.
Updated
Analytics scripts count a visit when JavaScript runs in a browser. AI crawlers mostly request the HTML and leave, so the script never runs. Google Analytics also filters known bots out on purpose. The visits happen; your analytics just can't see them.
Every request is in your access logs with its user agent and IP address. Search for the crawler names (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and so on). Logs are complete and free, but you're left to parse, verify and chart them yourself, and many hosts keep them for only a couple of weeks.
If your site is proxied through Cloudflare, its AI Crawl Control dashboard shows which AI crawlers visit and lets you allow or block them. It only covers traffic that passes through Cloudflare.
A few lines in your middleware, CDN worker or server can send each AI crawler request to an analytics tool. WysLeap's server-side crawler guide has copy-paste examples for Cloudflare Workers, Next.js and Express. The SEO & GEO dashboard then shows visits by crawler and page, next to the AI-referred visitors those crawlers lead to.
Anyone can send a request that says it's ChatGPT. On one of our own sites in October 2026, 76 of 79 requests calling themselves ChatGPT-User were scanners probing for secret files such as .env. We still see requests named OAI-SearchBot coming from cloud servers outside the ranges OpenAI publishes.
OpenAI, Perplexity, Google, Microsoft and Apple publish the IP ranges their crawlers use, so a request outside them is fake. WysLeap checks every server-reported hit against those lists and drops the ones that don't match. Anthropic, Amazon and Meta publish no lists; for those, a single IP claiming to be crawlers from several companies is a reliable sign of a scanner.
On hosts that serve static files straight from a CDN, requests for files such as robots.txt never reach your middleware or server code, so server-side reporting won't count them. If you want to know whether a crawler checked your robots.txt, look in the CDN's access logs.
robots.txt or by your firewall.Side by side
| Feature | WysLeap | Access logs | Cloudflare AI Crawl Control | GA4 |
|---|---|---|---|---|
| Sees crawlers that don't run JavaScript | With server reporting | |||
| Checks crawler IPs against published ranges | Manual | |||
| Visits by crawler and page | Manual | |||
| Next to your visitor analytics | ||||
| Works on any host | If your host keeps logs | Cloudflare sites only |
FAQ
Other guides
AI crawler visits by crawler and page, checked against published IP ranges.