forked from the-distro/infra
41 lines
520 B
Plaintext
41 lines
520 B
Plaintext
|
AI2Bot
|
||
|
|
Ai2Bot-Dolma
|
||
|
|
Amazonbot
|
||
|
|
anthropic-ai
|
||
|
|
Applebot
|
||
|
|
Applebot-Extended
|
||
|
|
Bytespider
|
||
|
|
CCBot
|
||
|
|
ChatGPT-User
|
||
|
|
Claude-Web
|
||
|
|
ClaudeBot
|
||
|
|
cohere-ai
|
||
|
|
Diffbot
|
||
|
|
FacebookBot
|
||
|
|
facebookexternalhit
|
||
|
|
FriendlyCrawler
|
||
|
|
Google-Extended
|
||
|
|
GoogleOther
|
||
|
|
GoogleOther-Image
|
||
|
|
GoogleOther-Video
|
||
|
|
GPTBot
|
||
|
|
iaskspider/2.0
|
||
|
|
ICC-Crawler
|
||
|
|
ImagesiftBot
|
||
|
|
img2dataset
|
||
|
|
ISSCyberRiskCrawler
|
||
|
|
Kangaroo Bot
|
||
|
|
Meta-ExternalAgent
|
||
|
|
Meta-ExternalFetcher
|
||
|
|
OAI-SearchBot
|
||
|
|
omgili
|
||
|
|
omgilibot
|
||
|
|
PerplexityBot
|
||
|
|
PetalBot
|
||
|
|
Scrapy
|
||
|
|
Sidetrade indexer bot
|
||
|
|
Timpibot
|
||
|
|
VelenPublicWebCrawler
|
||
|
|
Webzio-Extended
|
||
|
|
YouBot
|