# AI search bots allowed

> Part of [Lensy by Perseverance AI](https://perseveranceai.com/). The index of every page for AI agents is at [https://perseveranceai.com/llms.txt](https://perseveranceai.com/llms.txt).

Last updated: 2026-11-12

AI search check S1, worth 20 points.

## What it measures and why

20 times the share of six AI search and retrieval bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-SearchBot, Claude-User) that robots.txt allows for this path. Most specific group, longest matching rule wins.

These bots fetch pages for ChatGPT search, Perplexity and Claude answers. A page they may not fetch cannot be cited.

Evidence:

- [48 days of AI bot logs (Wislr)](https://www.wislr.com/articles/ai-bot-behavior-log-analysis/)

## How it counts

S1 is one of six AI search checks, worth 20 of the score's 100 points. It gives full, partial or no points, and the AI search score is the sum of the six. A page that can't be reached at all gets no score; the reachability line above both scores says why.

## Check it yourself

Scan the page on the [home page](https://perseveranceai.com/#scan), or run `lensy check <url>` from the [command line tool](https://perseveranceai.com/docs/cli) and look for `S1` in the report. A failed check comes with the fix worth the most points first. `lensy explain S1` prints what it measures, why it matters and the evidence in your terminal, and [every check](https://perseveranceai.com/docs/checks) lists the rest.

[All checks](https://perseveranceai.com/docs/checks)
