See what a crawler finds on your page.
Check one public URL. You get four clear answers and concrete next steps.
Fetching and checking the page safely …
The check could not be completed.
Crawlers can access it
Indexing allowed
Main content detectable
What to do next
Technical details
Observed crawler rules
| Crawler | Purpose | robots.txt |
|---|
This check examines only the server response for this one URL. No logged-in content, no JavaScript rendering, and no guarantee of Google indexing or AI citations.
Check the whole website
Use a full technical crawl when you need to find and prioritise problems across many pages.
Open Contextter Site AuditCheck questions and sources in AI search
See which follow-up questions and sources appear in real OpenAI and Gemini queries.
Open AI FanoutAnalyseSpider, Contextter and AI Fanout are operated by Matthias Ramahi and are not independent recommendations.
How the check works
- Check a URLEnter one public URL and start the check.
- Read the crawler responseWe fetch the server response like a simple HTTP crawler and explain it.
- Act on the resultYou get a few concrete pointers instead of an invented overall score.
Crawlers in plain language
What is a crawler, and what can this check actually show?What do you want to check?
What does a crawler find on my URL?Fetch a public URL and check access, indexing and main content.Where does this redirect lead?Build the redirect steps one by one.Can this crawler access my page?Test one robots.txt rule for a bot.What do my server logs show?Find status codes, paths and claimed bots.
Two sections. One clear purpose.
Use a tool when you already have data. Read a guide when you first need to understand what a result means.
Crawlers do different jobs.
Googlebot, Bingbot and AI crawlers are built for different purposes. We show what each operator publishes, how a bot can be checked and when the information was reviewed.
Compare crawlersWhy is the site called AnalyseSpider?
The domain was formerly used for log analysis and IP tools. Today's website is a new implementation under new ownership. Former software, copy, data and customers did not transfer.
Read about the new ownership