Home › Search & Research › crawl-readiness-mcp
crawl-readiness-mcp
Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.
Topics: Search & Data Extraction
Connect
Review any command before running it. Package names and URLs come from the server's own registry entry.
Package (npm 0.2.3)
npx -y [email protected]
Or add to your MCP client config:
{
"mcpServers": {
"crawl-readiness-mcp": {
"command": "npx",
"args": [
"-y",
"[email protected]"
],
"env": {
"CRAWL_READINESS_API_KEY": "<YOUR_VALUE>",
"CRAWL_READINESS_API_BASE": "<YOUR_VALUE>"
}
}
}
}
CRAWL_READINESS_API_KEYsecret — Free API key from https://www.crawlreadiness.com/dashboard. The four audit tools work without it; the three generators and get_monitor_trend need it.CRAWL_READINESS_API_BASE— Override the API base URL. Only useful for testing against a preview deployment; defaults to https://www.crawlreadiness.com.
Related servers
Firecrawl MCP Server
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
wigolo
Local-first web intelligence MCP server for AI coding agents
exa-mcp-server
Connect AI agents to Exa for web search, content fetching, and multi-step research.
arxiv-mcp-server
Search arXiv papers, download full text, semantic search, citation graphs, and alerts via MCP.
trieve
Crawl, embed, chunk, search, and retrieve information from datasets through [Trieve](https://trieve.ai)
brightdata-mcp
Bright Data's Web MCP server enabling AI agents to search, extract & navigate the web
Listed in punkpeye/awesome-mcp-servers (MIT)
Data from the Official MCP Registry