robotstxt-ai

Indexed by io.github.sharozdawa

About

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

Use this server

Add it to your MCP client. The catalogue lists this config verbatim from its source — it does not run, download, or vouch for the server. Review the command before you run it.

Transportstdio (runs a local command)

Claude Code

claude mcp add robotstxt-ai -- npx robotstxt-ai-mcp

Cursor

Add to Cursor

If the button doesn't open Cursor, use the JSON below — Cursor accepts the same mcpServers config.

VS Code

code --add-mcp '{"name":"robotstxt-ai","command":"npx","args":["robotstxt-ai-mcp"]}'

JSON

{
  "mcpServers": {
    "robotstxt-ai": {
      "command": "npx",
      "args": [
        "robotstxt-ai-mcp"
      ]
    }
  }
}