# Site discovery

> Fetch a site's agent-facing files (robots.txt, llms.txt, sitemap, well-known MCP and agent cards, OpenAPI, security.txt) and say what was found.

`site_discovery`

Example: Discover agent files for example.com

MCP: https://botthru.com/mcp?tools=site_discovery

Alias: https://botthru.com/mcp/site_discovery

```json
{
  "mcpServers": {
    "botthru-site_discovery": {
      "url": "https://botthru.com/mcp?tools=site_discovery",
      "transport": "streamable-http"
    }
  }
}
```

REST: `GET https://botthru.com/v1/tools/site_discovery?domain_or_url=example.com`
