User-agent: * Allow: / Disallow: /api/ Disallow: /ops/ Disallow: /account/ Disallow: /admin/ Disallow: /analytics/ # Scout Premium engagement boards: private token URLs (X-Robots-Tag noindex # is on every response too; this stops crawlers fetching a leaked link at all). Disallow: /scout/ # Faceted-navigation control: block crawling of filter/search/pagination # query-string URLs (?areas_required=, ?outcomes=, ?max_bedrooms=, ?page=, ...). # They are ephemeral search state, never a canonical page. Every indexable URL # is CLEAN (verified: 0 param URLs across all sitemaps), and clean URLs are # discovered via the sitemap below (NOT via param links) -- so this stops crawl # waste + self-canonical filter-combo index bloat WITHOUT gating discovery of # any real page. See CLAUDE.md "faceted-navigation". Disallow: /*? Sitemap: https://unitrank.com/sitemap.xml