User-agent: *
Allow: /
# /dashboard is intentionally NOT Disallow'd — blocking it via robots.txt would
# keep crawlers from seeing the in-page noindex meta, leaving it indexable from
# external anchor text with no snippet. The CSR shell carries
# `noindex, nofollow` in its <head>. /v1/ is the proxied API surface.
Disallow: /v1/
# …except the keyless ortofoto tiles the frontpage hero paints as its photo (its largest
# paint). Without them Googlebot renders the hero with an empty frame. Only the tile path:
# every other /v1/ path, the demo property lookup included, stays disallowed.
Allow: /v1/demo/orto/tile/
# /admin/* is the session-authenticated management API (proxied to grundfast-api). It answers
# unauthenticated crawls with 401, which Search Console surfaces as "Blocked due to unauthorized
# request (401)". It must never be indexed anyway — disallow it so Google stops attempting it.
Disallow: /admin/

# NB: no non-standard directives here on purpose. The `Content-Signal:` line was
# removed because robots.txt validators (incl. Lighthouse SEO) flag any unknown
# directive as invalid, which silently capped the SEO score on every page. AI /
# answer-engine guidance lives in /llms.txt instead — index for search + answer
# engines is the intent, and we want to be discoverable everywhere.
#
# The agent-discovery files are deliberately crawlable (nothing above disallows them):
# /llms.txt, /llms-full.txt, /.well-known/api-catalog, /.well-known/mcp/server-card.json,
# /.well-known/agent-skills/index.json, and every page as Markdown via ?format=md.

# The /ejendom property-page sitemap was retired 2026-08-06: listing ~25 000 near-duplicate
# template pages got the whole domain demoted, and none of them ended up indexed anyway. Those
# pages stayed public and crawlable, carrying `noindex, follow`, until 2026-10-15 (below).
#
# 2026-10-05: a capped, reviewed trial re-admitted a small list of rich property pages.
# 2026-10-15: every property page is indexable. Each one is served from a stored snapshot, and the
# property sitemap (/sitemap-ejendom-<n>.xml, listed by /sitemap-index.xml) lists a property once
# its snapshot exists. Switches: apps/web/src/lib/ejendom-trial-config.ts.
#
# One Sitemap line: the index lists every sitemap (sitemap.xml, the postnummer stage, the API
# reference and the property files), so listing sitemap.xml here as well only made crawlers read it twice.
Sitemap: https://grundfast.dk/sitemap-index.xml