User-agent: * Disallow: /privacy Disallow: /terms ## block crawlers with no value User-agent: Amazonbot Disallow: / User-agent: MJ12bot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Riddler Disallow: / User-agent: SemrushBot Disallow: / User-agent: Barkrowler Disallow: /