Robots.txt generator for blocking AI crawlers

비슷한 프로덕트

설명이 비슷한 순

1. A bot-free alternative to Otter.ai

   bluedothq.com

   HNHacker News5점

2. CockroachCrawler–1 package for crawling,browsers,PDF,extraction,and MCP

   github.com

   Give AI agents bounded eyes on the public web: crawl, search, and normalize evidence without an unrestricted browser.

   HNHacker News5점

3. A Node.js CLI tool to generate ai.txt

   npmjs.com

   llms.txt, robots.txt, humans.txt

   HNHacker News7점

4. GuardRail

   github.com

   shell guards that stop Claude Code before it pushes to main

   HNHacker News5점

5. I built an SDK that scrambles HTML so scrapers get garbage

   obscrd.dev

   Built this because I got tired of AI crawlers reading my HTML in plain text while robots.txt did nothing.

   HNHacker News16점