Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yes, but given the lack of generic "robot types" (e.g. "allow algorithmic search crawlers, allow archival, deny LLM training crawlers"), neither opt-in nor opt-out seems like a particularly great option in an age where new crawlers are appearing rapidly (and often, such as here, are announced only after the fact).


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: