Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Because this is the lowest-hanging fruit imaginable, why not take the 5 minutes to add a robots.txt and then use https://www.bing.com/webmasters/robotstxttester to confirm that Bing can fetch/parse it?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: