This rather bluntly runs up against the fact that permitting crawling is an expe... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		ToucanLoucan 8 months ago \| parent \| context \| favorite \| on: Scraperr – A Self Hosted Webscraper This rather bluntly runs up against the fact that permitting crawling is an expense the web operator is taking on, ergo, receiving that content is by definition a privilege not a right.

p3rls 8 months ago [–]

I think my team has spent about a quarter of this year's dev time on defending against scrapers. Their arrogance is the stuff of ancient greek dramas.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact