Soon, quick and dirty solution: User-Agent: \* Disallow: / | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		DataDaemon 7 months ago \| parent \| context \| favorite \| on: Google is burying the web alive Soon, quick and dirty solution: User-Agent: * Disallow: /

cr125rider 7 months ago [–]

So then the internet police know to zoom up the fiber line and arrest the user-agent for not following robots.txt!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact