But the API is HTTP and the news stories are syndicated across a hundred different sites. How do you limit the crawlers under this scheme? It seems like any serious attempt to limit crawling will require major software redeployment, cooperation of crawlers, widespread authentication, or some combination of these. Is there actually a feasible way to do this without breaking the web?
These are all good points. I don't have answers to any of them. Feasibility is a whole other issue.
My point is that if you look at online newspapers as online services, then they should be able to charge people for programmatic access to their service, just like any other tech service does through its API.
If I want to build an app on the back of Yahoo BOSS, I have to pay Yahoo.
If I want to build an app on the back of the New York Times, maybe I should have to pay the New York Times.
