This analogy only muddles the picture. We are talking about private property, but not a home, more like a shop or pub.
Its only raison d'etre is to serve certain guests. We are talking about a commercial server that is managed by a professional admin, and is explicitly open to only some of the public, and it already screens and filters guests, based on various criteria. For example, the server may disallow hotlinking of images, serve some content only upon receipt of a HTTP cookie confirming payment and serve different content for mobile devices, some movies blocked in certain countries, based on IP geolocation. The decision is based on HTTP request data and will of the owner, encoded as server's configuration. Ask yourself, how much different is Google News bot?
How about this analogy instead: ``if I hire a professional bouncer for my high-street shop, and the bouncer stops some people from entering and lets others in, that doesn't mean those let in can indeed walk in''?
Which would be crazy, and press would have a field day with such a shop.
Some access control is really easy to implement, other types are not, or they're not cost-effective. It's perfectly reasonable for a content provider to combine technological measures (like e.g. IP geolocation) with legal ones (terms of use, copyright).
That said, you're absolutely right: how hard can it be to block the Google News bot for a news organization? Not very. My point is merely that you can't draw any legal or moral conclusions from a 200 OK response, that's absurd.
To riff on your example, if my high-street shop has a sign that says "only people over 18" but doesn't have a bouncer, does that mean people under 18 should feel free to enter because nobody's physically stopping them?
Its only raison d'etre is to serve certain guests. We are talking about a commercial server that is managed by a professional admin, and is explicitly open to only some of the public, and it already screens and filters guests, based on various criteria. For example, the server may disallow hotlinking of images, serve some content only upon receipt of a HTTP cookie confirming payment and serve different content for mobile devices, some movies blocked in certain countries, based on IP geolocation. The decision is based on HTTP request data and will of the owner, encoded as server's configuration. Ask yourself, how much different is Google News bot?
How about this analogy instead: ``if I hire a professional bouncer for my high-street shop, and the bouncer stops some people from entering and lets others in, that doesn't mean those let in can indeed walk in''?
Which would be crazy, and press would have a field day with such a shop.