It's a cost thing. It costs more to render a website than it does to consume it. When you have some bot traffic mixed in with human traffic, that is fine.

When you have egregious bot traffic, say 10k requests per minute sustained load, it becomes a real problem for webmasters.