Where are you getting this from? Outside of o3, every AI provider's API is super cheap, with most productive queries I do coming in under 2c. We have no reason to believe any of them are selling API requests at a loss. I think <2c per query hardly counts as "quite expensive".
The reasoning people have for them selling API requests at a loss is simply their financial statements. Anthropic burned $3B this year. ChatGPT lost $5B. Microsoft has spent $19B on AI and Google has spent close to $50B. Given that revenue for the market leader ChatGPT is $3.7B, it's safe to say that they're losing massive amounts of money.
These companies are heavily subsidized by investors and their cloud service providers (like Microsoft and Google) in an attempt to gain market share. It might actually work - but this situation, where a product is sold under cost to drum up usage and build market share, with the intent to gain a monopoly and raise prices later on - is sort of the definition of a bubble, and is exactly how the mobile app bubble, the dot-com bubble, and previous AI bubbles have played out.
Not sure if it matters at this point. There will need to be many more rounds of CapEx to realize the promises that have been put forth about these models.
The implication would be that those API requests are being sold at a loss. Amodei wrote in January that Claude 3.5 Sonnet was trained for only a few $10Ms, but Anthropic has been losing billions.
That would be a killer for the current and near future generations of LLM as a business. If they are having to pay many times in compute what they are able to get for the API use (due to open models being near comparable?), then you definitely can't "make up for it in volume".
