If you have work happening in a part of your latent space that's got a much lower representation in your dataset then it's pretty plausible to include it. It doesn't actually matter who the user is if there's not a lot of people in the world working on problem X and you have a dataset of work on problem X.
If you don't trust the labs directly, you can always use AWS Bedrock or Azure Foundry which should have a much stronger incentive to not train. They make money from asset rental, not selling models. I'd be shocked if they were training.
That's exactly the argument of the people calling it plagiarism machines. No-one ever really did refute it there was just a bunch of settlements for elite institutions so they weren't left empty handed like the various small time creators/authors etc were.
I think the bigger issue here is this feels like some PR smoothing happening that after all the work that went into "it's safe to use for enterprises" now we have what looks like openAI using private user data to scoop novel research and the question of why couldn't they do it for an enterprise with much more money on the line.
> now we have what looks like openAI using private user data to scoop novel research
Is there any actual evidence of that? All I've seen so far are empty accusations because "it would be in their interests" or whatever. Personally I'm inclined to believe that they honor their terms until it's demonstrated otherwise.
If the work done is just "we made other people's work searchable without their consent" it's not quite the same as what they're implying in the marketing of "our model solved this problem".
A time posted sort is an algorithm too. But that's obviously not what's meant here even if the "gotcha" is there for that too. It's the combination of profitability motivations and manipulation of dopamine responses that are the outcomes people object to.
The vast majority of projects don't care what IDE people use either. But the difference is to use AI well you need context files and other artifacts in the project. So now the project is having to support the additional markdowns and make sure those are still working for Claude/cursor ecosystems etc.
Well, he dismisses any value whatsoever to GenAI. That's immediate bozo bit criteria to me. And, well, if Anthropic revenue doesn't grow 5x between now and the end of the decade, I'll be pretty surprised. But, sure, if it doesn't, then someone will keep them around anyway. AMD almost died in the 2010s as one example, but they kept getting propped up and now they're back in the game swinging. There are people who can see alpha beyond the next 10Q. Ed Zitron isn't that sort.
> Well, he dismisses any value whatsoever to GenAI.
I didn’t read it that way. I see a lot of value in it.
I just don’t see us justifying the amount of infrastructure being built or current valuations. Or in the unlikely event that we do, the societal upheaval is going to take away the ability to monetize it meaningfully.
OpenAI and Anthropic may make it through. But that is different from saying valuations are justified or that all this infrastructure will pay off.
"Those excited about generative AI are either the victim or the perpetrator of a con centered around a technology to ingratiate at the highest cost possible."
How else would you read the above statement? He's just preaching to his own choir IMO.
My take: like any gold rush, a lot of dumb ideas will get backed and they will all fail. And then we'll keep the ones that worked. SSND. Good luck picking the winners a priori.
I read it in context as being about the market prospects of genai.
The problem is, when there is so much overinvestment, everything gets wrecked. In the aftermath of the dotcom boom there was at least a bedrock of fiber and still useful equipment to build upon amid the rubble. This time we are going so much further; also many of the durable assets are misplaced bets and the depreciating ones will depreciate more steeply.
Someone should do the analysis of a decade and a half of Nvidia datacenter GPUs from Fermi to Kepler to Maxwell to Pascal to Volta to (Turing) to Ampere to Hopper to Blackwell and generate some hard depreciation numbers. Fiddling around a bit, 16-20% annual depreciation (so 5-6 years total and then any further revenue is bonus goods) it would appear, but that's a fiddle number.
But confounding this, K80s and V100s are still offered by cloud providers 13 and 9 years after their releases and academia still loves their GTX 1080 Pascals in their desktops. At companies, the beancounters take a computation and find the best architecture !/$ for that calculation. It does not need to be brand new shiny. It's Nvidia's job to make that case, not them. But anyway, the real data is right there. And those old GPUs demonstrate the dark fiber is already in place (and it's not so dark or they'd pull their racks).
AI is the special case. New GPU generations are the only way to access HW implementations of last year's research on precision modes and matrix math. If that slows down, that would be the first real bellwether of a slowdown. It hasn't happened yet. I'm a little surprised myself, but I also think coding agents are the vanguard of general design agents and that's going to hit a lot of industries at once.
So as long as the next generation of GPU halves the price of tokens and doubles throughput (or better), the demand for tokens will continue to rise IMO.
What I don't think is that AI can come for anyone's job successfully no matter what the C-suite sorts insist.
In summary, if you're a bear, you can point to the depreciation cycle and scream the sky is falling. And if you're a bull you can point to GPUs staying in production for a very long time despite the depreciation. Guess we have to wait for 2030.
Sure, according to an unnamed "GenAI principal at Alphabet" of whom "We could not verify the name of the person who describes themselves as 'GenAI principal architect at Alphabet' and therefore we cannot 100% trust their claims."
But let's run with Deep Layer(tm)'s hot take from 2024, GPUs all die in 2 years, no exceptions*. Poor guy just spent $400,000 on a DGX with 8 B200s, each of those B200s generates a piddly ~$3,000 in profit monthly spewing tokens, netting $576K in 2 years, that's a pathetic 20% annual return. Oh no... Won't someone call Michael Burry!
*Never mind the 3-year warranty or any extended service contract, that GPU is D E D and you're S O L.
You gotta include 50k+ in power plus other expenses. Still looks like an ok return on capital, but you are betting heavily that cost of computing doesn’t fall much.
If token prices fall a bunch then it may not even be worth leaving on, depending on your facility’s relative power, cooling costs.
If we push far into oversupply eventually a bunch of firms building this infrastructure are going to lose out.
Chinese providers realized that LLMs have peaked and have started trying to reduce the price per token. Deepseek pro v4 can easily add tests to my complicated code and costs cents for a million tokens.
I can ask Claude or ChatGPT architecture questions and then use Deepseek for the rest.
How are these businesses going to pay to price of energy and GPU depreciation again?
I love nonsense like this. If using a larger model to plan a chain of thought task for a smaller model works, what makes you think Anthropic can't do the same thing and offer it as one of their effort level settings? It reminds me of all those fallen AI ASIC startups insisting they can crush Nvidia until they found out the hard way that the rules of the game are dynamic.
The real challenge IMO is whether enterprise will want to run the models on-site for 100% security and privacy, but even then, what stops Anthropic from offering such an option on-prem or in the cloud?
He implies $400 billion in revenue by the end of 2029 is unrealistic when in fact it's very doable if you look at the trajectory of this technology since ChatGPT 4.0 launch. Google and Meta bring in around $500 billion in ad revenue between two of them annually. ChatGPT will easily bring 100s of billions in ad revenue if fully monetized given 1. it has billion weekly active users 2. ChatGPT conversation provides even better context for ad targeting vs search or social media. Enterprise AI revenue is going through the roof already, and with computer use companies will literally be able to fire large percentage of white collar workers and replace them with AI agent without updating their software infra.
Does that '100s of billions' come from a big bucket somewhere called 'spare cash', or does it correlate to a commensurate reduction in the 'around $500 billion in ad revenue' that Google and Meta are extracting?
Do your assumptions - " if you look at the trajectory " - factor in a slowing economy, a slowing growth in quality improvements in the tech, and/or the asymptote of market saturation for punters happy to stump up more than $50 a month?
What about a few hundred billion in salary and benefits reductions due to mass layoffs?
Not saying this would be good (qualitatively) or even good business in any sense, but we’ve already seen companies willing to sacrifice headcount to cover CAPEX for these models.
A few hundred billion in salary and benefits reductions equates to millions of layoffs. At minimum, we'd be looking at something about the same magnitude as the 2008 financial crisis. That scale of workforce reduction would have profound implications for the broader economy.
In a consumption-driven economy, businesses need consumers. Any gains from these layoffs would be short term at best.
I took that as exactly his point. Learning the syntax of Python or C++ or whatever isn't particularly hard. If reading the manual or being the one guy in the company who knew how the yamls worked was your "moat" then your career was built on sand. The challenge has always been leveraging that use of the tool into actually doing something useful with it.
reply