Over the last 20 years the economy has become dysfunctional. It no longer really resembles a free market; monopolies have established barriers to entry everywhere. And the biggest investors are awash with helicopter money that's been doled out for favors by the political class.
So those investors have tons of cash to burn and surprisingly few opportunities. Even within Silicon Valley/VC there are surprisingly few who seem to really understand the fundamental economics of software. Or perhaps those economics just aren't that important when you have billions of dollars on hand and cash is obviously not going to get you a return. Any whisper of possible exponential growth is worth throwing money at. Crypto? Why not. AI? Why not. Datacenters? Why not. Tulips? Why not.
This is by all definitions an empire in decline. Everything is broken or fake. Everyone is afraid to do what needs to be done. Power forbids it. So we're all just waiting for the other shoe to drop. Our secret police aren't as bad as the late stage USSR's yet, but hold Uncle Sam's beer...
(That's not a recommendation to try and time the nadir, by the way, as it could easily be 50 years away.)
I feel like my point is being over extrapolated a bit. All I was saying is that people in 2020 believed AI companies had much more of a moat then they actually do - that other companies would struggle to obtain enough data (y’know, downloading the internet to train models wasn’t legal back then) or resources to compete
It started way before the last 20 years, the past 40 years have created the conditions for this to happen.
Overfinancialisation of the economy while removing safeguards to keep markets functional, like anti-trust enforcement, are the main forces behind this erosion. Through finance the focus of companies is completely shifted away from producing good products and services, and into how to extract more paper wealth from existing structures to the detriment of products.
Not enforcing anti-trust and letting behemoths to form which cannot be competed against since with their amount of capital they can either buy their competition outright or just price dump for long enough to make competition non-viable.
It's the failure of neoliberalism, and that agenda has been pushed into Western countries since the 1980s-1990s, it made enormous wealth for the few at the top while eroding whole societies, economically and socially, it's all dysfunctional.
I feel like my point is being over extrapolated a bit. All I was saying is that people in 2020 believed AI companies had much more of a moat then they actually do - that other companies would struggle to obtain enough data (y’know, downloading the internet to train models wasn’t legal back then) or resources to compete
I replied to the extrapolation, not to your comment directly, I think your point might be more fitting under that comment than mine, I just drilled down from the points above mine.
The initial growth of the internet was a boom bust cycle, and as you see the internet is still here. This is more about the economic paradigm that is being used to grow a technology versus the economic paradigm the technologies rate of growth can support over time.
Just to give some rough numbers in the past 5 years the amount of GPU compute installed in FLOPS is somewhere over 5 times all the CPU flops that have ever existed.
This has nothing to do with AI being good or bad or being able to produce things and economic value. It is by far the largest and fastest growth of any technology ever and we have zero clue about the economic stability of this grand experiment we're performing.
The internet is a much more appropriate comparison than crypto or tulips, which bring no or negligible value.
Some differences to the market at that time seems to be that during the dotcom bubble, many of the companies had little to no revenue.
Leading AI labs already generate enormous revenue. The investments into capacity are needed to address the current demand.
The situation seems somewhat less speculative.
That said, I cannot predict how AI capabilities will develop and how demand will respond.
Should capabilities plateau hard and soon, maybe the demand will not be there for the compute investments.
If it does not, and instead AI applications in robotics, science, and self driving expand, chances seem reasonably good that the demand will be there, no?
And as for the economic stability, much of the investment comes from existing giants like Microsoft, Alphabet, Amazon, and Meta, who have the necessary cash flow.
These companies are less likely to collapse than some of the ones during the dotcom bubble.
Eh, I'd say it's closer to something like internet + tulips.
It's the total amount of money in the economy that's been invested toward a potential outcome. AI represents the largest amount of money, and largest fractional part of the economy invested ever.
Because of this AI could be the biggest economic boon ever, yet still not recover the full amount invested. This will have deep economic impacts that affect everyone and everything. At this point AI must achieve all its stated economic impacts or there will still be a huge economic crash that kills off any company that is over invested and cannot make a profit.
Worse, the many of the perceived economic impacts of AI are not for humans like you or I, but the huge companies you listed. Even if they economically win, everyone else made out of meat could still lose.
They say history doesn't repeat, but it does rhyme. This, at least to me, sounds like a mixtape of "internet" + "tulips" + "1920s financial world leading to global political instability".
Every potential outcome I see occurring pushes us closer to further instability, even if the economics on it work out on paper.
Out of this number how much of this goes to payroll that goes to humans versus how much of this goes to non-human infrastructure costs. The numbers suggest this is anywhere from 60 to 90%, with 70% being a reasonable average figure. Just under a trillion dollars paid directly to humans would disappear if AI somehow captured all of this market.
Moreso the AI industry needs to capture these gains very fast or they will be crushed by interest payments on the massive debts they've accrued.
>ven though Amodei hopes to cure most major disease in the next 5-10 years
The average development time for medicine is 10 to 15 years before a single dollar is earned from said medicine. AI will shave very little off that as human testing and our stupidly complex bodies introduce all kinds of problems. Furthermore running head long into blindly using AI medicine is how you produce X risks from super intelligence AI.
The speed at which AI has to develop in which to get profitability is the biggest risk, a potentially catastrophic risk at that.
You can theorize all you like, and many people like yourself are expecting to see GDP growth pick up as a result of AI, maybe if only because of the boom in datacenter construction etc (regardless of whether AI ends up actually boosting the economy), BUT ...
The reality is that so far there has been no sign of GDP growth picking up. It is basically flat at 2.5% +/- over the last few years.
In a similar vein one might have expected that the internet (think of all the e-commerce and efficiencies!) might have shifted GDP growth into a higher gear, but it did not, although in that case there was at least a significant boost in the 1996-2000 "dot com" era when the build out was happening (then to be followed by the crash and all the unused dark fiber etc).
So, maybe the hoped-for AI boom will be just as much of a dud (as it appears to be so far) as the internet boom. New day, different tools, same growth.
It's perhaps odd that we're not even seeing datacenter/etc build out register on GDP, but perhaps the scale of it is not as large as the internet build out?
>but perhaps the scale of it is not as large as the internet build out?
In the past 5 years 5 times as much GPU compute capacity has been installed as total CPU capacity that has ever existed. When it comes to capital intensity the growth of AI has been one of, if not the largest capital expenditure for a class of project in this short of time frame. Things like the Manhattan project which was absolutely monstrous expense at the time is dwarfed by this.
Of course measuring GDP growth is difficult when you have an idiot in chief trying to destroy the global economy with the dumbest set of actions contrived by a human ever. Disentangling this factors will take a lot of work on someones part.
Most of the GPU cost is in the GPUs themselves (and in the space and maintenance costs of the building). Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use.
There is cost, but the cost is mostly the opportunity cost of not being able to do something else.
> Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use.
I don't have any insight on modern GPU datacenters, but in decades past, some owned and operated datacenters didn put effort into making sure power management worked because the cost savings were worth it. I'm pretty sure I saw plans to shed load and power off servers if a utility made a demand response request or in case of loss of cooling. I wouldn't be surprised if some owned and operated data centers do regular full shutdowns at off peak... WOL, IPMI or RTC wakeup can bring them back when needed and if you already have a dynamic service orchestrator and setup times are acceptable, why not shut down if there's no actual priority work and there's also no idle priority opportunistic load either...
Unfortunately “being the same everywhere” only works if your application is distinctive enough in functionality that it can’t be anything else but itself. Firefox is just a browser. Safari is also a browser. Chrome is also a browser. And none of these apps are more important or relevant than the operating system they are working under.
If my OS says windows and tabs close on the left, then I expect every single app to be a good platform citizen and not make me relearn default controls just to operate simple functions. Windows and tabs close on the left. End of story.
The number of options has to be small and bounded. The query planning is more of a search/optimization problem than a classification problem since the number of options increases wildly based on query size.
Also in aviation, but with caveats; you want to take off and land with a headwind, because the headwind gives a greater airspeed which means greater lift.
This is true for takeoffs but not for landings. You want to land with a headwind because this means that for the same airspeed you have a lower groundspeed, i.e. when you actually touch down you're going slower on the runway than if you touched down at the same airspeed but with a tailwind.
No. Your calculated landing speed doesn't change depending on the winds, so you'll always be touching down at roughly the same airspeed for the same aircraft type, weight, and flap setting. Touching down with a headwind just means you don't need to use the brakes as hard.
It's beneficial on takeoff because headwind already factors into your airspeed before you even start rolling, which gives you more lift for the same groundspeed yes, enabling you to rotate sooner than you otherwise would.
It seems like the main disadvantage would be that you have to load all the autocomplete data / model weights on the client-side and your webpage might be CPU/memory limited
> main disadvantage would be that you have to load all the autocomplete data / model weights on the client-side
Not sure what "model weights" you're talking about, but yes, that is the trade-off. Although complete autocomplete data for the entirely of the JavaScript APIs would be what, in an efficient format, easily below 1MB at least.
reply