It was pretty disheartening to hear that only a single scientist quit the Manhattan Project after the Nazi's were defeated. I'm pleasantly surprised that the people working on this seem wiser. He is not the first, and hopefully will not be the last to do this.
I'm not sure what happens (I think no one does), but with every other technology, I can opt-out if I don't want to use it.
Even if I get unemployed, I don't see how I can opt-out when it comes to everyone everywhere using it.
Governments historically have considered humans as assets - it's even called "human capital", and they require us for tax revenue and to run the state. What happens when we aren't needed? Even if the AI is magically aligned to the state, I don't see how I end up the winning side of that. It's not like we're getting a voice in how this is shoved down our throats today. Why would it get any better when AGI can do everything humans can do? What still gives us a voice in the world?
The 'voice' anybody has is some function of their capital, ability to leverage it and ability to prevent its erosion by theft, fraud, seduction/politics scams and honeytrapping. Humans can still acquire capital in the future in several ways, although their ability to not be parted from it is in doubt.
1: verbal/literary labour with your vocal cords, or the use of an output tool, to playback existing or newly created works in environments where mechanical methods are disallowed or disfavored,
2: aesthetic labour through precise recreations of other's mental labour in the performing arts,
3: legal labour in acting as an agent to help or hinder existing legal processes in systems that give humans standing,
4: biological labour as a substrate for growing certain transplantable products,
5: smuggling labour (either non-invasive or surgically invasive),
6: political labour to incrementally sway electoral results in any polity where you are still enfranchised,
7: security labour as a canary for infohazards (such as 'diplomat' programs produced by another ASI) inside of a quarantined environment,
8: frontman work as a deniable patsy to enable economic forgery, deception, revenge or warfare by some hypergraph of patron AGIs and the superorganisms they compose with, or without, your knowledge of what your work entails.
Some of these have a strong factor of supply and demand. You can increase your market value by reducing the supply of other humans competing for these revenue sources.
National governments which can print their own currency don't require people for tax revenue. They use taxes to compel people to acquire their currency to give it value. If they can instead compel AIs to acquire their currency, they won't need humans for that either.
I'm not sure if I'm misunderstanding you, but the purpose of taxes isn't to "compel people to acquire their currency", it's to remove that currency from circulation and prevent the currency they've printed from inflating itself out of existence.
Reducing inflation is indeed one of the reasons, but what I was getting at is that the government doesn't "need" people for tax revenue to pay for its services, contrary to popular belief. It needs taxes to make its money valuable.
L. Randall Wray, "What Are Taxes For? The MMT Approach" (2014):
> Sovereign government does not really need revenue in its own currency in order to spend. It is the tax that 'drives' the currency. If we eliminated the tax, people probably would not immediately abandon use of the currency, but the main driver for its use would be gone.
There are two types of citizens: debtors and creditors. Printing currency pays the debtors. Raising interest rates pays the creditors. Income taxes take from debtors. Property taxes and printing money take from creditors.
I agree and I think uncertainty causes people to miss this. A dystopian world is good for no-one. The shitty things people do to each other are scarcity motivated and call me delusionally optimistic but I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
> I think a world of total and complete abundance doesn't incentivize people at the top (who are there because people at the bottom allow it) to mercy kill everyone else with AI drones.
They will have one of the strongest incentive: to remain at the top. Moreover, it will not be that hard. In Soviet Union one honest socialist with four revolvers (major general Vasily Blokhin) could easily kill up to a thousand people per day personally (if he were laboriously working overtime). And back then there wasn't even a global governmental surveillance or AI-drones with explosives.
> They will have one of the strongest incentive: to remain at the top.
At the top of what? After all the massacres the remaining billionaires and their families would be all that's left of humanity. Who would want to rule over a human-empty, machine-led Earth for the rest of their lives? Not to mention the attitude and morality of their families and those close to them.
I can think of several billionaires who would be quite fine ruling over a devastated and machine-led earth. Their families are chosen or bred to agree with their amorality, and they and their dynastic families would rule with pleasure.
Yes, they can print, but they can also tax capital. It's not actually impossible. We already do it a tiny bit, we've done it more in the past, and contrary to the dogma blasted from the billionaire-owned media outlets and drilled in billionaire-think-tank-approved Econ 101 curricula the universe will not actually explode if we do it again.
We're assuming the oligarchs control the government so they can be rich and powerful and use that to lead lavish lives. Well with these hypothetical AGI imbued robots, they could just construct lavish lives directly.
Not much different from the feudal era. You still need access to good lands (energy production, minerals) but an army of computers and robots instead of serfs and knights. Like Asimov's world of Solaria.
Depends on the political system. Under socialism authority will simply improve the environment by reducing your carbon footprint with eight grams of lead. Under capitalism you're already not needed to anybody.
> Why would it get any better when AGI can do everything humans can do?
Because that "better" we are building ourselves with our own hands. How it can get worse? Only if authority forbid us from building ourselves like we always did and decide to improve the environment.
A major problem with closed mobile platforms is that if you have an App Store monopoly, why do you need to care? It's why all the app stores are absolute trash.
Not sure how Apple handles DMCA but I know they're known for capricious arbitrary actions and being impossible to talk to. Their search and navigation is also total trash.
That hasn’t been my experience. The first couple of layers are sort of “inflexible bureaucrat,” but it doesn’t take long to get to an actual human, with authority to act. In my experience, I’m usually on the phone with someone, within a day or so.
Great idea. I'm thinking if it could make sense to send the output to a cheap / local model to filter out only the bits that "matter" and pass that through - for the cost some extra time, but maybe it's worth it for saving tokens in the larger model.
It has nothing to do with the Industrial Revolution. Put yourself in their shoes. They are fresh entrants to a job market and these companies seem hell-bent on making them useless in it.
Which was a legitimate criticism and took decades of fighting to resolve through organized labor, and protests. There was no 5 day work week or laws against child labor or wage requirements or against nearly any exploitation of employees at its advent.
If AI is like an Industrial Revolution which is yet unproven, but is being marketed as such, then you would expect exactly this reaction.
Believe me, I get it, I get how scary it is, for fresh grads, for people with not a lot of work experience, to be bombarded by this stream of AI-centric talk. Even discounting what the hype men will have you believe, this technology WILL have a huge effect on employemtn in an information services-heavy economy. Many of the grads probably used AI themselves a ton to write their papers, which ironically may have increased their fear of the technology rather than making them see it as a multiplier of their own skills, as they saw first-hand how effective it is at coming up with ideas they struggled to articulate themselves.
Also implicit in my response was the assumption that college grads don’t generally boo their commencement speakers. Perhaps I am wrong, perhaps that’s pretty routine. And yes, I agree these are just platitudes, that have gotten very tired - but isn’t that how most commencement speakers are? Platitudes, feel-good stories, motivational cliches, all coming from people who lucked their way to the top of some pile and thereby qualified to be invited to graduation? To go back to my earlier assumption - surely the vast majority of commencements don’t lead to booing, in spite of how disconnected and distanced most of the speakers probably are?
I am genuinely trying to understand the nature of the polarized reaction to AI. Just from the perspective of its value to me, I feel buoyed by it, and not worried that it’ll depress my personal value, my financial future, etc. And yes, again, believe me, I get it - I get that I have a lot of cushions and privileges that allow me to feel that way. I am by no means imagining everyone else is in the same position.
But I’m really fascinated at how much more negative the reaction to this technology is in the US, vs. elsewhere [1]. Are Americans somehow more perceptive and alive to AI's danger, than, say, the Germans and Indians that seem to be more willing to find something to be “excited" about, with this technology? Others commenting here have said that the speaker's message, that this technology has just made their college debt much harder to work off, is naturally off-putting. Okay, I confess, I guess I just don’t see LLM AIs as representing such a significantly more dangerous threat than so many other factors that have roiled the American economy over the last 2-3 decades. Offshoring, rising costs, stagnant wages, loss of labor power, inequalities compounded by a severely tiered educational system, constant cuts in public investments in social services - these have been secular trends since the 70s. Have commencement speakers been booed all these years, every time they spout some boilerplate language like, “globalization is the new reality, let's get prepared for it,” or, “Finding your own brand is essential to participating in the 21st century economy”?
I think part of the answer lies in this quote from the NYT article [2]: “Earlier in the speech, Caulfield [the speaker] had lost some of the crowd by praising wealthy corporate leaders, including Jeff Bezos. It didn’t go over well. [A fine arts Bachelor’s graduate that day said,] “Using Walt Disney instead of Bezos would have felt generic to me, yet still demonstrated she understood she was speaking to a crowd of artists.”
There’s an aspect here I think not of opposition to the technology per se but what it says about which large-scale social investments seem to matter to those in power. I think it’s somewhat amusing, and telling, that this person thought to reference Disney of all people - someone that by no stretch of the imagination could be described as a model of corporate responsibility or of support for labor - because in this view, Bezos is so diabolical, even Walt Disney, by virtue of his creative talent, is a benign example of corporate success.
What “Bezos” represents is the idea that the financial and political elites would rather gamble the economy on multi-trillion dollar bubbles, that aggressively extract even more natural resources from our shared commons, or send the privileged few to gaze at this planet from the “edges of space,” than figure out how to keep, say, the cost of insulin low. That’s what got booed, I think, more than the what “AI” represents.
All the websites currently blocking Claude Code or other AI agents are fighting a losing battle. Computer-use is in the early stages, and the thing preventing mass-adoption seems to be the number of tokens it takes. Agents can fumble around trying 10 CLI commands that don't work before finding the right one and we barely notice. But other visual agents (browser use / computer use etc) end up eventually fumbling on to the right thing, but we don't have the patience to wait 20 mins. to click a button. As tokens get cheaper + faster, we probably get the models that can use a UI interface just as natively as a CLI.
Tokens cheaper? I don't think that seems to be the case ... VC funded tokens were there to build user base and token price will go up as they eventually switch from growth to profitability.
I wish I could place a lot of money on the opposite side of this bet.
I don't think many realize how could the cheap, alternative models are becoming. I prefer SOTA models for key work, but I can also spend 10X as many tokens on an open model hosted by a non-VC subsidized provider (who is selling at a profit) for tasks that can tolerate slightly less quality.
The situation is only getting better as models improve and data centers get built out.
Yes, but how cheap is it to run four at the same time? It’s tough to run one good model locally, but running four at the same time which I commonly do with Claude and Codex just doesn’t seem to be happening anytime soon.
I'm referring to hosted models such as via OpenRouter or from the model providers' own services.
I think everyone making claims that inference is getting more expensive are unaware that there are more LLM providers than Google, Anthropic, and OpenAI.
Fair - there are bets both ways though I wouldn't consider it to be a certainty. That revenue drive on this AI build out is going to be real and multifold.
It will take a few years until scheduled data center construction finishes, and together with software optimizations that may come up in the meantime, it may cause a significant decrease in token price.
And the lethal trifecta but I suppose that's all agents as of now anyhow. Every AI provider has major warnings about letting AI have access to PII in the browser.
If only General Relativity had such an ironclad defense of being as unfalsifiable as Foom Hypothesis is. We could’ve avoided all of the quantum physics nonsense.
it doesn't mean it's unfalsifiable - it's a prediction about the future so you can falsify it when there's a bound on when it is going to happen. it just means there's little to no warning. I think it's a significant risk to AI progress that it can reach some sort of improvement speed > speed of warning or any threats from AI improvement
