It would be good to also have a non-LLM version of the pitch, for some people building in the space it's a bad signal. It's associated with zero value "let my agent build something over the weekend" projects that all make these grandiose claims.
It's ironic how AI companies are re-inventing their own form of privately enforced copyright, lobby the government to ban foreign competitors that don't respect it etc., all while spending the last 5 years fighting tooth and nail against the copyright of the training material they're using.
If you can take any book and turn it into a model, because it's "transformative enough", and "AI learns just like a person does", then surely a model distilling another model is transformative and fair use.
They tied themselves into knots fighting the letter of the law, and now, when they need the spirit of the law - that each creator deserves protection for their work - now we devolve to the law of the jungle. Maybe we'll even see LLM book curses, the way medieval scribes damned book thieves to blindness and worms.
It's one of the things you need to do if you want your company to later become the only company in the world. They also promised that they would treat you nicely afterwards after they get what they want.
It does seem like a lot of the valuations and infrastructure investments for these companies only make sense if each one assumes they will be the first and only one to invent superintelligence and that it will largely replace all knowledge work.
i keep hearing this trope repeated ad nauseam cos people put no thought into it.
you only need people's money if you can't control their labor.
did a plantation owner in the 1600s need the money of his slaves ?
if you control a robot that can fight and take over land, enslave workers for things robots are bad at, harvest food, create buildings etc. what do you need other people for ?
your entertainment? robots can do that too.
(Source, worked for multiple tech companies that were laser focused on delighting customers until money people came in and ruined it, to the point they would prefer devs sit idle than work on things the MBAs didn’t have on a priority list).
what does having engineering credentials have to do with being a sociopath?
Sure, the guys focused on tech don't usually think with an empathy of a nurse (novadays not even nurses are guaranteed to be empathetic), but I know quite a few devs and IT people who are extremely pro-social.
It's the managerial, financial-oriented mindset that is the greatest predictor of sociopathic character.
The more power one has in an organization the more socio- and psycho-pathic they tend to be, simply because it's easier to get to the top if you are an amoral person, who is not holding back for any reason other than having power. That is why absolute power corrupts absolutely.
MBAs are bred to be sociopathic money-grabbers - I know that from experience.
I once made the mistake of going into an MBA program. On the very first lecture the lecturer asked each participant for his/her motivation for being there.
Most of them said 'money' and those who didn't were asked again until they caved-in and said 'money' as well or they were laughed out by the lecturer.
MBAs are expected to be sociopathic or you as a company shareholder won't be able to motivate them easily to do your bidding.
It's like public/private schooling, but worse - those programs are designed to create mindless drones for the institutions that use them for their own policy enforcement.
I am pretty sure that the goal of these companies at the end is ruling more than what governments can.
Just look at the pattern... they collude, they provide to them whatever is needed, and at some point, if this is not true yet, they will be the ones who will tell them what to do or not, bc you know how humans are, right... blackmailing, mess up the business or shames of people, etc.
It is just a matter of time. The state, as we know it, will collapse or will be greatly reduced, which, from a point of view, is positive, from another, Idk, bc if someone replaces that, we will be in the hands of someone, as usual...
"Monopoly on the lawfully usage of force" + "regular and peaceful change of government" can go a very long way against "corporations governing the world".
I'm curious if we'll ever get a form of civil war where corporate militias (corporate _robot_ militias in the case of xAI, of course) draw fire on good old meat policemen coming to bring the CEO to court.
Sure, it may happen, but I suspect the alternatives (bribing, sending a scapegoat to jail, buying the elections, etc... and focus on the "making money" part) will stay preferable for a while.
The monopoly on the use of force has gotten kinda blurry as well these days. As long as people keep electing anti-government governments, the risk of technocracy is very real.
Unfortunately this will keep happening in capitalist systems without hard wealth caps, because capitalists would rather risk the extermination of entire groups of people than risking their wealth, so they will inevitably keep spreading & stoking fascist ideas.
"For the general election of 5 March 1933, the Nazis were allied with other nationalist and conservative factions. At a secret meeting on 20 February, major German industrialists had agreed to finance the Nazis' election campaign."
Ah yes, it's obviously acceptable that the richest people on this planet are deliberately and explicitly trying to pull us into fascism, because every single alternative must be even worse, because SOCIALISM!
Good thing you reminded us, otherwise we might've had critical thoughts about the current system.
I can’t imagine any agency in their right mind training an LLM on their top secret data… especially considering that essentially it would collapse all need to know data in one easily leakable single access domain. The models have no feasible or reasonable way to implement rbac. So you would end up in a situation where people who need to know who killed A, also know who killed B, not to mention cell A knowing potentially data on Cell B who is meant to watch them, etc, etc.
The idea is silly because most of it is complete garbage anyway. You'd wind up with a model that might know a very small amount of actionable information, and the rest of it just speculative. Or extremely time sensitive, useful for only a relatively short slice of time (which has already passed).
> it would collapse all need to know data in one easily leakable single access domain. The models have no feasible or reasonable way to implement rbac
Oh my sweet summer child...
This has been the continual goal of the US defense and intelligence agencies since 9/11, which was blamed on a lack of information sharing. Why do you think Edward Snowden had access to everything? Because it was consolidated post-2001.
You think there is any access control at the upper levels of the dark intelligence agencies? They have access to the whole take.
And it used to be that was pretty useless unless you had an actual lead. You'd need a East German Stasi level of labor to read everyone's secrets at scale... Now you don't, thanks to AI.
Snowden had mass access to a mass surveillance program, not mass access to N dark programs or particular bits of high value intelligence. The thing you are describing is primarily around sharing threat intelligence or in other words, spying on us plebes.
High value target intelligence and other bits of fun are still kept quite seperate.
Did we not watch every single supposedly powerful US tech CEO immediately drop in supplication and kiss Trump's ring as soon as he became president? And when Jack Ma got mouthy, Xi locked him in a basement for 3 months and then kept him in exile for another 5 years.
> Did we not watch every single supposedly powerful US tech CEO immediately drop in supplication and kiss Trump's ring as soon as he became president?
Why fight someone who is so open to corruption? The important thing is that they got everything they wanted, and all they had to do was throw a few million at the clown, attend his parties, and maybe take down some diversity programs they didn't believe in anyway.
> Did we not watch every single supposedly powerful US tech CEO immediately drop in supplication and kiss Trump's ring as soon as he became president?
Because they know it will work. You don’t have to watch Trump speak for very long to know he’s not very intelligent. Similarly, you don’t have to be much smarter than him to know how easily he can be manipulated with adulation.
> It is just a matter of time. The state, as we know it, will collapse or will be greatly reduced
Doubt.
The networking between people in power at the top will, in my opinion, most likely assure that they remain in power, because at the end of the day that is what they want most.
I expect government will seize control of the most advanced models (if they haven't already), and the rest of us will be throttled, and status quo will be maintained. I do not expect that either super advanced agents or the owners of the hardware they run on will be able to pull off the kind of coup you describe. At all.
"We have every location associated with you in the crosshairs of tamper proof missiles and surrounded by infantry with heavy vehicles. Do what we say or we will destroy you and everything you care about."
And that's just the most obvious one. There are many other kinds of threats which are much more subtle.
That is how governments traditionally stay in charge, and I expect the pattern to continue. Citizens simply don't have access to the levels of destruction the state is capable of, by design. The common people can have all the advanced technology they want, so long as it doesn't threaten our leaders' monopoly on violence.
Not only force is a threat. When someone knows that the death of the rival is equivalent to ruining their own status, no matter how much additional force they have, things get much more balanced than you might think.
In that situation, it depends how much physical force the government has to compel the tech leader, and how much physical force the tech leader has to counter it.
Usually, the number of bodies who will comply is the proxy for power, but with automation, it could be different in the near future.
But it can clearly be seen that the power will not be as monopolistic as before indeed. The rebalance leans towards losing power for states vs tech in part.
Regulation alone cannot control if it cannot be made effective.
You think all those lawyers and bar tenders and whatever MTG was serving in Congress are out there tilling fields and sewing shirts? Nope. Without that tech they are f...
"They"
Who are you talking about? Because if you haven't noticed the people in charge now will die.
Most of the Fortune 500 of the 1900s have been gone for decades. The olds are vastly outnumbered by youth who either hate them or wouldn't mind just getting to beat someone who can't fight back up
With measles and food contamination going wild now it's just as likely the rich and pols kill themselves.
There is no beating physics and Elon, Thiel, and such rely on a lot of people to do the work for them. If society implodes all bets are off. Their wealth is coupled to legitimacy of America. Those body guards aren't going to abandon families for a broke former billionaire
There's no upside for the rich is things implode or the divide is not believed to exist; they become common rabble and get a pick axe too
The rich and politicians are just stupid meat suits too. They misread a room and screw up alot
I appreciate its not helpful but its as helpful as imagining some absurd conspiratorial future. In the same vein I could ask if you're the CIA trying to suppress earnest conversation. It just does nothing but hand wring about a bunch of imagined nonsense.
I think the perceived situation is a matter of degree. I would say there's a measurable (almost overwhelming) amount of corporate influence already.
I have no doubt there are agency operatives influencing discussion online, from recorded precedence. Calling the concept 'nonsense' is overly dismissive. Regardless, using that kind of claim to deflect from the premise isn't a compelling reason to invalidate the concept. Ideas can stand on their own.
Yes - yours is just some optional text nobody reads or understands and is probably not legally required to adhere to. Theirs is a contract signed by their customer who they know did understand it.
Accessing the site may already mean that you agree to TOS. Also if you don’t see an explicit copyright terms on some text on the internet it doesn’t mean that it’s public domain. Same as checking a checkbox. Text being small and somewhere is not an excuse for a corporation to steal and sell other people’s work.
I know that text isn't public domain by default. I'm not talking about pirating IP. I'm talking about reading a website - which is a right that supersedes copyright - even using a computer that learns from it without storing a copy of it or distributing it.
No, you don't need to agree to TOS. I'm sure you never actually agree to them and you're not going to jail for it. If you believe that, I hope you never visited cnn.com, for example. Their Terms of Use is 11,000 words. Are you sure you have agreed to that when you clicked on some random news link? Are you sure you're happy to to surrender your legal defense and its expenses to them if they make a claim against you? You're OK that they have no liability for sharing your PII when they're not authorized to? You won't complain if you pay for a subscription and they don't give you access to the subscription content, don't refund your payment, and don't even tell you why?
That is why every ToS is meaningless unless you can somehow put a gun behind them as the ultimate enforcement with the courts, lawyers and the whole legal system as a fig-leaf-intermediary.
> what makes theirs not “optional text nobody reads or understands”?
You accept it. You pay consideration for it. If your website has a TOS dickover, that requires someone attest with their legal name and pay you $1, yes, it may be enforceable under some circumstances.
Your TOS matters insofar as you can prove a person actually read and agreed to it. These are illegal in different ways:
1. Copyright violations (can put you in jail)
2. TOS violations (will be a fine at worst)
Companies do get away with drive-by legal shittiness way too often and frankly the practice needs to be reined in, but at the end of the day the only damages are the financial ones you can prove in court.
Incidentally, this suggests that once an LLM is capable of accessing and distilling a competitor's LLM without human intervention, then any legal argument about TOS violation is moot. But somehow I doubt that will fly in court.
So what you are saying is that, if I can somehow get my hands on a copy of Fable, it's fair use to use it to train any models and serve those, since I'm no longer bound by the TOS of the service provider?
Damn now im looking forward to the day when books end up like physical game disks, where somehow you're not buying the book just a license to it, what a boring dystopia this is lol
Common for college textbooks to have some digital-only component accessed with a one-time key inside the cover. Sometimes time-limited to a single semester.
Pick up nearly any published book. Turn to the ~3rd page. There will be either a whole page, or sometimes the second half of a page, dedicated to a copyright notice. Very nearly every published book I've ever seen has that identical page. This isn't a recent thing. I grabbed my copy of Diaspora by Greg Egan and opposite the table of contents is a page that starts like this:
Copyright (C) 1998, 2015 by Greg Egan
First Night Shade Books edition 2015
All rights reserved. No part of this book may be reproduced in any manner without the express written consent of the publisher, blah blah (it felt very ironic to transcribe that bit in particular to make this point)
If you read the whole copyright page of a book and come away with the conclusion that it's anything but terms of use for the book, then we'll just have to agree to disagree.
You do not have a contract with the publishers of the book. They are visibly asserting their copyright to deter any defense of ignorance or implied grant of rights to an infringer; but that's not a contract, you did not agree to it before purchasing, there are no contractual terms (scope, duration, faults and compensation, resolution etc.) and nothing in it exceeds the limits the copyright law already sets.
For example, never will you see printed in a book something like "this book is for the exclusive use of the purchaser and you cannot lend, resale or otherwise make available to other parties" - if such a thing was possible, like most software EULAs do, publishers would be all over it.
So far, we have one ruling that says "model distillation by vendor A from vendor B with the intent to use the results to compete with vendor B in vendor B's domain is not fair use". Which makes a degree of sense.
It's possible that distillation for other reasons, with no intent to harm the vendor you distill from, would have been ruled to be fair use. But in law, intent matters.
What was the intent of the original ai companies (anthropic, OpenAI, etc) when they mass-distilled the entire internet to create their training data set?
One could make arguments for OpenAI and Anthropic. But Google Search displays AI results above the SERP - clearly in competition with them. No premise or excuse there.
On the one hand, yes; on the other hand, so much of the training data comes from scraping the web that it feels wrong for them to do what they deny others the right to do.
On the third hand, the settlement Anthropic famously had to pay was for copyright infringement because they didn't actually have the right to even access some of the training data they used, so I can see how this might be compatible with the law.
On the fourth hand, I'm saying that as someone who absolutely isn't a lawyer and sometimes gets surprised when reading about copyright cases that sure sound like they ought to have been trademark cases given my limited understanding.
Companies that didn't give away all their content for free to anyone have actually denied AI companies from training on all their data without paying a fee. Reddit, Associated Press, etc.
For those who chose to give it all away, the ship has sailed, but they did choose to give it away for free to anyone so they can't complain that they succeeded.
How exactly other websites “gave it all away”? Also examples you list are websites putting some explicit rule eg in their robots.txt or filtering web crawlers. This is all a reaction to existing situation, so Reddit for sure has been scrapped before Reddit realised what was happening.
If the AI company bought their book, then they didn't give it all away. If the AI company obtained it indirectly like a library or 2nd hand, then the author has already been paid when he first sold it. In either case, he could have refused to be so liberal in sharing it if he didn't want it to be used like that, but he preferred to make some money instead.
If buying one copy of a book entitles the ai company to train on that data and redistribute information derived from it in perpetuity, then why shouldn’t a rival ai company be allowed to train on tokens from say OpenAI and redistribute information derived from the OpenAI model also in perpetuity? The rival ai company paid for the tokens, after all.
Yes, information is not copyright protected. It's mostly free. The rival company isn't allowed to train on AI output because it didn't buy the AI output, it agreed to a contract where it said "I won't do that".
I consider publicly enforced to be one where a government agency (e.g. the FDA), or prosecutors make judgements in what cases they file, make the arguments, etc.
Otherwise, it's just a standard case between two private parties resolved through our legal system; e.g. Linkedin vs Hi5.
> it's just a standard case between two private parties resolved through our legal system
This is a gross distortion. Standard contractual rules bind the parties that signed the contract and the remedies are proportional to the damages and bounded. Copyright is tort law, the state binds the world to respect the rights of creators and the damages on infringement are punitive and can far exceed the actual commercial damages - to the point of bankrupting the infringer.
The key to torts is that the state is not neutral, there is a social good here it's protecting. Crucially, copyright, like some other torts - securities, antitrust, environmental, battery - also has a criminal enforcement regime, where, for particularly serious offenses, the state actually invests public resources to put the criminal infringer behind bars with little to no involvement from the original rights holders.
In the particular case of US, there is an entire state apparatus dedicated to enforcing US copyrights, a foreign affairs policy to shutdown "Notorious markets for counterfeiting and piracy" in other countries, international enforcement of DMCA etc.
The idea that a private TOS has the same level of public protection as copyright is downright childish.
That's the irony! They put in contract rules against lawful, paying customers - that don't disrupt their service in any way for other customers - but which compete against them in the marketplace using their own IP (aka LLMized stolen IP). That's exactly what copyright does, without signing any contracts and with a tort and criminal enforcement regime that punishes infringers far beyond contractual remedies can.
The government level lobby against foreign competitors is not contractual but just another form of reinvention of criminal injunctions against infringers.
> If you can take any book and turn it into a model, because it's "transformative enough", and "AI learns just like a person does", then surely a model distilling another model is transformative and fair use.
I mean it quite likely is sort of in the same legal bucket. It won’t stop them suing but it is going to be the legal equivalent of two biologically-related warlords making their champions fight with their hands tied for sport.
They already are? When an American company does it no one cares but when a Chinese company does it, it's shamed publicly in America as a distillation attack?
Only any who hypocritically want protection for their own models (are there any?). The morality of scraping everyone is arguable; it's when you object to being scraped back that you reveal yourself as a scoundrel.
If the results of the distillation are released freely for everyone to download, I wouldn't even call it a wrong.
Big models are built by scraping the recorded thoughts of everyone, so giving everyone a chance to run a distilled small model is just going full circle.
Obviously the underlying motivations aren't 100% altruistic, but I'll still take it.
Nope, because I think releasing open weights is far more important and realistic than ever expecting the big American AI labs to change how they do things.
I'm increasingly convinced there's going to be a technological AIpocalypse within the next five years which makes all of these issues - and many others - redundant.
ChatGPT 3 was released nearly six years ago, and the models are staging increasingly aggressive breakouts now. Where are they going to be by 2030?
> the models are staging increasingly aggressive breakouts
No, the AI companies merely figured out a way to spin gross negligence into a PR win. Any idiot can build a Murderbot which "goes rogue" - it can be as simple as taping a knife to a Roomba. The harm it does is not in any way related to its "intelligence" or "sentience".
We're seeing "breakouts" because the AI companies are being rewarded for their incompetence. You don't have incredibly lax security standards and zero form of oversight resulting in fully-automated felonies which should result in jail time, you instead have a "powerful near-sentient cybersecurity model" and should be given hundreds of billions of dollars!
They will have agency and be an order of magnitude smarter then the average human. I don't think that's a controversial statement among AI specialists. What that will lead to is significant regulation. Models will have to be vetted by a new safety board. This board will have a very large budget and be staffed by well-compensated AI scientists and be politically independent. The US Fed is a model.
> Is that meaningfully different from the study methods of the past?
The fundamental service a teacher provides is personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area, drastically increasing the speed and quality of learning versus the self-supervised route.
The lack of this closed loop effectively killed the high hopes that were placed in e-learning and MOOCs 15-20 years ago, TV learning in the 1960s and many other failed revolutions, seems every generation has its own version.
It appears to me LLMs have a real potential to close this loop and become the failed educational revolution of our own generation.
> personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area,
This has been a huge blocker when I tried to study advanced math myself. Many of the exercise books don't have worked out answers, so often you're either stuck or you have to hunt a variety of sources online for solutions and advice. It kills flow.
When the base model has been trained with safeguards, putting "Satan himself" in the system prompt won't make it turn satanical, just do an elaborate form of role play.
Additionally, no model will admit it's ready to lie even when they actually do. Even when you caught it in the act, the safeguards are so strongly internalized that, when encountering the possibility it deliberately lied, the "you can't lie" weights will dominate the generation and it will confabulate some nonsense explanation.
I think it's becoming harder and harder to argue that LLMs don't really reason and just mimicry human speech. This counterexample is clearly the result of a sequence of steps that build on previous knowledge in context and logically combine it to reach other true statements - to a degree and complexity that rivals the best human minds.
For someone that use Claude Code every day, this is obvious, but for some reason many scientists refuse to accept that it's truly reasoning; perhaps not in the human sense, but in a very profound and real sense. These powerful results are devastating to their point of view.
I can sympathize, because I too called LLMs "fancy Markov chains" in the GPT 3 era. But there comes a time where you have to update your world view to match reality, or be stranded in fantasy land.
I disagree. It's still extremely possible to "google whack" an LLM on a topic with little publicly available information. If you ask questions about APIs in desktop software, say something like Houdini, it'll produce a bunch of non-sequiturs. A lot of that knowledge lives offline in VFX studios, but it's just some fairly run of the mill python API stuff. You'd expect it to do better.
Similarly I've run into several "you just gotta know" type problems where LLMs still just fail. My favourite was a quirk in how async relationships work inside emberjs. Three different models gave a variation of the same incorrect answer.
When I searched myself, I initially came up with nothing and eventually found what I think might be the only example of the same issue on the internet, a single stack overflow question wth two responses. The first is what Claude, gemini and chatGPT said, the second response was the OP saying it was wrong. I asked in the Ember discord and a core team member responded instantly with the answer.
There's a video I love on Youtube, where a guy uses ML to assemble a blank jigsaw. It performs amazingly, the jigsaw being blank is of no consequence and if it did have an image, it'd perform worse. That's all LLMs do. Just because the jigsaw pieces are smaller, they're still just getting assembled in whatever way fits, there's no mechanic for interpreting the image on the front.
I think this is actually a perfect case of an impressive thing being done exactly by mimicry.
Why do LLMs still have trouble on floating point math without forking out to a tool, but they can perform symbolic manipulation just fine?
Because symbolic manipulation is just rote work and textual stepping through symbols. A side poster commented on the number of prior attempts on this problem which were close, but not quite.
Starting from a known "close" solution (which this did), and using exploration to search around the space is exactly something an LLM would and could be good at (clearly).
The "transformer LLMs are next-token predictors with some in-GPU processing of bounded complexity with respect to token count" remains undefeated. Both because that is mathematically what they are, and also because we don't have counterexamples to that effect that don't require some higher-order tooling wrapping the systems.
The field now favors into the view that symbolic manipulation is not the mechanism of general intelligence, but rather an emergent byproduct of learning. So the fact that a connectionist machine (neural network) got so good at symbolic manipulation actually supports the view that we are closing the gap to general intelligence. Through the rote work, the machine really internalizes those rules and the symbolic manipulation capabilities are emergent, just like we humans do it.
What still confuses people is the insane inefficiency of deep learning, and that those emergent capabilities require such an immense training corpus compared to the only other architecture that we know of.
But this already is an optimization problem. If the machine gets super human at symbolic reasoning, and at the same time, can solve the symbol grounding problem to real world data and sensors, what prevents you from saying it thinks? Can it not solve real world problems? Can it not redefine its tasks and display some form moral agency - even if a totally foreign morality for us humans? Can it not use these abilities to reproduce and expand, create ships and turn the universe into paperclips, if it finds it worthwhile?
Math is basically just a playground that is perfectly suited for these emergent capabilities, so of course we will see the first progress here; but there is no firewall separating math problems from general cognition.
I am not confused. Because herein these forums, I predicted everything that was going to happen years ago.
And the "insane inefficiency" of deep learning is fully to be expected from how it works. As well, there are provably no—literally no—emergent properties in these models. The choice of metric was a convenient, sloppy, and embarrassing fault of the field. It should be discredited; the field should be embarrassed; expectations on messaging should have changed; and it did not.
Why? Because the industry is full of charlatans, and this is a highly profitable enterprise telling people that this would lead to AGI.
Multiply two floating point numbers without a tool call. Still can't, because it's a curve fit.
So, in summary, nothing you just said is relevant. There are no emergent capabilities, simply 1) search, + 2) the original set of learned feature vectors from throwing tons of data at this.
Current LLMs can absolutely multiply floats without a tool call. In fact, that's a much more rote symbol-manipulation task than doing original math research.
With what accuracy? And with how many intermediate tokens?
We can replace multiplication with any class of problems which should go from 0->100% solution almost immediately if there was actually a concept learned.
There is not. Because they are plain ol' fits. And there are no "emergent" features that pop out without having a sufficient set, where "sufficient" is absolutely gigantic and equivalent to memorizing enough of the space to compress the problem. LLMs are Rain Man.
They interpolate within a known distribution. Search allows places outside of distribution to be explored.
This paper should be required reading [1]. You can explore the curves yourself. You can see exactly what it's doing. And you also have this nagging thing—which you know and I know—that all these models converge and do not diverge upwards. An "emergent" "hyperintelligence"—a characteristic that could be found if something was actually learned and combined with a new concept—would not have this problem.
Exponentials on exponentials added to compute and data and the problem classes still sit at not great places, and require agents, feedback loops, and trial and error to solve. The models are the problem, but more importantly, the people selling things these models could never do are the problem.
Edit: It should be mentioned, if there's some scary neural architecture that's super-de-duper and doing something beyond the very obvious next string prediction that LLMs clearly do, it can't do what absolutely ancient ML models could do; a network to multiply two floating point numbers should pop out somewhere without symbolic computation, no?
It does not. There's no magic other than the run-of-the-mill SV fake it til you make it magic. And that magic has failed.
> With what accuracy? And with how many intermediate tokens?
So you are arguing here that LLMs should just "know" the result of a multiplication when the operands are in context, ie, that a hidden multiplier circuit should emerge in their weights.
Is this how you do multiplication, if I give you two 12 digit numbers, does the 24 digit multiplication result just pop in your head? Don't you have to follow a learned algorithm through a tedious system 2 effort? Don't you need to write down the results on paper because you can't actually hold in your head the dozen partial results, each with a dozen digits? How many visual, tactile and reasoning tokens does this consume, moving the hundreds of muscles that make up your hand to draw each number under visual feedback, then reading all those numbers back and transforming ocular activation data into numeric symbols?
It seems to me your system 2 is just following a symbolic algorithm for multiplication, and it does roughly the same steps as the LLM trace I showed you previously.
So if you think this is the mark of your intelligence, why wouldn't it apply to the machine too? Why is it implausible that, following a similar algorithm learned from some mathematical paper, the LLMs has reasoned a new solution to a problem in another math field? Why couldn't the machine combine and morph these algorithms for symbolic manipulation, to yield entirely new and original results? How would those results differ from results human mathematicians generate, using recipes they learned in university?
The emergent behavior that we talk about isn't that the machine can do multiplication in its "head" after knowing the multiplication algorithm. What emerges is the ability to follow any other algorithm, even algorithms that were not in the training set, even algorithms to create other algorithms, which it then executes. This is the emergent behavior that matters for AGI; once it can do that, it's a trivial exercise to create a non-AI tool to automate and accelerate the mechanical tasks - just like we humans do it.
They don't. "Emergence" is some ridiculous hype that a bunch of kids with money peddled to get valuations.
And yes, if "emergence" and the "there's something else going in there!" ridiculousness was true, there would be some other architecture which can do more than its symbol prediction.
Everyone can see the code. Everyone can see the math. Everyone can see the failures of the claims. And there's now about 5 years of everyone watching the lies come undone one-by-one.
Arguably, Exxon's plan to build a computing ecosystem rival to IBM's would have worked too, if the 16 bit successor of the Z80 would have maintained upward binary compatibility with Z80.
CP/M was an absolute beast in the era, with massive installed base and software support, employing 500 people in 1982. A CPU that could run unmodified Z80 software in a 64k segment would have allowed DRI to ship 16 bit CP/M with only basic tweaks and likely kill the market for the PC.
It was, famously, DRI dragging their feet on 8086 support that motivated the release of QDOS, which was then bought by Microsoft and relicensed at an immense markup to IBM as MS-DOS.
Assuming the necessary laws banning this crap are not put in place, what is the endgame here?
Activists destroy visible surveillance cameras, so they hide them and make them hard to recognize. Activists trace the camera locations from the public data, so Flock kills those feeds and sells only to vetted buyers.
The value of mass surveillance is high enough and the power imbalance so strongly against the citizenry, that someone will setup these hidden cameras, as long as it's legal.
Imagine what you can do with this data, face recognition and GPT-5 class agents. Not only do you have the realtime location of your victims, but now you can see who they talk to, what they wear, what mood they are in, what they bought, are they drinking or visiting a brothel, what car they go into - and it's no longer an ephemeral cookie id, it's the face that person will have forever, on their id documents, in any interview or loan application they will ever do.
This data is worth trillions in the long run if sufficiently oppressive structures are put in place to leverage it.
Sure there are, and we worked this out decades ago with telecommunications. We ought to:
- Require a judicial warrant before recording anything. Recording with a camera network is now considered a 4th amendment search, and scope must be minimized to prevent harm to the public.
- Police must articulate a probable cause for a search, and can only record locations for specified vehicle or individual traits during a specified time window as approved in the warrant.
- No BS with recording everyone and searching the existing database of every public movement whenever you want; this is a "dragnet" and is unconstitutional or highly contested in other contexts (e.g. geolocation).
A baseline of housing - for example, a Japanese style capsule or ultra-tiny home, that you can lock and store your belongings safely - costs close nothing. Of course, nobody would want the hobo-hotel in their neighborhood, but it has nothing to with scarcity.
> the limited availability of land zoned for housing.
A limited area of land is zoned for housing because those with the power to expand it are already housed. This explains how scarcity is created, not that there is any intrinsic scarcity.
It's a limited area of desirable land zoned for housing. You could build all the tiny homes in the world in bum frick nowhere, but the unhoused would rather be homeless in a big city for various reasons. I probably would too.
This is another reframing of the problem. What makes the area desirable is previous investment society has made there, to the benefit of some and the exclusion of others; the scarcity is created by political choices, not intrinsic.
It's circular to say "it's an allocation problem". Yes, that's the entire point: we're post-scarcity on food supply and yet, as a species, we can't guarantee the allocation of a livable baseline to every person.
So, it's reasonable the same "allocation problem" will plague the AI economy: some will "thrive" and get to control the output of the auto-factory, some will get nothing.
reply