I cannot even begin to express how many times I've seen engineers working super hard to optimize happy-paths so that we turn 3 nines of availability into 4 nines but introduce unintended emergent behaviors in unhappy-paths that turn 1 nine into zero nines via thundering herds, retry storms, etc.
You have my empathy for this kind of sentiment. Personally this seems somewhat rare in practice. That being said I'm curious if anyone has anecdotes they can share about these kinds of things?
1. People add retries w/o backoff. Now we have a retry storm.
2. People don't add jitter so we get huge waves of highly correlated retries that cause self-perpetuating overload cycles and failure.
3. People add retries (even w/ backoff) at more than one layer (e.g. one in process, one in envoy), so now we have a deeply confusing multi-level of retries with super weird n^2 patterns.
4. People find ways to fetch from multiple data sources to make a composite object but don't cache/reuse data they fetched, so one data source being down causes DOS on _other_ data sources because of retries.
5. People add failover mechanisms where all failures happily pick the same failover target because, ya know, it has the most free capacity.
6. People underestimate connection setup cost so "failover" causes huge increase in overall load due to connection setup (often tls setup) causing huge influx of "new" cpu work in a loaded system. CPU spike causes unhealthy destinations, causing more failovers, causing metastable failure.
7. People realize purely local decisions aren't optimal so add a layer of global health metrics (e.g. global retry budgets) but these systems add a time delay to the metric (ie its the view of the world 5ms ago). This delay makes their naive control algorithm go into oscillations or divergence spraying stuff everywhere.
All of these things have a similar pattern that when things are almost all good, they will make the system more robust. You'll get an extra 9 of reliability on good days. But they make everything far worse when things are bad.
Configuring postgres to automatically failover instead of doing it manually. The automated system caused more downtime in a few months than manually doing it did for years before. All in the name of more automations and less downtime
For resolution of a problem you can’t prevent categorically, you have to spend a good bit of time pretending to be the computer before you can trust the computer to do it. A run book is a draft of a requirements doc for a program to do the same thing. And you can’t afford to test that program in production without loads of simulations first. And that’s running the process manually and checking aggressively for any signs of problems.
Some coworkers snaked the autoscaling work out from under me, and then ignored the advice I offered on low water marks and weeks of testing and rehearsals. All I can figure about their irrational exhuberence is something to do with claiming something for annual reviews? All I do know is we didn’t make it 40 hours from when they flipped it on until we had our worst outage in two years. Classic FAFO.
They were hoping to eventually get to a low water mark of just over 30% of our static cluster size, and they decided to use that for their initial go instead of my estimate of 40% being the low, and a recommendation that they start with 50-60% for the initial weeks and then ramp it down. When I refused to let it go in the status meeting, the team decided we should vote on it. Two guesses how that went.
A bug in the deployment logic the Ops team had for doing things like upgrading VMs caused the next update during daylight hours to spin up the new instances at the minimum cluster size, instead of the current cluster size. It was done outside peek hours but that still had us cut over with just under half of the hardware we needed at that moment. Because we were in such a fucking hurry to be something like third in line to use the new autoscaling support, nobody else had run into this problem yet (or system did a pretty good job of buffering everyone else).
A couple weeks later they’d looked at the stats and decided that we were spending less than a half hour a day running at the low water mark, and the additional shutdowns were causing churn that made it harder for us to detect problems like memory leaks. Surprise, surprise, they increased the min cluster size to exactly what I’d told them two weeks before.
Since “listen to me” isn’t a lesson that transfers to other teams, I will instead say, don’t transfer initiatives to a new team when there is less than 10% of the project left. The lack of friction you encounter may result in a lack of respect for the danger, and attention to potential problems. And if you must transfer, if your ideas about rollout timeline end up being less conservative than theirs, be patient and do it the slow way. They probably have encountered problems you haven’t seen yet.
I think the "happy path" might be a slightly wrong classification in GP, since the post is in reply to a retry-storm issue and explicitly talks about retry storms and thundering herds.
I've seen many cases where engineers optimize the sad path, but pessimize the wretched path. Or in less flowery language, they cut the occurrence rate of common non-critical failures, but by doing that they introduce code that can make rare failures much worse.
The cases I've seen generally boil down to naive retry logic or poorly tested and poorly maintained fallback paths (such as killswitches that break their environment[1], graceful degradation turned graceless, dormant feature flags that get reactivated).
The case you see with a retry storm here is the most classic one and the one that annoys me the most. I've seen engineers adding aggressive retries even into places where the impact is minor (you could show an error and let the user manually retry instead). Retries that improve user experience can be great if done correctly, but I've never seen the authors of such pull request addressing the risk and mitigation techniques for retry storm or retry amplification.
I've seen cases which had:
1. Retries on the client side (browser or mobile app).
2. Retries on the BFF.
3. Retries on Microservice A used by the BFF.
4. Retries on Microservice B used by Microservice A.
5. Retries on Critical Service C used by Microservice B.
Most of these retries had very short timeouts (e.g. 100ms), in order to keep latency SLOs during normal operations (not a good idea on retries). Every time QA saw a layer without retries, that would be a bug, and adding retries is easy, so we'd get a new retry without much thought. But the first time Critical Service C became overloaded, Microservice B started timing out a couple of times and retrying. This was too much too much for Microservice A that had a short timeout that couldn't hold the 3 retries done by Microservice B, so it making doing its own retries, all of them dropped in the middle of the way. Eventually you'll get a full-blown retry storm where every request from the client side got amplified with 3^5 retries, easily bringing down Critical Service C.
We'd usually introduce a circuit breaker for the particular path that caused the issue, but a variation of this kept happening several times because designing safe retries across a vast collection of microservices takes a lot of effort, and it's always easier to just add a quick-and-dirty retry at any point where you think you might need one and call it a day.
A proper solution (which I've never seen implemented) would be an mandating a corporate-wide inventory of retry-paths, and monitoring it for any path that is at risk of triggering a retry storm, or adding mandatory headers that cross microservices and track the amount of retries done up the chain and the time spent in total waiting for previous retries. You could have a budget for both and automatically stop performing more retries. Both solution require extra effort and a large degree of coordination.
> I've seen many cases where engineers optimize the sad path, but pessimize the wretched path.
As said GP, yes this is exactly what I meant and "happy-path" was an unclear choice of words. They optimize for the unhappy path on the good days, and make the bad days much worse.
There is a huge amount of misinformation on this topic, including in this thread, at the minute.
Some index funds have a very long horizon before they include them (e.g. a year). Others are "fast-tracked" (e.g. notably VTI). Most of those, however, are float-adjusted, so only the stock available for trade is considered part of the marketcap. So e.g. VTI / VTSAX will buy spacex relatively quickly after the IPO but at the float-adjusted weight of ~$75B because that's the % of stock available.
If you care alot about this, now is the time to understand how your index fund treats IPOs wrt to delays + float adjustment.
Specifically, I do a typical 3FP and own VTSAX, but I don't read bogleheads or anything. True set-it-and-forget-it, but I do want to read more if things are shifting.
You should not trust me, but here's my understanding. I wish there was a really good writeup somewhere to explain this authoritatively but I'm not sure there is one. Would also love to see one. Frankly vanguard should do it.
VTSAX (and VTI) follow the CRSP index. This is float-adjusted but they likely will be fast tracked (these are two separate rules in how this index chooses to weight things and participate in new stocks). At ~5% float, these companies will be in the 50-100B range. So under all those assumptions, they'll be bought quickly but represent less than 1% of VTSAX (until they float more shares on the public market).
At least part of the answer is this doesn't meet the requirement of the proposed law. You need to actually provide the functionality of the server-side, not just its API.
I have no idea how you can assert what is necessary/sufficient for consciousness in this way. Your comment reads like you believe you understand consciousness far more than I believe anyone actually does.
I believe consciousness needs some kind of mutable internal state because otherwise literally everything is conscious, which makes the concept useless. A rock "computes" a path to fall when you drop it but I don't believe rocks are conscious. Panpsychism is not a common belief.
Essentially, something is conscious iff "there is something that it is like to be" that thing. Some people find that completely unsatisfying, some people think it's an insight of utter genius. I'm more in the latter camp.
Also, I think consciousness is non-binary. Something could be semi-conscious, or more or less conscious than something else.
Anyway, I don't think that there's anything that it's like to be an LLM. I don't see how anybody who knows how they actually work could think that.
> Anyway, I don't think that there's anything that it's like to be an LLM. I don't see how anybody who knows how they actually work could think that.
While I have almost zero belief that LLMs are conscious, I just don't think this is so trivially asserted.
The easy half of this is thinking that LLMs aren't conscious given what we know about how they work. The hard part (and very, very famously so) is explaining how _you_ are conscious given what we know about how you work. You can't ignore the second half of this problem when making statements like this... because many of the obvious ways to argue that clearly LLMs aren't conscious would also apply to you.
I wouldn't say that we actually know how our brains work. Based mainly on my neuroscience minor from 10 years ago I'd say that understanding feels hopelessly far away.
> My point isn't that coordination is easy - it's that treating it as impossible becomes self-fulfilling.
While I see what you are getting at, and I think its super important we come up with philosophical frameworks to push back on the central idea in question (ie, the moral hazard of "its gonna happen anyway so why not pour a little more into the river").... I think your writing/responses miss the central point.
As I see it, the fundamental issue with this essay, and your responses, is you keep conflating impossible with probability zero. People are saying "this is inevitable" to mean this has probability 1 of occurring, with basic game theory reasoning (its a giant iterative prisoners dilemna), and your response "but it's possible". Yes, with measure zero.
Telling us that such a path surely exists isn't useful. If you want to push back on "inevitability" you need to find a credible path with probability > 0 (which is not the same as impossible).
Thanks for your thoughtful response. I think there's a misunderstanding (maybe my text wasn't clear. If so please point out where so I can fix it).
We actually agree: even if the probability of successful coordination is only 10%, accepting inevitability makes it 0%. That difference matters enormously given the stakes.
My argument isn't "coordination is definitely possible" but rather "believing it's impossible guarantees failure." When tech leaders say "AGI is inevitable," they're not describing reality; they're shaping it by discouraging attempts to coordinate.
Human cloning hasn't happened because we maintain active resistance despite technical feasibility.
You're asking for credible paths with P > 0. I'm saying: knowing P with certainty is impossible, so accepting P = 1 narratives makes alternative paths invisible. The path emerges through trial and error, not before it.
> When tech leaders say "AGI is inevitable," they're not describing reality; they're shaping it by discouraging attempts to coordinate.
No, they're describing reality. As I posted in another comment, progress in technology drops capital requirements for innovation. Even if there's global coordination to stop AGI development right now, progress in tech means that in 30 years someone in their basement could do what OpenAI is doing right now but with commodity hardware. Preventing this would require an oppressive regime controlling basic information technology and knowledge to an extent that isn't palatable to anyone.
"They're describing reality" - No, they're making predictions about the future. If AGI requires 30 years of compute improvements as you say, then it's not reality, it's a forecast contingent on those 30 years of development continuing unimpeded.
As for "oppressive regime", we already do this for nuclear and biotech, and most people find it quite palatable! Nuclear materials are tightly controlled globally. Cloning humans is illegal almost everywhere. We've had the knowledge for both for decades, yet basement nukes and basement human clones aren't happening.
I'm not saying we should make it illegal, I'm just saying there are more gray areas than what's generally accounted for.
> No, they're making predictions about the future. If AGI requires 30 years of compute improvements as you say, then it's not reality, it's a forecast contingent on those 30 years of development continuing unimpeded.
The idea that you can develop "good" information technology without enabling the creation of "bad" information technology is pure fantasy, and if your idea is actually that we could halt the progress of information technology wholesale, then that's laughable, sorry to say. Hence the inevitability.
> As for "oppressive regime", we already do this for nuclear and biotech, and most people find it quite palatable!
You mean we do this for raw materials that have inherent scarcity because they are somewhat rare, difficult to mine, and difficult to refine? And you think this natural scarcity is somehow comparable to natural abundance that follows from digital information that can be trivially copied at perfect fidelity?
Furthermore, you've misunderstood what was meant by "oppressive regime". The same technologies that allow us to email each other, make family photo albums and forecast the weather or the stock market are what also enable AI. There is no way in which to suppress AI without also suppressing these other benign uses that everyone enjoys and that enable considerable productivity. This is not comparable to the technology and raw materials for nuclear weapons.
> We've had the knowledge for both for decades, yet basement nukes and basement human clones aren't happening.
You seem awfully confident about declaring the non-existence of something that's inherently underground and thus difficult to measure.
But let's make the comparison of AI to cloning more apt: how confident would you be that cloning won't happen once the knowledge of how to construct artificial wombs is discovered? Now reconsider those probabilities about if such wombs also easy to construct with readily available materials. That's the reality of information technology.
If this were so, I've yet to see any explanation that accounts for why human cloning has been so successfully prevented, even though it's relatively inexpensive technology, to the point that we have (had?) pet cloning companies selling it as a consumer service.
1. How do you know human cloning has been prevented? Maybe you mean it's not provided as a commercial service, but does that entail it's not happening at all?
2. Preventing the manifestation of physical objects is a lot easier than preventing the dissemination of pure information. AIs are easy to copy, easy to run, and can assist in their own creation, advancement and proliferation, and it only gets easier over time. For an apt analogy, consider a cloning lab where every clone that escaped was compelled to create their own cloning lab, and everything you needed could be bought at any corner store.
3. All cloning requires existing biological organisms to participate at various stages. You need not only the biologists on board, but also the surrogate that has to carry the fetus to term. What do you think will happen when artificial wombs become available?
1. Because neither Elon Musk nor Kim Jong-Un have a clone. That is, if human cloning were being carried out, we'd see clones of famous and/or powerful people. Is there some chance it happened in a lab somewhere and kept extremely secret? Sure. Is it a technology that's available to the groups that could pay for it? No, it would be very visible (eventually, at least).
2. Computers powerful enough to train AI are also physical objects, ones that consume gigantic amounts of power as well. Maybe some day we'll have computers that can train Claude on 50kW of power running in your pocket, but maybe not. There are fundamental limits to how much computation you can get per watt, and we're getting closer to them. So, preventing AI may be as simple as banning use of any computer cluster that consumes more than some wattage, say 1000kW, without government audit, while also banning research into more computationally efficient ways of doing AI.
3. This is not a real problem, since some biologists that are into cloning may have wombs of their own to gestate the clone. Artifical wombs, even cheap ones, would change nothing in relation to cloning (except maybe reduce the diversity of rogue cloning research teams - angering the criminal enterprise DEI department, I'm sure).
I was struck how the argument is also isomorphic to how we talked about computers and chess. We're at the stage where we are arguing the computer isn't _really_ understanding chess, though. It's just doing huge amounts of dumb computation with huge amounts of opening book and end tables and no real understanding, strategy or sense of whats going on.
Even though all the criticism were, in a sense, valid, in the end none of it amounted to a serious challenge to getting good at the task at hand.
The premise that an AI needs to do Y "as we do" to be good at X because humans use Y to be good at X needs closer examination. This presumption seems to be omnipresent in these conversations and I find it so strange. Alpha Zero doesn't model chess "the way we do".
Both that, and that we should not expect LLMs to achieve ability with humans as baseline comparison. It’s as if cars were rapidly getting better due to some new innovation, and expecting them to fly within a year. It’s a new, and different thing, where the universality of ”plausibly sounding” coherent text appeared to be general, when it’s advanced pattern matching. Nothing wrong with that, pattern matching is extremely useful, but drawing the equal sign to human cognition is extremely premature, and a bet that is very likely be wrong.
> The premise that an AI needs to do Y "as we do" to be good at X because humans use Y to be good at X needs closer examination.
I don't see it being used as a premise. It see it as speculation that is trying to understand why this type of AI underperforms at certain types of tasks. Y may not be necessary to do X well, but if a system is doing X poorly and the difference between that system and another system seems to be Y, it's worth exploring if adding Y would improve the performance.
Suppose this is as good a place to pile-on as any.
Though this was not the post I was expecting to show up today, it was super awesome for me to get to have played my tiny part in this big journey. Thanks for everything @je (and qi + david -- and all the contributors before and after my time!).
On the other hand the top ten comments as a whole are 3 bullish, 5 neutral, 2 bearish which is certainly not an overwhelming sentiment in any direction. That's despite the fact that bullish comments on start-ups tend to get more votes because it's a start-up community.
Sure, plenty of people thought that it was a good purchase, but my point is nobody thought of it as buying out their competition. The transition into a social media platform and algorithmic content machine occurred under Facebook's direction.
> "Where's the money in Instagram?" Preventing Instagram from developing into something that has a negative effect on Facebook. It's a "keep your enemies closer" move.
HN is notorious for this kind of thing, such as the iPod: "less space than a nomad, no wireless, lame". Due to not understanding how much consumers value simplicity.
Every time I see a comment accusing HN of having some specific consensual position like a hive mind, I go back and see comments both contradicting and supporting the stance. In other words, different opinions. Every single time I check, and every single time it shows the original commenter engaged in selection bias.
This case is particularly wrong, as that iPod quote is from Slashdot. HN didn’t even exist in 2001.
But hindsight of HN is different from hindsight of FaceBook.
FaceBook was literally collecting data on what apps people were using on their phones and empirically saw the rise of Instagram. Of course the rise of Instagram didn't need to continue but that's why you buy all the realistic competitors so even if most of them fail you have a moat of dead companies.
After trying to set up wireless on my printer interface and enter a password with up and down arrows rotating through an entire set of keys, I'm fairly convinced that no wireless on the iPod was massively correct. If people were expected to set up wifi by entering a password with a rotation device adoption would be miniscule.
Dunno why the internet enjoys dunking so much on a poor anonymous poster who guessed wrong about a product that would catch on, whether that's the iPod, the iPad, Dropbox, etc.
We don't seem to spend half as much energy taking major news outlets to task when they similarly guess wrong, unless we feel that somehow adding a question mark negates any responsibility (i.e. "The Ouya will revolutionize gaming" vs. "Will the Ouya revolutionize gaming?").
It's a recurring cognitive dissonance between cynical tech people and the actual mass market. I mean I'm cynical about most things but the ones that aren't and who get onto the hype train earn big money off of it.
Those people aren't putting in their own money. The people who did put their money into Instagram got to see behind the corporate covers, and they decided to buy anyway. It's very easy to say whether you'd invest $1B if you're not putting in any of your money.
