I did notice that on the `products/github-actions` page, the metrics for the zed-industries/zed repo are missing. It's showing a `NaNm NaNs` for both the With and Without Depot sections. Might be worth fixing it, as the current state reads as Depot providing no improvement for zed.
Genuinely, it was one of the worst parts of working at Amazon. Especially since I often interacted with people who only used Chime. Messages would be missed for weeks because they'd never check slack, or I'd never check chime. Awful experience.
"The goal isn’t to rank models, but to understand how they fail."
The goal isn't to write an informative blog post describing what you learned, but to generate slop and expect other folks to read it.
I really wish people would stop doing this. I love reading about your side projects and all of the cool things you're doing. But, it just feels insulting to open up something that's so obviously completely AI generated. If you aren't willing to write it in your own voice, why would it be worth reading?
You know the meme where a concise sentence is translated by an LLM into a loquacious formal email which is then again summarized to a concise statement by another LLM on the receiving end?
I believe that's what we need to do here. People have some interesting information to share, but they don't care about penmanship and that's not just being lazy. It takes a lot of time to produce a nice post. I cannot guarantee the author used an LLM but there sure is a suspicious amount of em-dashes.
Anyway, there are still some interesting data points so I'd recommend to run the website through an LLM to get a nice summary if the prominent TL;DR is too short for you. Times are a-changing.
I agree somewhat. My issue is primarily that, without the author actually penning the post themselves, we have little to no evidence that they've actually done anything. Maybe the data is all AI generated or hallucinated, maybe the validations weren't thorough. I could determine all of these things myself, via rigorous review of the blog post. But at that point, I'm just doing the research myself, of what use is the post?
For work communications, I agree with you. There's an inherent accountability there. If you send me AI slop, and something goes terribly wrong, you'll be held accountable for the slop. Here, the slop is just noise that prevents us from finding the truly interesting posts.
It's a very interesting issue you raise here. Notice that even if he typed it all out himself we wouldn't be any wiser. Literally nothing would have changed. Using the "I can verify that he performed a lot of work" as a quality signal always was a questionable - albeit understandable - choice but in the LLM-age it's useless.
<super_weird_rant>
I don't think I like it, but I think we are heading towards a situation where all information is filtered, reviewed and validated before it even becomes available to you. We need to do a lot of work to define what "reviewed" and "validated" mean here, but I don't see many ways around it. This would, however, require a vast attitude shift whereby we have some way of proclaiming "facts" and "arguments" tied to "proofs" that can be automated in some fashion, not just for code, but for all communication in general.
Stuff like "X is true in 50% of cases" need to be automatically and transparently tied to some part of a system that supports your claim which itself can be tied to some greater system, etc. If we have UIs that support this cleanly we can inspect the veracity of claims ourselves as so far the validation is feasible/practical/economical. Perhaps some sort of "this claim is true under X,Y conditions"-fingerprint made by some trusted VerificationAgency, a chain of trust so to speak, like our certificate systems. Or perhaps a P2P network of open-source "ClaimVerifiers". If everything is by default written with verification in mind, not just code, but literally everything that needs to be correct, I think that would be quite interesting.
OK, this is super weird so I'll let myself out now.
Sort of agree, yea. The issue is we allow high wealth individuals to use their assets as collateral for low interest loans, without having that be a taxable event. They can cash in on their equity, effectively withdrawing from their stake in these companies, and avoid taxes altogether.
I think just closing this loophole, and having personal loans over a certain % that are backed by specific equities (stocks, land, etc), be taxable. This fixes the loophole, in that they now need to pay taxes on these loans, on top of interest. It would incentivize them to then increase their base pay, where they'd pay income tax like the rest of us.
I agree with you on closing the loophole. However, I would count any money taken through this loophole as income. Another limit I would remove is the current limit on Social Security/medicare taxes. It doesn't matter how much you make; you should pay that tax.
I and many other HN participants are in the privileged few who would actually see higher taxes from that change in SS and Medicare taxation. I'm on board. But most probably are not. It is weird and still somewhat foreign to me that my paycheck increases in the late fall / early winter. I appreciate the extra money coming in, but realistically it wouldn't make a material difference in my life if it didn't.
It is being used in performance reviews, source: recent Amazon SWE.
They also use a bunch of dumb metrics like, total PRs submitted, total comments made on PRs, etc. To the point that, there are multiple heavily used internal tools to game these metrics. Eg, auto-comment LGTM on any approved PR. Thus, making the metrics even worse than they would have been prior.
> Amazon has told employees that the AI token statistics would not be used in performance evaluations.
> Managers are discouraged from using token use to measure performance, according to a person familiar with the matter.
Like CAD and architects, if you're not using LLM's while coding it's an issue, but Amazon is very clear that this isn't an official metric. I would believe managers know how many tokens you're using, but it sounds like they just interviewed a disgruntled employee who didn't like AI and published it.
>but Amazon is very clear that this isn't an official metric.
You're replying to an amazon employee who says they are being used in performance reviews, in comment thread on an article where 2 other Amazon employees say that their token usage is being tracked and they feel pressure to maximize token usage.
Do you have first hand knowledge to refute these 3 people with first hand knowledge?
The CAD thing is incredibly weird. I've never known an architect who had their CAD usage minutes tracked.
Btw I'm a big tech company and I know many people who are "token maxing". It's very common.
You too can avoid Claude AWOL by subscribing to our premium package, Claude VACATION. Claude will show up on time, and give you at least one months notice before any outages.
Ah, the old, "This thing happened to me, and even if other people say it didn't happen to them IT MUST have happened to them and their denial of it just makes me even more justified" argument.
reply