> The only time that isn't the case is if the market is irrational.
There are other situations, such as when measuring actual ability is sufficiently difficult or impossible (subject to the constraints you pointed out - via a process that's repeatable by a largely undifferentiated set of interviewers).
In those situations, you have basically no choice but to look for proxy measures.
Now, obviously, proxies can be gamed to various degrees. Leetcode is not actually such a terrible proxy, though. Here are some points in favor:
1) On average, better engineers will require less practice time to achieve similar results,
2) There is a certain minimum level of capability required to even "grind it out"; that minimum doesn't serve as a sufficient floor for being a net productive engineer, but it gets you a reasonable chunk of the way there, and that's why so much of the evaluation criteria for these interviews focuses on communication ability, since that's the rest of what they care about
Taken together, you get a process that is reasonably good at eliminating false positives and still manages to tilt the field in favor of more skilled candidates. On the candidate's side, it has the benefits of being generalizable across multiple interviews and also getting easier each time you do it.
While I'm somewhat sympathetic to the claim that the process favors candidates who have more free time, I think it's a largely overstated concern. First, this is true of any interviewing process wherein a candidate can improve their performance through practice. To a first approximation this is all interview processes. Second, it doesn't take _that_ much time. If you put in 100 hours (which is basically "made it a moderately important priority for 2-3 months") and you still can't clear any interviews, there are a few possible explanations.
1) You're failing at the communication side of things. Thankfully this is also something you can practice!
2) You fall into an unfortunate edge case, e.g. extreme performance anxiety, which isn't reflected in your day-to-day work. This sucks! I don't think there's a process that doesn't have unfortunate edge cases; all we can do is try to minimize them.
3) You're not able to learn the material well enough to generalize it to novel interview questions. This is the process working as intended.
I've also heard a reason that goes something like "I can totally solve those problems, just not in 45 minutes". Problem-solving speed is going to be positively correlated with other traits of talented engineers; obviously some otherwise talented engineers will still fall on the right side of the curve. Again, this sucks, but please bring me a process that effectively rules out false positives without bringing in some false negatives. If you can manage it there's a huge market opportunity there.
These are interesting points, but they very much read like post-hoc rationalisations to me. Let me take a different tact, I want to try challenging some of the more fundamental assumptions.
Do you really believe in the quality of Leetcode as an evaluation criteria, or are you trying to justify it? Google used lateral thinking puzzles for something like 8 years before realising they were completely useless (not just unreliable - they had zero predictive power). We have precedent for these companies using selection methods that are bananas, and smaller companies copying them.
>There are other situations, such as when measuring actual ability is sufficiently difficult or impossible
Sure, that's an example of why the market might be irrational. But then I'd ask: do you think it's extremely difficult/impossible to judge other programmers? I mean that seriously. I don't find that difficult. I can usually size up another programmer pretty quickly. If I sniff around a bit and look at some example code, my intuitive read will be much more three-dimensional and reliable than giving them two Leetcodes. You don't find that to be the case?
How much do you spend on hiring one candidate? I'm guessing tens of thousands, if not more. You can't invest a day into trawling that candidate's GitHub? "But not all candidates have a GitHub." Ok, but some do, so that's not the reason.
What I think is actually extremely difficult is a large, beurocratic organisation developing a replicable, second-hand evaluation system. It's a standardised test, and it has the same problems as most standardised tests.
> you get a process that is reasonably good at eliminating false positives
Right, it's good as a filtering mechanism. But if it's just a filtering mechanism, it shouldn't take up the majority of the interview.
Like, here is one of the assumptions you're making: most good engineers grind Leetcode before a new job. I don't think that's true. I think the number of people who grind Leetcode, even among the best candidates, is extremely small. Like <10%. If that's the case, it's inherently a very flawed criteria. If nothing else, you're filtering for the candidates that everyone else is filtering for - which means you're filtering not for capable candidates, but for candidates who are overpriced.
Here's another thing that's absent from these discussions: algorithms questions were developed before Leetcode existed. They're not designed for a world where you can game them. Leetcode is a bug in the system, but a lot of your justifications are, "it's fine, because the selection criteria is actually designed to select people who game it." Really? You want people to game the criteria? I don't buy it.
There are other situations, such as when measuring actual ability is sufficiently difficult or impossible (subject to the constraints you pointed out - via a process that's repeatable by a largely undifferentiated set of interviewers).
In those situations, you have basically no choice but to look for proxy measures.
Now, obviously, proxies can be gamed to various degrees. Leetcode is not actually such a terrible proxy, though. Here are some points in favor:
1) On average, better engineers will require less practice time to achieve similar results,
2) There is a certain minimum level of capability required to even "grind it out"; that minimum doesn't serve as a sufficient floor for being a net productive engineer, but it gets you a reasonable chunk of the way there, and that's why so much of the evaluation criteria for these interviews focuses on communication ability, since that's the rest of what they care about
Taken together, you get a process that is reasonably good at eliminating false positives and still manages to tilt the field in favor of more skilled candidates. On the candidate's side, it has the benefits of being generalizable across multiple interviews and also getting easier each time you do it.
While I'm somewhat sympathetic to the claim that the process favors candidates who have more free time, I think it's a largely overstated concern. First, this is true of any interviewing process wherein a candidate can improve their performance through practice. To a first approximation this is all interview processes. Second, it doesn't take _that_ much time. If you put in 100 hours (which is basically "made it a moderately important priority for 2-3 months") and you still can't clear any interviews, there are a few possible explanations.
1) You're failing at the communication side of things. Thankfully this is also something you can practice!
2) You fall into an unfortunate edge case, e.g. extreme performance anxiety, which isn't reflected in your day-to-day work. This sucks! I don't think there's a process that doesn't have unfortunate edge cases; all we can do is try to minimize them.
3) You're not able to learn the material well enough to generalize it to novel interview questions. This is the process working as intended.
I've also heard a reason that goes something like "I can totally solve those problems, just not in 45 minutes". Problem-solving speed is going to be positively correlated with other traits of talented engineers; obviously some otherwise talented engineers will still fall on the right side of the curve. Again, this sucks, but please bring me a process that effectively rules out false positives without bringing in some false negatives. If you can manage it there's a huge market opportunity there.