Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yeah, if the benchmark is actually predictive of the tasks you have then it is trivial to conclude that the cheapest-per-benchmark-task model will be the cheapest one for your tasks…


It might vary between tasks though. A model that’s great at abstract reasoning might be great at writing math proofs but struggle to write software in <insert language>.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: