Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think part of the reason why nobody seems serious about replication is that it's actually mostly not worth it.

Most research is useless and pointless, with only a few exceptions. We don't have a way to figure out which topics are the exceptions, so someone has to do the research. It's not worth it (or rather: extremely high financial risk) for companies or individuals to do it, so governments gave to fund it.

At this point, the current amount of fraud does not justify replicating even 1% of studies. We would get less scientific advancement in total. The current situation likely does justify some small investments in shaping incentives.



>>It's not worth it (or rather: extremely high financial risk) for companies or individuals to do it

The problem is that it's hard to reliably capture value from research. A good example is LLM's. If OpenAI, Google, Meta and AWS had been able to build a wall round GTP3.5 Turbo and above models then I expect that they could have captured all the value of the research effort.. as it is I don't think that is/will be the case - it's almost too easy to replicate as Mistral have shown. Note: I'm not saying it's trivial or something, but if you spend a few $million on it you can get close enough, and then spending a few $million more will get you all the way. Also, I am not talking about building a frontier model today (which requires $100millon or so and some difficult skills/organisation) but rather a model in say 3 years time with the frontier performance of todays models.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: