A recent review of mine had a LLM-ism at the very end, "would you like me to format this into a formal peer review report?" So they very likely copy-pasted their whole review :). I'm pretty down on academia atm ;-;
I think this is more of a systematic issue. I review now since 2-3 years, I do not get paid, which is fine. However, it takes always a huge amount of time without really having anything from it, but I do it because it is important work.
There now so many researcher that need to publish which explains the flooding, LLM only speed it up, so reciprocal reviews take place. So now you are forced to review and you are having less and less time. So it’s a natural choice for you if you already took an LLM to write a paper to use it to review.
Perhaps one solution would be much harder entry barriers, and enforcing some guidelines. For example that a supervisor can not have more than 5 papers and PhD students only need one real paper on a major conference/journal.
Super unfortunate. On the other side, I reviewed four papers for a top-tier AI conference and three were clearly fully Claude generated, as in all text, figures, results, everything. Actual good reviewer time is wasted on such papers and your (i hope) human written good paper receives AI responses. It's a sad state of affairs for sure.
I find it incredibly sad that this is what the world is becoming, and I mostly blame people for this, not the LLMs.
It’s the same type of people that would have no issue letting an LLM open a pull request on GitHub wasting valuable time of other humans, and whatnot.
I’m using LLMs all the time myself, but it’s so incredibly important to use it to improve the quality of your work, not degrade it. People seem to be totally oblivious about this.
On the flip side, it does make it easier to recognize people who are wasting my time.
I think we underestimate the psychology of this kind of use of LLMs. I don't think these people (students, academics, lawyers, etc etc etc) are all just lazy morons. I think that using an LLM gives a strong knee-jerk feeling of "OMG this is exactly what I have to say. This is MY idea, these are my thoughts, this is - in a real sense - MY writing!" The feeling floods you when you see the output being churned out, way before you you read the thing (if you ever do) - it's the initial *seeing* it. Then when you hand the generated text in, unread by you, you do not have the feeling of cheating, you have the feeling of having exercised your powers, pushed right to the edge of your expertise and thoughtfulness, you successfully overcame obstacles because of your experience and unique capabilities.
This is not always the psychological situation, but I think it might be a lot of the time.
I also think that until we recognize this, we won't be able to help people to not do it. Calling them lazy or being bewildered by them or feeling rage or contempt towards them or threatening them is not going to be practically helpful.
I can't help but feel like all the tech glasses are basically a way to get more ai training data where they can get like a constant stream of human POV stuff.
Ah! That's actually exactly why I don't use WhatsApp, Google, etc anymore.
When Meta bought Oculus a decade ago I though "why the heck are they buying this VR company..." which prompted me to read "The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power" 2018 Shoshana Zuboff's book.
The idea is precisely that they don't need more of the same data, they have enough (even though they do need to keep them up to date) but they instead need more KINDS of data to keep on earning money from selling ads.
They don't care for the technology itself, only for ways to distinguish needs better and that's exactly what a new digital medium provides.
That % should increase substantially with each violation too. It's hard to make fines hurt when the corporation you want to punish has more money than most countries.
I know in electrical engineering / chip design, it's actually quite hard to do "real" research that's useful for industry simply because of how expensive it is to make chips... I think most research for improving LLMs will go in a similar direction.
I've personal almost stopped reading papers in my area, which is in ML but not related to LLMs or CV. I do look for work related to whatever I'm doing, but it's kind of depressing how uncommon it is for (say) neurips papers to actually have anything useful...
(It's also kind of annoying how basically all funding agencies are only funding research into or using AI, but don't provide enough funding for lots of gpu time lol)
Yeah it's very short, but I can't remember if I read all of it, or just flipped through pieces. It has good advice and I think fairly "famous." (edit: this reminded me I still have my copy :-D)
I know someone in a profession that does a lot of writing, and it blew my mind how clearly feedback was communicated by a superior. Made me wish tech people had better written communication skills. ;-;
yeah, it would be almost shocking if an open source benchmark was NOT used ~somewhere in training. Perhaps just pre-training, but still.
Neural networks can be fairly robust to some mistakes in their training data, so maybe it doesn't even matter if some of them are incorrect. Who knows.
1. US a lot of international PhD applicants, so traveling before even being accepted is difficult, and 2. lots of people don't do a masters.
when I applied to PhD programs (not in math) it was basically CV + personal statement + recommendation letters + short chats with interested faculty :shrug: Maybe it was because my CV was "strong" but the chats were more see if interests were aligned, rather than actually interviewing me.
Remote talks are also a common alternative in such cases. Zoom etc.
I also know that another major difference is that American universities tend to hire without a professors involvement, into a generic "program", then the PhD student seeks an advisor after being accepted, so applicants have to woo some unconnected committee pursuing various goals misaligned from the PIs instead of convincing the PI. In much of Europe a professor basically "owns" a chair and really is boss and decides hiring pretty much alone.
Anyways, if they don't have a masters' yet, they could present their bachelor thesis. But I already think it's a bad model to combine the masters (courses) and the phd (research) into this American hybrid that's the direct-bachelor-to-PhD jump, but that is somewhat unrelated.
I agree the combined masters + phd is weird tbh. I think almost everyone treats it like "just focus on research and spend as little time on course work as possible." I would prefer it if courses were more flexible.
I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
> I'm not a big fan of the US application setup. IDK how it is in Europe, but in the US, it feels like there's a lot of not-very-meritocratic "secret" stuff you need to know to up your chances.
In Germany (countries differ quite a bit on these details) a significant chunk (though not necessarily majority) is directly hiring a master's student who worked at the same chair on a part-time student research assistant job and/or a master's thesis and did a good job and typically turned the thesis into a paper submission. In other cases it's still somewhat inside baseball like getting recommendations from other scientists that the PI knows, or coming from a university that the professor knows well. It can also be from an impressive email, from a chat at a conference poster, etc. There is much less of this global casting that committees do, like "we need this and this type of character now" to have a vibrant student body composition, since each PI makes a decision just about their lab, without having a global view, and their goals are basically to get research out of that person, papers, results useful for funding applications etc. So they want to see aptitude in that. Through talk quality, communication efficiency when talking with existing team member one on one, recommendations, sometimes even phone calls with the PI with whom the student worked (these are small worlds). I think it's more organic and natural, though still prone to exclusionary gatekeeping, than the HR-style narrative essay-and-extracurriculars-based admissions though maybe that's more prominent for undergrad admissions even in the US.
I think people probably assume that openai / anthropics use of their data is probably like google's """limited""" use, in the sense that historically google wouldn't trivially be able to just take something from google cloud or someone's search history and insta-convert into some competing project... But LLMs are quite strong at approximately "memorizing", so I think that risk is wayyy higher.
numpy is pretty much all C and python. They may dispatch to some Fortran libraries, but I think basically all the internal implementations they have are C.
IIRC, scipy does (did?) actually use a lot of fortran fwiw.
I sort of wish they used c++ for some of the template stuff, especially since the code base already seems to have some C++ iirc.
I also contributed a tiny bit in the past, and their C template system was a surprise. (but this is cool and I love numpy anyway lol)
reply