> And I found a fascinating pattern: the AI gives artificially high scores to reports written with AI [...] it was giving very high marks to poorly reasoned, error-filled work simply because it was elegantly written. Too elegantly... Clearly written with ChatGPT.
This is an interesting phenomenon, but I would have liked to see some quantitative evidence for this N=24 sample, e.g. would a paper ordinarily get an 80% score but the LLM gives it a 95%?
I also wonder how accurate a professor's perception of style is. I tend to write in a formal style, even in online forums like this one, and I wonder if people assume I use LLMs as a result (I don't).
This is an interesting phenomenon, but I would have liked to see some quantitative evidence for this N=24 sample, e.g. would a paper ordinarily get an 80% score but the LLM gives it a 95%?
I also wonder how accurate a professor's perception of style is. I tend to write in a formal style, even in online forums like this one, and I wonder if people assume I use LLMs as a result (I don't).