I'm tired of these posts; LLMs are good for happy-path demos, that's it. And even then, their success rate depends on the prompter already knowing the answer!
Literally any out-of-distribution project in which I used LLMs lead to catastrophic failure. The models can't "see" stuff outside their training data.
by micheles
0 subcomment
As a former Theoretical Physicist, this result is remarkable. I myself I tried to use AI for calculations in Perturbative Quantum Field Theory and I was impressed. I agree with the authors: it looks like the future of Theoretical Physics would be more in verification and consistency checking of AI-assisted results rather than in manual calculations.
Hacker News