Why LeCun’s World Model Won’t Save AI
After the unexpected divorce between LeCun and Meta, there is a lot of talk that the dead-end in LLM progress will be overcome through the physics of the world. That is, having a neural network work with physical data from the surrounding environment will allow the model to acquire meaning and an understanding of its actions. LeCun has a foundational paper that nobody is going to read. So, I'll summarize it as best I can. Essentially, the idea is that the current trajectory of LLM development is doomed. As long as they are predicting the next token, real understanding — the emergence of real meaning — is impossible. LeCun proposes training neural networks on physical world data, assuming that building a model of it will allow the system to discard details and focus on meaning.
