Is LLM Interpretability Proven?
A recently published research essay claims that the difficulty of neural network interpretability has been overstated. In his essay on goodfire.com, Nick Cammarata argues that neuron polysemanticity is caused not by superposition but by ordinary, mostly fixable issues — and that most neurons, even in massive models, perform a single, understandable job. The author himself calls the text a working hypothesis: the technical paper with a full description of the experiments has not been released yet. However, as usual, a bold headline and strong claims have sparked numerous retellings in which the hypothesis has turned into established fact.Below is a quick look at this hypothesis.Spot-checking Cammarata’s claimsSummarizing the essay is probably unnecessary. It is written clearly and persuasively, so it is easy to follow the arguments and the preliminary description of the experiments on your own.
