A number of linguists have in recent decades suggested that the distinction between the kinds of phrasings we regard as “nativelike,” or “natural”, and those that strike us as jarring, stilted or just plain wrong, has more to tell us about language than the time-honored opposition between grammar and vocabulary. In the light of these trends, my paper revisits Coleridge’s critical claim that “it would be scarcely more difficult to push a stone out from the pyramids with the bare hand, than to alter a word, or the position of a word, in Milton or Shakespeare . . . without making the poet say something else, or something worse, that he does say.” The paper argues that the strange sense of naturalness Coleridge is evoking in this metaphor, and in the Biographia Literaria’s theorization of poetic diction more generally, can be clarified by that recent linguistic work, and discusses the writings of linguists Joan Bybee, Adele Goldberg, Michael Hoey, Paul Hopper, and Andrew Pawley and Frances Hodgetts Syder, to this end. The paper concludes by suggesting that the Biographia Literaria can contribute to those linguists’ discussions in turn, by illuminating the compositional processes through which new language is coined.