As mentioned elsewhere, researchers have done brain scans while asking people to imagine something, and for the majority of people the visual cortex lights up, but for a small number of people the visual parts of the brain are not so active.

This is very much a real thing, but largely goes unnoticed because it doesn’t really affect anything, except for people going about their lives thinking that the word ‘visualise’ is a metaphor.