That's just it, the reason I noticed it was because they didn't continue at that speed. They pass, then they slow down to slower than my original speed (I'm also careful not to speed up at all if someone is passing me, that's stupid and reckless). I had thought about it being my own perception, but the fact is that I didn't first assume it was because I was driving a minivan. For the first few months I was just sort of annoyed that people seemed to be driving stupid, but mostly when I drove the minivan. Then when thinking about it it occurred to me that it was likely the minivan and perceptions that surround it that cause the problem.

My working theory is that people have bad experiences being stuck behind a slow minivan (which I'll admit, happens), and those experiences can be particularly annoying because it's hard to see past minivans. As people find themselves behind a minivan in a situation where their speed isn't quite yet stabilized (such as shortly after entering the freeway), they either assume it's going slow or just decide they want to not be behind a minivan, so pass.

I have had corroboration from other minivan owners I know, but I am open to being wrong or misinterpreting the information. I suspect it's less to do with that and more to do with specific traffic patterns that make it more or less likely (such as light traffic allowing easy passing), which may be more or less prevalent in certain areas.