I don't think, for accelerating a desktop graphics card used across your entire desktop environment, IOMMU changes my argument at all.
I believe it *does* help if, say your desktop's integrated video works natively and you have a second card you want to pass through to a gaming VM. Or if you have a server with no card shared between users and a one-card-per-user scenario.
You can use IOMMU to restrict the access to the appropriate memory sections.