I've lead remote teams and gathering the team 1x or 2x per year was absolute necessity to have everyone working well together.

Obviously you do this in a sensible manner (a lot of heads up, locked down dates so people can plan vacation around it, full expenses paid).