If you're still so far apart that you can't see your neighbors it's still "empty". The amount of space available in the United States boggles the mind when you actually go out there and drive for hours seeing almost no one. The damage to nature comes from large industrial activities like farming (fields or trees) or strip mining. Simply living there would actually help preserve that wilderness.
i use to wonder how you could detonate a nuclear bomb and no one notice, then I drove through New Mexico. The shear amount of nothing is hard to put in words. I spend a lot of time in far West Texas and Brewster county seems like beehive of activity compared to New Mexico.