One practical aspect of the same clothes until age 6 is that kids grow so fast you need to re-use clothes. In the 18th and 19th century when having clothes made was much more expensive, and the number of children a family had was high, the re-use would have been essential. Any parent today who has had two kids of one sex and then the third comes along of the other sex finds they have a bunch of boxes of things that they don't want to use.
We sought to keep our baby clothes especially and up to about age 3 clothes as neutral as possible for that reason.
While boys and girls wore "dresses" till about age 6 or 7, till they were breached[1], they carried gender markers --at least in the upper classes. The clothes were not interchanged. The styles were different and people of the time would be able to identify the gender.
2. Boys sometimes wore bands (http://en.wikipedia.org/wiki/Bands_(neckwear) ). Due to their frequent use of lace they often are perceived as feminine today, but were perceived as masculine at the time. You can sort of see that on Roosevelt in the linked article. Some clearer examples are visible in the Wikipedia gallery of unbreeched boys: http://en.wikipedia.org/wiki/Breeching_(boys)#Gallery
Source, unfortunately paywalled (and it's a very minor part of the article, so not necessarily worth digging up unless you're interested in the main article subject): http://www.jstor.org/stable/1923418
Both of those gender markers look like they would be easy to alter with a bit of sewing, so the idea that clothing is gender neutral to make reuse easier still stands.
I find neutral clothing is not necessary because parents with older children are always handing down clothes of all colors. I would guess that having differently colored clothes means more are produced overall, but I think it is an acceptable price to pay for variety.
We sought to keep our baby clothes especially and up to about age 3 clothes as neutral as possible for that reason.