Using phone audio in a way that benefits from that extra increase in audio quality is the minority case (based on my observations). Most people listen on the their phone when they are 'out and about', walking in town with car noise in the background, on the subway etc.. In those environments, the returns diminish quite quickly past a decent pair of noise isolating in ear headphones.

This is the case for which the new situation is the unarguable improvement, but I think it's rare enough that consumers as a group have lost more than they gain.