Indeed. Are there any 'good' strategies for this (loading ES6 for supported browsers)? I imagine you'd have to do UA sniffing on the server side, and I suppose it's kind of a moot point if you're using React/JSX.
There's no need unless you need runtime-only concepts, like new string methods. Use https://babeljs.io/ to turn nice es6 into es5. (I imagine you already know this, if you're using react/jsx)
Just try to eval a sample of ES6 syntax you use, and if it throws error, load ES5 code. You'd need to feature check all the features that you use, to cover the browser versions with incomplete ES6 support.