If you want to be an amateur chemist I recommend not getting your instructions from an LLM that might be hallucinating. Chemistry can be very dangerous if you're following incorrect instructions.
From experience as a failed organic chemist (who happily switched to computational chemistry for reasons of self preservation) I can tell you it's plenty dangerous when you're following correct instructions :^)
I was talking about cow eggs specifically! When ChatGPT et al got out, one of the funniest things to do was ask it about the best recipes for cow egg omelette or camel egg salad, and the LLM would provide. Sadly, most of it got patched somehow.
Oops... Yep, I missed that too. (On the internet, no one knows you're a dog.)
That's funny. It makes me wonder how these statistical mad libs machines will handle the gradual boundaries nature gives us. Almost all mammals give birth live, but not all. Nearly all mammals had mammalian parents, but not all.
Daniel Dennett was making this argument for why we haven't developed reasonable models for the nature of consciousness. It's because we're so sure there will be an absolute classification, and not a gradual accumulation of interacting systems that together yield the phenomenon.