To start a fiber, you allocate some memory, set RSP to the end of that memory, set your other registers to some arbitrary initial state, and jump to your fiber routine. To switch fibers, you set RSP to some other block of memory, restore your registers, and set PC to whatever it was when you last switched away from that fiber. There's nothing magical, and it works with almost all existing code. If you hold a mutex and switch to a different fiber, the mutex stays held. How could it be otherwise?
I was thinking of a situation where thread-aware but not fiber-aware code uses mutex to synchronize with itself, which breaks with fibers because they reuse the same thread, and the mutex is bound to that thread (so if another fiber tries to acquire that mutex, it's told that it already has it, and proceeds to stomp over shared data with impunity).
But upon further consideration, I realize that in this narrow scenario - where fibers are used in conjunction with callback-based APIs - this shouldn't apply, because you can't synchronize concurrent callback chains with plain mutexes, either.
Having said all that, are there any actual implementations that seamlessly marry fibers with callbacks? I don't recall seeing any real world code that pulled that off. Which seems to imply that there are other problems here.
Of note is that CLR tried to support fibers, and found it to be something that was actually fairly expensive. By extension, this also applies to any code running on top of that VM:
"If you call into managed code on a thread that was converted to a fiber, and then later switch fibers without involvement w/ the CLR, things will break badly. Our stack walks and exception propagation will rely on the wrong fiber’s stack, the GC will fail to find roots for stacks that aren’t live on threads, among many, many other things." (http://joeduffyblog.com/2006/11/09/fibers-and-the-clr/)
GC is a sticking point here, it seems - clearly it needs to be fiber-aware to properly handle roots in switched-out fibers.
