An interesting take on Operating System design is the Synthesis Kernel that makes use of techniques like lockless data structures and run-time code synthesis:

http://valerieaurora.org/synthesis/SynthesisOS/