I just asked a guy working there, he confirmed that the screen and the touch inputs are solely driven by the SE. Basically the OS only stores a few fonts and all the graphical shapes are rebuild at runtime, it's a time for space trade-off.
Wow, and all that runs on the SE? That would be really impressive!
It's still not ideal though, given that every additional feature or library blows up the trusted code base and increases the scope of any audit as a result.
Well, I just hope the existing wallets will remain supported going forward.
They have a modular approach where the OS is not updated very often, and over it are running userland applets that define their own UI and features and can be audited separately. These applets are built using Ledger's SDK which make use of the security features of the OS through a bunch of scrutinized syscalls.