Basic state machines are useful for controlling flow between different menus/game screens - this "current mode" state exists across iterations of the game loop so you need some kind of data based state machine.
e.g. in say a jrpg you have "new game/continue" menu -> world map -> battles and character menu and back again.
Edit: oh I misunderstood you - you were asking about behaviour trees. I haven't used them for anything other than controlling game unit behaviour (although not necessarily just enemy ai - one other example is using them to combine a bunch of simple actions into one thing that the user can tell a unit to do - so like an attack move in an RTS is actually "loop (is enemy nearby? -> no -> move towards location -> yes -> attack (which is probably it's own behaviour tree))
