You're assuming that the AGI will communicate with the agents directly instead of through an LLM. If the agents are actually intelligent agents then the AGI may not be able to assume that the agents are not human, in which case it's safer for the AGI to use the LLM to define instructions for all tasks. And if that's the case then it will want to do all the work itself, if it's generally intelligent.