What's the second remote for? ATV remote drives everything.
Even if you have, say, an PS5 or Xbox Series X plugged to same TV, the ATV remote screen button causes TV to take over from console and turning on a console controller causes it to take over from ATV.
Even if you have, say, an PS5 or Xbox Series X plugged to same TV, the ATV remote screen button causes TV to take over from console and turning on a console controller causes it to take over from ATV.
Assuming you turned on all the CEC stuff anyway.