I saw on level1techs, a 4 Bay chassis with an AMD APU plenty capable of transcoding multiple streams. Can't remember who made it, and you'd have to DIY the OS install, but that's pretty ideal to me.
What i do, and will continue to do, is use a USB-C disk box (Level 1 also recently reviewed some they quite liked, despite the usual fears around USB) and whatever PC i have laying around. 5 years strong running ZFS over USB 3 with a 4770K, regularly serving 4 Plex streams at once without complaints and no failures (i mean, usual disks wearing out, but nothing caused by USB).
So if a 4770 can transcode 2x1080 and direct play 2x more, any old anything with hardware transcoding these days should be just fine.
What i do, and will continue to do, is use a USB-C disk box (Level 1 also recently reviewed some they quite liked, despite the usual fears around USB) and whatever PC i have laying around. 5 years strong running ZFS over USB 3 with a 4770K, regularly serving 4 Plex streams at once without complaints and no failures (i mean, usual disks wearing out, but nothing caused by USB).
So if a 4770 can transcode 2x1080 and direct play 2x more, any old anything with hardware transcoding these days should be just fine.