I also have a TrueNAS, but because of its limitations (read-only root file system), I came to the conclusion that, if I ever need to reinstall it, I would switch to Proxmox and install TrueNAS as one virtual client, next the other clients for my home lab.
I have found workarounds for the read-only root file system. But they aren't great. I have installed Gentoo with a prefix inside the home directory, which provides me with a working compiler and I can install and update packages. This sort of works.
For running services, I installed jailmaker, which starts a lxc debian, with docker-compose. But I am not so happy about that, because I would rather have an atomic system there. I couldn't figure out how to install Fedora CoreOS inside a lxc container, and if that is even possible. Maybe NixOS would be a another option.
But, as I said, for those services I would rather just run them in Proxmox and only use the TrueNAS for the NAS/ZFS management. That provides more flexibility and better system utilization.
I use TrueNAS Scale as root OS and have it run a Linux VM, which is easily done via their 'Virtualization' feature. No need for Proxmox. Afaik it works a lot better to give zfs direct access to underlying hdds.
TrueNAS also has an 'Apps' feature, which are basically glorified helm chart installs on k3s that TrueNAS installs for you. But I prefer more control so I have k8s on the Linux VM. Whats also great is that the k8s on the Linux VM can use the TrueNAS storage via democratic-csi.
The deprecation caused me to move to something more neutral and stay away from all 'native' apps of TrueNAS and migrated to ordinary docker-compose, because that seem to be the most approachable.
I was also looking into running a Talos k8s cluster, but that didn't seem to be as approachable to me and a bit overkill for a single-node setup.
> I also have a TrueNAS, but because of its limitations (read-only root file system)
It isn't really the case. TrueNAS wants you to look at it as an appliance so they make it work that way out of the box.
On the previous release, they had only commented out the apt repos but you could write to the root filesystem.
On the latest release, they went a little further and did lock the root filesystem by default but using a single command (`sudo /usr/local/libexec/disable-rootfs-protection`), root becomes writable and the commented out apt repos are restored. It just works.
I have both of these releases running side by side for multiple years by now. It will not auto-update between releases anyway similarly to how nobody would do a dist-upgrade on you automatically. Neither have ever overwritten my changes to enable rootfs rw + apt repo fix and other changes to the filesystem, no more than a normal Debian would. Enabling apt actually gets you a more up to date system than you'd get otherwise.
I have found workarounds for the read-only root file system. But they aren't great. I have installed Gentoo with a prefix inside the home directory, which provides me with a working compiler and I can install and update packages. This sort of works.
For running services, I installed jailmaker, which starts a lxc debian, with docker-compose. But I am not so happy about that, because I would rather have an atomic system there. I couldn't figure out how to install Fedora CoreOS inside a lxc container, and if that is even possible. Maybe NixOS would be a another option.
But, as I said, for those services I would rather just run them in Proxmox and only use the TrueNAS for the NAS/ZFS management. That provides more flexibility and better system utilization.