n00b question, I always see service meshes used in the context of containers and mostly with kube. Would they work with more monolithic/traditional n-tier architecture deployed directly on host OS as well? Or maybe put another way, are there likely to be pain points that don't exist in containerized architectures?
The main pain point at the moment is that meshes were written for containerized environments first, and attempts to extend full functionality to other environments are pretty immature at the moment.
Meshes are a lot more than just sidecar proxying -- they are what make sidecar proxying manageable, and they add a lot of other features like authentication, network policies, various other traffic control policies, service discovery, etc. They are an attempt to do for service-to-service communication what Kubernetes has done for container deployment -- make it abstract and declarative, with configurations that are independent from the underlying implementation.
The underlying implementation that works right now is the Kubernetes API and etcd, and alternate implementations need to be provided for those features to work well outside of Kubernetes. I think it will happen sometime in the next few years.
Meshes are there to abstract away stuff that is much more manageable in a monolith. If you have 100 microservices implemented using 5 different platforms you need libraries and programming discipline to implement retries with backoffs, circuit breakers, health checks, tracing, service discovery and other similar stuff in every single one of them.
In a monolith you need to implement some of this stuff only once and you don't need a lot of it at all because you are not making remote procedure calls.