You could do a lot worse than modern GC. For instance, in Rust, lack of GC may cause you to reallocate memory or use reference counting, both of which are likely slower than GC. For instance, building a tree in Rust will do a lot of reference counting, which is a lot of heap accesses.
> building a tree in Rust will do a lot of reference counting
This isn't true in most cases. If every subtree is only referenced by its (unique) parent, then you can use a standard Rust "Box", which means that during compilation, the compiler inserts calls to malloc() (when the Box is created) and free() (when the Box goes out of scope). There will be no reference counting — or any other overhead — at runtime.
