And the size will be less and less until a certain information compression limit: Emad tweeted that their optimized model is already just 2.1 Gb and he hopes to make it less, around 100 Mb:

https://twitter.com/EMostaque/status/1564655464406650881

With all the model optimization, distillation papers already out, a few hundred Mb doesn't look impossible with similar quality outputs.