Excuse my laziness for not looking this up myself, but I have two 8G RAM M1 Macs... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		mark_l_watson on March 12, 2023 \| parent \| context \| favorite \| on: Using LLaMA with M1 Mac and Python 3.11 Excuse my laziness for not looking this up myself, but I have two 8G RAM M1 Macs. Which smaller LLM models will run with such a small amount of memory? any old GPT-2 models? HN user diimdeep has commented here that he ran the article code and model on a 8G RAM M1 Mac, so maybe I will just try it. I have had good luck in the past with Apple's TensorFlow tools for M1 for building my own models.

simonw on March 12, 2023 | [–]

The LLaMA 7B model will run fine on an 8GB M1.

recuter on March 12, 2023 | [–]

Rather remarkably, Llama 7B and 13B run fine (about as fast as ChatGPT/bing) if you follow the instructions in the posted article.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact