Viewing a single comment thread. View all comments

wywywywy t1_j151o6u wrote

You could run a cut-down version of such models. I managed to run inference on OPT 2.7B, GPT-Neo 2.7B, etc on my 8GB gpu.

Now that I've upgraded to a used 3090, I can run OPT 6.7B, GPT-J 6B, etc.

5