Viewing a single comment thread. View all comments

Zondartul t1_j8hd29v wrote

Reply to comment by Disastrous_Elk_6375 in [R] [P] OpenAssistant is a fully open-source chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so. by radi-cho

The plan is to make it kinda good and train in (on industrial hardware) and then distill it down to a smaller model that ideally can fit in a consumer GPU. It's going to be big at first but they do want to make it small eventually.

5

Disastrous_Elk_6375 t1_j8hdb2r wrote

Do you know if distilling will be possible after instruct finetuning and the RLHF steps? I know it works on "vanilla" models, but I haven't searched anything regarding distillation of instruct trained models.

2

Zondartul t1_j8i1bdg wrote

Sorry, I just casually watch Yannic Kilcher's YT videos, so I don't know much else.

3