Viewing a single comment thread. View all comments

nmfisher t1_jdyeyit wrote

IMO the area most ripe for picking is distilling larger pretrained models into smaller, task-specific ones. Think extracting a 30mb LM from Lllama that is limited to financial terminology.

There's still a huge amount of untapped potential.

40