What. The. ***k. [less than 1B parameter model outperforms GPT 3.5 in science multiple choice questions]VeganPizzaPie t1_j9kjuth wrote
Reply to comment by Lawjarp2 in What. The. ***k. [less than 1B parameter model outperforms GPT 3.5 in science multiple choice questions] by Destiny_Knight
But if you have to reason out the correct answer, and do that over and over again, it doesn't matter if there are 4 options or 1000. Think about it. The bar exam and other post graduate tests have multiple choice. You think anyone could pass those? Why do they take years of study?
Lawjarp2 t1_j9liaa5 wrote
No. Once an LLM gets a keyword a lot of related stuff will come up in probabilities. Also you can go backwards on reasoning. This makes it easier for an LLM to answer if trained for this exact scenario.
Viewing a single comment thread. View all comments