Viewing a single comment thread. View all comments

PassingTumbleweed t1_jeh0p1j wrote

Reply to comment by KD_A in [P] CAPPr: use OpenAI or HuggingFace models to easily do zero-shot text classification by KD_A

I'm curious to get your thoughts about a simple example where you have three classes: cat, dog, and bird. What happens if the top-1 prediction is "eagle"? Does that probability mass get discarded? Because it should probably go into the bird category

1

KD_A OP t1_jeh0ygl wrote

Yup it gets totally discarded. Hopefully, the conditional probability of bird is higher than cat or dog.

2

PassingTumbleweed t1_jeh1248 wrote

One thing I've seen with these LLMs is that you can prompt them with the classes using sort of a multiple choice style. It would be interesting to experiment with whether this can stabilize the outputs and reduce the amount of out of vocabulary predictions you get

2