What do you recommend for a text summarization task?

Submitted by average-joee t3_11gz0te in deeplearning

Hello there, I have a dataset of product reviews. At least 10 reviews per product. And their respective summary of those 10 reviews (Ground Truth).

I'm looking to use transfer learning from a pre-trained model for the summarization task. Just wanted to ask for some recommendations.

6

Comments

You must log in or register to comment.

LetMeGuessYourAlts t1_jar11st wrote

How many of them are there? If you've got 100's of them, you could do it for a couple bucks or less with gpt-3 davinci without a ton of prompt engineering. If I had a ton of them I'd probably go with gpt-j and see if could do a serviceable job with few-shot learning.

2

Mkboii t1_jas438i wrote

If you have the budget, gpt3/gpt3.5 are pretty great at it.

If you want to do it on your hardware/cloud instances.

You can go for gpt-j or flan-T5, they do great with basic prompts.

Or you can try a finetuned model, you might be able to get a finetuned T5 or there's BART and pegasus as well.

For summarisation even smaller models are pretty good so don't just jump to the biggest models. Compare the performance.

2

fundamental_entropy t1_jasohit wrote

Fine-tuning flan T5 xl or XXL can give you decent results. From my experience these are best open source models to fine-tune on . However they won't match results of larger models like gpt 3.5. But if you have millions of such reviews then chatgpt or gpt 3.5 may not be financially feasible.

1

average-joee OP t1_jaspyf6 wrote

Since you mentioned Huddingface, What do you think of Pegasus for Abstractive Summarization?

1

fundamental_entropy t1_jasqy64 wrote

Flan models are trained in almost every open dataset available in Generic English tasks. Recent research suggests models trained to perform multiple tasks (in fact ratios of different tasks too affect see flan 2022 paper) are better than models trained only on a given task. Flan T5 beats T5 in almost every task and sometimes Flan T5 XXL matches gpt3 type of prompt generation.

3

boosandy t1_jb0btu3 wrote

Flan t5 xl is a very good transformer finetuned on instructions. T5 is also good. You might want to tweak around some generation parameters to get good results.

1