Viewing a single comment thread. View all comments

UnderstandingDry1256 t1_j5c0y0o wrote

What are the training strategies used for GPT models? Are transformer blocks or layers trained independently? Are they trained using some subset of data and fine tuned then?

I would appreciate any references or details :)

2