Comments

You must log in or register to comment.

step21 t1_jabzt1w wrote

If you say you had a good understanding until then, what changed? The GPT architecture as far as I know in newer editions didn’t change completely, but made smaller changes and spent a lot of time on better data, better curation/guidelines etc.

27

professorlust t1_jacfxvl wrote

Regarding ChatGPT, I believe OP is frustrated not by the Transformer architecture but by the improvements made in the inference functionality.

That’s the real “black box” of GPT style LLMs and the least open

1

jamesj t1_jac05ai wrote

Look up Andrej karpathys YouTube videos of building makemore from scratch

15

Borky_ t1_jac1gy9 wrote

he also had videos on building mini chat-gpt, man's a treasure

7

RingoCatKeeper t1_jac20hg wrote

Vote for Midjourney. I don't know how they improved their performance, no paper or publications.

7

Magnesus t1_jac83a7 wrote

There was some discovery made recently about something to do with offset noise during training - people are speculating that MJ did that while others didn't. Here is video explaining how it works: https://m.youtube.com/watch?v=cVxQmbf3q7Q

On the other hand if that was it MJ would be better at generating dark images, so maybe not? Shame they don't share how they do it.

7

RingoCatKeeper t1_jac8qzr wrote

Thanks for the link, this methods sounds workable. The earilest version of MJ results was somehow blurry and noisey, I wonder if it was because of this method.

2

Far-Butterscotch-436 t1_jad1g2v wrote

Just about every discussion i get notifications for is deleted , what s up with that?

1