beezlebub33 t1_j03sufl wrote
Reply to comment by cole_braell in Is it just me or does it feel like GPT-4 will basically be game over for the existing world order? by Practical-Mix-4332
>GPT4 - 100 Trillion Parameters
I must be out of the loop. Where is that rumor from?
__ingeniare__ t1_j03tzp3 wrote
It's just a rumour and I think Sam Altman basically denied that this was the case. Another, perhaps more plausible, rumour is that GPT-4 will have a very different architecture where the parameter count between it an GPT-3 doesn't say much because it's no longer just about bruteforce scaling.
beezlebub33 t1_j04k5c9 wrote
That would, IMHO, be a big win. Even if the scaling hypothesis is correct, why would you want to solve the problem that way, when there are probably far better ways to solve it.
Sure, we could fly an interstellar spacecraft to another solar system, but it would be a bad idea to do it, because in the time that it would take to get there, some other ways of getting there would be invented. IF you left for the stars now, people would be waiting for you when you got there.
In the same way, simply scaling compute and data may get you to a certain amount of intelligence. But the costs and effort would be huge. It would probably be better to spend that time and effort (and money) on making the underlying ideas better. And even if it turns out that, yes, we have to scale, waiting until computational costs come down further is probably a good idea.
cole_braell t1_j03tlv3 wrote
Here, but now I’m reading elsewhere that they may have pulled that number out if their ass.
Viewing a single comment thread. View all comments