Viewing a single comment thread. View all comments

Jaohni t1_irajujt wrote

Reply to comment by AnimalNo5205 in Samsung announces 36 Gbps GDDR7 memory standard, aims to release V-NAND storage solutions with 1000 layers by 2030 by Avieshek

I wouldn't say that HBM never went anywhere; it was a high bandwidth, high latency alternative to GDDR's (relatively) low bandwidth, low latency, which was achieved by essentially overclocking the interconnects in GDDR, leading to HBM being much more power efficient. And then they overclocked their Vega series through the moon, but anyway...

...HBM is still alive and well, but it's more commonly used in server and workstation applications ATM, where bandwidth is worth as much as the compute in the right workload. We might actually see some high end gaming GPUs in a year and a half to two and a half years here, as certain incoming trends in game rendering (raytracing, machine learning, and so on), can benefit from increased bandwidth, though at least on the AMD side I think they'd prefer to do 3d stacked cache as beyond having a higher effective bandwidth, it also basically improves the perceived latency, and power efficiency is more heavily improved than via using HBM.

3

oscardssmith t1_irdexbw wrote

As I understood it, HBM isn't higher latency. It's just more expensive. Is that incorrect?

2

Jaohni t1_irdfkqr wrote

So, imagine you have one lane to transfer data from memory to a processor. You're probably going to clock that lane as quickly as you possibly could, right? Well, that means it'll have the lowest latency possible, too. But, if you added a second lane, you might not be able to totally double bandwidth, because you might not be able to clock both lanes as high as just the one, but maybe you get 1.8 or 1.9x the bandwidth of just the one...At the cost of slightly higher latency, in this case, 1.1x the latency.

The same idea is basically true of HBM versus GDDR. GDDR essentially has overclocked interconnects to get certain bandwidth targets, and as a consequence has lower latency, but with HBM it's difficult to clock all those interconnects at the same frequency, so you get higher bandwidth and higher latency overall. Because it's less efficient to overclock those lanes, though, HBM ends up being less power hungry (usually).

1