Submitted by Outrageous_Room_3167 t3_zmqp2q in deeplearning
Hey just purchased 6 3090 cards, I'm looking to build out a machine. Most likely will build out 2 separate machines.
​
I need some advice, the first build I want to do 4x3090 (with 2x NVLink Bridge's), I have a AMD threadripper as well, 84 pcie lanes.
​
I will probably need to build out a custom chassis, this is no problem but any advice on the pcie riser front?
I had a box fan in front of my mining rig with 6 cards. Was hot but worked great.
A mining rig has a load significantly different from DL loads. I work with these cards, we have like 10 rigs in the office with dual/triple 3090s, some Tis, I'm well aware of their limits.
Do you have a recommendation for 3x? At least the mobo and case?
Probably ROG Strix X570-E. Case doesn't matter much as long as it's a full tower, but as for the brand, Fractal usually makes good cases. Specifically, Fractal Torrent is the best all around case IMO, but they're sort of hard to find.
>NVLink probably won't matter much since your CPU will be bottlenecked trying to send 5.2 TB/s of data to your GPUs. But again, there are no benchmarks to show how much, maybe the gains from NVLink will be noticeable.
I guess the bigger benefit of the NVLink is the larger memory, but aside from that, I don't think the performance gains are huge from what I've read. My thinking was to build out a chassis with external fans as well to cool everything down.
There is no larger memory. NVLink only increases bandwidth by up to 300 GB/s unless there is a software implementation of memory pooling, which there isn't for any relevant DL framework.
Every week this has to be explained to yet another aspiring system integrator...
Not op but that’s good to know, so it would only speed the process and not make a big gpu with 24+24 gbs :(
You can always make a model parallelized model and have it on any card, not that hard. Your biggest problem is load balancing in that case, but it can be done with a bit of benchmarking and heuristics.
thanks for the keyword, I’ll keep this in mind when/if I buy another 3090
You know typing it out, i had some doubts LOL thanks
Mining community has been building these for quite some time now. Depending on you TR mobo, you might get away with pcie 3.0 risers. Cooler master has some quality models.
To avoid taking too much advice from miners, the thing with mining rigs is that the amount of data transfer between CPU and GPU is minimal so they can get away with 1x risers, which definitely won't be a good idea on a deep learning rig. This means pretty much no mining rig would function as a deep learning rig.
Exactly why I suggest Cooler Master x16 risers. (they don't have x1...) I use the 20 and 30 cm PCIe 3.0 ones as my motherboard does not support PCIe 4.0.
Chassis is the same as a mining rig.
My point is that OP can not just pick any mining rig and expect it to work. Mining rig chassis are designed so that you can use those USB risers that are long and easy to route. Now when you have the wide and relatively short risers it can limit the physical configuration.
Gotcha. Mining cases are not a good option.
Open frames like this https://www.amazon.com/Mining-Computer-Currency-Bitcoin-Accessories/dp/B09CNG58R1/ should work. Building your own base on available space is best though.
Mining rigs are not best for deep learning. A lot of mining rigs run on x1 pcie lane.
Look to get a HP Ml350 G9
It should be able to use 3 x 3090 easily at x16. A lot of the mining rigs only runs at x1 and not the best idea for deep learning.
Only problem I see is you will not get pcie4 with ml350 G9
I run 3x 3090 in a single case, without water cooling, but using one PCI riser and keeping the case open to allow for airflow. This is on a single 1600w PSU, no NVLink.
Anything more would be tough without a custom loop, and dual PSU.
Works great!
edit: I use a Fractal Design Design XL, and mount one 3090 FE vertically with a riser. Its janky but works.
>I run 3x 3090 in a single case, without water cooling, but using one PCI riser and keeping the case open to allow for airflow. This is on a single 1600w PSU, no NVLink.
Oh sweet. So maybe we go the route of 2, 3x3090 machines. Might be the better direction. Was considering 2000W PSU & 4x3090. How much memory & what CPU are you using?
We have a Ryzen 3950 max ram (128gb) - but I’d like to get a system that supports more ram and 16x on all pci slots. But alas - $$$
Isn't that amount of RAM good enough for the setup? Or you think it's a bottle neck in the training, just new to all of this :)
It’s fine - we haven’t had a huge issue due to it but we work on video related projects so memory is always a boon. That’s all!
I enjoy this type of build. Depending on your work space. They sell extruded aluminum meant for GPU Mining. It's got room for eATX mobos and space for 2 big PSUs. They are spaced far enough apart that if you get the 3000 rpm noctua industrial fans and line them up you can cool that many 3090s on a single rig. If you are willing to cough up enough for 6 PCIe 4.0 x16 risers. Remember they are finicky at best so make sure you keep your warranty papers.
Yeah we have a ton of extrude here our company does robotics stuff so bunch of tools and parts to build anything.
The first question is did you get two slot blower cards or 3+ slot gaming cards.
The second is do you want to build an actual server where noise levels are irrelevant or a somewhat quiet box.
The FE is 3 slot, 2 slot blower were too expensive. No, we don't care about noise.
Usually you'd get something like this https://www.supermicro.com/en/products/system/GPU/4U/SYS-420GP-TNR to host many cards, but with FE one I think they are too large. My best guess is to go for a 5H or even 6H mining server chassis.
suflaj t1_j0cxviw wrote
Chassis is not a problem, it's the heat.
Generally anything above 2x 3090 will need to be underclocked or in an open case to be under 90°C.
I don't think a 4x 3090 rig is possible without water cooling, since even with riser cables and an open case the cards are going to be fairly close to one another. The cards will need to be underclocked heavily and you will need the best power supply on the market, and you would still risk shutdowns or even hardware failure if 4 cards go into a transient spike at the same time. I would not risk it if you're building 2 rigs anyways, there is little benefit from a 4x and 2x configuration instead of a 3x and 3x configuration.
NVLink probably won't matter much since your CPU will be bottlenecked trying to send 5.2 TB/s of data to your GPUs. But again, there are no benchmarks to show how much, maybe the gains from NVLink will be noticeable.