I recently upgraded my 5 year old computer from a AMD Ryzen 5950x to a AMD Ryzen 9950X3D together with everything that comes with that, new motherboard, new memory, etc..
As a storage minded person I wanted to get the “ultimate” storage setup for what was possible! Turns out the X870e chipset has some quirks to take into account! Let’s go through it including some benchmarks for the Samsung 9100 PRO and the Samsung 990 Pro!
Fast (over the top) storage
I want to start with that I realize very well that the storage setup I’m going to talk about is ridiculous for about 98%+ of the people out there. This is more of a “because I can” then a “this is absolutely” needed setup and configuration. But hey, if you like fast storage, you are in the right place!
The hardware
(Links to Amazon are provided if you wish to support me through affiliate links!)
- Motherboard
- Asus X870e ProArt Creator
- Memory
- 96GB 6000Mhz
- G.Skill Ripjaws M5 Neo RGB F5-6000J3036F48GX2-RM5NRK
- Storage
*All tests performed with a fresh install and updated version of Windows 11 24H2!
*Yes the photo shows a 9100PRO and a 980PRO, the tests were really done using 990PRO drives!
Understanding the chipset (AMD X870e)
The AMD Ryzen platform provides up to 28x PCIe Gen5 lanes which can be divided to your hardware but the AMD desktop platform has some assigned “limitations” which the mobile platform actually didn’t have and thus I was able to use 24 of these lanes in my recent server build!
28 Lanes Total
- x16 to Graphics card
- Or x8x8 for 2 PCIe slots
- Can de divided up more if required and motherboard has a switch to do so
- x4 to CPU connected NVMe slot
- x4 to Chipset (AMD X870e)
- x4 to USB4 output (AMD 870e mandatory)
Now here things get interesting! Since USB4 is mandatory for X870e this means we don’t have 28 lanes but 24, that’s actually less then before (or like I remarked, in my server build!). Now the chipset also needs some PCIe lanes which means it uses another 4 and thus now only 20 lanes are left to assign. That’s a graphics card with x16 and a single NVMe drive with x4 and it’s all gone!
We want to connect 3x x4 NVMe SSDs
So the premise of this article is to connect lots of NVMe storage! In our case that’s 1x 9100 PRO 4TB which needs x4 Gen5 lanes and 2x 990 PRO 4TB which need x4 Gen4 lanes each so 8 in total. So that’s 12 PCIe lanes… but wait, there is a problem!
20 lanes available (after USB4 and chipset) -16 for the GPU = 4 lanes left so we can only connect a single drive? Hmmm, that covers the 1x 9100 PRO 4TB which required Gen5 lanes anyway but where do the 2x 990 PRO 4TB go?
Chipset also provides NVMe storage! (Nope)
The AMD X870e also makes some lanes available as PCIe or in our case NVMe so that could solve our problem maybe?
Turns out, no. The chipset is connected using x4 Gen4 in total, not Gen5! So once you put a single drive behind the chipset, it’s bandwidth is fully saturated. The Asus X870e ProArt Creator actually has 2 NVMe slots behind the chipset, each designated as x4 Gen4 but when doing benchmarks it was quite obvious that the bandwidth was shared! Running a single 990 PRO 4TB or two in RAID0 basically made no performance difference whatsoever. (sadly I lost this benchmark data).
Desktop platform means compromises….
Since this is a desktop platform the amount of PCIe lanes you get is quite limited as I described above. You can basically run a Graphics card and a single CPU connected NVMe drive and that’s it, all lanes gone!
But most higher end motherboards (like this Asus X870e ProArt Creator) will add PCIe bifurcation options. In the case of my motherboard it allows you to run the x16 in x8x8 or as I ended up doing in x8x4x4! This means the primary graphics card get x8 Gen5 lanes, the PCIe slot gets x4 Gen5 lanes and an extra NVMe slot on the motherboard gets x4 Gen5 lanes!
This way I was able to connect all the 3 NVMe drives without limited bandwidth to them but this does mean the graphics card is running with less PCIe lanes then it possibly can. Luckily since we’re on Gen5 using x8 is basically the same as Gen4 x16 so still an enormous amount of bandwidth. TechPowerUp! always does PCIe scaling tests and even the mighty RTX 5090 only loses about 1% of performance running with x8 Gen5 lanes vs x16 Gen5 lanes so it basically doesn’t make any noticeable difference.
End result configuration
The end result is as described above:
- Graphics card
- x8 Gen5
- 9100 PRO 4TB
- x4 Gen5 CPU
- 990 PRO 4TB – 1
- x4 Gen5 CPU running in Gen4
- 990 PRO 4TB – 2
- x4 Gen4 X870e chipset
With this configuration I’ve done lots of benchmarks on the drives! Including testing Single drive vs AMD RAID vs Windows Dynamic Disk RAID0.
Single Drive benchmarks
The single drive benchmarks speak for themselves really. The difference between the 990 PRO 4TB that was connected to the CPU or through the chipset also showed minimal differences!
Single Samsung 9100 PRO 4TB
Single Samsung 990 PRO 4TB – CPU
Single Samsung 990 PRO 4TB – X870e port
AMD RAID benchmarks (a horrible experience)
So the next option I wanted to try is using AMD RAID which is built into the X870e chipset to see what kind of performance results that would bring.
Well whatever numbers down here say, don’t go for it, it was truly a horrible experience and you’re talking to someone who worked with RAID controllers professionally for over 25 years! I’ve seen clunky and horrible interfaces, trust me but the AMD one actually made me loose data (hence I’m missing some benchmark figures!).
So first you go into the BIOS and enable NVMe RAID mode in the SATA menu (lol). Reboot and then the RAIDXPERT2 menu becomes available in the BIOS. I wanted to leave my 9100 PRO drive alone and just make a RAID0 array of the 2x 990 PRO. This seemed to work just fine, it allowed me to create an array of just those 2 drives and not touch the 9100 PRO, save, reboot and booting into the OS.
Oddly I couldn’t install the drivers or the tool available from AMD since it said my boot drive wasn’t in a RAID configuration, well duh, but another is… so?
Also the OS would still see the native drives with weird partitions on there, don’t like that either, easy to make a mistake!
Anyway, after force installing the 3! drivers manually I was able to install the software tool (which took 10 minutes?) but a reboot was required before I could use it, ok.
Reboot…. system no longer boots, no system partition available. Huh? Going into RAIDXPERT BIOS part it showed the 9100 PRO was there but it was “offline” and there was no way to get it working again. I figured I’d add the drive as a “volume” which in theory should leave it alone but then it would at least be known in the RAID configuration and boot again. Yeah no, IT WILL DESTROY ALL YOUR DATA. In my case it wasn’t a big loss since it was a new install anyway but be warned!
Well ok, we’re in the AMD RAID rabbit hole now, guess we get to install the OS too. Boot the Windows install stick and add another stick with the drivers on there, no big deal.
Took me an hour of multiple tries to figure it out, you need to load all 3 drivers (no indication of this) into the setup program? Sometimes it will throw an error, hang, just reboot or other weird stuff. But if you do happen to do it in the right order (while the wizard seems to not acknowledge anything) your ARRAY drives will actually show up!
Ok, install! I quickly benchmarked it and then absolutely disabled this shit again… holy crap, I know with RAID0 your data isn’t safe but with this horrible layer around it you know you will loose it at some point, including your non-RAID OS drive since it’s encapsulated in the same way and can’t be read normally anymore by another system, BE WARNED!
RAID 9100 PRO 4TB
I believe the lower results vs direct NVMe is caused by the extra encapsulation layer and with storage this fast, any extra steps will cause extra latency and thus lower performance.
It might have been my imagination or maybe bad after taste of the install experience but things also felt just ever so slightly less snappy. Sadly the benchmarks I did won’t really reflect latency in that way.
RAID 2x 990 PRO 4TB – RAID0
Hmm, ATTO snafu, I did setup bytes/sec but seemingly it didn’t switch over in the screenshot, I have added IO/sec for all the other configurations too to compare!
Compared to a single drive the RAID (when connected correctly PCIe wise as we talked about above!) does show a big improvement in performance. Looking at large sequential transfers we basically see about a doubling of performance for low and high queue depths! The randoms don’t scale the same way though but do also see a bit of an improvement. Interestingly the 2x 990 PRO in RAID0 are about as fast as the 9100 PRO is on it’s own! Given the available bandwidth this makes sense!
But whatever the results here, the (imagined?) sluggishness and horrible software experience, I wouldn’t want to run this way! Let’s try a different solution!
Windows Dynamic Disk
Another option is the tried and true Windows Dynamic Disks – RAID0! This has been available since Windows 2000 I believe and while not awesome (like for instance ZFS) it isn’t bad either and has a proven track record for sure! It’s also fully software so if I pop the same drives in any other windows system I can just read the drives, no problem.
I also tried the newer Windows Storage spaces but for some reason they’ve removed the RAID0 option from the Windows 11 version? Yeah, no dice.
Another advantage this approach has above using AMD RAID is that the drives SMART information and things like firmware upgrade stay working!
And as I’ve highlighted in the RAID part, Windows Dynamic disks can easily be transported between systems, a big plus in my opinion if I do ever need to recover something or parts of this system die!
Software – 9100 PRO 4TB
No need to repeat the results, the same as above!
Software – 2x 990 PRO 4TB – RAID0
Quickly glancing at these results vs the AMD RAID results we see that Q8T1 results are about the same but the Q1T1 results are quite a bit slower. This is likely because the AMD driver was doing a little bit of caching or read-ahead, accelerating these normally single threaded operations.
We see the same in the other results, the Windows Dynamic disk solution is a little bit slower vs the AMD RAID results. So win for AMD RAID in that sense but because of the horrible software experience, I’m not going to go there!
Conclusion
This article was mostly meant to take you along on the journey I had building, optimizing and testing my new system in regards to it’s 12TB of NVMe storage! As you can see there are some caveats to workaround because of it being a desktop system but once you get there performance is quite fast!
I believe for general purposes the 9100 PRO 4TB and 2x 990 PRO 4TB in RAID0 are about as fast, for anything latency sensitive though the single 9100 PRO will win, hands down since there isn’t a software layer in between it. I could also see this doing benchmarks, sometimes it would just fill up a single core completely!
Still results are good and I’m happy running the 990 PRO 4TB in software RAID0 using Windows Dynamic disks, it’s going to hold my Nextcloud folder, downloaded games and other things like that, no data I’m going to miss and even if it does, I run hourly off-site backups on anything near to sad if I’d loose it, so even no big loss then.
And hey, using the 2 drives in RAID0 actually “doubles their TBW”! The TBW for all drives is the same at 2.4PB per drive but with them in a RIAD0 that’s now 4.8PB with an even wear spread over the 2 drives.
The end the article I’ll re-iterate, 98%+ of people out there don’t need this kind of storage setup, hey even I don’t, not really, I just enjoy this stuff! 14GB/s storage performance is insanely fast and everything flies but then again, a x4 Gen3 drive would likely or a decent x4 Gen4 drive like the 990 PROs just by themselves, would likely be just as fast for 98% of tasks!
14GB/sec sure is awesome though! 😀