Testing options SATA expansion via M.2

Post Reply
brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Testing options SATA expansion via M.2

Post by brad »

Happy new year all,

Im currently looking to setup SAN/NAS and considering physical disks.

Not 100% sure if this will work but planning to have 6 x ssd sata discs connected.

M.2 to mini SAS adaptor - https://www.ebay.com.au/itm/StarTech-U- ... 3884696603
mini SAS to 4x SATA - https://www.ebay.com.au/itm/Mini-SAS-SF ... 3522114198

This should hopefully give 6 high speed SATA ports that I can use in total on the H2+. Ive ordered another H2+ and products listed above for a test before I go out and buy all the drives.

RAID 10: 3 disk stripped together and then mirrored
RAID 50: 2 x RAIDZ mirrored

RAID 50 should allow fastest read speeds and most available disk space whereas RAID10 will have better write performance. Plan is to use 6x 500Gb SSD's which in RAID10 is 1.5Tb usable.

I looked at connecting M.2 to 5x sata adapter but they all use M.2 connectors with only 2x pcie lanes so going to test if the path of M.2 to mini SAS to 4x SATA is possible.

Plan was to install FreeNAS as use it as block / file storage (which is now TrueNAS CORE) but the new TrueNAS SCALE which is about to be released looks better. It allows horizontal scaling of the cluster by adding more machines and some other nice features like running containers or vm's.
These users thanked the author brad for the post:
odroid (Mon Feb 15, 2021 3:40 pm)

TurdFurguson
Posts: 7
Joined: Sat Jul 18, 2020 4:07 am
languages_spoken: english
ODROIDs: H2+
Has thanked: 26 times
Been thanked: 0
Contact:

Re: Testing options SATA expansion via M.2

Post by TurdFurguson »

One concern / question. The "mini SAS" adapter is actually a U.2 adapter. Now, I could be wrong (and often am), but while the connectors are the same is the SAS protocol the same as U.2?

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

TurdFurguson wrote:
Mon Feb 15, 2021 5:20 pm
One concern / question. The "mini SAS" adapter is actually a U.2 adapter. Now, I could be wrong (and often am), but while the connectors are the same is the SAS protocol the same as U.2?
This is the question I need to answer with my test I think, from all my research I'm unsure at this stage it might some down to firmware even.
These users thanked the author brad for the post:
TurdFurguson (Tue Feb 16, 2021 1:41 am)

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

The mini SAS to sata test did not work, essentially the adapter requires the SATA lanes to exist on the connector.

I do believe this will work https://www.amazon.com.au/Internal-Non- ... B07T3RMFFT as it has a proper pcie to sata controller (JMB585).

The only issue I don't like here is only 2xPCIE lanes at pcie 2.0 speeds. Will likely be a bottleneck I think for 4 x SSD but I think it would work well for up to 2 SSD's (or 4+ HDD). Maybe I'm wrong?

User avatar
odroid
Site Admin
Posts: 36930
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 1595 times
Been thanked: 1072 times
Contact:

Re: Testing options SATA expansion via M.2

Post by odroid »

Sad news. :(
I think that HDDs will not have a problem while SSDs will have some band width bottleneck certainly.

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

I had a good look around at M.2 cards but could find nothing with pciex4 interface they are all max pciex2 (even if they don't make that obvious). I also looked at standard full pcie to sata adaptors and all (most) of these are pciex2 only.

Final option I think is a ServeRAID H1110 SAS/SATA Controller https://lenovopress.com/tips0831-serveraid-h1110

A relatively small pcie card with 4 x 6G sata, can fit in the case (type 5 case) along with 6x 2.5 SSD. Has a PCIEx4 version 2.0 connection, can be flashed with non raid firmware and is quiet cheap. Drivers exist for windows, linux & BSD. Think ill order one and try my luck.

defcon
Posts: 22
Joined: Tue Feb 02, 2021 10:48 pm
languages_spoken: english
ODROIDs: H2+
Has thanked: 0
Been thanked: 3 times
Contact:

Re: Testing options SATA expansion via M.2

Post by defcon »

brad wrote:
Fri Feb 19, 2021 3:21 pm
I do believe this will work https://www.amazon.com.au/Internal-Non- ... B07T3RMFFT as it has a proper pcie to sata controller (JMB585).
Yep, works nice for me on H2+ with 3x 12TB disks, more disks coming. JMB585 is quite fast, some tests https://forums.unraid.net/topic/41340-s ... ity-check/
But you will need custom heatsink on the chip, or you'll get I/O errors after a while. Like this. There is blue silicone thermal pad 1.5mm thick between chip and heatsink and its zip tied. And the black wire stuffed in heatsink is temp sensor from case fan control board.
IMG_0159.jpg
IMG_0159.jpg (248.8 KiB) Viewed 171 times
These users thanked the author defcon for the post (total 2):
brad (Sun Feb 21, 2021 4:43 pm) • odroid (Mon Feb 22, 2021 9:21 am)

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

defcon wrote:
Sun Feb 21, 2021 10:53 am
brad wrote:
Fri Feb 19, 2021 3:21 pm
I do believe this will work https://www.amazon.com.au/Internal-Non- ... B07T3RMFFT as it has a proper pcie to sata controller (JMB585).
Yep, works nice for me on H2+ with 3x 12TB disks, more disks coming. JMB585 is quite fast, some tests https://forums.unraid.net/topic/41340-s ... ity-check/
But you will need custom heatsink on the chip, or you'll get I/O errors after a while. Like this. There is blue silicone thermal pad 1.5mm thick between chip and heatsink and its zip tied. And the black wire stuffed in heatsink is temp sensor from case fan control board.
Thanks for the details on JMB585 M.2 board. Interesting that the heatsink was needed to prevent IO errors (I guess the small M.2 board had difficulties dissipating the heat).

Also the page with all the tests is very interesting and helpful as well. Just need to remember that the odroid H2 / H2+ is limited to PCIE version 2.0 but the page shows the JMB585 in PCIE 3.0 with 2 channels so results will be much different on Odroid. The tests look good for the H1110 though with PCIE 2.0 x4 :)

defcon
Posts: 22
Joined: Tue Feb 02, 2021 10:48 pm
languages_spoken: english
ODROIDs: H2+
Has thanked: 0
Been thanked: 3 times
Contact:

Re: Testing options SATA expansion via M.2

Post by defcon »

Nice find, I did not know about PCIe 2.0. Anyway, 2x lanes give 1000MB/s, 200 per disk. It does hit performance of spinning hdds, but very little. SSDs are out of luck

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

I have ordered a ServeRaid M1015 and ServeRaid H1110 card for testing, hopefully they don't take too long to arrive.

The H1110 is 4x pcie 2.0 4 port SATA/SAS and the M1015 is 8x pcie 2.0 8 port SATA/SAS. They can be flashed with either RAID or non RAID firmware. I grabbed the 8 port sata version as well so I can try to connect up some HDD's along side the SSD's for an additional storage pool.

I also did some initial testing with TrueNAS SCALE (It is the linux based alpha version of TrueNAS CORE). Unfortunately it would not detect an emmc based device so I needed to install the OS to a SSD meaning it could not be used for the main storage pool (although OS could be installed to USB). TrueNAS scale alpha looks like it will be very nice once complete but at the moment the nighty builds are missing features.

Next to test if TrueNAS CORE (BSD based) will detect the emmc for installation of the OS.

defcon
Posts: 22
Joined: Tue Feb 02, 2021 10:48 pm
languages_spoken: english
ODROIDs: H2+
Has thanked: 0
Been thanked: 3 times
Contact:

Re: Testing options SATA expansion via M.2

Post by defcon »

brad wrote:
Sun Feb 28, 2021 10:51 am
test if TrueNAS CORE (BSD based) will detect the emmc for installation of the OS
It will. I tested FreeNAS initially, different SBC though.Then OMV and Unraid and next going back to TrueNAS (bsd).
brad wrote:
Sun Feb 28, 2021 10:51 am
M1015 is 8x pcie 2.0 8 port SATA/SAS
Its not for Odroid H2+, right?

brad
Posts: 1380
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 N1 N2 N2+ H2 H2+ (64 bit ftw)
Location: Australia
Has thanked: 120 times
Been thanked: 181 times
Contact:

Re: Testing options SATA expansion via M.2

Post by brad »

defcon wrote:
Sun Feb 28, 2021 10:56 am
brad wrote:
Sun Feb 28, 2021 10:51 am
test if TrueNAS CORE (BSD based) will detect the emmc for installation of the OS
It will. I tested FreeNAS initially, different SBC though.Then OMV and Unraid and next going back to TrueNAS (bsd).
brad wrote:
Sun Feb 28, 2021 10:51 am
M1015 is 8x pcie 2.0 8 port SATA/SAS
Its not for Odroid H2+, right?
It is actually, the card can run with PCIE x4 bus. Although it will mainly serve the 4 x SSD's i'm also thinking about 2x HDD's as well for another storage pool and maybe a hot SSD spare drive.

I was also going to consider testing out the raid 10 function of the card, as RAID 10 has mirrored disks only half the bandwidth is needed on the PCIE interface.

Post Reply

Return to “Hardware and peripherals”

Who is online

Users browsing this forum: No registered users and 0 guests