Building 6 Bay NAS

[tmihai20]

I just finished setting up and migrating stuff from XU4 to H2. For anybody thinking about making their own NAS with oDroid H2, in the form shwon here in the thread or in the classical form (with 2 HDDs), while transferring from an USB 3.0 HDD to the SATA HDD I got speeds of 120-130 MB/s (it took about 8 hours to transfer 3.5 TB from the external HDD to the internal one just because I had small size files, besides large multimedia files). I am more than pleased with it. It looks very, very cool with the transparent case and it handles operation gracefully. I got around 100 MB/s on Samba transfers to my desktop. H2 is a beauty and a beast.

[RomaT]

I got around 100 MB/s on Samba transfers to my desktop.

This is not the limit, some have already received 14...15 Gbit/s (1.6-1.8 GB/s)
viewtopic.php?p=263622#p263622

[mad_ady]

@RomaT: don't spoil the man's fun Now 100MB/s will look inadequate

[tmihai20]

@RomaT @mad_ady: I am more than ok with transfers over or around 100 MB/s in my local network (I do not plan to go over 1 Gigabit for now, it would cost too much). I know that this setup is capable of more or that we can achieve more. That test is a theoretical speed obtained with iperf, not bad, but hardly not a real life scenario. I got arount 930-950 MB/s with my XU4 in the past, I did not test H2 as I am more than pleased with its setup now.

[Mullcom]

@RomaT @mad_ady: I am more than ok with transfers over or around 100 MB/s in my local network (I do not plan to go over 1 Gigabit for now, it would cost too much). I know that this setup is capable of more or that we can achieve more. That test is a theoretical speed obtained with iperf, not bad, but hardly not a real life scenario. I got arount 930-950 MB/s with my XU4 in the past, I did not test H2 as I am more than pleased with its setup now.

I need 1gbit in local network. We can have up to 1gbit fiber in to the house. And soon all children going to download and streaming a lot of things.

So i have now start to upgrade my local network between server and router so it can Handel all workload.

Skickat från min SM-G955F via Tapatalk

[RomaT]

Have you thought about thin and fat clients and booting PXE?

[mad_ady]

Where can you find a thin client terminal these days?

[RomaT]

this must be sought not in the store, but in the head, the rest is done by hands
start, for example, by moving client drives to the server, and the client simply mounts them as network drives.
and the client can be both a PC and laptop and a smartphone and TV and Odroid boards, etc.
i.e. just create your own data cloud in your basement.

[mad_ady]

Yes, I meant a laptop-like device that acts as the client. Back in the day SUN would sell thin clients without a disk where you would do everything over the network.

But today, I'm not so sure about use cases. A gaming pc needs local cpu and fast local storage. A "chromebook" doesn't need much local storage, but may need loads of ram. The disadvantages of a thin client in a chromebook role are:
1. you need ethernet
2. you can't use it outside of your home.

The big advantage of thin clients was client roaming from terminal to terminal. Unless you have a really big house/loads of terminals, it's not an advantage...
But maybe I'm wrong and there are other valid use cases.

[RomaT]

A gaming pc needs local cpu and fast local storage.

This is a fat client, which has a powerful processor and memory and a graphics card and a small SSD for OS
fast storage is just a high-speed network and a mounted network drive.
I repeat, "thin" or "fat/thick" clients are just a name, and the implementation depends only on savvy.

[mad_ady]

I don't know... Unless you go with @domith's 40/56Gbps network, 1Gbps over LAN is much slower throughput than 400MB/s local ssd. I agree, it vastly depends on one's needs. Maybe 10Gbps will bridge the gap without burning a big hole in our wallets.

[RomaT]

the essence of is if you invest in infrastructure, you will save on each item separately.
which ultimately costs less than with poor infrastructure.
IMHO: I’m saying that doing NAS with the 1GbE network interface today is a bad idea,
5GBE in the short term is normal, 10GbE - good,
more than 10GbE- it's overkill for today, but it's good for the long term and not with Odroid-H2.
P.S.: in odroid H2 limitation PCIe bus - four-lanes v2.0 is the maximum speed of 2GB/s (16Gbps)

[tmihai20]

To overcome a possible saturation of the 1 Gbit network I would use several small servers like the HC1 or HC2, all connected through Gigabit. I mean you really need like constant transfers from capable devices to actually fill 1 Gigabit. Internet providers offer 300 and 500 Mbit connections for a fair price. If everybody is streaming their own 4K content independently, downloading through Steam and through on several PCs, then 1 Gbit will definitely not be enough. I would actually use managed switches, load balancing and so on, there is much optimization to be done. There are still people with 100 Mbit wired houses and 56 Mbit Internet connections. I think 5 Gigabit would become the norm in about 5 years, when 8K content would probably be as mainstream as 4K content is now.

[RomaT]

1. Actually, that’s about NAS, not about Internet content (speed Internet access nothing to do with).
2. motherboard manufacturers have already launched integrated network interfaces 2.5/5Gbps
NAS manufacturers already release with 10GbE
i.e. it’s already going on, and after 5 years everyone will have it, as 1GBe now.
3. Well, about the speed of Internet access https://www.speedtest.net/result/7914234600

[Mullcom]

To get good stability in local network you need have good backbone capacity. The other is how how setup up the connections.

I have AA house that are tre floors and basement.
So backbone need to be slot more BW then out to the clients. Then if you use one Nas that constant uses you also need to have overflow of bandwidth. If you not you going to notis that you not get that capacity that you Wish you.

Less jump between to Mitch switch and switch. And don't put all floor on same switch. I am going to put one switch on every Flor and fiber between this switch. An fiber between router and fiber switch. I can have then 10 clients att the same time running 1gbit locally streaming or copy between clients Without noticing any loss.

Notis we have 3 children and one on the way

Skickat från min SM-G955F via Tapatalk

[mad_ady]

Congratulations! Parenting these days involves doing backbone capacity upgrades it seems
I took the opposite approach. The kids are on dial-up until further notice. Not bandwidth hogs in my network

[RomaT]

https://youtu.be/MgNpI6VAAhI

[RomaT]

Finally I got a network cards 10GBE, more details here:
viewtopic.php?p=265986#p265986
I begin installation in the chassis:
.

P8091053s.JPG (277.99 KiB) Viewed 2309 times

.

P8091071s.JPG (261.53 KiB) Viewed 2306 times

.

P8231098s.jpg (196.09 KiB) Viewed 1975 times

.

P8101095s.JPG (172.26 KiB) Viewed 2272 times

.

P8101088s.JPG (371.31 KiB) Viewed 2273 times

[RomaT]

SATA SSD Caching array LVM2 on SATA HDD

Code: Select all

# References
# https://www.digitalocean.com/community/tutorials/how-to-use-lvm-to-manage-storage-devices-on-ubuntu-18-04
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=773731
# http://man7.org/linux/man-pages/man7/lvmcache.7.html
# http://strugglers.net/~andy/blog/2017/07/19/bcache-and-lvmcache/
# https://manpages.ubuntu.com/manpages/precise/man8/update-initramfs.8.html
#
# /dev/sda is the big slow hard disk (ada1)
# /dev/sdb is the fast small SSD (sdb1) use 99% for meta 1/1000
sudo apt update
sudo apt install lvm2 thin-provisioning-tools
# view devices supports LVM2
sudo lvmdiskscan
# create storage
sudo pvcreate /dev/sda1 /dev/sdb1
sudo vgcreate storage-vg /dev/sda1
sudo vgextend storage-vg /dev/sdb1
# Let's use the dm-cache based version (write through caching)
sudo lvcreate -n storage-origin-lv -l 100%PVS storage-vg /dev/sda1
sudo lvcreate -n storage-cache-meta-lv -L 512M storage-vg /dev/sdb1
sudo lvcreate -n storage-cache-lv -l 99%FREE storage-vg /dev/sdb1
sudo lvconvert --type cache-pool --poolmetadata storage-vg/storage-cache-meta-lv storage-vg/storage-cache-lv
sudo lvconvert --type cache --cachepool storage-vg/storage-cache-lv storage-vg/storage-origin-lv
sudo mkfs -t ext4 /dev/storage-vg/storage-origin-lv
sudo mount /dev/storage-vg/storage-origin-lv /mnt/storage
# copy UUID device storage from view:
sudo blkid /dev/storage-vg/storage-origin-lv
sudo vi /etc/fstab
# insert new line storage-origin-lv as UUID device, paste from "blkid"
UUID=1234-5678-91011 /mnt/storage auto nosuid,nodev,nofail 0 0
# save and exit

# Check all one last time
sudo pvscan
sudo vgscan
sudo lvscan

# Update the initial ram fs so that the LVM2 volume shows up
sudo vgchange -a y storage-vg
sudo update-initramfs -u
sudo reboot
# if not /dev/storage-vg/* then 
sudo vgchange -a y storage-vg

speed transfer network file system:
.
954 MBytes/s = 5840 Mbits/s (from NAS Odroid-H2 to PC)

test5_h2_from_LVM2_to_pc.jpg (20.92 KiB) Viewed 1619 times

.
730 MBytes/s = 7632 Mbits/s (from PC to NAS Odroid-H2)

test8_pc_to_h2_on_LVM2.png (12.53 KiB) Viewed 1619 times

.
for comparison, the network speed of a single SATA HDD
157 MBytes/s = 1256 Mbits/s = 1.2 Gbps (from Odroid-H2 to PC)

test6_h2_to_pc.png (12.99 KiB) Viewed 1612 times

[RomaT]

Alternative chassis for 6-disks NAS FRACTAL DESIGN Node 304
.

case.jpg (43.7 KiB) Viewed 1573 times

[doughnut]

OK, here is my dilemma. When I use the ADP6ST0-J05 port multiplier, I am only able to get about 80-100MBs throughput to a raid 5 (mdadm) 5 disk array. Based on this level of throughput, I think the device is stuck in CIS (command) mode. In order to get 2-400 MBs throughput on writes to my 5 disk array, I think I need to be using FIS mode. I turn write cache on for each drive. Is there a trick to get this device to go into FIS mode on the H2? I am using linux 18.04 lts.

I picked up a 5-port PCIe 3.0 x4 sata card and I am able to get 375-400 MBs on my Raid-5(mdadm) writes, but then I lose the ability to have >1Gbs network. So I know the H2 is capable of sustaining that level of throughput. Does anyone have any suggestions how I can get that level of Raid-5 write performance using the ADP6ST0-J05 port multiplier, so I can add a multi-gigabit network interface to the H2 to feed it the data needed to sustain that level of IO performance?

[doughnut]

By the way, I have one of these left over from my old NAS. This is a nice box, so I am sure the Node 304 is nice too!. Plenty of room for fans to cool the drives etc. I will probably reconfigure this with an H2 inside to act as my backup server.

This beast holds 10-3.5 in Hdds. Not hot swap,, but they all fit nicely inside. I like the drive spacing, so there is plenty of air flow over the drives. I also have a Lian-Li PC-08? It doesn't show as an active product, but it holds 6 HDDs plus a 5.25 bay for optical (itx) . It will probably be repurposed for my 5-6 drive NAS.

[RomaT]

5-port PCIe 3.0 x4 sata card

there are not x4, but 2 lanes of PCIe

PCIe 3.0 x2 to 5-port sata card, but then I lose the ability to have >1Gbs network. Does anyone have any suggestions how I can get that level of Raid-5 write performance

To do this, persuade HARDkernel to throw out two network interfaces 1GbE, thereby freeing up two PCIe lanes, install a second connector M.2, for connecting a SATA controller,
then the first M.2 connector will remain for connecting a 10GbE network card.
It is for this reason that I had to abandon H2, selling her, acquiring a regular miniITX motherboard, with two M.2 M connectors, one M.2 E and PCI-E x16 slot,
installing in the first connector M.2 M SSD NVMe, in the second M.2 M network card 10GB, in the M.2 E connector an additional SATA controller, In the same M.2 E connector there is an I2C interface for LCD
.
connector pin M.2 E:

m2e.jpg (94.46 KiB) Viewed 1282 times

.
Now I have this speed when copying files over the network (1.14 GB/s = 9.12 Gbits/s):

netspeed01.png (11.51 KiB) Viewed 1268 times

.
network speed Test:

netspeed04.jpg (80.75 KiB) Viewed 1268 times

[Mullcom]

5-port PCIe 3.0 x4 sata card

there are not x4, but 2 lanes of PCIe

PCIe 3.0 x2 to 5-port sata card, but then I lose the ability to have >1Gbs network. Does anyone have any suggestions how I can get that level of Raid-5 write performance

To do this, persuade HARDkernel to throw out two network interfaces 1GbE, thereby freeing up two PCIe lanes, install a second connector M.2, for connecting a SATA controller,
then the first M.2 connector will remain for connecting a 10GbE network card.
It is for this reason that I had to abandon H2, selling her, acquiring a regular miniITX motherboard, with two M.2 M connectors, one M.2 E and PCI-E x16 slot,
installing in the first connector M.2 M SSD NVMe, in the second M.2 M network card 10GB, in the M.2 E connector an additional SATA controller, In the same M.2 E connector there is an I2C interface for LCD
.
connector pin M.2 E:

m2e.jpg

.
Now I have this speed when copying files over the network (1.14 GB/s = 9.12 Gbits/s):

netspeed01.png

.
network speed Test:

netspeed04.jpg

This is good results.. good work.

Skickat från min SM-G955F via Tapatalk