White wood mini-tower of 3 x H2

Post Reply
domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

White wood mini-tower of 3 x H2

Post by domih » Tue May 05, 2020 8:36 am

Little new project: a white wood mini-tower of 3 x H2 with 5-disc cage with which you do not need to disassemble the whole thing to replace a drive. I'll use only 3 hard disks, one for each H2. An ATX SFX PSU will power the 3 x H2 and the disks. It will be used for a replicated NAS and a 3-node Cassandra server. The latter is because any home must have a Cassandra server :-)

See picture below. As the French says "y a plus qu'à"
.
20200504_161821.png
20200504_161821.png (2.9 MiB) Viewed 1515 times

User avatar
rooted
Posts: 7742
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 718 times
Been thanked: 207 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by rooted » Tue May 05, 2020 10:28 am

Nice project, why do you need a distributed database?
These users thanked the author rooted for the post:
domih (Tue May 05, 2020 3:50 pm)

User avatar
mad_ady
Posts: 8136
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 565 times
Been thanked: 399 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by mad_ady » Tue May 05, 2020 1:56 pm

rooted wrote:Nice project, why do you need a distributed database?
For the shopping list and cupboard items, what else?
These users thanked the author mad_ady for the post:
domih (Tue May 05, 2020 3:50 pm)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Thu May 07, 2020 3:52 am

rooted wrote:
Tue May 05, 2020 10:28 am
Nice project, why do you need a distributed database?
Mostly testing ideas, writing HOWTOs in relation to professional work.

Also I'm really interested in evaluating whether or not the algorithms used to distribute PB of data on thousands of commodity PCs are ALSO useful to distribute storage and data processing on SBCs with limited resources.
These users thanked the author domih for the post:
rooted (Thu May 07, 2020 11:01 pm)

User avatar
rooted
Posts: 7742
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 718 times
Been thanked: 207 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by rooted » Thu May 07, 2020 11:02 pm

And here I was thinking (because of @mad_ady) you were creating a shopping list for the entire country
These users thanked the author rooted for the post:
domih (Sun May 10, 2020 8:36 am)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sun May 10, 2020 8:36 am

Update
- Corner braces to assemble shelves with side panels,
- Velcro tape to fix PSU, H2 column and HDD cage to shelves: easy removal for maintenance without fooling around with cramped space for (un)screwing, dampened vibrations for HDD cage and resonance,
- 3.8 x 36" wood to build front mesh frame,
- Plastic back front mesh,
- Loctite to assemble front mesh frame and glue the mesh to it.
20200509_162418.png
20200509_162418.png (2.86 MiB) Viewed 1306 times
These users thanked the author domih for the post:
odroid (Mon May 11, 2020 10:49 am)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Wed May 27, 2020 2:46 pm

Update

See overall view of the case shown below (build in progress).
.
case_overall_view.png
case_overall_view.png (1.39 MiB) Viewed 1121 times
.
I dropped the usage of corner braces (with the original goal of leaving the sides virgin of any marks), not easy to fix correctly though, result not solid enough. So I went with Philllips pan head 6x1 1/4, three on each side for each shelf. Very solid. I'll mask the screw holes with white putty or plastic round covers.

At the top there is a nice aluminum handle to make the handling/transport of the case easy. The 3 x H2 LED power button will also be on the top. I'll need a 5/8" drill bit, the closest to the 16mm button diameter. Yeah, these buttons are available again. I ordered the 3rd button (plus a few other things) from Ameridroid.com and as usual got it 2 days later with the usual Haribo Goldbears. Kudos to Ameridroid.com! Disclosure: they are not far from the San Francisco Bay Area so I'm a privileged customer in terms of delivery time :-)
.
case_top.png
case_top.png (1.19 MiB) Viewed 1121 times
.
The bottom section of the tower will hold the 5-disk SATA cage (see https://www.ebay.com/sch/i.html?_nkw=Venus+TL+backplane). I'll use only 3 slots for now. 1 x HGST 10TB He10 (HUH721010ALE604) per H2. I bought them used on eBay for about the same price as a new WD Red 10TB. See https://www.ebay.com/sch/i.html?_nkw=HUH721010ALE604.
.
case_bottom.png
case_bottom.png (1.24 MiB) Viewed 1121 times
.
I'm also working on setting up the 3 x H2 and their powering. See picture shown below.
.
psu_3xh2_power_distrib.png
psu_3xh2_power_distrib.png (1.87 MiB) Viewed 1121 times
.
It's basically an expanded version of the setup I used in https://magazine.odroid.com/article/net ... odroid-h2/. Originally, I wanted to go with 2.5 Gbe using USB 3 adapters in order to pass the 1 Gbe limit. But following testing it appears that these USB 3 adapters are having teething issues, see viewtopic.php?f=172&t=38713.

So I revisited the networking solution with the following options: (a) Wait for Hard Kernel in case they would make a new SBC with 1+ Gbe NICs (b) Try 10 Gbe SolarFlare NICs (i.e. the SFN5122F which you can find for $20 or less on eBay, soon they'll give you one free when you buy detergent or toilet paper) with SFP+ DAC cables and a Microtik CRS317-1G-16S+RM switch, overall unbeatable prices compared to 10Base-T RJ-45 solutions - as of this writing (c) Continue with the Mellanox InfiniBand solution. In the Odroid Magazine article I wrote:

.../...
Note: do Mellanox ConnectX-3 cards also work on the ODROID-H2? As of this writing only the Pro version works. The non-pro versions do not. Know anyway that in terms of max speed achievable on the H2, the ConnectX-3 will provide the same result as the ConnectX-2. The ConnectX-2 provide 40 Gb/s in IB mode but only 10 GbE in Ethernet mode. The connectX-3 provide 40 Gb/s in IB mode and 40 GbE in Ethernet mode. However IP over IB provides a speed as good (if not a little bit better) with a port configured for IB as IP on a port configured for Ethernet. So really, no need to run a ConnectX3 Pro on the H2 (unless you already have a 20 or more GbE network running at home based on Ethernet).
.../...

Since then I upgraded my IB switch from a QDR (about $100) to an FDR10 (about $150) and finally an FDR (about $200) all from eBay. As a result from modern PC to modern PC my IPoIB speed jumped from about 32 Gbe, to 38 Gbe and finally 45 Gbe. However, the Connect-XB or Connect X2 (too old) cards I used on the H2 do not work well with the FDR10 or FDR switches. On the other hand, the ConnectX3 Pro on the H2 works 100% OK with these switches, still obviously with the limited speed from 10 Gbe to 14 Gbe depending on direction, which is the best the H2 + Celeron J4105 + PCIe 2.0 x4 can do. So with the ConnectX3 Pro the 3 x H2s can still be part of the IB network with the modern PCs :-) I finally decided to adopt this solution when I found an eBay seller selling used ConnectX3 Pro cards for $45 each. See these cards having their thermal paste re-done in the image shown below.
.
connect_x3_pro.png
connect_x3_pro.png (1.98 MiB) Viewed 1121 times
.
I also found less expensive M.2 NMVe M key to PCIe x4 adapters on eBay, see https://www.ebay.com/itm/PCIe-x4-3-0-PC ... 4398886092. Hopefully they work as well as the ones I originally bought. I'll know in a few weeks, right now they are in China...

Also note the 80mm fans on top on each H2. They are fixed with a small square of Velcro to the heat sinks. No vibration noise. The speed of the fans is set using the small fan board you can see in the picture shown earlier.
These users thanked the author domih for the post:
hominoid (Thu May 28, 2020 10:07 pm)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Mon Jun 01, 2020 11:00 am

Update

See front and back view of the case shown below. Work in progress. The LED power buttons left the top of the case to go down with the 3 x H2 shelf. One more shelf TBD for the PSU + breakout board + fan board.

Note: I dropped using a mesh for the front side of the case. I'll instead use a dark grey or blue acrylic door.
.
case_front.png
case_front.png (1.29 MiB) Viewed 1024 times
.
case_back.png
case_back.png (1.32 MiB) Viewed 1024 times
.
Note the oblong opening at the back of the shelf to allow cables to go through. The PSU shelf will have the same opening.

1 of 3 Connect X3 Pro NIC with custom fan fixed on acrylic sheet. Two bands of Velcro will allow to stick the acrylic sheet to the side of the case. Side and position will depend on the M.2 NMVe M key to PCIe x4 adapter cable which I have not received yet.
.
nic_front.png
nic_front.png (1.8 MiB) Viewed 1024 times
.
nic_back.png
nic_back.png (1.7 MiB) Viewed 1024 times

User avatar
venkatbo
Posts: 537
Joined: Mon Feb 25, 2013 3:10 pm
languages_spoken: english
ODROIDs: C0/C1+/C2; U3+; XU3-Lite/XU4; H2; VU/VU7+; Touchscreen Show TFT Cloudshell SmartPower mAHRS DAC GPS Multiscope Ocam
Location: Bay Area, California
Has thanked: 1 time
Been thanked: 5 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by venkatbo » Mon Jun 01, 2020 10:14 pm

Nice effort!

Wondering if the fans will be effective (with airflow restricted) when the H2's are stacked. Small fans vertically mounted behind the fins may draw more air and cool better and eject the air freely at the back of the stack. Just a thought...

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sun Jun 07, 2020 7:28 am

domih wrote:
Wed May 27, 2020 2:46 pm
.../...
I also found less expensive M.2 NMVe M key to PCIe x4 adapters on eBay, see https://www.ebay.com/itm/PCIe-x4-3-0-PC ... 4398886092. Hopefully they work as well as the ones I originally bought. I'll know in a few weeks, right now they are in China...
.../...
For the people who are interested by these adapters:

I received the 3 adapters today, so that's relatively quick from CH/HK/TW. I tested one: works AOK. There branded ADT Link. They come with the SATA to 4-pin FDD cable for power as well as the small M.2 screw. In other words, they are ready for use once received, no extra cost. I bought straight 30 cm so that I can pivot the cards 180. The pricing is much less than what I found before: $23.39 (eBay) a piece vs $30.40 or $55.10 (AliExpress). Free shipping.

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sun Jun 07, 2020 7:42 am

venkatbo wrote:
Mon Jun 01, 2020 10:14 pm
Nice effort!

Wondering if the fans will be effective (with airflow restricted) when the H2's are stacked. Small fans vertically mounted behind the fins may draw more air and cool better and eject the air freely at the back of the stack. Just a thought...
Thank you.

Yes, we'll see at usage because predicting air flow is a tall order. The fans are pushing air to the heat sinks so the air will come out by the sides. The back of the case will stay open. The front will be mesh - I reverted to mesh instead of using a blue acrylic door(*). Given the size of the H2 heat sink, it is supposed to work OK without fan. I'm adding fans because as a rule of thumb they lower the cooled CPU or IC temperature by 20 C and also to prevent #1 cooking #2 cooking #3 by convection. The 80 mm are oversized for the task, allowing me to reduce their RPM for silent operation. The main source of heat coming from the above H2 will be the 32 GB of memory. Finally there is also the 3 Mellanox NICs which will blow hot air from the side. So I have no clue what the final result will be.

(*) I experimented with small pieces of acrylic and I do not trust myself able to pull off a nice door with perfect cuts...

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sun Jun 07, 2020 8:00 am

Update

The wood work is quasi-complete. See picture below. Screw caps on the side to give a good enough look. Disks cage installed with Velcro taping at the bottom.

Now I'm working on the PSU, PSU break out card and fan card installation, then the front mesh panels. Last to go in will be the 3 x H2 and Mellanox cards.
.
20200606_155320.png
20200606_155320.png (1.28 MiB) Viewed 895 times
These users thanked the author domih for the post:
odroid (Mon Jun 08, 2020 9:40 am)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Tue Jun 16, 2020 10:56 am

Update

Top compartment front black mesh completed. Still working on the middle compartment one.
.
top-mesh.png
top-mesh.png (1.22 MiB) Viewed 715 times
These users thanked the author domih for the post (total 2):
odroid (Tue Jun 16, 2020 11:00 am) • Luke.go (Tue Jun 16, 2020 11:46 am)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Wed Jun 17, 2020 1:52 pm

Update

This is only of interest to the people who want to know how it's done.

Once the middle compartment black mesh frame is assembled and it corners smoothed, it is time to paint black mat of the 2 sides of the bars that will be visible through the mesh. Acrylic or Enamel paint will do. Acrylic is easier because you clean the brush with warm water once you are done. However I had a small bottle of Enamel handy so I used it. The brush to be cleaned with Enamel thinner. A light layer of paint is enough.
.
paint-it-black-said-mick-1.png
paint-it-black-said-mick-1.png (1.23 MiB) Viewed 681 times
paint-it-black-2.png
paint-it-black-2.png (1.3 MiB) Viewed 681 times
paint-it-black-3.png
paint-it-black-3.png (1.3 MiB) Viewed 681 times
.
Let it dry for some time (1+ hour) and then start to glue the mesh to the 2 unpainted sides of the frame bars, one bar and one side at a time, it's just a question of patience, letting the glue cure for 15 to 30 minutes at each step. Use small clamps a sacrificial piece of wood to keep the mesh and the bar stuck together during curing.
.
mesh.png
mesh.png (1.35 MiB) Viewed 681 times
.
Tip: how to get the mesh to bend cleanly at 90 degrees along a predetermined line ? It's like in paper modeling, if you want a clean bend of the paper, just trace a line with a fine point pen on the other side of the paper pressing the pen hard enough (but not too hard), then start bending the paper, the bend will exactly occur along the line, clean and perfect. For the mesh which is plastic I used a cutter, pressing it lightly enough to trace the bending line without actually fully cutting the mesh. How to know which pressure to apply: rehearse on a small piece until you obtain the intended result. Then the bend will be clean and good enough.

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Wed Jun 17, 2020 2:09 pm

Reality Check

<hopeful>Wow! The case looks as good as a commercial case.</hopeful>

Mmm... well it hopefully looks good in the pictures where photography eliminate a lot of little defects, especially because the pictures are 25% reduction of the originals.

The apparent quality is about the same if you look at it from 3 or more feet. The defects start to appear if you look closer and it is obvious this is amateur handmade (with tragi-comic mistakes) and not a work of art.

Just saying so because other forum members who want to build their own case from scratch should NOT feel intimidated to make the plunge. It's fun!

Just temper your expectations of perfection, unless you are a high quality handyman.

User avatar
mad_ady
Posts: 8136
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 565 times
Been thanked: 399 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by mad_ady » Wed Jun 17, 2020 2:57 pm

Don't worry - it looks better than my solution - putting a big sheet of cardboard to hide the devices/cables :)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Fri Jun 19, 2020 7:47 am

For the people tempted in using the H2 or H2+ M.2 NVMe slot as a PCI2.0 x4 slot, I just saw that Ameridroid is now selling two accessories to simplify you life:

- https://ameridroid.com/collections/new/ ... pcie-cards
- https://ameridroid.com/collections/new/ ... r-straight
These users thanked the author domih for the post (total 2):
odroid (Fri Jun 19, 2020 9:40 am) • Ameridroid (Sat Jun 20, 2020 12:38 am)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Mon Jun 22, 2020 8:08 am

Update

The case now has its final look with the middle compartment front mesh completed.

Next step: fix the electronics inside the case.

final-front-view.png
final-front-view.png (1.45 MiB) Viewed 480 times

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Thu Jun 25, 2020 9:33 am

Update

Initial integration testing without the HDDs

See below the 3xH2's powered on with the 3xMellanox ConnectX-3 Pro network cards and plugged to the Mellanox 18-port FDR10 switch.

3xH2 with Mellanox network cards.png
3xH2 with Mellanox network cards.png (3.62 MiB) Viewed 366 times
2xPCs+3xH2 plugged into the FDR10 switch.png
2xPCs+3xH2 plugged into the FDR10 switch.png (1.94 MiB) Viewed 366 times

For a quick introduction about the Mellanox InfiniBand networks, see https://magazine.odroid.com/article/net ... odroid-h2/

Let's check that the IB local net shows all the expected devices (it does):

Code: Select all

domih@h2a:~$ sudo iblinkinfo
CA: h2b mlx4_0:
      0xc4346bffffdc3691      7    1[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>      12    9[  ] "SwitchX -  Mellanox Technologies" ( )
CA: h2c mlx4_0:
      0xc4346bffffdc5561      5    1[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>      12   11[  ] "SwitchX -  Mellanox Technologies" ( )
CA: i5-a mlx4_0:
      0x0002c90300426771      2    1[  ] ==( 4X          10.0 Gbps Active/  LinkUp)==>      12    3[  ] "SwitchX -  Mellanox Technologies" ( Could be FDR10 (Found link at QDR but expected speed is FDR10))
CA: ripper HCA-1:
      0xf452140300346ea1      3    1[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>      12    1[  ] "SwitchX -  Mellanox Technologies" ( )
Switch: 0xf452140300789a60 SwitchX -  Mellanox Technologies:
          12    1[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>       3    1[  ] "ripper HCA-1" ( )
          12    2[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    3[  ] ==( 4X          10.0 Gbps Active/  LinkUp)==>       2    1[  ] "i5-a mlx4_0" ( Could be FDR10 (Found link at QDR but expected speed is FDR10))
          12    4[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    5[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    6[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    7[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    8[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12    9[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>       7    1[  ] "h2b mlx4_0" ( )
          12   10[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>       1    1[  ] "h2a mlx4_0" ( )
          12   11[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>       5    1[  ] "h2c mlx4_0" ( )
          12   12[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   13[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   14[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   15[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   16[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   17[  ] ==(                Down/ Polling)==>             [  ] "" ( )
          12   18[  ] ==(                Down/ Polling)==>             [  ] "" ( )
CA: h2a mlx4_0:
      0xc4346bffffdc55b1      1    1[  ] ==( 4X  10.0 Gbps (FDR10) Active/  LinkUp)==>      12   10[  ] "SwitchX -  Mellanox Technologies" ( )
Let's check that one of the H2s can ping the i5-a PC via the regular 1 GbE network and via the IPoIB network (it can):

Code: Select all

domih@h2a:~$ ping i5-a -c 10
PING i5-a (192.168.1.20) 56(84) bytes of data.
64 bytes from i5-a (192.168.1.20): icmp_seq=1 ttl=64 time=0.804 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=2 ttl=64 time=0.829 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=3 ttl=64 time=0.825 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=4 ttl=64 time=0.638 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=5 ttl=64 time=0.819 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=6 ttl=64 time=0.827 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=7 ttl=64 time=0.826 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=8 ttl=64 time=0.826 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=9 ttl=64 time=0.829 ms
64 bytes from i5-a (192.168.1.20): icmp_seq=10 ttl=64 time=0.823 ms

--- i5-a ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9037ms
rtt min/avg/max/mdev = 0.638/0.804/0.829/0.055 ms
domih@h2a:~$ ping i5-a.ib -c 10
PING i5-a.ib (10.10.56.20) 56(84) bytes of data.
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=1 ttl=64 time=0.140 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=2 ttl=64 time=0.190 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=3 ttl=64 time=0.179 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=4 ttl=64 time=0.175 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=5 ttl=64 time=0.177 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=6 ttl=64 time=0.177 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=7 ttl=64 time=0.176 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=8 ttl=64 time=0.177 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=9 ttl=64 time=0.178 ms
64 bytes from i5-a.ib (10.10.56.20): icmp_seq=10 ttl=64 time=0.178 ms

--- i5-a.ib ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9207ms
rtt min/avg/max/mdev = 0.140/0.174/0.190/0.012 ms
From there let's perform iperf3 sessions over IPoIB from the 3xH2's toward the i5-a PC:

Code: Select all

domih@h2c:~$ iperf3 -c i5-a.ib --bind 10.10.56.72
Connecting to host i5-a.ib, port 5201
[  4] local 10.10.56.72 port 40529 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   1.00-2.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   2.00-3.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   3.00-4.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   4.00-5.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   5.00-6.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   6.00-7.00   sec  1.31 GBytes  11.3 Gbits/sec    0   3.18 MBytes       
[  4]   7.00-8.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   8.00-9.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  4]   9.00-10.00  sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec                  receiver

iperf Done.
domih@h2c:~$ iperf3 -c i5-a.ib --bind 10.10.56.72 -R
Connecting to host i5-a.ib, port 5201
Reverse mode, remote host i5-a.ib is sending
[  4] local 10.10.56.72 port 49109 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   1.00-2.00   sec  1.57 GBytes  13.5 Gbits/sec                  
[  4]   2.00-3.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   3.00-4.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   4.00-5.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   5.00-6.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   6.00-7.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   7.00-8.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   8.00-9.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  4]   9.00-10.00  sec  1.67 GBytes  14.4 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  16.6 GBytes  14.3 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  16.6 GBytes  14.3 Gbits/sec                  receiver

iperf Done.


domih@h2b:~$ iperf3 -c i5-a.ib --bind 10.10.56.71
Connecting to host i5-a.ib, port 5201
[  5] local 10.10.56.71 port 42363 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   1.00-2.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   2.00-3.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   3.00-4.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   4.00-5.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   5.00-6.00   sec  1.31 GBytes  11.3 Gbits/sec    0   3.18 MBytes       
[  5]   6.00-7.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   7.00-8.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   8.00-9.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   9.00-10.00  sec  1.31 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec                  receiver

iperf Done.
domih@h2b:~$ iperf3 -c i5-a.ib --bind 10.10.56.71 -R
Connecting to host i5-a.ib, port 5201
Reverse mode, remote host i5-a.ib is sending
[  5] local 10.10.56.71 port 46419 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.54 GBytes  13.2 Gbits/sec                  
[  5]   1.00-2.00   sec  1.56 GBytes  13.4 Gbits/sec                  
[  5]   2.00-3.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   3.00-4.00   sec  1.66 GBytes  14.2 Gbits/sec                  
[  5]   4.00-5.00   sec  1.67 GBytes  14.3 Gbits/sec                  
[  5]   5.00-6.00   sec  1.67 GBytes  14.3 Gbits/sec                  
[  5]   6.00-7.00   sec  1.65 GBytes  14.2 Gbits/sec                  
[  5]   7.00-8.00   sec  1.65 GBytes  14.1 Gbits/sec                  
[  5]   8.00-9.00   sec  1.67 GBytes  14.3 Gbits/sec                  
[  5]   9.00-10.00  sec  1.67 GBytes  14.3 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  16.4 GBytes  14.1 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  16.4 GBytes  14.1 Gbits/sec                  receiver

iperf Done.

domih@h2a:~$ iperf3 -c i5-a.ib --bind 10.10.56.70
Connecting to host i5-a.ib, port 5201
[  5] local 10.10.56.70 port 57781 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   1.00-2.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   2.00-3.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   3.00-4.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   4.00-5.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   5.00-6.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   6.00-7.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   7.00-8.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   8.00-9.00   sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  5]   9.00-10.00  sec  1.31 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  13.1 GBytes  11.2 Gbits/sec                  receiver

iperf Done.
domih@h2a:~$ iperf3 -c i5-a.ib --bind 10.10.56.70 -R
Connecting to host i5-a.ib, port 5201
Reverse mode, remote host i5-a.ib is sending
[  5] local 10.10.56.70 port 48207 connected to 10.10.56.20 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.67 GBytes  14.3 Gbits/sec                  
[  5]   1.00-2.00   sec  1.65 GBytes  14.2 Gbits/sec                  
[  5]   2.00-3.00   sec  1.67 GBytes  14.3 Gbits/sec                  
[  5]   3.00-4.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   4.00-5.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   5.00-6.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   6.00-7.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   7.00-8.00   sec  1.67 GBytes  14.4 Gbits/sec                  
[  5]   8.00-9.00   sec  1.64 GBytes  14.1 Gbits/sec                  
[  5]   9.00-10.00  sec  1.66 GBytes  14.3 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  16.7 GBytes  14.3 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  16.6 GBytes  14.3 Gbits/sec                  receiver

iperf Done.
As expected we get about 11 Gbits/sec and 14 Gbits/sec depending on direction. In the reverse direction the i5-a PC is running a 9600K which is obviously faster at sending packets than the Celeron J4105.

Let's do iperf3 sessions from H2 to H2 over IPoIB:

Code: Select all

domih@h2c:~$ iperf3 -c h2a.ib --bind 10.10.56.72
Connecting to host h2a.ib, port 5201
[  4] local 10.10.56.72 port 42445 connected to 10.10.56.70 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.26 GBytes  10.8 Gbits/sec    0   3.06 MBytes       
[  4]   1.00-2.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   2.00-3.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   3.00-4.00   sec  1.30 GBytes  11.1 Gbits/sec    0   3.06 MBytes       
[  4]   4.00-5.00   sec  1.29 GBytes  11.1 Gbits/sec    0   3.06 MBytes       
[  4]   5.00-6.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   6.00-7.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   7.00-8.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   8.00-9.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.06 MBytes       
[  4]   9.00-10.00  sec  1.29 GBytes  11.1 Gbits/sec    0   3.06 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  12.9 GBytes  11.1 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  12.9 GBytes  11.1 Gbits/sec                  receiver

iperf Done.
domih@h2c:~$ iperf3 -c h2a.ib --bind 10.10.56.72 -R
Connecting to host h2a.ib, port 5201
Reverse mode, remote host h2a.ib is sending
[  4] local 10.10.56.72 port 42283 connected to 10.10.56.70 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.30 GBytes  11.2 Gbits/sec                  
[  4]   1.00-2.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   2.00-3.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   3.00-4.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   4.00-5.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   5.00-6.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   6.00-7.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  4]   7.00-8.00   sec  1.30 GBytes  11.2 Gbits/sec                  
[  4]   8.00-9.00   sec  1.30 GBytes  11.2 Gbits/sec                  
[  4]   9.00-10.00  sec  1.30 GBytes  11.2 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  13.0 GBytes  11.2 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  13.0 GBytes  11.2 Gbits/sec                  receiver

iperf Done.


domih@h2b:~$ iperf3 -c h2a.ib --bind 10.10.56.71
Connecting to host h2a.ib, port 5201
[  5] local 10.10.56.71 port 44087 connected to 10.10.56.70 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.30 GBytes  11.1 Gbits/sec    0   3.18 MBytes       
[  5]   1.00-2.00   sec  1.30 GBytes  11.1 Gbits/sec    0   3.18 MBytes       
[  5]   2.00-3.00   sec  1.30 GBytes  11.1 Gbits/sec    0   3.18 MBytes       
[  5]   3.00-4.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   4.00-5.00   sec  1.30 GBytes  11.2 Gbits/sec    0   3.18 MBytes       
[  5]   5.00-6.00   sec  1.30 GBytes  11.1 Gbits/sec    0   3.18 MBytes       
[  5]   6.00-7.00   sec  1.23 GBytes  10.6 Gbits/sec    0   3.18 MBytes       
[  5]   7.00-8.00   sec  1.24 GBytes  10.6 Gbits/sec    0   3.18 MBytes       
[  5]   8.00-9.00   sec  1.25 GBytes  10.7 Gbits/sec    0   3.18 MBytes       
[  5]   9.00-10.00  sec  1.29 GBytes  11.1 Gbits/sec    0   3.18 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  12.8 GBytes  11.0 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  12.8 GBytes  11.0 Gbits/sec                  receiver

iperf Done.
domih@h2b:~$ iperf3 -c h2a.ib --bind 10.10.56.71 -R
Connecting to host h2a.ib, port 5201
Reverse mode, remote host h2a.ib is sending
[  5] local 10.10.56.71 port 45811 connected to 10.10.56.70 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.30 GBytes  11.1 Gbits/sec                  
[  5]   1.00-2.00   sec  1.25 GBytes  10.7 Gbits/sec                  
[  5]   2.00-3.00   sec  1.30 GBytes  11.2 Gbits/sec                  
[  5]   3.00-4.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  5]   4.00-5.00   sec  1.31 GBytes  11.2 Gbits/sec                  
[  5]   5.00-6.00   sec  1.30 GBytes  11.1 Gbits/sec                  
[  5]   6.00-7.00   sec  1.28 GBytes  11.0 Gbits/sec                  
[  5]   7.00-8.00   sec  1.29 GBytes  11.1 Gbits/sec                  
[  5]   8.00-9.00   sec  1.30 GBytes  11.2 Gbits/sec                  
[  5]   9.00-10.00  sec  1.31 GBytes  11.2 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  12.9 GBytes  11.1 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  12.9 GBytes  11.1 Gbits/sec                  receiver

iperf Done.
As expected we get around 11 Gbits/sec in both directions.

CONCLUSION
The three H2s behave as expected with IPoIB, at least from the iperf3 point of view.

FINAL REMARK
IPoIB with FDR10 is able to offer about 38 GbE connections with modern PCs. We lose a lot of bandwith on the H2 due to the expected hardware limitations. So does it makes sense to even contemplate such a solution? It sounds like putting a Lotus car body around a moped engine. As a matter of fact, it does. It would be totally stupid/idiotic/crazy to spend thousands of $$$ for brand new Mellanox hardware for an H2.

However, it DOES totally makes sense with the used hardware you can find on eBay:
- 18-port FDR10 Mellanox: got one for $150.
- 36-port FDR Mellanox: got one for $200.
- Mellanox ConnectX-3 Pro cards: got for example 4 cards at $45 each. Other Pro and non Pro cards for similar prices ($45 to $65).
- FDR 56 Gbps IB cables: got for example at once 12 x 2M DAC copper for $100, that's around $8.5 per cable.

The tricks are:
- You have to know what you are looking for.
- You have to be patient for the very good deals.
- You really have to test all the hardware upon reception.

Yes we get "only" 11 GbE with H2. But on the same local net we get around 44+ GbE with FDR with modern PCs (Intel 8/9th gen. or higher, AMD Zen+ or Zen 2).

Compare this to the current RJ-45 new 10 GbE hardware. An 18 or 36-port 10 GbE switch = $$$$, 10 GbE cards are still quite expensive $$$ + CAT6-A cables $$.

Obvious conclusion: so you DO get more while spending less.

So as the French say: "Qu'importe le flacon, pourvu qu'on ait l'ivresse !" which can translate as "What matters the jug, if euphoria be within?"

Next Steps

(1) test and observe differences for Samba, NFS, GlusterFS(?) and Cassandra over:
- 1 GbE
- 2.5 GbE
- SFP+ 10 GbE
- 11+ GbE IPoIB

(2) Finalize case project:
- Put all the electronics in the tower case,
- Enjoy a 3-node server over 11+ GbE :-)

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Wed Jul 01, 2020 1:50 pm

Update

This is only of interest to the people who want to know how it's done.

Final step of the case building proper: gluing little pieces of wood with Velcro to fix the front mesh panels while keeping them removable for easier future inside maintenance. See pictures below.

Next step: install the boards and PSU inside the case. However it will only happen in a few weeks because I'm going to do testing for 1 GbE, 2.5 GbE, 10 GbE SFP+ and 11+ GbE IPoIB with Samba and NFS, meaning a lot of configuration changes easier to make with the boards out of the case. For IPoIB the testing will be made with the FDR10 switch because I'm still working on customizing the fan set with "silent" Noctua fans instead of the crazy noisy Delta ones on the full FDR switch.

Optional ultimate step for the case: when I'm in the mood I'll try to smooth a few of the seams, fill them with some putty and paint them white insignia to make them look better. As of now I see it as optional. Once the tower is placed at its operational location nobody will look at it closely, maybe the occasional roaming house spider.


1.front-mesh-velcro-attachments-painted-white.png
1.front-mesh-velcro-attachments-painted-white.png (1.66 MiB) Viewed 201 times
2.gluing-them-to-the-case.png
2.gluing-them-to-the-case.png (1.33 MiB) Viewed 201 times
3.how-it-looks-finished.png
3.how-it-looks-finished.png (855.51 KiB) Viewed 201 times
4.velcro-loops-on-the-front-mesh-back.png
4.velcro-loops-on-the-front-mesh-back.png (1.3 MiB) Viewed 201 times

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sat Jul 04, 2020 4:22 am

Update

Change of plan: I can proceed with the planed testing with the assembled system, so here we are, see photos shown below. Work in progress, spaghetti cables all over the place. Middle compartment front mesh to be fixed once the LED power buttons are connected. HDDs are finally plugged in.

partially-assembled-front-view.png
partially-assembled-front-view.png (1.51 MiB) Viewed 68 times
partially-assembled-rear-view.png
partially-assembled-rear-view.png (1.78 MiB) Viewed 68 times
<humor>
Thank you NZXT, Phanteks, etc(*) for the inspiration but my tower rocks compared to yours :-)

3 x H2 computers for distributed processing
12 cores
96 GB of memory
5 x 3.5" bays
30 TB of disk storage
Enough remaining space for 3 x SATA SSD
192 GB eMMC
1 GbE, 2.5 GbE and 11+ GbE networking
Self-made & 15 μs of fame = priceless

(*) https://www.nzxt.com/product-overview/h1, http://phanteks.com/Enthoo-Evolv-Shift.html.

Long term aspiration: find a the pretext and the adequate SBCs to make something ultra-cool like this one http://www.gamerstorm.com/product/CASES ... 4840.shtml
</humor>

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sat Jul 04, 2020 4:53 am

Ambient temperature: 22C. 3 x H2's running with HDDs, idle: power 38.5W, CPUs temperature: 32 to 34C. H2's off but fans and HDDs still powered: 11.8W. All devices turned off: PSU alone shows 0W at the wall.

I connected the power buttons and fixed the middle compartment mesh. See picture below.

With the the middle compartment mesh installed the CPUs thermal stay the same at idle: 32 to 34C (after 30 minutes). I'm going to let them run idle for a few hours.

led-power-buttons.png
led-power-buttons.png (1.6 MiB) Viewed 65 times

domih
Posts: 207
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 69 times
Been thanked: 77 times
Contact:

Re: White wood mini-tower of 3 x H2

Post by domih » Sun Jul 05, 2020 10:03 am

Update

Testing the network cards. Some definitive cable management has to be done before I stick the cards inside the case.

cable-management-needed.png
cable-management-needed.png (1.82 MiB) Viewed 5 times
I tested running 3 concurrent iperf3 sessions from a PC toward the 3 x H2. Because the 3 connections were going through the same port on the PC side I reached the max FDR10 IPoIB speed of about 38 GbE. So I ended up with slightly more than 12 GbE throughput on each H2. I guess I'll reach 14+ GbE with the FDR switch.

On the PC side the System Monitor showed an outgoing network throughput of 4.7GB/sec :-)

parallel-3x12GbE.png
parallel-3x12GbE.png (1.69 MiB) Viewed 5 times

Post Reply

Return to “Projects”

Who is online

Users browsing this forum: No registered users and 1 guest