[HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post Reply
domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

[HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

Although I'm using Mellanox InfiniBand for my local Intranet(*) where an H2 can communicate from 10Gbe to 14Gbe depending on direction, there are two other way to do 10G with the Odroid H2 if this is what you want to achieve.

(*) see https://magazine.odroid.com/article/net ... odroid-h2/ and viewtopic.php?f=172&t=38711. Yes this is quite exotic (at the beginning), but works very well and it is cheaper than the current RJ-45 10GBase-T solutions you can find (as of this writing).

1. The RJ-45 10GBase-T way: use an M.2 NMVe M key to PCIe x4 adapter(**) and plug in an RJ-45 10G PCIe card. See https://www.newegg.com/p/pl?d=10+Gbe+PC ... sdeptsrh=1 and https://www.amazon.com/s?k=10+Gbe+PCIe+card. This will cost you from $75 to $250+ depending on brand, features and number or ports.

(**) see https://www.ebay.com/itm/PCIe-x4-3-0-PC ... 4398886092

WARNING: not all PCIe cards will work with the PCIe 2.0 x4 available on the H2. Your mileage may vary. Dig into the Odroid forums to leverage the experience of other users.

At this point, if you only have one computer you want to connect to your H2, just connect them directly with a CAT6A (max length 100M = 330ft) and you're done. If you want to use many more computers, you'll need a switch and it will cost you $$$ (from about $800 at best up to +$2500 depending on number of ports and capabilities). Common brands are slowly climbing the 1+ G mountain. Models with 2.5GBase-T, 5GBase-T and 10GBase-T ports are appearing each month. Expensive. IMHO: wait for the no brand models from China, South Korea and Taiwan. Most of them will work as well... because most of them are exactly the same hardware minus the brand added to it.

If you are planning to have a mix of 2.5, 5 and 10G devices for the next few years, RJ-45 10GBase-T is the way to go for minimizing hassles and headaches.

2. A more exotic way is to chose an SFP+ solution.

Cards: You can find still functioning well 10G SFP+ NICs on eBay for less than $20. Example: https://www.ebay.com/itm/SolarFlare-SFN ... 3129025454. The SolarFlare SFN5122F was one the 10 Gbe networking work horse in data centers years ago. There are more recent models if you want to spend more than $20 (but still much less than $75, depending on features you need). Data centers today (the big ones) are standardizing on 100 or 200 Gbe. From these data centers point of view 10G is like an antique from the Merovingian period. But what is trash from some is treasure for others (the others being us here). There is a cornucopia of used 10, 25, 40 Gbe or InfiniBand 56 Gbps networking hardware on eBay. From our point of view, this is the Golden Age of reusing this hardware for individual purpose. Yes there is at least something trickling down in our dear US of A...

SFP+ cables: A 3M SFP+ DAC (Direct Attach Cable) copper cable can be find for cheap on eBay too. See for example: https://www.ebay.com/sch/i.html?_from=R ... =0&_sop=15

WARNING: 10 Gbe DAC cables are not all equal. One brand may or may not work with particular models of NICs. The Molex SFP-H10Gb-Cu3M works fine with the SolarFlare SFN5122F. Note: did the first cable I bought work with the card? Nope! Then I did my home work. The 2nd cable I bought works AOK :-)

Switches: There is also a plethora of 10G SFP+ switches on eBay. Starting from $70 up to $600 and more. Do you home work: check out how many ports are actually 10G SFP+, often the cheap ones will have 2 or 4 x 10G SFP+ and 24 or 48 1G ports. They were used to connect a corporate department computers to the backbone of the corporate Intranet. Or racks to the backbone network in a data center.

WARNING: Another issue to deal going the SFP+ route if the fact that the various forms of the 2.5G or 5G networking (i.e. BaseT) are relatively recent and "old" used 10G hardware you can buy on eBay will know nothing about of these protocols. In other words, they will NOT negotiate 2.5G or 5G making it impossible to connect a 2.5G or 5G device.

I did not want to spend too much time finding the perfect "old" switch so I settled on a very recent one from MicroTik (a Latvian company well-known in the networking business).

MicroTik CRS317-1G-16S+ Product Page
https://mikrotik.com/product/crs317_1g_16s_rm
MSRP: $399.00

Review from STH
https://www.servethehome.com/mikrotik-c ... rm-review/

On Amazon: $349.90
https://www.amazon.com/Cloud-Router-Swi ... B0747TC9DB

On NewEgg: $369.00
https://www.newegg.com/mikrotik-crs317- ... 002R-000D9

On eBay: $328.76 + $22.00 shipping
https://www.ebay.com/itm/Mikrotik-CRS31 ... 3710103713

I was lucky to stumble on a bidding new one which I got for $275 + shipping + tax = $313.66.
.
ebay-bidding.png
ebay-bidding.png (84.88 KiB) Viewed 4109 times
Hands-on
See the H2 and switch in the picture shown below.
.
H2-and-SFP+-switch.png
H2-and-SFP+-switch.png (1.46 MiB) Viewed 4109 times
See the SolarFlare SFN5122F connected to the H2 via the m.2 / PCIe adapter cable in the picture shown below:
.
SolarFlare SFN5122F.png
SolarFlare SFN5122F.png (1.85 MiB) Viewed 4109 times
.
The SFP+ cable connects the SolarFlare SFN5122F to the first port of the switch:
.
switch ports 1 and 2 in use.png
switch ports 1 and 2 in use.png (1.89 MiB) Viewed 4109 times
Note the red CAT6-A cable connected to the SFP+ switch. This "magic" is possible thanks too a small piece of hardware (transceiver) which converts back and forth SFP+ signals to RJ-45 signals. This thing was also bought from eBay. See https://www.ebay.com/itm/10G-SFP-RJ45-C ... 4214937752. There are multiple brands and models of such a transceiver. For a general overview of these transceivers, see this article and the related reviews from STH: https://www.servethehome.com/sfp-to-10g ... ers-guide/

With everything setup, one can see the connections in the switch web-based admin:
.
admin-1.png
admin-1.png (107.24 KiB) Viewed 4109 times
CONTINUED IN NEXT POST
Last edited by domih on Sun Mar 21, 2021 6:51 am, edited 2 times in total.
These users thanked the author domih for the post (total 2):
rooted (Wed Jun 10, 2020 2:03 pm) • mad_ady (Wed Jun 10, 2020 2:58 pm)

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

CONTINUING FROM PREVIOUS POST.

In the screenshot shown below, the first device is the H2 with the SolarFlare SFN5122F, the second device is a PC whose motherboard has an onboard Aquantia 10G Ethernet.
.
admin-2.png
admin-2.png (70.36 KiB) Viewed 4103 times
Here is a detailed view of the Ipolex transceiver allowing the Aquantia AQC-107 10GBase-T to talk with the SFP+ switch:
.
ipolex-1.png
ipolex-1.png (1.75 MiB) Viewed 4103 times
ipolex-2.png
ipolex-2.png (1.92 MiB) Viewed 4103 times
Let's now see the way the SolarFlare SFN5122F shows on the H2 with lspci:

Code: Select all

domih@h2a:~$ lspci
00:00.0 Host bridge: Intel Corporation Device 31f0 (rev 03)
.../...
01:00.0 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
01:00.1 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
Let's check whether or not the card does use the 4 lanes (it does, see LnkSta line):

Code: Select all

domih@h2a:~$ sudo lspci -vv -s 01:00.0 | grep Width 
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
		LnkSta:	Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
After configuring the NIC IP settings with nmtui, let's see the result (see enp1s0f1np1 section):

Code: Select all

domih@h2a:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1e:06:45:0d:47 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.70/24 brd 192.168.1.255 scope global noprefixroute enp2s0
       valid_lft forever preferred_lft forever
    inet6 fe80::7938:96d2:adee:503b/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: enp3s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 00:1e:06:45:0d:48 brd ff:ff:ff:ff:ff:ff
4: enp1s0f0np0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0f:53:07:89:80 brd ff:ff:ff:ff:ff:ff
5: enp1s0f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0f:53:07:89:81 brd ff:ff:ff:ff:ff:ff
    inet 172.16.25.69/16 brd 172.16.255.255 scope global noprefixroute enp1s0f1np1
       valid_lft forever preferred_lft forever
    inet6 fe80::af67:8d14:c279:c9da/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
Is the H2 "seeing" the switch? It does:

Code: Select all

domih@h2a:~$ ping 172.16.25.1
PING 172.16.25.1 (172.16.25.1) 56(84) bytes of data.
64 bytes from 172.16.25.1: icmp_seq=1 ttl=255 time=0.188 ms
64 bytes from 172.16.25.1: icmp_seq=2 ttl=255 time=0.141 ms
64 bytes from 172.16.25.1: icmp_seq=3 ttl=255 time=0.326 ms
64 bytes from 172.16.25.1: icmp_seq=4 ttl=255 time=0.142 ms
64 bytes from 172.16.25.1: icmp_seq=5 ttl=255 time=0.335 ms
^C
--- 172.16.25.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4102ms
rtt min/avg/max/mdev = 0.141/0.226/0.335/0.087 ms
Is the card configured for 10G with its link up? It is:

Code: Select all

domih@h2a:~$ ethtool enp1s0f1np1
Settings for enp1s0f1np1:
	Supported ports: [ FIBRE ]
	Supported link modes:   1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: No
	Supported FEC modes: Not reported
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  Not reported
	Link partner advertised pause frame use: No
	Link partner advertised auto-negotiation: No
	Link partner advertised FEC modes: Not reported
	Speed: 10000Mb/s
	Duplex: Full
	Port: FIBRE
	PHYAD: 255
	Transceiver: internal
	Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
	Current message level: 0x000020f7 (8439)
			       drv probe link ifdown ifup rx_err tx_err hw
	Link detected: yes
Is the H2 "seeing" the PC with the Aquantia? It does:

Code: Select all

domih@h2a:~$ ping 172.16.25.36
PING 172.16.25.36 (172.16.25.36) 56(84) bytes of data.
64 bytes from 172.16.25.36: icmp_seq=1 ttl=64 time=0.534 ms
64 bytes from 172.16.25.36: icmp_seq=2 ttl=64 time=0.296 ms
64 bytes from 172.16.25.36: icmp_seq=3 ttl=64 time=0.250 ms
64 bytes from 172.16.25.36: icmp_seq=4 ttl=64 time=0.264 ms
64 bytes from 172.16.25.36: icmp_seq=5 ttl=64 time=0.299 ms
64 bytes from 172.16.25.36: icmp_seq=6 ttl=64 time=0.259 ms
64 bytes from 172.16.25.36: icmp_seq=7 ttl=64 time=0.254 ms
64 bytes from 172.16.25.36: icmp_seq=8 ttl=64 time=0.326 ms
^C
--- 172.16.25.36 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7158ms
rtt min/avg/max/mdev = 0.250/0.310/0.534/0.089 ms
It looks like everything is OK. Let's measure the throughput with iperf3:

Code: Select all

domih@h2a:~$ iperf3 -c 172.16.25.36 --bind 172.16.25.69
Connecting to host 172.16.25.36, port 5201
[  4] local 172.16.25.69 port 43329 connected to 172.16.25.36 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.09 GBytes  9.33 Gbits/sec    0   1.31 MBytes       
[  4]   1.00-2.00   sec  1.09 GBytes  9.33 Gbits/sec    0   1.41 MBytes       
[  4]   2.00-3.00   sec  1.09 GBytes  9.32 Gbits/sec    0   1.41 MBytes       
[  4]   3.00-4.00   sec  1.09 GBytes  9.33 Gbits/sec    0   1.41 MBytes       
[  4]   4.00-5.00   sec  1.08 GBytes  9.30 Gbits/sec    0   1.60 MBytes       
[  4]   5.00-6.00   sec  1.08 GBytes  9.31 Gbits/sec    0   1.60 MBytes       
[  4]   6.00-7.00   sec  1.09 GBytes  9.33 Gbits/sec    0   2.16 MBytes       
[  4]   7.00-8.00   sec  1.07 GBytes  9.22 Gbits/sec    1   1.21 MBytes       
[  4]   8.00-9.00   sec  1.06 GBytes  9.15 Gbits/sec    1   1.28 MBytes       
[  4]   9.00-10.00  sec  1.08 GBytes  9.26 Gbits/sec    0   1.34 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  10.8 GBytes  9.29 Gbits/sec    2             sender
[  4]   0.00-10.00  sec  10.8 GBytes  9.28 Gbits/sec                  receiver

iperf Done.
domih@h2a:~$ iperf3 -c 172.16.25.36 --bind 172.16.25.69 -R
Connecting to host 172.16.25.36, port 5201
Reverse mode, remote host 172.16.25.36 is sending
[  4] local 172.16.25.69 port 37157 connected to 172.16.25.36 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  1.09 GBytes  9.39 Gbits/sec                  
[  4]   1.00-2.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   2.00-3.00   sec  1.09 GBytes  9.40 Gbits/sec                  
[  4]   3.00-4.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   4.00-5.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   5.00-6.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   6.00-7.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   7.00-8.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   8.00-9.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  4]   9.00-10.00  sec  1.10 GBytes  9.41 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  11.0 GBytes  9.41 Gbits/sec  408             sender
[  4]   0.00-10.00  sec  11.0 GBytes  9.41 Gbits/sec                  receiver

iperf Done.
Yeah! We get a respectable 9.28 Gbits/sec in one direction and 9.41 Gbits/sec in the other.

However, see the high number of segment retries (408) in the reverse test (-R parameter). I will have to do my home work on this one to mitigate the issue. It does not seem to affect much the throughput though.

CONCLUSION: if you are NOT planning to use 2.5G and 5G devices and want to use 10G as soon as possible an SFP+ switch and a mix of SFP+ and RJ-45 NICs might be the solution you are looking for.

With SFP+, it will cost you about $20 (card) + $15 (3m DAC cable) + $360 / 16 (switch) = $57.50 per port and $50 (transceiver) + $15 (CAT6-A cable) + $360 / 16 = $87.50 per port (not counting the price of the 10GBase-T NIC when it is onboard a motherboard). This means that you want to go with an SFP+ NIC when the PC does not have 10G. All you need is a PCIe 2.0 x8 or higher (x4 is OK as shown above). Expect the price of the SFP+ to RJ-45 transceivers to go down.

Other considerations:
- SFP+ DAC cables are cheap enough for 3m or 5m or 7m. For much longer distances, you'll have to go with a fiber cable and corresponding transceiver. In this case, the 100m max length possible with CAT6-A makes the RJ-45 solution better in terms of cost.
- For casual usage, plugging-in an RJ-45 NIC in a PCIe slot is a simple thing to do (plug and play). The consumer products are easy to use, you will not have to perform tweaking in most of the cases. Old data centers cards based on SFP+ provide many options, the SolarFlare Server Adapter User Guide is 393-page long. But for the hands-on in this post, I did not even read it, the card drivers are part of the Linux kernel and I did not need to change settings on the card beyond setting the IP address and mask. It was also plug and play. So it could be more complex to set up an SFP+ solution but it does not have to be. You only need to start tweaking when you start to do specialized things like VMware servicing or esoteric networking configuration. Also in the RJ-45, it would be the same thing with cards costing > $200.

Do you home work by comparing this price per port to a full RJ-45 solution. The RJ-45 10G NICs are expensive and the switches even more expensive. But then again, if you are planning to have a mix of 2.5G, 5G and 10G devices, you will want to use RJ-45 all along and ignore SFP+.

To conclude: which is better for 10G networking? As you might expect, it depends!

HTH.
Last edited by domih on Wed Jun 10, 2020 3:17 pm, edited 1 time in total.
These users thanked the author domih for the post:
mad_ady (Wed Jun 10, 2020 3:00 pm)

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by mad_ady »

Great networking overview.
Regarding
However, see the high number of segment retries (408) in the reverse test (-R parameter). I will have to do my home work on this one to mitigate the issue. It does not seem to affect much the throughput though.
I think packet drops and retransmits are normal and a part of TCP's exponential backoff network congestion avoidance algorithm. Simply put - TCP will always try (as long as the transmission window allows it) to send more data on a connection until there is packet loss/need for retransmission. When that happens (based on the current implementation of the backoff algorithm) it will reduce its throughput and ramp it up again slowly until another loss happens, and so on. Unlike UDP who will happily congest every link.
So packet loss is normal in the life of a tcp session. You can see it on 1G iperf tests too.
These users thanked the author mad_ady for the post:
domih (Wed Jun 10, 2020 3:20 pm)

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

mad_ady wrote:
Wed Jun 10, 2020 3:09 pm
Great networking overview.
Regarding
However, see the high number of segment retries (408) in the reverse test (-R parameter). I will have to do my home work on this one to mitigate the issue. It does not seem to affect much the throughput though.
I think packet drops and retransmits are normal and a part of TCP's exponential backoff network congestion avoidance algorithm. Simply put - TCP will always try (as long as the transmission window allows it) to send more data on a connection until there is packet loss/need for retransmission. When that happens (based on the current implementation of the backoff algorithm) it will reduce its throughput and ramp it up again slowly until another loss happens, and so on. Unlike UDP who will happily congest every link.
So packet loss is normal in the life of a tcp session. You can see it on 1G iperf tests too.
I will have to read more about it because I'm really used to NOT seeing segment retries. With InfiniBand I only saw them when testing a damaged cable. A handful is acceptable but several hundred means potential issue, like for instance what I saw when testing the 2.5 Gbe USB adapters. But once again, I'll have to read about it, contrary to what it may seem, I am definitely NOT a networking guy ;-)

kingtut
Posts: 1
Joined: Sun Aug 23, 2020 5:03 am
languages_spoken: english
Has thanked: 0
Been thanked: 0
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by kingtut »

@domih Can the Solarflare or even the RJ45 10gbe NIC be powered by the onboard Odroid H2 SATA power connecting to the m.2 to PCIe breakout adapter?

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

kingtut wrote:
Sun Aug 23, 2020 5:06 am
@domih Can the Solarflare or even the RJ45 10gbe NIC be powered by the onboard Odroid H2 SATA power connecting to the m.2 to PCIe breakout adapter?
I don't know. The cards I used during testing where SolarFlare S5122-R66, the 5122 is stated to "typically" use 4.9W. What "typical" means, beats me.

solarflare-5122.png
solarflare-5122.png (44.29 KiB) Viewed 3617 times
So the next question is to know the max power the H2/H2+ SATA III power connector can deliver. If OK, then test idle and full speed with iperf3.

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

kingtut wrote:
Sun Aug 23, 2020 5:06 am
@domih Can the Solarflare or even the RJ45 10gbe NIC be powered by the onboard Odroid H2 SATA power connecting to the m.2 to PCIe breakout adapter?
To give a more precise answer: yes you can power a Solarflare 10 GbE NIC from the H2/H2+ SATA power connectors.

I just tried it. Using the regular Hardkernel SATA power cables connecting to the H2/H2+ board, one is forked with a split extension cable to power the HDD and SSD. I connect the M.2 adapter floppy-like female to SATA male cable to the other Hardkernel SATA power cable connected to the H2/H2+ board. Works fine: I ran multiple Samba tests, no issue. The 5W NIC does not overload the H2+ 12V rail.

CONCLUSION: The 15V/4A power brick provides 60W, as long as the H2/H2+ board + disk(s) + network card are let's say <= 50W you're fine. Otherwise, if necessary upgrade to a 19V 90W power brick as described in other threads.

For a 10 GbE RJ-45 NIC, I don't know, I don't have one. I guess it is OK AS LONG AS the rule stated in the CONCLUSION is respected.

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

A timid first salvo at democratizing 10GbE at home:

TP-Link's Cheap 5-port and 8-port 10GbE Switches Now Available
https://www.tomshardware.com/news/tp-li ... gbe-switch

IMHO, the "cheap" in the title is candid:
- $275 MSRP for the 5 ports.
- Projected $450 for the 8 ports.

Product pages
https://www.tp-link.com/en/home-network ... /#overview
https://www.tp-link.com/en/home-network ... tl-sx1008/

In the article, the comparison with the ASUS XG-U2008 is useless because it's a 2x10Gbe + 8x1Gbe and there is an endless series of switches offering such a mix.

IMHO, as of this writing, the cheapest solutions are still looking for used 10Gb switches on eBay and choose the best adapted to your needs. Example: the Aruba Networks S3500 24-Port PoE S3500-24P Gigabit Switch S3500-4x10G 10GB at $135 with 24x1GbE RJ-45 + 4x10GbE SFP+ coupled with used SolarFlare cards and cheap SFP+ cables for the 4 lucky PCs... or H2+ :-) See https://www.ebay.com/sch/i.html?_nkw=10gb+switch
These users thanked the author domih for the post:
TurdFurguson (Sun Mar 21, 2021 7:32 am)

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

Inside the MicroTik CRS317-1G-16S+ and silencing the two fans

The two fans inside the switch can be noisy, so I replaced them with two 12V Noctua NF-A4x20. Mission accomplished: the switch is now quasi-silent even with the two fans at full speed.

The construction of the MicroTik CRS317-1G-16S+ is of pretty good quality and the servicing is very easy: 5 screws and you can slide the top cover open, my rating is 10/10. The fans plugs are standard :-)

Internal view of the switch

01-inside-the-switch.png
01-inside-the-switch.png (1.45 MiB) Viewed 2123 times

The noisy stock fans

02-the-noisy-beasts.png
02-the-noisy-beasts.png (1.4 MiB) Viewed 2123 times

Noctua replacement
Using the rubber fasteners to minimize noise from resonance.

03-noctua-1.png
03-noctua-1.png (1.49 MiB) Viewed 2123 times
04-noctua-2.png
04-noctua-2.png (1.71 MiB) Viewed 2123 times

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

Other 10GbE cards opportunities

If you are not a fan of the Marvell/Aquantia chipset, there are used INTEL-based (X540 RJ-45) at sub-$100 on eBay, see https://www.ebay.com/sch/i.html?_nkw=in ... 45&_sop=15

Same thing for INTEL-based (X520 SFP+) again on eBay, see https://www.ebay.com/sch/i.html?_nkw=in ... fp&_sop=15

WARNING
INTEL-based NICs on eBay: make sure you identify the counterfeit ones, see https://www.bing.com/search?q=counterfeit+intel+10g+nic, note that these used cards are already out of warranty so if a user says they might be counterfeit but work fine, you might not care anyway...

Again, if you go with SFP+, the used SolarFlare are inexpensive and all over the place.

Note: drivers for all these cards are probably already present in the Linux upstream kernels. Same for Windows via updates.

Last but not least: the 10G NICs on eBay probably ran quite a few years (decade+ ?) in a data center, so re-pasting the heat sink is a good idea.

elatllat
Posts: 1886
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1, N2, C4, N2+, HC4
Has thanked: 67 times
Been thanked: 137 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by elatllat »

I just had a nearby lightning strike induce voltage in an ethernet cable and take out ports on some gear. The Odroids were all fine as they are not grounded, but it's making me consider optical or Metal Shield RJ45...

User avatar
rooted
Posts: 8999
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 745 times
Been thanked: 422 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by rooted »

elatllat wrote:I just had a nearby lightning strike induce voltage in an ethernet cable and take out ports on some gear. The Odroids were all fine as they are not grounded, but it's making me consider optical or Metal Shield RJ45...
Yeah I had a nearby lightning strike take out the Ethernet on one of my XU4, I have to use USB Ethernet now :/

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

Nothing stops capitalism (read greed). The SolarFlare SFN5122F Dual Port 10GbE I've got for less than $20 on eBay one and a half year ago, are now floating around eBay at $35.

I guess the shortage of IC in 2021 traveled back in time to affect the manufacturing of these cards in the 2000's *&^%$#@! :D

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

You want to have 10 GbE on your computer but you are out of PCIe slots for a 10 GbE PCIe card?

If you have a free NVMe PCIe Gen 3 x2 or x4 slot, you can use this new thing:

https://www.techradar.com/news/this-tin ... ur-mini-pc
https://www.innodisk.com/en/products/em ... /egpl-t101

Do not expect a low nice price. Regular "consumer" 10 GbE PCIe cards range from $100 to $500++.

While compatible with PCIe Gen 2, you won't get 10 GbE with Gen 2, maybe 7.5 GbE at best.

Reminder: a 1 GbE or 2.5 GbE chipset can get "warm", a 10 GbE will get quite hotter.
These users thanked the author domih for the post:
odroid (Fri Dec 17, 2021 9:36 am)

domih
Posts: 445
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4 - 1GbE, 2.5GbE, 10GbE, 40+GbE
Location: San Francisco Bay Area
Has thanked: 167 times
Been thanked: 169 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by domih »

More new 10 GbE hardware that can auto-negotiate to 2.5 and 5 GbE at relative affordable pricing:

TP-Link JetStream 8-Port 10GE SFP+ L2+ Managed Switch
https://www.tp-link.com/us/business-net ... l-sx3008f/
https://www.newegg.com/p/0E6-002W-007R1 $249

QNAP QHora-301W AX & 10Gbe
https://nascompares.com/2020/11/12/qnap ... er-review/
https://www.newegg.com/qnap-qhora-301w- ... 6833831028 $329

Dual 10GbE router supports WiFi 6 (AX3600)
https://www.cnx-software.com/2021/12/23 ... -6-ax3600/
https://www.acelink.com.tw/products_det ... br-6889ax/ $???
These users thanked the author domih for the post:
odroid (Tue Dec 28, 2021 8:55 am)

User avatar
rooted
Posts: 8999
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 745 times
Been thanked: 422 times
Contact:

Re: [HOWTO] Yet another exotic way to do 10G networking with the Odroid H2

Post by rooted »


I'm interested in this router and hope the price isn't crazy

Post Reply

Return to “Hardware and peripherals”

Who is online

Users browsing this forum: No registered users and 5 guests