[HOWTO] 2.5Gbe or 5Gbe with H2 and/or N2 and/or C4?

domih · Post by **domih** » Tue May 05, 2020 9:39 am

I know for a fact that the H2 can handle 2.5Gbe Ethernet. I'm pretty sure the N2 and C4 should also be able too, although I have not tried them yet. To be more precise I did try them but unsuccessfully, not because of the N2 or C4 but because of the adapters. For 5Gbe, the H2 will handle it, the question is open for the N2 and the C4.

By "handle it" I mean: the USB 3 port is fast enough to support the expected Ethernet speed and the onboard USB chipset as well as CPU still have enough power in reserve to for instance handle a hard disk in a timely manner if the network operation is a file copy.

So how do you get 2.5Gbe Ethernet on the H2, N2 and C4? You can use a USB 3 2.5Gbe adapter, they have started to appear on the market. You have to find a USB 3.0 type A or a USB 3.1 type C + an active 3.1 C -> 3.0 A adapter.

Important: for the latter a dumb cable converter will not work, you need an active one that includes some electronics actually converting 3.1 and 3.0 back and forth. See this one which works: https://www.ebay.com/itm/USB-3-1-Type-C ... 4026583245

See the different models I tried in the picture at the end of this post.

There are all using the same Realtek chipset, the RTL8156... and they are "stability-challenged" (euphemism for not working well, at all.) What you will see:
- Using iperf3, weird variation of the speed from 2.35 Gbe (what you should expect) to whatever 150Mb/s and anything in the middle.
- Incredible amount of segment retries, several hundreds sometimes during a same few minutes test.
- Hard (fatal and final) or soft (might recover) disconnections, for the hard ones the only way to have the computers seeing each other again is to unplug and plug back the USB 3 adapter.
- Any attempt to run a 2+ hour iperf3 test ending up with a lost connection, usually of the hard kind.

Tested with Linux 18.04, 19.04, 19.10 and 20.04 with manually installing the most recent Realtek drivers.

The tests were made with direct connections from PC to PC. There are currently no CHEAP 1/2.5/5/10 Gbe BaseT (RJ45) switches.

I know for a fact that the H2, N2 and C4 are totally innocent in this Mr. SNAFU meets Ms. FUBAR story:
- An H2 with a PCIe 2.0 2.5Gbe Ethernet card works OK providing a sustainable 2.35 Gb/sec. The chipset is a Realtek RTL8125 (for PCie) which is an entire different beast from the RTL8156 (for USB 3).
- I also tried the USB models on PCs (Intel and AMD) to see similar issues.

CONCLUSION
Is it worth to update to 2.5Gbe on SBCs such as the H2, N2 or C4? Yes, in the not so far future. But do not waste your hard won kopeks on RTL8156 based USB 3 2.5Gbe adapters.

Wait for the ones based on the RTL8156B. Notice the B at the end and yes when a chip maker releases a B version, you know what that means about the "A" version

.

: 20200504_162006.png (3.57 MiB) Viewed 8109 times

PigLover · Post by **PigLover** » Wed May 06, 2020 1:49 am

What would be really nice would be for the H2 to support it natively. The RTL8125 is pin compatible with the RTL8111G currently used on the H2. Dead simple to spec the new part and test. There is a small power profile change, but should be well inside the tolerances already in the design. Few if any changes would be required.

domih · Post by **domih** » Thu May 07, 2020 3:47 am

Let's lit a candle and hope that Hard Kernel thinks the same way!

zupet · Post by **zupet** » Tue Jul 07, 2020 4:01 pm

viewtopic.php?f=97&t=30839&p=288901&hil ... 56#p288901

I've tested USB 3.0 RTL8156 with XU4, and it worked well even with SBCs.

https://www.realtek.com/en/component/zo ... 0-software

As I know, realtek does not shipping binary linux drivers for RTL8156.
It have to be merged into kernel source and compile.

I wonder what driver you used.

rooted · Post by **rooted** » Wed Jul 08, 2020 7:47 am

Disregard

domih · Post by **domih** » Thu Jul 09, 2020 5:58 am

zupet wrote: ↑
Tue Jul 07, 2020 4:01 pm
viewtopic.php?f=97&t=30839&p=288901&hil ... 56#p288901

I've tested USB 3.0 RTL8156 with XU4, and it worked well even with SBCs.

https://www.realtek.com/en/component/zo ... 0-software

As I know, realtek does not shipping binary linux drivers for RTL8156.
It have to be merged into kernel source and compile.

I wonder what driver you used.

At the time of testing (Apr 2020), I used: r8152.53.56-2.12.0.tar, it looks like they released an update r8152.53.56-2.13.0.tar.bz2. I'll try it next week.

iperf3 will "work" for the default 10 sec test. Run 1H, 2H or more and it is probable you'll start experiencing transmit issues (which show up post-mortem in syslog/dmesg).

Depending on models and tries I was getting these errors sooner or later.

Important Note: If anyone finds a way to make them work OK on Linux even under torture-like testing, I will be very grateful. I have seven of these duds, I had 8 but I sacrificed one opening it for getting at the chipset and check it was an RTL8156, not a B one.

domih · Post by **domih** » Thu Jul 09, 2020 1:58 pm

If anyone feels adventurous and wants to try it for the rest of us: https://www.amazon.com/2500Mbps-Etherne ... B088N74HBY

zupet · Post by **zupet** » Thu Jul 09, 2020 4:05 pm

domih wrote: ↑
Thu Jul 09, 2020 5:58 am
At the time of testing (Apr 2020), I used: r8152.53.56-2.12.0.tar, it looks like they released an update r8152.53.56-2.13.0.tar.bz2. I'll try it next week.

iperf3 will "work" for the default 10 sec test. Run 1H, 2H or more and it is probable you'll start experiencing transmit issues (which show up post-mortem in syslog/dmesg).

Depending on models and tries I was getting these errors sooner or later.

Important Note: If anyone finds a way to make them work OK on Linux even under torture-like testing, I will be very grateful. I have seven of these duds, I had 8 but I sacrificed one opening it for getting at the chipset and check it was an RTL8156, not a B one.

I've tested it more than 4 hours. You can find my test result in first link, 4 hours average 279MB/s.

I had issue with nfs but iperf3 works well for both pc -> xu4 or xu4 -> pc.

domih · Post by **domih** » Sat Jul 11, 2020 6:12 am

zupet wrote: ↑
Thu Jul 09, 2020 4:05 pm

domih wrote: ↑
Thu Jul 09, 2020 5:58 am
At the time of testing (Apr 2020), I used: r8152.53.56-2.12.0.tar, it looks like they released an update r8152.53.56-2.13.0.tar.bz2. I'll try it next week.

iperf3 will "work" for the default 10 sec test. Run 1H, 2H or more and it is probable you'll start experiencing transmit issues (which show up post-mortem in syslog/dmesg).

Depending on models and tries I was getting these errors sooner or later.

Important Note: If anyone finds a way to make them work OK on Linux even under torture-like testing, I will be very grateful. I have seven of these duds, I had 8 but I sacrificed one opening it for getting at the chipset and check it was an RTL8156, not a B one.
I've tested it more than 4 hours. You can find my test result in first link, 4 hours average 279MB/s.

I had issue with nfs but iperf3 works well for both pc -> xu4 or xu4 -> pc.

I'll give a try to v2.13 but I'm still very skeptical. The fact that NFS does not work well is worrying. Note that I was able to run several hours tests with no "apparent" issues. So it could be what your are witnessing. To be sure, I ended up redirecting the console output of iperf3 into a text file to do post-mortem examination of the throughput all along the session and here it was: while the session end up apparently well, there were non-fatal incidents right in the middle. Could it be that the USB NIC issues are less apparent on an XU4 compared to H2, N2 and C4 ?

domih · Post by **domih** » Thu Jul 30, 2020 5:21 am

There are new RTL8156B-based USB 3.0 Type A 2.5GbE adapters available on eBay:

https://www.ebay.com/itm/USB-3-0-Ethern ... 3057155924
US $20.86 - Estimated between Tue. Aug. 25 and Fri. Sep. 18 - From Shen Zhen, China
The description does specify "Chipset:Realtek RTL8156B"

I ordered two of them. I will let you know how they work when I received them.

There are other models Type A and Type C, just make sure to check the chip set is RTL8156B:
https://www.ebay.com/sch/i.html?_from=R ... =0&_sop=15

domih · Post by **domih** » Tue Aug 11, 2020 7:34 am

I have received the new RTL8156B-based USB 3.0 Type A 2.5GbE adapters. See product label photos shown below.
I'm currently in the process of testing them. Initial testing looks OK but as described in this thread, long tests are necessary to show that they do not have the flaws of their predecessors. I'll let you know of the results in the next few days.

: 20200810_143719.png (2.86 MiB) Viewed 7062 times

: 20200810_143737.png (3.16 MiB) Viewed 7062 times

domih · Post by **domih** » Tue Aug 11, 2020 9:21 am

First tests.

h2a: H2+
h2b: H2 with R8156B-based USB 3 2.5 GbE adapter
h2c: H2 with R8156B-based USB 3 2.5 GbE adapter
h2d: H2+

h2a and h2d are iperf3 servers.
h2b and h2c are iperf3 clients.

h2a.25, h2b.25, h2c.25 and h2d.25 are the host names for the 2.5 GbE connections.

1-hour Test

Code: Select all

domih@h2b:~$ iperf3 -c h2a.25 -t 3600 > iperf3-h2.25-1h.text
.../...
# Check post-mortem the output:
domih@h2b:~$ less iperf3-h2.25-1h.text
# Scroll with space bar looking for any visual disruption revealing errors or change of speed
.../...
Connecting to host h2a.25, port 5201
[  5] local 172.16.25.71 port 58172 connected to 172.16.25.70 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   277 MBytes  2.32 Gbits/sec    0   1007 KBytes      
[  5]   1.00-2.00   sec   278 MBytes  2.33 Gbits/sec    0   1007 KBytes
.../...
[  5] 3598.00-3599.00 sec   278 MBytes  2.33 Gbits/sec    0   2.15 MBytes      
[  5] 3599.00-3600.00 sec   278 MBytes  2.33 Gbits/sec    0   2.15 MBytes
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-3600.00 sec   975 GBytes  2.33 Gbits/sec   12             sender
[  5]   0.00-3600.00 sec   975 GBytes  2.33 Gbits/sec                  receiver

domih@h2b:~$ iperf3 -c h2a.25 -t 3600 -R > iperf3-h2.25-1h-reverse.text
.../...
# Check post-mortem the output:
domih@h2b:~$ less iperf3-h2.25-1h.text
# Scroll with space bar looking for any visual disruption revealing errors or change of speed
.../...
Connecting to host h2a.25, port 5201
Reverse mode, remote host h2a.25 is sending
[  5] local 172.16.25.71 port 58176 connected to 172.16.25.70 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   253 MBytes  2.12 Gbits/sec                  
[  5]   1.00-2.00   sec   275 MBytes  2.31 Gbits/sec
.../...
[  5] 3598.00-3599.00 sec   274 MBytes  2.29 Gbits/sec                  
[  5] 3599.00-3600.00 sec   275 MBytes  2.31 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-3600.00 sec   950 GBytes  2.27 Gbits/sec  7341             sender
[  5]   0.00-3600.00 sec   950 GBytes  2.27 Gbits/sec                  receiver

So in reverse, the speed varies from +-950 to +- 2.32 Gbits/sec with 7341 segment retries and a total of 950 GBytes compared to 975 GBytes in the other direction.

CONCLUSION: 1-hour test and its reverse between h2b (H2) and h2a (H2+) showed no errors nor disconnections. The segment retries in reverse mode can be explained by the fact that the throughput via the PCIe-based RTL8125B is slighlty superior to the USB-based RTL8156B?

2-hour Test

Code: Select all

domih@h2c:~$ iperf3 -c h2d.25 -t 7200 > iperf3-h2d.25-7200.txt
.../...
# Check post-mortem the output:
domih@h2c:~$ less iperf3-h2d.25-7200.txt
# Scroll with space bar looking for any visual disruption revealing errors or change of speed
.../...
Connecting to host h2d.25, port 5201
[  4] local 172.16.25.72 port 40204 connected to 172.16.25.73 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   123 MBytes  1.04 Gbits/sec    6    417 KBytes       
[  4]   1.00-2.00   sec   263 MBytes  2.21 Gbits/sec    0    759 KBytes
.../...
[  4] 7198.00-7199.00 sec   261 MBytes  2.19 Gbits/sec    0   2.40 MBytes       
[  4] 7199.00-7200.00 sec   262 MBytes  2.20 Gbits/sec    0   2.40 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-7200.00 sec  0.00 (null)s  2.18 Gbits/sec   15             sender
[  4]   0.00-7200.00 sec  0.00 (null)s  2.18 Gbits/sec                  receiver

Note: the 2-hour test in reverse is still running.

CONCLUSION: 2-hour test went OK. The test h2c->h2d is slower than the test h2b->h2a by around 6.44%. Not sure why but good anyway.

Note: I started a 7-hour test with h2b->h2a.

Post by **odroid** » Tue Aug 11, 2020 9:28 am

Very promising

domih · Post by **domih** » Tue Aug 11, 2020 10:37 am

...and while analyzing the 2-hour reversed test (speed all over the place) I just found out that the r8156 driver was not even installed. I'll redo that test with the driver installed. It looks to me that the kernel 5.4.0-42-generic still does not integrate the RTL8156B correctly into the r8169 kernel module. I have not seen any hard errors or spurious disconnections yet though like I was seeing with the RTL8156 not B

domih · Post by **domih** » Tue Aug 11, 2020 10:51 am

Yep, it is much better with the r8152-2.13.0 driver. I even get 2.35 Gbits/sec in reverse mode. I guess that installing the driver on the other H2 will also eliminate the witnessed discrepancy in reverse and bring the speed to 2.35 Gbits/sec to both directions.

domih · Post by **domih** » Tue Aug 11, 2020 3:01 pm

More testing

2-hour test in reverse with r8152-2.13.0 driver

Code: Select all

domih@h2c:~$ iperf3 -c h2d.25 -t 7200 -R > iperf3-h2d.25-7200-reverse.txt
.../...
less iperf3-h2d.25-7200-reverse.txt
.../...
Connecting to host h2d.25, port 5201
Reverse mode, remote host h2d.25 is sending
[  4] local 172.16.25.72 port 58030 connected to 172.16.25.73 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  46.2 MBytes   387 Mbits/sec                  
[  4]   1.00-2.00   sec  35.0 MBytes   294 Mbits/sec                  
[  4]   2.00-3.00   sec  39.3 MBytes   329 Mbits/sec                  
[  4]   3.00-4.00   sec  36.3 MBytes   305 Mbits/sec                  
[  4]   4.00-5.00   sec  28.9 MBytes   242 Mbits/sec                  
[  4]   5.00-6.00   sec  42.9 MBytes   360 Mbits/sec                  
[  4]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec                  
[  4]   7.00-8.00   sec   280 MBytes  2.35 Gbits/sec
.../...
[  4] 7198.00-7199.00 sec   280 MBytes  2.35 Gbits/sec                  
[  4] 7199.00-7200.00 sec   280 MBytes  2.35 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-7200.01 sec  1.92 TBytes  2.35 Gbits/sec  2506             sender
[  4]   0.00-7200.00 sec  0.00 (null)s  2.35 Gbits/sec                  receiver

No errors, no disconnections.

5.5-hour test with r8169 driver

Code: Select all

domih@h2b:~$ iperf3 -c h2a.25 -t 25200 > iperf3-h2.25-7h.text
.../...
# I interrupted the test to install the r8152-2.13.0 driver for the overnight test
# domih@h2b:~$ less iperf3-h2.25-7h.text
.../...
Connecting to host h2a.25, port 5201
[  5] local 172.16.25.71 port 58180 connected to 172.16.25.70 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   279 MBytes  2.34 Gbits/sec    0   1.01 MBytes       
[  5]   1.00-2.00   sec   278 MBytes  2.33 Gbits/sec    0   1.06 MBytes
.../...
[  5] 19917.00-19917.08 sec  20.0 MBytes  2.21 Gbits/sec    0   3.89 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-19917.08 sec  5.27 TBytes  2.33 Gbits/sec   44             sender
[  5]   0.00-19917.08 sec  0.00 Bytes  0.00 bits/sec                  receiver

No errors, no disconnections.

CONCLUSION: The RTL8156B seems to eliminate the issues of hard errors and disconnections I witnessed with the RTL8156.

I'm now going to run an overnight test. If pristine I will consider these new RTL8156B-based USB 3 2.5 GbE as validated and my new BFF.

OTHER CONCLUSION: The r8169 kernel module "handles" the RTL8156B and gives the appearance to be working BUT to get full speed and eliminate the "all over the place" speed in the reception direction make sure to install the r8152 kernel module which you can get from the Realtek website https://www.realtek.com/en/component/zo ... 0-software. Tip: Do not confuse this driver with the one for the onboard RTL8125B. The drivers for USB and PCIe NIC are two different beasts.

Post by **odroid** » Tue Aug 11, 2020 3:12 pm

r8152 and r8125 make a lot of confusions.

Thank you for sharing the valuable test result. I hope we can test the same USB NIC within a couple of weeks.

domih · Post by **domih** » Tue Aug 11, 2020 3:23 pm

<<...CONCLUSION: 2-hour test went OK. The test h2c->h2d is slower than the test h2b->h2a by around 6.44%. Not sure why but good anyway...>>

The r8152 kernel module fixed it, both h2c->h2d and h2b->h2a now cruise at 2.35 Gbits/sec.

Overnight 10-hour tests are now running. Results tomorrow morning around 9 AM (US CA time)

domih · Post by **domih** » Tue Aug 11, 2020 3:35 pm

odroid wrote: ↑
Tue Aug 11, 2020 3:12 pm
r8152 and r8125 make a lot of confusions.
Thank you for sharing the valuable test result. I hope we can test the same USB NIC within a couple of weeks.

Yep, sometimes I screw up myself. In addition to deceive the "candid" users the kernel module/driver for the RTL8156B is named 8152

The install of the r8152 kernel module on Ubuntu is:

tar -xf r8152.53.56-2.13.0.tar.bz2
cd r8152-2.13.0
make
sudo make install
# Check whether or not the r8152 module is loaded
lsmod | grep r8
# If OK, update initramfs
sudo update-initramfs -u
# Optional: reboot to check whether or not the r8152 module is loaded
sudo reboot
lsmod | grep r8

Suggestion: I guess you could make a .deb for the 8152 too for v18.04 and v20.04 to ease the life of the users who are not savvy in manually building and installing kernel modules.

Post by **odroid** » Tue Aug 11, 2020 3:48 pm

Yep. we will make a DKMS deb package once we test the RTL8125B adapter.

domih · Post by **domih** » Wed Aug 12, 2020 4:41 am

More testing

10-hour test with r8152-2.13.0 driver, h2b->h2a

Code: Select all

domih@h2b:~$ iperf3 -c h2a.25 -t 36000 > iperf3-h2a.25-36000.text
.../...
domih@h2b:~$ less iperf3-h2a.25-36000.text
.../...
Connecting to host h2a.25, port 5201
[  5] local 172.16.25.71 port 57488 connected to 172.16.25.70 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   284 MBytes  2.38 Gbits/sec    0    814 KBytes       
[  5]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    854 KBytes
.../...
[  5] 35998.00-35999.00 sec   280 MBytes  2.35 Gbits/sec    0   3.14 MBytes       
[  5] 35999.00-36000.00 sec   281 MBytes  2.36 Gbits/sec    0   3.14 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-36000.00 sec  9.63 TBytes  2.35 Gbits/sec    0             sender
[  5]   0.00-36000.00 sec  9.63 TBytes  2.35 Gbits/sec                  receiver

10-hour test with r8152-2.13.0 driver, h2c->h2d

Code: Select all

domih@h2c:~$ iperf3 -c h2d.25 -t 36000 > iperf3-h2d.25-36000.txt
.../...
domih@h2c:~$ less iperf3-h2d.25-36000.txt
.../...
Connecting to host h2d.25, port 5201
[  4] local 172.16.25.72 port 58038 connected to 172.16.25.73 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   268 MBytes  2.25 Gbits/sec    0    817 KBytes       
[  4]   1.00-2.00   sec   281 MBytes  2.36 Gbits/sec    0    956 KBytes
.../...
[  4] 35998.00-35999.00 sec   281 MBytes  2.36 Gbits/sec    0   4.19 MBytes       
[  4] 35999.00-36000.00 sec   281 MBytes  2.35 Gbits/sec    0   4.19 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-36000.00 sec  0.00 (null)s  2.35 Gbits/sec    2             sender
[  4]   0.00-36000.03 sec  9.63 TBytes  2.35 Gbits/sec                  receiver

CONCLUSION: No errors, no disconnections. Initial discrepancies eliminated by Realtek driver. I consider these RTL8156B-based USB 3 2.5 GbE adapters to be the right stuff

johnnyde94 · Post by **johnnyde94** » Thu Sep 03, 2020 8:42 am

Any news on making the driver package, as I’m sure I found a way to mess things up even things as easily as this lol. I downloaded the file from realteks website, cd to the directory and ran “Sudo make && Sudo make install” I also followed the read me instructions.

tobetter · Post by **tobetter** » Thu Sep 03, 2020 9:22 am

johnnyde94 wrote: ↑
Thu Sep 03, 2020 8:42 am
Any news on making the driver package, as I’m sure I found a way to mess things up even things as easily as this lol. I downloaded the file from realteks website, cd to the directory and ran “Sudo make && Sudo make install” I also followed the read me instructions.

Have you looked into this page?
https://wiki.odroid.com/odroid-h2/hardw ... r8125_dkms

domih · Post by **domih** » Sun Sep 06, 2020 11:43 am

johnnyde94 wrote: ↑
Thu Sep 03, 2020 8:42 am
Any news on making the driver package, as I’m sure I found a way to mess things up even things as easily as this lol. I downloaded the file from realteks website, cd to the directory and ran “Sudo make && Sudo make install” I also followed the read me instructions.

If you are talking about the r8152-2.13.0 driver for a USB 2.5 GbE adapter and the ReadMe.txt file it contains: do NOT follow what it says. Hardware manufacturers are the worse offenders in terms of writing documentation

What I wrote in a previous post works OK:

tar -xf r8152.53.56-2.13.0.tar.bz2
cd r8152-2.13.0
make
sudo make install
# Check whether or not the r8152 module is loaded
lsmod | grep r8
# If OK, update initramfs
sudo update-initramfs -u
# Optional: reboot to check whether or not the r8152 module is loaded
sudo reboot
lsmod | grep r8

The easiest way to configure the NIC is then to use the desktop Network Connections or issue a sudo nmtui in a console.

If you are talking about the driver to install for getting the H2+ onboard PCIe 2 x NIC working, follow: https://wiki.odroid.com/odroid-h2/appli ... _on_h2plus

domih · Post by **domih** » Thu Sep 10, 2020 3:41 pm

Using the same methodology described there viewtopic.php?p=305296#p305296, here are the Samba test results for an overclocked N2+ using a SSD and HDD as shared volumes. The tests are made using the onboard 1 GbE NIC and using an RTL8156B-based USB 3.0 Type A 2.5GbE adapter described there viewtopic.php?p=302427#p302427 for the 2.5 GbE tests.

The SSD is a WD 3D Blue 1 TB. The HDD is a Seagate IronWolf 8 TB. Both are initially empty. Both are connected via USB 3 and not using UAS:

domih@n2pa:~$ lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
|__ Port 2: Dev 5, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
|__ Port 3: Dev 4, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
../..

The N2+ is overclocked as described there: https://wiki.odroid.com/odroid-n2/hardware/overclocking

domih@n2pa:~$ lscpu
Architecture: aarch64
Byte Order: Little Endian
CPU(s): 6
On-line CPU(s) list: 0-5
Thread(s) per core: 1
Core(s) per socket: 3
Socket(s): 2
Vendor ID: ARM
Model: 4
Model name: Cortex-A53
Stepping: r0p4
CPU max MHz: 2400.0000
CPU min MHz: 500.0000
BogoMIPS: 48.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32

iperf3

: n2+ iperf3.png (18.86 KiB) Viewed 6342 times

Samba (SSD)

: n2+ samba ssd.png (87.6 KiB) Viewed 6342 times

Samba (HDD)

: n2+ samba hdd.png (87.04 KiB) Viewed 6342 times

CONCLUSIONS
1. The Odroid N2+ performs 1 GbE networking with flying colors.
2. It is slightly under-powered for 2.5 GbE by not being able to sustain 2.35 GbE in one iperf3 direction but not by much. Depending on the test using a USB 3.0 Type A 2.5GbE adapter enables you to accelerate a Samba file copy (upload) from 1.22 to 1.62 times faster which is not negligible.
3. Given that an RTL8156B-based USB 3.0 Type A 2.5GbE adapter costs about $20 on eBay, this is a cheap way to integrate an Odroid N2+ into a 2.5 GbE local net and to go faster than 1 GbE.

Continued in viewtopic.php?p=306055#p306055

johnnyde94 · Post by **johnnyde94** » Sat Sep 12, 2020 10:48 am

domih wrote: ↑
Sun Sep 06, 2020 11:43 am

johnnyde94 wrote: ↑
Thu Sep 03, 2020 8:42 am
Any news on making the driver package, as I’m sure I found a way to mess things up even things as easily as this lol. I downloaded the file from realteks website, cd to the directory and ran “Sudo make && Sudo make install” I also followed the read me instructions.
If you are talking about the r8152-2.13.0 driver for a USB 2.5 GbE adapter and the ReadMe.txt file it contains: do NOT follow what it says. Hardware manufacturers are the worse offenders in terms of writing documentation

What I wrote in a previous post works OK:

tar -xf r8152.53.56-2.13.0.tar.bz2
cd r8152-2.13.0
make
sudo make install
# Check whether or not the r8152 module is loaded
lsmod | grep r8
# If OK, update initramfs
sudo update-initramfs -u
# Optional: reboot to check whether or not the r8152 module is loaded
sudo reboot
lsmod | grep r8

The easiest way to configure the NIC is then to use the desktop Network Connections or issue a sudo nmtui in a console.

If you are talking about the driver to install for getting the H2+ onboard PCIe 2 x NIC working, follow: https://wiki.odroid.com/odroid-h2/appli ... _on_h2plus

Just switched my dosto arbian on the xu4 and no luck so far

make
make -C /lib/modules/4.14.195-odroidxu4/build M=/home/odroid/Downloads/r8152-2.13.0 modules
make[1]: Entering directory '/usr/src/linux-headers-4.14.195-odroidxu4'
rm: cannot remove '/home/odroid/Downloads/r8152-2.13.0/.tmp_versions/r8152.mod': Permission denied
make[1]: *** [Makefile

prepare] Error 1
make[1]: Leaving directory '/usr/src/linux-headers-4.14.195-odroidxu4'
make: *** [Makefile:24: modules] Error 2
odroid@odroidxu4:Downloads/r8152-2.13.0 $ sudo make install
make -C /lib/modules/4.14.195-odroidxu4/build M=/home/odroid/Downloads/r8152-2.13.0 INSTALL_MOD_DIR=kernel/drivers/net/usb modules_install
make[1]: Entering directory '/usr/src/linux-headers-4.14.195-odroidxu4'
INSTALL /home/odroid/Downloads/r8152-2.13.0/r8152.ko
DEPMOD 4.14.195-odroidxu4
make[1]: Leaving directory '/usr/src/linux-headers-4.14.195-odroidxu4'
modprobe r8152
odroid@odroidxu4:Downloads/r8152-2.13.0 $ lsmod | grep r8
nothing showing on that command

johnnyde94 · Post by **johnnyde94** » Sat Sep 12, 2020 10:57 am

The NIC doesnt want to work at all

domih · Post by **domih** » Sat Sep 12, 2020 3:20 pm

johnnyde94 wrote: ↑
Sat Sep 12, 2020 10:57 am
The NIC doesnt want to work at all

I have a spare XU4, I'll give it a try.

---

Otherwise you got a "rm: cannot remove '/home/odroid/Downloads/r8152-2.13.0/.tmp_versions/r8152.mod': Permission denied" because the 1st time you issued a "sudo make" instead of make. Retry with first deleting the whole folder:

cd /home/odroid/Downloads
sudo rm -fr r8152-2.13.0
tar -xf r8152.53.56-2.13.0.tar.bz2
cd r8152-2.13.0
make
sudo make install
lsmod | grep r8

If this works, and if the module does not load at reboot, add "r8152" in modules.conf: sudo vi /etc/modules-load.d/modules.conf or sudo vi /etc/modules.conf. Not sure which one it is on the Ubuntu running on your XU4.

domih · Post by **domih** » Sat Sep 12, 2020 3:21 pm

johnnyde94 wrote: ↑
Sat Sep 12, 2020 10:57 am
The NIC doesnt want to work at all

What does "lsusb -t" return?

johnnyde94 · Post by **johnnyde94** » Sat Sep 12, 2020 10:44 pm

domih wrote: ↑
Sat Sep 12, 2020 3:21 pm

johnnyde94 wrote: ↑
Sat Sep 12, 2020 10:57 am
The NIC doesnt want to work at all
What does "lsusb -t" return?

lsusb -t
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
|__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/: Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 5000M
|__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 4: Dev 7, If 0, Class=Vendor Specific Class, Driver=, 5000M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 480M
|__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 3: Dev 9, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 9, If 2, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 9, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 1: Dev 5, If 2, Class=Vendor Specific Class, Driver=btusb, 12M
|__ Port 1: Dev 5, If 0, Class=Vendor Specific Class, Driver=btusb, 12M
|__ Port 1: Dev 5, If 3, Class=Application Specific Interface, Driver=, 12M
|__ Port 1: Dev 5, If 1, Class=Vendor Specific Class, Driver=btusb, 12M
|__ Port 2: Dev 8, If 2, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 2: Dev 8, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 2: Dev 8, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 2: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M

Odd so the driver is there even though after following you directions the lsmod command is still blank. Im going to try the edit you recommended and see. Im on the focal port btw

johnnyde94 · Post by **johnnyde94** » Sat Sep 12, 2020 10:58 pm

no luck even after adding what you said to vi /etc/modules-load.d/modules.conf, that was the correct one. what is odd is on the same board just with "pure" ubuntu focal the adapter was at least working. My exact config is the xu4 with the same white 2.5 r8152b chip using Armbian Focal desktop stock kernel 4.14.y. I may try it on armbian Armbian bionic 5.4.y and or change to 5.4.y now to test if the kernal or the distro is the main issue and foward this to Armbian.

johnnyde94 · Post by **johnnyde94** » Sun Sep 13, 2020 3:25 am

no luck on new kernal ether

domih · Post by **domih** » Sun Sep 13, 2020 4:30 am

johnnyde94 wrote: ↑
Sat Sep 12, 2020 10:58 pm
no luck even after adding what you said to vi /etc/modules-load.d/modules.conf, that was the correct one. what is odd is on the same board just with "pure" ubuntu focal the adapter was at least working. My exact config is the xu4 with the same white 2.5 r8152b chip using Armbian Focal desktop stock kernel 4.14.y. I may try it on armbian Armbian bionic 5.4.y and or change to 5.4.y now to test if the kernal or the distro is the main issue and foward this to Armbian.

Yes, I'm seeing the same thing with kernel 4.14.x. One more thing I did on the C4 was:

cd Downloads/r8152-2.13.0/
sudo cp 50-usb-realtek-net.rules /etc/udev/rules.d/
sudo reboot

to tell the kernel to use the Realtek driver (module) instead of the built-in CDC/NCM driver (tip from @odroid), no luck: eth0 itself an RTL8153 gets handled by r8152 but the USB adapter does not show up. This is vexing.

According to my notes (another tip from @odroid):

# Make sure you have the latest kernel
sudo apt update
sudo apt full-upgrade -y

# Follow instructions https://wiki.odroid.com/odroid-xu4/soft ... tive-build to rebuild the kernel
# After downloading the sources and BEFORE building, replace the kernel source's r8152.c driver with the latest Realtek one (from the r8152-2.13.0)

I have not tried it so I'm punting and leave to @odroid to detail the magical procedure

domih · Post by **domih** » Sun Sep 13, 2020 1:46 pm

Follow-up of viewtopic.php?p=305819#p305819

Samba SSD & HDD (this time it's about downloading files, meaning copying files from the NAS, from Samba running on the N+2)

: copy-download-from-nas.png (185.15 KiB) Viewed 6215 times

CONCLUSION
1. The acceleration from 1 GbE to 2.5 GbE is much more pronounced

SSD: 2.352, 2.359,1.374 and 2.042 times faster.
HDD: 2.328, 2.373, 1.374 and 2.034 times faster.

More than twice faster means that the RTL8156B-based USB 3.0 Type A 2.5GbE adapter at $20 on eBay investment is a good investment.

2. Not much difference between SSD and HDD in comparison to the difference between 1 GbE and 2.5 GbE.

(*) The N2+ having "only" 4 GB of memory I wanted to know if zram would have an influence on the Samba tests. Mmm... it does not. I consistently saw a few tenths of a second less but it is within the margin of errors, could be something else, so it would be wishful thinking to say it has an influence. However it does not hurt either and will help elsewhere so I let it run. Anyway as an off-topic note, here is the procedure for installing zram.

Before zram is installed, configured and running:

Code: Select all

domih@n2pa:~$ free -mth
              total        used        free      shared  buff/cache   available
Mem:           3.6G        192M        100M        5.8M        3.3G        3.4G
Swap:            0B          0B          0B
Total:         3.6G        192M        100M

Installation and configuration:

Code: Select all

sudo apt install zram-config
sudo vi /usr/bin/init-zram-swapping
.../...
# Comment out this line
# NRDEVICES=$(grep -c ^processor /proc/cpuinfo | sed 's/^0$/1/') 
# Add this line
NRDEVICES=4 
.../...
# initialize the devices
for i in $(seq ${NRDEVICES}); do
  DEVNUMBER=$((i - 1))
  # Add this line
  echo lz4 > /sys/block/zram${DEVNUMBER}/comp_algorithm    
  echo $mem > /sys/block/zram${DEVNUMBER}/disksize
  mkswap /dev/zram${DEVNUMBER}
  swapon -p 5 /dev/zram${DEVNUMBER}
done
.../...
sudo reboot

zram is now running:

Code: Select all

domih@n2pa:~$ free -mht
              total        used        free      shared  buff/cache   available
Mem:           3.6G        185M        3.2G        5.5M        220M        3.4G
Swap:          1.8G          0B        1.8G
Total:         5.4G        185M        5.0G

References
https://www.cnx-software.com/2018/05/14 ... able-zram/
https://askubuntu.com/questions/1044976 ... -algorithm

After running the Samba tests, only 32K landed in the zram swap:

Code: Select all

domih@n2pa:~$ zramctl
NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram3 lz4         464.1M  12K  171B    8K       6 [SWAP]
/dev/zram2 lz4         464.1M  12K  344B   12K       6 [SWAP]
/dev/zram1 lz4         464.1M  12K  171B    8K       6 [SWAP]
/dev/zram0 lz4         464.1M  12K  171B    8K       6 [SWAP]

domih@n2pa:~$ cat /proc/swaps
Filename				Type		Size	Used	Priority
/dev/zram0                              partition	475188	8	5
/dev/zram1                              partition	475188	8	5
/dev/zram2                              partition	475188	8	5
/dev/zram3                              partition	475188	8	5

domih@n2pa:~$ free -mth
              total        used        free      shared  buff/cache   available
Mem:           3.6G        194M        331M        5.8M        3.1G        3.4G
Swap:          1.8G         32K        1.8G
Total:         5.4G        194M        2.1G

OVERALL CONCLUSION
Does it make sense to use 2.5 GbE with an Odroid N2+?
Yes it does.
Samba writing files to NAS is from 1.22 to 1.62 times faster, depending on usage.
Samba reading files from NAS is from 1.374 to 2.373 times faster, depending on usage.

igorpec · Post by **igorpec** » Sun Sep 13, 2020 2:57 pm

You can skip all the troubles - kernel recompilation and zram install. In Armbian - with modern kernel - all you need to do is to plug the device. Nothing else - all other settings are anyway optimal OOB.

Sadly I only have model without b which doesn't work very well.

Refference:
https://github.com/armbian/build/blob/m ... #L184-L196

my 2c

igorpec · Post by **igorpec** » Sun Sep 13, 2020 3:36 pm

I also have a 5G USB3 dongle

and that one performs better, but the price of just a little below 100 EUR is not very friendly

Tested on Odroid N2+

Code: Select all

[  5]   0.00-1.00   sec   310 MBytes  2.60 Gbits/sec    5    542 KBytes       
[  5]   1.00-2.00   sec   313 MBytes  2.63 Gbits/sec    0    803 KBytes       
[  5]   2.00-3.00   sec   312 MBytes  2.62 Gbits/sec    0    853 KBytes       
[  5]   3.00-4.00   sec   312 MBytes  2.62 Gbits/sec    0    870 KBytes       
[  5]   4.00-5.00   sec   314 MBytes  2.63 Gbits/sec    0    872 KBytes       
[  5]   5.00-6.00   sec   312 MBytes  2.62 Gbits/sec    0    875 KBytes       
[  5]   6.00-7.00   sec   312 MBytes  2.62 Gbits/sec    0    877 KBytes

domih · Post by **domih** » Mon Sep 14, 2020 1:07 pm

igorpec wrote: ↑
Sun Sep 13, 2020 2:57 pm
Sadly I only have model without b which doesn't work very well.

Yep, the non-B can go to the electronics recycle center.

igorpec wrote: ↑
Sun Sep 13, 2020 2:57 pm
You can skip all the troubles - kernel recompilation and zram install. In Armbian - with modern kernel - all you need to do is to plug the device. Nothing else - all other settings are anyway optimal OOB.

Refference:
https://github.com/armbian/build/blob/m ... #L184-L196

my 2c

Thanks! I have the Odroid Ubuntu working OK on N2/N2+ and C4. The problem seems to be on the XU4 only. Is the Armbian build also solving this particular issue on the XU4? TIA!

domih · Post by **domih** » Mon Sep 14, 2020 1:17 pm

igorpec wrote: ↑

Sun Sep 13, 2020 3:36 pm

I also have a 5G USB3 dongle

and that one performs better, but the price of just a little below 100 EUR is not very friendly

Tested on Odroid N2+

Code: Select all

[  5]   0.00-1.00   sec   310 MBytes  2.60 Gbits/sec    5    542 KBytes       
[  5]   1.00-2.00   sec   313 MBytes  2.63 Gbits/sec    0    803 KBytes       
[  5]   2.00-3.00   sec   312 MBytes  2.62 Gbits/sec    0    853 KBytes       
[  5]   3.00-4.00   sec   312 MBytes  2.62 Gbits/sec    0    870 KBytes       
[  5]   4.00-5.00   sec   314 MBytes  2.63 Gbits/sec    0    872 KBytes       
[  5]   5.00-6.00   sec   312 MBytes  2.62 Gbits/sec    0    875 KBytes       
[  5]   6.00-7.00   sec   312 MBytes  2.62 Gbits/sec    0    877 KBytes

Which iperf3 direction? Forward? Reverse? TIA!

Yes, these prices are ridiculous. Same thing on eBay: https://www.ebay.com/sch/i.html?_from=R ... =0&_sop=15

igorpec · Post by **igorpec** » Mon Sep 14, 2020 7:49 pm

domih wrote: ↑
Mon Sep 14, 2020 1:07 pm
Is the Armbian build also solving this particular issue on the XU4?

No, for XU4 its disabled (even I got it working) since problems were reported with on-board network which uses the same driver. That's represent some poking within driver - perhaps someday.

domih wrote: ↑
Mon Sep 14, 2020 1:07 pm
Yep, the non-B can go to the electronics recycle center.

Strange they work perfectly in Windows 10. That is puzzling me.

Which iperf3 direction? Forward? Reverse?

All directions about the same.

johnnyde94 · Post by **johnnyde94** » Tue Sep 15, 2020 9:33 am

Any Ideas

domih · Post by **domih** » Sun Sep 20, 2020 3:15 pm

Using the same methodology described there viewtopic.php?f=172&t=38711&p=305296#p305296, here are the Samba test results for an Odroid C4 using a SSD and HDD as shared volumes.

The tests are made using the onboard 1 GbE NIC and using an RTL8156B-based USB 3.0 Type A 2.5GbE adapter described there viewtopic.php?p=302427#p302427 for the 2.5 GbE tests.

The SSD is a WD 3D Blue 1 TB. The HDD is a Seagate IronWolf 8 TB. Both are initially empty. Both are connected via USB 3 and not using UAS.

The overclocked N2+ tests are there:
viewtopic.php?p=306055#p306055
viewtopic.php?p=305819#p305819

iperf3

: c4-iperf3-1-vs-2.5-gbe.png (19.31 KiB) Viewed 5963 times

Samba Upload (writing on NAS)

: c4-samba-upload.png (177.32 KiB) Viewed 5963 times

Samba Download (reading from NAS)

: c4-samba-download.png (176.91 KiB) Viewed 5963 times

The acceleration from 1 GbE to 2.5 GbE is consistent and reaches more than twice faster.

Samba writing SSD: 1.393, 1.335, 1.142 and 1.870 faster
Samba writing HDD: 1.356, 1.341, 1.311 and 2.043 faster

Samba reading SSD: 2.328, 2.363, 1.434 and 2.010 faster
Samba reading HDD: 2.323, 2.362, 1.277 and 2.014 faster

Not much difference between SSD and HDD regarding to the difference between 1 GbE and 2.5 GbE.

OVERALL CONCLUSION
Does it make sense to use 2.5 GbE with an Odroid C4?
Yes it does.
The RTL8156B-based USB 3.0 Type A 2.5GbE adapter at $20 on eBay investment is NOT a waste of money at all.

domih · Post by **domih** » Sun Sep 20, 2020 4:51 pm

Additional thoughts

Both the Odroid N2+ and the Odroid C4 are good solutions to replace your XU4-based home and NAS server(s) if your objective is to overhaul your network to 2.5 GbE without spending $$$ and consuming the energy of X86-based servers.

The C4 is not really far from the N2+ in terms of networking. The N2+ is really close to be able to support full speed 2.5 GbE in both directions and it does with its CPU overclocked at 2.4 GHz.

Beyond regular SBC usage, both solutions are seductive for small home servers.

The Odroid N2+ Amlogic S922X and the Odroid C4 Amlogic S905X3 both have a 32-bit DRAM memory interface with dual ranks and a max 4GB total address space. Ref: https://republicaradio.com/s922x-datasheet/ and https://www.cnx-software.com/2019/04/12 ... k-diagram/

The Odroid H2/H2+ takes over for faster network speed (100% 2.5 GbE, as well as 5 and 10G via PCIe) as well as beefier home server(s) with up to 32 GB of memory.

One can only wonder what the future will bring with ARM-based CPUs at 3+ GHz and memory interface for 4+ GB.

2.5 GbE will be sooner than later the new 1 gigabit at home (only took 20+ years to kill the latter...) and we may even dream of 5 GbE on cheap SBCs in a few years while not sucking dry the wallet and blowing up the energy bill. The road might be circuitous though because DIY users (like us) are just a drop in the sea of TV appliances and other devices these CPU are targeted at...