Getting UAS working on HK 4.9

Test and fix the Kernel 4.9 features

Moderators: mdrjr, odroid

Getting UAS working on HK 4.9

Unread postby DarkBahamut » Thu Feb 23, 2017 3:08 am

I had previously mentioned in a thread that I wasn't able to get UAS working on the HK 4.9. It was pointed out that it worked on some other 3rd party 4.9 kernel (which it does) but I was a bit stumped at first as to why there was a difference. As we are getting closer to an official full 4.9 release I decided to revisit this and I believe I've found the cause and a possible fix. As i'm no expert on these drivers I would appreciate anyone taking a look to make sure it won't cause any potential side effects or weird behaviour.

The issue (as far as I can tell):

I came across a message from Samsung that stated that the Exynos 542X xhci host was 'quirky'. In the sense that it comes up in USB2 mode at first before the host driver switches it into USB3 mode. Samsung has some patches for this that but not all of it is used here, but that's fine as there is other code in mainline that sorts that out. The issue is that piece of code comes after the check for streams support (which are required for UAS). The result being that the xhci host controller streams support is checked while it's still in USB2 mode, and thus comes back as not supported. The fix is to move the code for the usb phy to before the streams support check occurs, to make sure it's in SS before that happens and thus reports it support correctly. This fixes the issue on my test build.

Before patch:

Code: Select all
[    4.252055] usb 4-1.1: Product: ASM1053E
[    4.255792] usb 4-1.1: Manufacturer: asmedia
[    4.260129] usb 4-1.1: SerialNumber: 123456789012
[    4.284074] usb 4-1.1: USB controller xhci-hcd.2.auto does not support streams, which are required by the UAS driver.
[    4.293583] usb 4-1.1: Please try an other USB controller if you wish to use UAS.
[    4.301530] usb 4-1.1: USB controller xhci-hcd.2.auto does not support streams, which are required by the UAS driver.
[    4.311553] usb 4-1.1: Please try an other USB controller if you wish to use UAS.
[    4.318968] usb-storage 4-1.1:1.0: USB Mass Storage device detected
[    4.326852] usb-storage 4-1.1:1.0: Quirks match for vid 174c pid 55aa: 400000
[    4.333152] scsi host0: usb-storage 4-1.1:1.0

darkbahamut@odroid:~$ lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M


After patch:

Code: Select all
[    4.416374] usb 4-1.1: Product: ASM1053E
[    4.420215] usb 4-1.1: Manufacturer: asmedia
[    4.424465] usb 4-1.1: SerialNumber: 123456789012
[    4.447567] scsi host0: uas

darkbahamut@odroid:~$ lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M


USB3 SSD read Performance before: 137MB/s

Code: Select all
darkbahamut@odroid:/media/darkbahamut/895e1fce-7ac1-4826-813e-4e02e546e3c1$ dd if=test of=/dev/null iflag=direct bs=8M
256+0 records in
256+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 15.6477 s, 137 MB/s


USB3 SSD read performance after: 307MB/s

Code: Select all
darkbahamut@odroid:/media/darkbahamut/895e1fce-7ac1-4826-813e-4e02e546e3c1$ dd if=test of=/dev/null iflag=direct bs=8M
256+0 records in
256+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.98684 s, 307 MB/s



Patch:

Code: Select all
From 96d7ebf3652892585f62e0090a73dd5901426aee Mon Sep 17 00:00:00 2001
From: DarkBahamut <darkbahamut@gmail.com>
Date: Wed, 22 Feb 2017 18:06:32 +0000
Subject: [PATCH] usb: host: xhci-plat: Bring USB phy up earlier to ensure
 streams supported

---
 drivers/usb/host/xhci-plat.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index df5d7d8..e51560d 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -210,6 +210,22 @@ static int xhci_plat_probe(struct platform_device *pdev)
          *priv = *priv_match;
    }
 
+   hcd->usb_phy = devm_usb_get_phy_by_phandle(&pdev->dev, "usb-phy", 0);
+   if (IS_ERR(hcd->usb_phy)) {
+      ret = PTR_ERR(hcd->usb_phy);
+      if (ret == -EPROBE_DEFER)
+         goto put_usb3_hcd;
+      hcd->usb_phy = NULL;
+   } else {
+      ret = usb_phy_init(hcd->usb_phy);
+      if (ret)
+         goto put_usb3_hcd;
+   }
+
+   ret = usb_add_hcd(hcd, irq, IRQF_SHARED);
+   if (ret)
+      goto disable_usb_phy;
+
    device_wakeup_enable(hcd->self.controller);
 
    xhci->clk = clk;
@@ -227,22 +243,6 @@ static int xhci_plat_probe(struct platform_device *pdev)
    if (HCC_MAX_PSA(xhci->hcc_params) >= 4)
       xhci->shared_hcd->can_do_streams = 1;
 
-   hcd->usb_phy = devm_usb_get_phy_by_phandle(&pdev->dev, "usb-phy", 0);
-   if (IS_ERR(hcd->usb_phy)) {
-      ret = PTR_ERR(hcd->usb_phy);
-      if (ret == -EPROBE_DEFER)
-         goto put_usb3_hcd;
-      hcd->usb_phy = NULL;
-   } else {
-      ret = usb_phy_init(hcd->usb_phy);
-      if (ret)
-         goto put_usb3_hcd;
-   }
-
-   ret = usb_add_hcd(hcd, irq, IRQF_SHARED);
-   if (ret)
-      goto disable_usb_phy;
-
    /* Get possile USB 3.0 type PHY (PIPE3) available with xhci */
    xhci->shared_hcd->phy = devm_phy_get(pdev->dev.parent, "usb3-phy");
    if (IS_ERR(xhci->shared_hcd->phy)) {
--
2.7.4

DarkBahamut
 
Posts: 241
Joined: Tue Jan 19, 2016 10:19 am
languages_spoken: english
ODROIDs: XU4

Re: Getting UAS working on HK 4.9

Unread postby mdrjr » Thu Feb 23, 2017 10:51 am

Ohh~~~

Amazing :) Just applied to our 4.9 kernel.. Thank you DarkBahamut!
mdrjr
Site Admin
 
Posts: 11627
Joined: Fri Feb 22, 2013 11:34 pm
Location: Brazil
languages_spoken: english, portuguese
ODROIDs: -

Re: Getting UAS working on HK 4.9

Unread postby odroid » Thu Feb 23, 2017 7:41 pm

After dist-upgrade and reboot, you will have a new kernel 4.9.11-10.
It contains DarkBahamut's patch as well as the USB_UAS driver.

We connected a UAS capable external USB 3.5inch 5TB HDD, the reading performance is boosted to 220MB/sec from 150MB/sec. :o
But our CloudShell USB-SATA bridge controller doesn't support the UAS mode unfortunately. :(
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby rooted » Fri Feb 24, 2017 5:29 am

odroid wrote:After dist-upgrade and reboot, you will have a new kernel 4.9.11-10.
It contains DarkBahamut's patch as well as the USB_UAS driver.

We connected a UAS capable external USB 3.5inch 5TB HDD, the reading performance is boosted to 220MB/sec from 150MB/sec. :o
But our CloudShell USB-SATA bridge controller doesn't support the UAS mode unfortunately. :(


Ah :)

Ugh :(

Good work finding and fixing this DarkBahamut.
User avatar
rooted
 
Posts: 3653
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)

Re: Getting UAS working on HK 4.9

Unread postby odroid » Tue Mar 07, 2017 7:02 pm

@DarkBahamut

We are making a two bay DIY NAS kit which can include two 3.5inch HDDs.
Our new prototype PCB has the JMS561 bridge controller which supports Hardware RADI0, RAID1, SPAN and JBOD modes as well as UAS.
We attached two 6TB WD RED NAS HDDs(EXT4 format) and got below "DD" test result. The UAS mode was enabled well on Kernel 4.9.
RAID0 : Write 145 MB/s , Read 183MB/s
RAID1 : Write 165 MB/s , Read 166MB/s
JBOD : Write 172 MB/s , Read 173MB/s
SPAN : Write 110 MB/s, Read 156MB/s

It looks okay to build a two-bay NAS platform nicely since Gbit Ethernet limits the speed at 110MB/sec approx.
But the actual Samba transfer speed was not good.

On the RAID 1 mode, the performance of reading (XU4 -> Win10 PC) was 100MB/sec.
But the writing speed was only 30MB/sec. :(

Did you assign the samba services on the A15 big core?
Can you share your SMB configuration file?
Any other things we need to consider for better writing performance?
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby memeka » Tue Mar 07, 2017 7:37 pm

@odroid - there was a blog post of someone who uses XU4 with openmediavault which has some performance tips for nas.
Btw, how is the power supply? Can you use just 1 cable to power the hdd and the Xu, or are there different power cables?
User avatar
memeka
 
Posts: 3561
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: Getting UAS working on HK 4.9

Unread postby rooted » Tue Mar 07, 2017 9:29 pm

I have a two bay SATA dock being delivered today, it's UAS capable. It's JMicron based but I don't know which chipset.

I will test it and post my results later.
User avatar
rooted
 
Posts: 3653
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)

Re: Getting UAS working on HK 4.9

Unread postby odroid » Tue Mar 07, 2017 10:56 pm

memeka wrote:@odroid - there was a blog post of someone who uses XU4 with openmediavault which has some performance tips for nas.
Btw, how is the power supply? Can you use just 1 cable to power the hdd and the Xu, or are there different power cables?

I found some posts but we need further information.
The NAS kit works with a single PSU 15V/4A for the HDDs, Bridge-chip, 2.8inch TFT-LCD and XU4 thanks to a nice power distribution circuitry on our NAS board.


rooted wrote:I have a two bay SATA dock being delivered today, it's UAS capable. It's JMicron based but I don't know which chipset.
I will test it and post my results later.

Great!
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby DarkBahamut » Thu Mar 09, 2017 10:49 am

I haven't done lots of testing on 4.9 yet, but it seem most of the same tweaks should apply for NAS/Samba. I'm finding that with UAS peak speed it a little higher than on 3.10, but the speed is a bit variable so far. On 3.10 I get an almost perfect ~100MB/s with no movement in the speed, but on 4.9 it hit 110MB/s, then will suddenly drop to ~70-80 before going back up. I've got a few theories to investigate on this, I can see things like the RPS is behaving different (it looks broken) and HMP can be suspect at times.

I only have a single drive enclosure with a ASMedia 1053 chipset (UAS enabled) with a 4TB WD Green drive, so not the fastest. Just about keeps up with the network though. I just did a quick test with a 40GB (38,000 file changes) backup run through the XU4 running 4.9 to see what the write speeds were like:

Image

Overall average on the big files is probably 95-100MB/s with the speed varying between 80-110MB/s. I haven't heavily optimised though and It can go faster for sure. At the minute I'm running:

Samba 4.5.5 (built from source)
Ondemand govenor - 600MHz min speed on both clusters, otherwise completely stock settings (bad :()
MTU 6975 and 9k Jumbo packets enabled on network and all clients
RPS set to 0e (I'm pretty sure this isn't working on 4.9 though)
EXT4 formatted drive.
Following samba config:

Code: Select all
write cache size = 524288
getwd cache = yes
use sendfile = yes
min receivefile size = 16384


The speeds will get even better if you:
Set the NIC interrupts to be on one A15 core (since RPS appears broken)
Lock Samba to a different A15 core than the interrupts
Make sure io_is_busy = 1 is set for the governor if using ondemand, or just use performance if power/noise is of no concern
DarkBahamut
 
Posts: 241
Joined: Tue Jan 19, 2016 10:19 am
languages_spoken: english
ODROIDs: XU4

Re: Getting UAS working on HK 4.9

Unread postby rooted » Thu Mar 09, 2017 1:37 pm

Try it without jumbo, perhaps it's struggling here.
User avatar
rooted
 
Posts: 3653
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)

Re: Getting UAS working on HK 4.9

Unread postby odroid » Sat Mar 11, 2017 11:01 am

@DarkBahamut
We could get around 50~60MB/sec of RAID1 write performance with your help. Thanks.
We will do further optimization. :)

It seems to be worth to try disabling all LITTLE cores.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby mad_ady » Sun Mar 12, 2017 9:07 pm

@odroid: try moving usb (and network) interrupts on different big cores. I'm doing that in my NAS tutorial published in february.
User avatar
mad_ady
 
Posts: 2395
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Getting UAS working on HK 4.9

Unread postby DarkBahamut » Sun Mar 12, 2017 10:20 pm

On 4.9 the need to move the interrupts is definitely greater. On 3.10 I could get the same performance (within 3%) on the A7's as I could with the A15's with RPS, but on 4.9 RPS doesn't appear to work. I'll regularly see the A7 core hit 100.0% busy with no load balancing happening at all, even though the RPS file is set correctly. There is then a noticeable uplift in transfer speed stability once you set the interrupt to a big core.
DarkBahamut
 
Posts: 241
Joined: Tue Jan 19, 2016 10:19 am
languages_spoken: english
ODROIDs: XU4

Re: Getting UAS working on HK 4.9

Unread postby ttyoo » Mon Mar 13, 2017 6:33 pm

odroid wrote:After dist-upgrade and reboot, you will have a new kernel 4.9.11-10.
It contains DarkBahamut's patch as well as the USB_UAS driver.

We connected a UAS capable external USB 3.5inch 5TB HDD, the reading performance is boosted to 220MB/sec from 150MB/sec. :o
But our CloudShell USB-SATA bridge controller doesn't support the UAS mode unfortunately. :(



Hi, I'm new here. I wonder where can I get the Jessie image including the odroid maintained mainline kernel.
I need this UAS and USB3.0 work for my NAS.
from the wiki, I only found the Jessie image by meveric.
thanks guys
ttyoo
 
Posts: 2
Joined: Mon Mar 13, 2017 5:56 pm
languages_spoken: english
ODROIDs: xu4

Re: Getting UAS working on HK 4.9

Unread postby crashoverride » Mon Mar 13, 2017 6:37 pm

ttyoo wrote:I need this UAS and USB3.0 work for my NAS.

Keep in mind this is an experimental[1] kernel. It may work or it may corrupt your data. Ensure you have adequate backups and conduct testing before depending on it.

[edit]
[1] Perhaps "experimental" is the wrong word. "Pre-release" may be a better term to describe it.
crashoverride
 
Posts: 3060
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Getting UAS working on HK 4.9

Unread postby ttyoo » Mon Mar 13, 2017 11:08 pm

crashoverride wrote:
ttyoo wrote:I need this UAS and USB3.0 work for my NAS.

Keep in mind this is an experimental[1] kernel. It may work or it may corrupt your data. Ensure you have adequate backups and conduct testing before depending on it.

[edit]
[1] Perhaps "experimental" is the wrong word. "Pre-release" may be a better term to describe it.


thanks for your update.
i think my NAS is also experimental :)

do you know what debian image should i use?
ttyoo
 
Posts: 2
Joined: Mon Mar 13, 2017 5:56 pm
languages_spoken: english
ODROIDs: xu4

Re: Getting UAS working on HK 4.9

Unread postby crashoverride » Mon Mar 13, 2017 11:31 pm

ttyoo wrote:do you know what debian image should i use?

I do not. I only use the official Ubuntu images provided by HardKernel.
crashoverride
 
Posts: 3060
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Thu Mar 16, 2017 2:51 pm

After switching to Windows 10 from Ubuntu 16 on our host PC, the Samba performance is quite different. :o

Downloading from PC to XU4 Dual HDD RAID-1 mode.
PC2XU4.png
(10.69 KiB) Downloaded 1628 times


Uploading from XU4 to PC. (same storage config)
XU4_to_PC.png
(10.61 KiB) Downloaded 1628 times


The NAS performance is really impressive.
We will share the hardware and software configurations two week later since we need some stability tests. ;)
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby crashoverride » Thu Mar 16, 2017 10:34 pm

odroid wrote:After switching to Windows 10 from Ubuntu 16 on our host PC, the Samba performance is quite different.

This is a very long standing issue. It continually gets reported (search for "gvfs samba slow transfer speed") and everyone claims its someone else's bug so they close it. :lol:

For this reason, I always state that speed testing should be done with a Windows client and not a Linux client.

The workaround for this is to use "smbclient" or to mount the network share with "mount" / fstab. Anything that uses Gnome Virtual File System (gvfs), like the Mate GUI, will be unreasonably slow.
https://www.samba.org/samba/docs/man/manpages-3/smbclient.1.html
https://www.samba.org/samba/docs/man/manpages-3/mount.cifs.8.html
crashoverride
 
Posts: 3060
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Fri Mar 17, 2017 3:09 am

Yes. we wasted almost two weeks to learn the famous issue :(
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby crossover » Fri Mar 17, 2017 6:58 pm

Amazing ARM NAS performance. Great job. When do you sell the new two-hdd NAS kit?
crossover
 
Posts: 105
Joined: Wed Jul 22, 2015 2:23 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, USB-IO, Tinkering kits

Re: Getting UAS working on HK 4.9

Unread postby Brian.K » Mon Mar 20, 2017 11:33 am

Can the 'reboot' command run with UAS device?
On JMicron 561, the system is hanging when the software reboot.
User avatar
Brian.K
 
Posts: 244
Joined: Tue Sep 15, 2015 7:30 pm
Location: South Korea
languages_spoken: English, Korean
ODROIDs: XU4, C1+, C2

Re: Getting UAS working on HK 4.9

Unread postby tmihai20 » Mon Mar 20, 2017 9:43 pm

Just as @ttyoo; I would also like to test the 4.9 kernel on a Debian Jessie (I updated to OMV 3, it is now based on Debian Jessie). I do not really want to use Ubuntu to be able to do this. I thought I could install OMV on Ubuntu (thank you @mad_ady for pointing it out). OMV as it is performs quite well for my taste. I do not want to spend hours to move to a new distro.

A little offtopic, which of the 2 USB 3.0 ports is the one that is not shared with the ethernet? The lower one or the top one (with the USB ports to the right, with the XU4 standing normally)? My guess is that it is the lower one, lsusb shows the external HDD connected to port 1 when I connect the HDD to it and port 2 when I connect the HDD to the upper port.
Last edited by tmihai20 on Wed Mar 22, 2017 5:41 am, edited 1 time in total.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 56
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: Getting UAS working on HK 4.9

Unread postby odroid » Tue Mar 21, 2017 12:31 am

Both USB 3.0 ports are independent from the Ethernet.
So you can use any one of them.
Refer the block diagram.
http://www.hardkernel.com/main/products ... &tab_idx=2

As far as I know, meveric has a trial Kernel package for his Debian Jessie image.
Search the forum.
Or download the kernel source and build/install it.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby tmihai20 » Tue Mar 21, 2017 3:23 am

Thanks a million. I will look for his posts.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 56
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: Getting UAS working on HK 4.9

Unread postby mad_ady » Tue Mar 21, 2017 4:18 am

Also, last I checked, OMV can't be installed on top of ubuntu yet - too many dependency conflicts
User avatar
mad_ady
 
Posts: 2395
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Getting UAS working on HK 4.9

Unread postby jobenvil » Fri Apr 07, 2017 4:41 pm

Um, strange, i though, i posted yesterday but it doesn't appear :?

I spent some time with UAS last year. I made some performance tests. Yesterday I tested the mdjr kernel 4.9.17 to compare results. If somebody is interested, can have a look here [url]http:xu4.keltike.de[/url]

It will be nice to compare SAMBA performance/parameters. What I observed is that frequency of A7 are not 1,4 GHz, like already commented here. Therefore results could be lower than should be.
jobenvil
 
Posts: 36
Joined: Tue Feb 09, 2016 7:11 pm
Location: Düsseldorf
languages_spoken: spanish, english, german
ODROIDs: odroidxu4

Re: Getting UAS working on HK 4.9

Unread postby odroid » Fri Apr 07, 2017 4:57 pm

I found your post. ;)
viewtopic.php?f=97&t=17349&p=185733#p185733

Anyway, thank you for the very detail performance test and comparison.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby jobenvil » Sun Apr 09, 2017 3:35 am

I found your post. ;)
viewtopic.php?f=97&t=17349&p=185733#p185733

ooops, this should not happen :o thanks! I see, I'm little lost.

@DarkBahamut thanks for share your samba parameters
jobenvil
 
Posts: 36
Joined: Tue Feb 09, 2016 7:11 pm
Location: Düsseldorf
languages_spoken: spanish, english, german
ODROIDs: odroidxu4

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Sun Apr 09, 2017 6:04 pm

odroid wrote:The NAS performance is really impressive.


It is given that Ethernet is also USB3 only ;)

Glad to see that UASP now gets the attention it deserves. Here you find some information how Windows Explorer is able to show that good performance numbers (and why that's irrelevant for most/many use cases): http://www.helios.de/web/EN/support/TI/157.html (I would also recommend to use their 'Helios LanTest' tool since this does not only moronically focus on sequential transfer speeds but also tests stuff like latency and so on)

And for an automatically optimized NAS distro please look here (takes some time): http://forum.openmediavault.org/index.php/Thread/17855-Building-OMV-automatically-for-a-bunch-of-different-ARM-dev-boards/
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Sun Apr 09, 2017 6:15 pm

mad_ady wrote:Also, last I checked, OMV can't be installed on top of ubuntu yet - too many dependency conflicts


Once OMV will support Debian Stretch I'll look into Xenial compatibility again. Currently Armbian's build system when used to generate a generic OMV image from scratch only supports Jessie but I hope this will change in a few months: https://github.com/igorpecovnik/lib/blob/907c8790f6f8345c163a630d9083de02d4b17991/scripts/customize-image.sh.template#L31-L126
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Sun Apr 09, 2017 9:30 pm

DarkBahamut wrote:On 4.9 the need to move the interrupts is definitely greater.


Can anyone please comment on setting IRQs the way Armbian currently does it on XU4: https://github.com/igorpecovnik/lib/blo ... fo#L80-L88

I'm currently preparing a fully automated OMV build for XU4 but wanted to ensure that IRQ affinity matches NAS demands. Thank you!
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Mon Apr 10, 2017 8:55 am

A maximum transmission unit (MTU) is the largest size packet or frame in mac layer. The MTU size is set to 1500 by default. ODROID-XU4 supports the large MTU size called jumbo frame.
Receive Packet Steering (RPS) is used to direct packets to specific CPUS for processing.
RPS helps to prevent the hardware queue of a single network interface card from becoming a bottleneck in network traffic.
Add the lines below in /etc/rc.local. They are adjusting MTU size, interrupt assignments and RPS settings.
Code: Select all
ifconfig eth0 mtu 6975
echo 5 > /proc/irq/143/smp_affinity_list
echo 6 > /proc/irq/144/smp_affinity_list
echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus


To improve samba performance, some configurations are necessary in the samba configuration file.
These configurations enable caches and adjust protocol feature for samba performance. Add the configurations below to [global] section in /etc/samba/smb.conf.
Code: Select all
write cache size = 524288
getwd cache = yes
use sendfile = yes
min receivefile size = 16384
socket options = TCP_NODELAY IPTOS_LOWDELAY


We've tested it on Kernel 4.9.20 with Ubuntu 16.04 image.

Also refer mad_ady's nice article in Magazine February issue.
http://magazine.odroid.com/wp-content/u ... pdf#page=8
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Mon Apr 10, 2017 2:42 pm

Thanks for the link to the article explaining the HMP problem and which cores build the little and which the big cluster. It would be still interesting for me to get a full '/proc/interrupts' output since with Armbian (or the OMV derivat of it) we want to try to use settings that work with either legacy or mainline kernel so numbers might change while 'search patterns' like 'usb5' still work:
Code: Select all
echo 8 >/proc/irq/$(awk -F":" "/usb5/ {print \$1}" </proc/interrupts | sed 's/\ //g')/smp_affinity


BTW: I don't believe jumbo frames are needed or even practical (it's a RTL8153 that can easily saturate Gigabit Ethernet with appropriate settings). Top performance should be possible by tweaking cgroups and IRQ affinity alone :)

BTW: What's also nice with real-world workloads (and not just sequential transfer benchmarks) is optimizing I/O scheduling classes and priority especially when we can make use of UASP:
Code: Select all
echo '* * * * * root for i in `pgrep "smbd|afpd|cnid"` ; do ionice -c1 -p $i ; done >/dev/null 2>&1' \
    >/etc/cron.d/make_samba_and_netatalk_faster
 chmod 600 /etc/cron.d/make_samba_and_netatalk_faster


And with kernel 4.4 or above it's always an option to use btrfs with 'compress=lzo' if the storage is slower than the network (unlikely with UASP now :) )
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby mad_ady » Mon Apr 10, 2017 7:12 pm

Yes, increasing the mtu only added a couple of Mbps to the transfer rates in my tests

Sent from my SM-J320F using Tapatalk
User avatar
mad_ady
 
Posts: 2395
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Getting UAS working on HK 4.9

Unread postby mattrix » Wed Apr 12, 2017 11:11 am

Anyone else getting this error

[131564.787696] xhci-hcd xhci-hcd.2.auto: ERROR Transfer event for disabled endpoint or incorrect stream ring
[131564.796001] xhci-hcd xhci-hcd.2.auto: @00000000b684e170 00000000 00000000 04000000 09038001
[131597.693837] sd 1:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN
[131597.700427] sd 1:0:0:0: [sdb] tag#1 CDB: opcode=0x28 28 00 22 c1 fd 30 00 00 20 00
[131597.708756] sd 1:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD
[131597.715756] sd 1:0:0:0: [sdb] tag#0 CDB: opcode=0x28 28 00 22 c1 fd 10 00 00 20 00
[131597.723839] scsi host1: uas_eh_bus_reset_handler start
[131597.809114] usb 4-1.1.3: reset SuperSpeed USB device number 7 using xhci-hcd

It happens when I'm streaming a movie on random occasions.

I tried the method to force usb-storage to be used instead of UAS,

/etc/modprobe.d/usb3-disk-bug.conf
options usb-storage quirks=0x0bc2:0x3321:u

but it wouldn't work.
UAS kept being used :(
mattrix
 
Posts: 88
Joined: Tue Jan 13, 2015 7:12 am
languages_spoken: english
ODROIDs: C1

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Wed Apr 12, 2017 6:20 pm

odroid wrote:Add the lines below in /etc/rc.local. They are adjusting MTU size, interrupt assignments and RPS settings.


In the meantime I ordered a XU4 to be able to test through things and ended up with a somewhat different approach compared to your or @mad_ady's suggestions: https://forum.armbian.com/index.php?/topic/3953-preview-generate-omv-images-for-sbc-with-armbian/&do=findComment&comment=29304

Please keep in mind that the performance numbers are made with a slightly outdated and not optimized Netatalk installation that could perform a little bit better. But this is stuff I'll discuss with OMV developers since potential fixes should be included upstream in OMV and not be part of 'Armbian tweaks' so any OMV user can benefit from.

TL;DR: UASP makes a real difference in real-world NAS situations :)
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby memeka » Wed Apr 12, 2017 6:31 pm

tkaiser wrote:
odroid wrote:Add the lines below in /etc/rc.local. They are adjusting MTU size, interrupt assignments and RPS settings.


In the meantime I ordered a XU4 to be able to test through things and ended up with a somewhat different approach compared to your or @mad_ady's suggestions: https://forum.armbian.com/index.php?/topic/3953-preview-generate-omv-images-for-sbc-with-armbian/&do=findComment&comment=29304

Please keep in mind that the performance numbers are made with a slightly outdated and not optimized Netatalk installation that could perform a little bit better. But this is stuff I'll discuss with OMV developers since potential fixes should be included upstream in OMV and not be part of 'Armbian tweaks' so any OMV user can benefit from.

TL;DR: UASP makes a real difference in real-world NAS situations :)


@tkaiser i am curious to see the CloudShell2 and (if you're getting one) your thoughts on it :D
User avatar
memeka
 
Posts: 3561
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Wed Apr 12, 2017 6:43 pm

memeka wrote:@tkaiser i am curious to see the CloudShell2 and (if you're getting one) your thoughts on it :D


I hope the ODROID OMV maintainer will comment on that over there: http://forum.openmediavault.org/index.php/Thread/17855-Building-OMV-automatically-for-a-bunch-of-different-ARM-dev-boards/?postID=141390#post141390

It was already somewhat stupid to buy the XU4 (I've 40 boards lying around and will use the XU4 for nothing else than looking into big.LITTLE issues from time to time) so I won't spend more money here, especially since I'm not a fan of home/SOHO RAID solutions anyway and will focus again on other ARM devices for NAS purposes (USB3 on XU4 performs not that bad but overhead and added latency can be seen in numbers)
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Thu Apr 13, 2017 5:19 pm

User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Thu Apr 13, 2017 6:20 pm

odroid wrote:The CloudShell2 is ready to ship.


Can both disks be accessed with hdparm/smartctl in every mode?
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Thu Apr 13, 2017 6:54 pm

JBOD mode should be possible.
But we need to check all modes. I will let you know tomorrow.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby odroid » Thu Apr 13, 2017 9:08 pm

Yesterday, we had a chance to use the Helious LanTest.
RAID 0 mode with two WD Red 6TB NAS HDDs(MWD60EFRX-68L0BN1) were used for this test.
Image
Segate 2TB HDDs also showed very similar result.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby rooted » Thu Apr 13, 2017 11:15 pm

I don't understand how writes are faster than reads?
User avatar
rooted
 
Posts: 3653
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)

Re: Getting UAS working on HK 4.9

Unread postby MastaG » Fri Apr 14, 2017 3:50 am

@odroid
Did you try installing the original AX88179 driver from here?
http://www.asix.com.tw/download.php?sub ... ItemID=131

If you use it instead of the one that ships with the kernel, you can set:
/sbin/ethtool -K eth1 sg on tso on

Maybe this will benefit the lan tests :)
MastaG
 
Posts: 157
Joined: Mon Aug 26, 2013 6:05 pm
languages_spoken: english

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Fri Apr 14, 2017 6:26 am

odroid wrote:Yesterday, we had a chance to use the Helious LanTest.
RAID 0 mode with two WD Red 6TB NAS HDDs(MWD60EFRX-68L0BN1) were used for this test.


Thanks for the test though those numbers are strange (a bit too high, also the orange triangles indicating slowest and fastest numbers are wrong). Seems I have to sacrifice a PC to be infected with Windows 10 to test myself. :)

It would be great if you could give the new OMV build for XU4 a try: http://kaiser-edv.de/tmp/NumpU8/ (smb.conf adoptions are missing but a few tweaks are active that might increase performance). Logon credentials are root:openmediavault (SSH) or admin:openmediavault (Web UI). In case you find the time to try it out, please test default performance with Cloudshell 2, then make the smb.conf changes, test again and provide also output of 'armbianmonitor -u' after tests have finished (debug logs)
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Getting UAS working on HK 4.9

Unread postby odroid » Fri Apr 14, 2017 4:25 pm

odroid wrote:JBOD mode should be possible.
But we need to check all modes. I will let you know tomorrow.


We could use the "smartctl" access with only JBOD/PM mode.
We will contact to Jmicron other possible way.

We need to update the TFT LCD display script for other modes if their answer is negative. :(
Image
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby odroid » Fri Apr 14, 2017 4:32 pm

tkaiser wrote:Thanks for the test though those numbers are strange (a bit too high, also the orange triangles indicating slowest and fastest numbers are wrong). Seems I have to sacrifice a PC to be infected with Windows 10 to test myself. :)

It would be great if you could give the new OMV build for XU4 a try: http://kaiser-edv.de/tmp/NumpU8/ (smb.conf adoptions are missing but a few tweaks are active that might increase performance). Logon credentials are root:openmediavault (SSH) or admin:openmediavault (Web UI). In case you find the time to try it out, please test default performance with Cloudshell 2, then make the smb.conf changes, test again and provide also output of 'armbianmonitor -u' after tests have finished (debug logs)


As far as I know, "ryecoaaron" is developing an official OMV image for XU4 board with CloudShell2 these days.
We will try the official one once it is available since we are terribly busy these days. Sorry about that,
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby odroid » Fri Apr 14, 2017 4:35 pm

MastaG wrote:@odroid
Did you try installing the original AX88179 driver from here?
http://www.asix.com.tw/download.php?sub ... ItemID=131

If you use it instead of the one that ships with the kernel, you can set:
/sbin/ethtool -K eth1 sg on tso on

Maybe this will benefit the lan tests :)


No, we are using RTL8153 chipset instead of AX88179 on the XU4.
Image
The first one is LAN9514, the second one is AX88179 and the third one is RTL8513.
User avatar
odroid
Site Admin
 
Posts: 24345
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Getting UAS working on HK 4.9

Unread postby tkaiser » Fri Apr 14, 2017 6:58 pm

odroid wrote:As far as I know, "ryecoaaron" is developing an official OMV image for XU4 board with CloudShell2 these days.
We will try the official one once it is available since we are terribly busy these days.


Well, it was about helping him since I still hope he'll rely on the 'let Armbian build OMV images for various ARM boards' attempt. This way a bunch of optimized settings will be automagically applied and it's also ensured that your kernel updates are available: https://github.com/igorpecovnik/lib/blob/master/config/sources/odroidxu4.conf#L32-L33 (Armbian's build system checks out your kernel sources, applies some patches and packages everything as .debs available through apt.armbian.com from then on)

Anyway: not so surprising that SMART doesn't work in RAID modes (same true for hdparm to set drive parameters). I hope you educate Cloudshell users to not use these modes but to use software RAID instead (if people are crazy enough to use RAID anyway)
tkaiser
 
Posts: 94
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Next

Return to Linux Kernel 4.9 Debugging Party

Who is online

Users browsing this forum: No registered users and 1 guest