USB3 and UAS -- anyone has tested it already?

[tkaiser]

As already outlined in another thread here viewtopic.php?f=93&t=15727#p113236 UAS [1] support is available with mainline kernel and this even helps with USB 2.0 setups.

In case of a board being capable of USB 3.0 it should make a huge difference regarding achievable performance. Anyone here who tried this using kernel 4.x, a fast SSD and an UASP capable USB-to-SATA bridge? In the aforementioned thread i linked to a wiki page where such chipsets are listed.

Thx, Thomas

[1] https://en.wikipedia.org/wiki/USB_Attached_SCSI

[tkaiser]

Ok, so neither users nor the manufacturer is using USB3 connected storage on the XU4? Really?

[Brian.K]

Tested on CloundShell for XU4

* Kernel version:

Code: Select all

odroid@odroid:~$ uname -a
Linux odroid 4.4.0-rc1 #4 SMP PREEMPT Wed Nov 18 17:53:26 KST 2015 armv7l armv7l armv7l GNU/Linux


* with UAS - HDD: SAMSUNG Spinpoint ST2000LM003 2TB

Code: Select all

odroid@odroid:~$ zcat /proc/config.gz | grep USB_UAS
CONFIG_USB_UAS=m
odroid@odroid:~$ lsmod
Module                  Size  Used by
cfg80211              210902  0
cpufreq_userspace       1716  0
cpufreq_ondemand        8810  0
cpufreq_powersave       1065  0
cpufreq_conservative     6754  0
joydev                  9349  0
mousedev               10616  0
evdev                  11785  0
input_leds              2852  0
uas                    13763  0
usb_storage            47236  1 uas
cdc_ether               4814  0
usbnet                 23844  1 cdc_ether
r8152                  32400  0
leds_pwm                3135  0
ipv6                  333279  42
autofs4                24109  2
odroid@odroid:~$ sudo mount /dev/sda1 /media/hdd
[  120.020691] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
odroid@odroid:~$ cd /media/hdd/
odroid@odroid:/media/hdd$ sudo dd if=/dev/zero of=test oflag=direct bs=8M count=64
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 4.61635 s, 116 MB/s
odroid@odroid:/media/hdd$ sudo dd if=test of=/dev/null iflag=direct bs=8M
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 4.54198 s, 118 MB/s

odroid@odroid:/media/hdd$ sudo hdparm -tT /dev/sda1

/dev/sda1:
 Timing cached reads:   638 MB in  2.00 seconds = 318.73 MB/sec
 Timing buffered disk reads: 342 MB in  3.01 seconds = 113.51 MB/sec


* without UAS - HDD: SAMSUNG Spinpoint ST2000LM003 2TB

Code: Select all

odroid@odroid:/media/hdd$ zcat /proc/config.gz | grep USB_UAS
# CONFIG_USB_UAS is not set
odroid@odroid:/media/hdd$ lsmod
Module                  Size  Used by
cfg80211              210902  0
cpufreq_userspace       1716  0
cpufreq_ondemand        8810  0
cpufreq_powersave       1065  0
cpufreq_conservative     6754  0
joydev                  9349  0
mousedev               10616  0
evdev                  11785  0
input_leds              2852  0
usb_storage            47236  1
cdc_ether               4814  0
usbnet                 23844  1 cdc_ether
r8152                  32400  0
leds_pwm                3135  0
ipv6                  333279  40
autofs4                24109  2 
odroid@odroid:/media/hdd$ sudo dd if=/dev/zero of=test oflag=direct bs=8M count=64
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 4.51778 s, 119 MB/s
odroid@odroid:/media/hdd$ sudo dd if=test of=/dev/null iflag=direct bs=8M
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 4.44664 s, 121 MB/s

odroid@odroid:/media/hdd$ sudo hdparm -tT /dev/sda1

/dev/sda1:
 Timing cached reads:   1480 MB in  2.00 seconds = 740.93 MB/sec
 Timing buffered disk reads: 356 MB in  3.01 seconds = 118.18 MB/sec


* with UAS - SSD: TOSHIBA SSD HDT312 128GB

Code: Select all

odroid@odroid:/media/hdd$ sudo dd if=/dev/zero of=test oflag=direct bs=8M count=64
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 3.22502 s, 166 MB/s

odroid@odroid:/media/hdd$ sudo dd if=test of=/dev/null iflag=direct bs=8M
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 3.47056 s, 155 MB/s

odroid@odroid:/media/hdd$ sudo hdparm -tT /dev/sda1

/dev/sda1:
 Timing cached reads:   640 MB in  2.00 seconds = 319.46 MB/sec
 Timing buffered disk reads: 394 MB in  3.01 seconds = 131.04 MB/sec


* without UAS - SSD: TOSHIBA SSD HDT312 128GB

Code: Select all

odroid@odroid:/media/hdd$ sudo dd if=/dev/zero of=test oflag=direct bs=8M count=64
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 3.1917 s, 168 MB/s

odroid@odroid:/media/hdd$ sudo dd if=test of=/dev/null iflag=direct bs=8M
64+0 records in
64+0 records out
536870912 bytes (537 MB) copied, 3.52293 s, 152 MB/s

odroid@odroid:/media/hdd$ sudo hdparm -tT /dev/sda1

/dev/sda1:
 Timing cached reads:   1478 MB in  2.00 seconds = 740.41 MB/sec
 Timing buffered disk reads: 408 MB in  3.01 seconds = 135.46 MB/sec


[odroid]

Ummm...
The UAS driver might not be working correctly or something wrong in the ARM platform.

[LiquidAcid]

Just because the UAS module is loaded, doesn't mean that the USB storage subsystem actually uses it for the device. There are quite some few requirements so that it can be used, and only a few USB-SATA bridges fullfill these.

[odroid]

Good point !
There is a GL3321G in our cloudshell as a USB-SATA bridge controller.
According to the GL3321G datasheet, it supports the USB Attached SCSI (UAS) and USB Attached SCSI Protocol (UASP).
So we might need to find missing items to activate it.

[tkaiser]

Ummm...
The UAS driver might not be working correctly or something wrong in the ARM platform.

I already tried to explain what's also necessary to get a clue whether UASP works or not in the other thread: viewtopic.php?f=93&t=15727#p113236

1) SSDs with just 128 GB capacity are always slow since not that much NAND chips will be accessed in parallel. If the device has always been used via USB then chances are great that it's even slower since TRIM does not work over USB. Maybe the SSD is the bottleneck (that's why you always need another fast system to compare with)

2) dd/hdparm are no benchmark tools . I would better use iozone instead (apt-get install iozone3 [1]) and then use the IOPS test. This is where UAS vs. BOT always a difference even on slow SSDs/disks

3) If the worker thread in question runs on one of the slow A7 cores then this might become the bottleneck. Did you check with htop whether one core is utilised with 100% when running iozone?

Thx

[1] In case the repositories you use do not contain iozone, I've made a deb some time ago: http://kaiser-edv.de/downloads/iozone3_ ... _armhf.deb

[tkaiser]

Just FYI: The dirt-cheap Orange Pi PC using Allwinner's H3 SoC shows a pretty slow USB 2.0 performance when running with Allwinner's boring outdated 3.4.39 kernel. Since first steps regarding mainline kernel support have been done and at least USB and UART work with kernel 4.0.0-rc4 I gave it a try again: It's the fastest USB 2.0 device I ever measured: 39 MB/s write and 41.5 MB/s read:

http://forum.armbian.com/index.php/topi ... #entry3244

And this is USB 2.0! You should really give it a try using an UASP capable USB-to-SATA bridge with the XU4. I'm really curious whether performance does increase or not.

[ard]

Tested on CloundShell for XU4

None of the cloudshells I have support UAS. For uas you need to have an usb endpoint supporting bInterfaceProtol 98 (class 8, subclass 6). The cloud shell only supports protocol 80. As storage protocol UAS is helpful for harddisks or having multiple usb3 storage devices, as the usb3 bus is freed after issueing the commands.
As monitoring UAS is essential since it passes the maintenance commands as is. The current cloudshell doesn't allow to fiddle with the smart settings of the disk since it's support for pass through other than read and write is poor.

[ard]

Good point !
There is a GL3321G in our cloudshell as a USB-SATA bridge controller.
According to the GL3321G datasheet, it supports the USB Attached SCSI (UAS) and USB Attached SCSI Protocol (UASP).
So we might need to find missing items to activate it.

Hi Odroid,

When I got the cloudshell and UAS was missing from the device descriptor, I've been looking on the GL site, and at that time the GL3310 supported UAS and the GL3321 did not. I thought because the GL3310 was a lot more complex to use in a design, this was a design trade off.
The current datasheet I see at alldatasheet says it supports UAS.

This probably means there has been a firmware upgrade to support UAS. As USB devices are very poor in firmware version showing, maybe we can do an inventory of people that have the GL3321 *and* have UAS...
I will make a thread of that.

[odroid]

I will check the feasibility of UAS on the GL3321 in next week.

Update: The GL3321 doesn't support the UAS mode.

[wolfii]

Hi all,

I read a few discussions regarding UAS support on odroid boards (in my case: XU4), but I did not find a definitive answer whether the hardware supports it or not and, if the hardware supports it, whether there will be a linux kernel that will support it also.

I would be glad about any news on this topic.

Thanks a lot

Wolfgang

[mad_ady]

@wolfii: One of my disks is seen as uas on the xu4 while running a custom 4.9 kernel. I've had no issues with it. I expect uas to work when hk releases their 4.9 kernel

[DarkBahamut]

UAS doesn't work on the current HK builds. It complains of missing features with the USB host controller.

Are the custom 4.9 kernels using patches to work around that?

[mad_ady]

I'm using the memeka branch from dmole

[DarkBahamut]

I tried a different 4.9 branch and indeed the UAS driver loads and works fine. I'll see if I can work out why it doesn't work on the HK branch.

Interestingly, HK have previously said that USB 3 is limited to ~160MB/s on the XU4 due to an architecture limit of the Exynos 5433 SoC, but this doesn't seem to be the case. Attaching an old and quite slow Crucial m4 64GB SSD to my enclosure yields read speed in excess of 300MB/s. The same drive mounted with the mass storage driver only reaches 145MB/s, so the main limit there is the driver and not the SoC.

This isn't any help for NAS applications since the network is the limit anyway, but if you need fast local storage then the UAS driver is a pretty nice deal.

Code: Select all

darkbahamut@odroid:/media/darkbahamut/895e1fce-7ac1-4826-813e-4e02e546e3c1$ dd if=test of=/dev/null iflag=direct bs=8M
256+0 records in
256+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 6.75137 s, 318 MB/s

[Pepes]

Odroid, can you tell us if DarkBahamut is right? Thanks!

[mad_ady]

@Pepes: uas support is already integraded in HK's 4.9 kernel due to a recent patch from DarkBahamut.
viewtopic.php?f=146&t=26016

[jobenvil]

I have done several time ago some UAS performance tests, if you are interested. I updated it, today, with 4.9.17@mdjr:

http://xu4.keltike.de/

BTW, thank you to you ALL regarding the incredible progress in the lasts months, this is a nice VOI.

[tkaiser]

I have done several time ago some UAS performance tests, if you are interested. I updated it, today, with 4.9.17@mdjr:

http://xu4.keltike.de/

Thank you for providing LanTest numbers (way better than just Windows Explorer sequential transfer speeds). Out of curiosity I ordered an XU4 just now, will test UAS storage in depth and then try to come up with reasonable settings for a general purpose NAS (not trying to optimize for bandwidth only but also low latency which can be way more important for specific workloads, eg. TimeMachine backups). Progress can be monitored over in OMV forum: http://forum.openmediavault.org/index.php/Thread/17855-Building-OMV-automatically-for-a-bunch-of-different-ARM-dev-boards/?postID=141273#post141273

I also changed the link to XU4 UASP performance numbers to keltike.de in Armbian's SBC storage performance overview: https://forum.armbian.com/index.php?/topic/1925-some-storage-benchmarks-on-sbcs/&do=findComment&comment=15318