Syncing two UAS devices causes Ubuntu to hang

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Tue Jul 02, 2019 1:48 am

My workmate, Joshua, had tried a lot and long test and he made a change on the Linux kernel based on his parametric test to change a value in a driver. So I am uploading the Image and kernel modules, so I hope some of you may try it out if it gives reliable performance. Before starting updating with uploaded images, please do back up the original files, specifically Image.gz and meson64_odroidn2.dtb for sure.

Here are the links which you need to download:
Image.gz - https://drive.google.com/open?id=1w3StN ... L4CghOTvRA
meson64_odroidn2.dtb - https://drive.google.com/open?id=1Ek3f2 ... MEse91m6GB
Linux kernel modules - https://drive.google.com/open?id=1NbUph ... -vMS2LBdIF

Please do copy the first two files Image.gz and meson64_odroidn2.dtb into "probably" /media/boot where the original kernel files are stored and please do tar xzvf linux-4.9.182-lib_modules.tar.gz -C /. After copying every files, what you have to do is adding one parameter usb-xhci.tablesize=8 to boot.init, it would be looked like this.
setenv bootargs "root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait rw ${condev} ${amlogic} no_console_suspend fsck.repair=yes net.ifnames=0 elevator=noop hdmimode=${hdmimode} cvbsmode=576cvbs max_freq_a53=${max_freq_a53} max_freq_a73=${max_freq_a73} maxcpus=${maxcpus} voutmode=${voutmode} ${cmode} disablehpd=${disablehpd} cvbscable=${cvbscable} overscan=${overscan} ${hid_quirks} usb-xhci.tablesize=8
Once everything is done, please do reboot then the kernel will use the new value added to the boot argument. If you are able to boot successfully with the kernel, please do set 32 to the max sector size as below before testing the storages. This was already been introduced earlier in this thread as a workaround.

Code: Select all

# echo 32 > /sys/class/block/sda/queue/max_sectors_kb
# echo 32 > /sys/class/block/sdb/queue/max_sectors_kb
What are we doing here?
So basically the parametric test I've mentioned are about two factors, one for the buffer size in USB driver and another is in the block device driver, we've performed multiple combos and eventually, 8 for USB driver and 32 for the block device drive gives better stability but the performance decreased but had run for a couple of days without any errors. Still, the test devices are running with the same factors and other combos as well, at the same time Amlogic is also looking into the same problem and trying a different approach.

Sorry for long instruction instead of giving you the kernel update, since this is a temporary approach it would be grateful if someone can try out this change and share the result if it works with another device as well. Also, you could try out other factors like from 8 to 16 for usb-xhci.tablesize or 32 to 64+ for max_sectors_kb. Whenever you change the value of usb-xhci.tablesize you must reboot your N2.

Please not that this change is not for a user who is using an single USB storage attached to ODROID-N2.

Thank you.
These users thanked the author tobetter for the post (total 2):
xabolcs (Tue Jul 02, 2019 4:51 am) • joshua.yang (Tue Jul 02, 2019 4:35 pm)

gahabana
Posts: 29
Joined: Wed Sep 07, 2016 1:47 am
languages_spoken: english
ODROIDs: odroid-c2
odroid-n2
odroid-xu4
Has thanked: 2 times
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by gahabana » Tue Jul 02, 2019 2:53 pm

@fonix232,
In my experience it does not matter which USB to SATA or NVME bridges/adapters you are using. I have not been using UAS and it does not matter - even with USB 3.0 (older) HDDs nto to mention newer USB 3.1/USB 3.2/USB-C interface - with 2 drives system will error sooner or later. Sometimes even with only one usb HDD. Something is wrong either with HW or with SW support.
Everyone is hopeful that Odroid wiht help from chipset manufacturors will deliver a sw fix (either kernel patch or some 'tweak') and until that does happen USB is at best unreliable (for HDD at least) on Odroid-N2.

Maybe it will change and maybe speeds will not go from 300+Mb/second down to USB 2.0 range but we will need to see.

gregeeh
Posts: 4
Joined: Wed May 08, 2019 8:09 am
languages_spoken: english
Has thanked: 0
Been thanked: 2 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by gregeeh » Tue Jul 02, 2019 3:02 pm

gahabana wrote:
Tue Jul 02, 2019 2:53 pm
@fonix232,
Sometimes even with only one usb HDD. Something is wrong either with HW or with SW support.
And it is not just HDD's that have an issue. My PS3 PlayTV Dual Tuner plugged into the USB 3 port will give errors after just a few minutes of watching FTA TV that only has a transfer rate of about 3500 kb/sec. There's no problem if the PS3 PlayTV Dual Tuner is plugged into the micro USB port it is just the USB 3 ports that have issues.

This has been an on going problem for several months now and it's time something was done to fix it once and for all so we can use what we paid for.
These users thanked the author gregeeh for the post (total 2):
thyandrecardoso (Tue Jul 02, 2019 10:31 pm) • thequietman44 (Wed Jul 03, 2019 3:19 pm)

Andrew Sayers
Posts: 31
Joined: Fri May 17, 2019 5:06 pm
languages_spoken: english
ODROIDs: N2
Has thanked: 6 times
Been thanked: 3 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by Andrew Sayers » Wed Jul 03, 2019 3:22 am

tobetter wrote:
Tue Jul 02, 2019 1:48 am
My workmate, Joshua, had tried a lot and long test and he made a change on the Linux kernel based on his parametric test to change a value in a driver. So I am uploading the Image and kernel modules, so I hope some of you may try it out if it gives reliable performance.
I'm now running this on my N2, and will let you know the results tomorrow. My system has been fairly stable with just the previous 32 fix (admittedly with a fairly low load), but at least this should tell you whether it makes anything worse :)

I do have a few questions though...
  • do you want to know anything more than whether it works and what the transfer speed looks like?
  • is it worth including your instructions in the top post, or do you plan to push this out through the normal channels once it's been tested?
  • this sounds like a temporary workaround - what are the chances the performance of this workaround is similar to the performance of the permanent solution?
  • do you guys want to start a new (sticky) thread about this whole issue? I'm happy to keep maintaining the top post in this thread either way, but recent posts have shown the title is confusing and people want to see a more official response

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Wed Jul 03, 2019 3:21 pm

Andrew Sayers wrote:
Wed Jul 03, 2019 3:22 am
tobetter wrote:
Tue Jul 02, 2019 1:48 am
My workmate, Joshua, had tried a lot and long test and he made a change on the Linux kernel based on his parametric test to change a value in a driver. So I am uploading the Image and kernel modules, so I hope some of you may try it out if it gives reliable performance.
I'm now running this on my N2, and will let you know the results tomorrow. My system has been fairly stable with just the previous 32 fix (admittedly with a fairly low load), but at least this should tell you whether it makes anything worse :)

I do have a few questions though...
Trying to figure out the combos with less IO traffic drops but reliable stability is the first.
[*]do you want to know anything more than whether it works and what the transfer speed looks like?
It would be great if you can share how your set up works at which performance.
[*]is it worth including your instructions in the top post, or do you plan to push this out through the normal channels once it's been tested?
If the way works and if it helps a bit having more stable performance, I will ask to add it for the next kernel update. So making less work for setting up.
[*]if you're planning to push it out, is there anything I can do to help get this unrelated issue with bootini fixed at the same time?
Not sure about the issue, honestly I haven't looked into it so far and I am not able to manage it. But let me look into the thread and try if there is what I can help.
[*]this sounds like a temporary workaround - what are the chances the performance of this workaround is similar to the performance of the permanent solution?
It's a bit hard to say at the moment, we can offer you the best combo which we tested that gives more stability of USB I/O at the same time we have a binary update from Amlogic which could a bit improve the stability which is being tested internally. We can commit later after having some test, sorry about the slow progress.
[*]do you guys want to start a new (sticky) thread about this whole issue? I'm happy to keep maintaining the top post in this thread either way, but recent posts have shown the title is confusing and people want to see a more official response
[/list]
Not sure which is better. I understand since there are a few threads with a similar issue, so maybe it could be better to invite the users to one thread for USB stability issue. IMHO, at the moment, thinking to keep tracking on this post and we could manage the USB problem once the patch is released if a similar issue can happen from other USB devices rather than storage.

Andrew Sayers
Posts: 31
Joined: Fri May 17, 2019 5:06 pm
languages_spoken: english
ODROIDs: N2
Has thanked: 6 times
Been thanked: 3 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by Andrew Sayers » Thu Jul 04, 2019 1:25 am

I've now done a day's testing, and the results are... a bit odd. I wasn't able to capture as much data as I would have liked, but I'll keep trying and get back to you.

I've now run the test described in the top post twice with the new kernel, using tablesize=8 and max_sectors_kb=32. In both cases, /dev/sdb went away after approximately 9500 seconds (+-100). But other USB devices weren't affected and sdb came back after a reboot instead of needing a full power cycle. I haven't tried unplugging it then plugging it back in, but I wouldn't be surprised if that works too. Obviously this is a serious issue, but the behaviour is completely different to the normal bug. I can't rule out the possibility this is an issue with the drive, although frankly it would be quite a coincidence.

I haven't run a test that long on the old kernel before, so I'm now doing that. Given that each test takes nearly three hours, it might be a while before I can give you much detail.

elatllat
Posts: 1355
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1, N2
Has thanked: 5 times
Been thanked: 19 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by elatllat » Thu Jul 04, 2019 8:56 pm

Anyone try this on the 5.2 kernel?
Is this an issue for non UAS devices? ( I have to add my devices to quirks to get smartctl to work anyway )

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Thu Jul 04, 2019 11:01 pm

Another update on U-boot, please download the recent update U-boot blob from the link and follow the instruction.
https://github.com/hardkernel/u-boot/re ... droidn2-51

Code: Select all

$ tar xzvf u-boot-odroidn2-51.tar.gz
$ cd sd_fuse
$ sudo ./sd_fusing.sh </dev/path/of/your/memory/card>
In the lab test, this binary update gives a bit better stability but not performance improvement, what we also observed is that an error rate could vary with the type of storage, SSD or USB stick. In case 2+ disks are going to run, I would recommend using 2 for the kernel parameter sg_tablesize which would give a slower performance.

I've asked to prepare the update package and expect to be done soon.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Fri Jul 05, 2019 2:17 am

Linux kernel and U-boot are updated with patches, please do the commands for HK's Ubuntu users.

Code: Select all

$ sudo apt update
$ sudo apt upgrade
Once you have done updating, please add one kernel argument usb-xhci.tablesize=2 to /media/boot/boot.ini. So it would be look like this.

Code: Select all

# Boot Args
setenv bootargs "root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait rw ${condev} ${amlogic} no_console_suspend fsck.repair=yes net.ifnames=0 levator=noop hdmimode=${hdmimode} cvbsmode=576cvbs max_freq_a53=${max_freq_a53} max_freq_a73=${max_freq_a73} maxcpus=${maxcpus} outmode=${voutmode} ${cmode} disablehpd=${disablehpd} cvbscable=${cvbscable} overscan=${overscan} ${hid_quirks} usb-xhci.tablesize=2"

Andrew Sayers
Posts: 31
Joined: Fri May 17, 2019 5:06 pm
languages_spoken: english
ODROIDs: N2
Has thanked: 6 times
Been thanked: 3 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by Andrew Sayers » Fri Jul 05, 2019 3:44 am

tobetter wrote:
Fri Jul 05, 2019 2:17 am
Linux kernel and U-boot are updated with patches, please do the commands for HK's Ubuntu users.
Thanks for the updates, I'll try them tomorrow and report back. Today's tests showed that the bug I mentioned in my previous post was also present with the old kernel, but happened about twice as quickly. In other words, the July 1st kernel and tablesize are an improvement but still don't produce a fully stable system. Hopefully the new u-boot will improve things further.

Here's the kernel log from the moment the hard disk fails (this is with the old kernel, but the error from the July 1st kernel is basically the same):

Code: Select all

Jul  3 21:46:37 andrews-house kernel: [16770.701997@0] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
Jul  3 21:46:37 andrews-house kernel: [16770.707044@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf000 trb-start 00000000cf7d4fc0 trb-end 0000000000000000 seg-start 00000000cf7d4000 seg-end 00000000cf7d4ff0
Jul  3 21:46:37 andrews-house kernel: [16770.707048@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf000 trb-start 00000000cf7d3000 trb-end 00000000cf7d3000 seg-start 00000000cf7d3000 seg-end 00000000cf7d3ff0
Jul  3 21:46:37 andrews-house kernel: [16770.707053@0] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 13
Jul  3 21:46:37 andrews-house kernel: [16770.717784@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf020 trb-start 00000000cf7d4fc0 trb-end 0000000000000000 seg-start 00000000cf7d4000 seg-end 00000000cf7d4ff0
Jul  3 21:46:37 andrews-house kernel: [16770.717788@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf020 trb-start 00000000cf7d3000 trb-end 00000000cf7d3000 seg-start 00000000cf7d3000 seg-end 00000000cf7d3ff0
Jul  3 21:46:37 andrews-house kernel: [16770.717792@0] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
Jul  3 21:46:37 andrews-house kernel: [16770.728444@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf060 trb-start 00000000cf7d4fc0 trb-end 0000000000000000 seg-start 00000000cf7d4000 seg-end 00000000cf7d4ff0
Jul  3 21:46:37 andrews-house kernel: [16770.728448@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf060 trb-start 00000000cf7d3000 trb-end 00000000cf7d3000 seg-start 00000000cf7d3000 seg-end 00000000cf7d3ff0
Jul  3 21:46:37 andrews-house kernel: [16770.728484@0] xhci-hcd xhci-hcd.0.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 6
Jul  3 21:46:37 andrews-house kernel: [16770.739104@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf070 trb-start 00000000cf7d4fc0 trb-end 0000000000000000 seg-start 00000000cf7d4000 seg-end 00000000cf7d4ff0
Jul  3 21:46:37 andrews-house kernel: [16770.739108@0] xhci-hcd xhci-hcd.0.auto: Looking for event-dma 00000000cf7bf070 trb-start 00000000cf7d3000 trb-end 00000000cf7d3000 seg-start 00000000cf7d3000 seg-end 00000000cf7d3ff0
Jul  3 21:47:07 andrews-house kernel: [16800.967917@0] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:47:38 andrews-house kernel: [16831.932018@1] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:47:48 andrews-house kernel: [16842.076226@1] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:48:04 andrews-house kernel: [16858.459982@1] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:48:04 andrews-house kernel: [16858.559862@0] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:48:15 andrews-house kernel: [16868.699941@1] usb 2-1.4: reset SuperSpeed USB device number 5 using xhci-hcd
Jul  3 21:48:15 andrews-house kernel: [16868.720356@0] sd 1:0:0:0: Device offlined - not ready after error recovery
Jul  3 21:48:15 andrews-house kernel: [16868.720395@0] sd 1:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Jul  3 21:48:15 andrews-house kernel: [16868.720410@0] sd 1:0:0:0: [sdb] tag#0 CDB: Read(16) 88 00 00 00 00 00 25 07 23 20 00 00 00 40 00 00
Jul  3 21:48:15 andrews-house kernel: [16868.720417@0] blk_update_request: I/O error, dev sdb, sector 621224736
Jul  3 21:48:15 andrews-house kernel: [16868.721426@1] sd 1:0:0:0: rejecting I/O to offline device
Jul  3 21:48:15 andrews-house kernel: [16868.726921@1] blk_update_request: I/O error, dev sdb, sector 621224800
Jul  3 21:48:15 andrews-house kernel: [16868.733381@4] sd 1:0:0:0: rejecting I/O to offline device
Jul  3 21:48:15 andrews-house kernel: [16868.738660@4] blk_update_request: I/O error, dev sdb, sector 621224864
Jul  3 21:48:15 andrews-house kernel: [16868.747008@4] sd 1:0:0:0: rejecting I/O to offline device
Jul  3 21:48:15 andrews-house kernel: [16868.750458@4] blk_update_request: I/O error, dev sdb, sector 621224424
Jul  3 21:48:15 andrews-house kernel: [16868.757183@4] sd 1:0:0:0: rejecting I/O to offline device
Jul  3 21:48:15 andrews-house kernel: [16868.762421@4] blk_update_request: I/O error, dev sdb, sector 621224424
Jul  3 21:48:15 andrews-house kernel: [16868.768897@4] Buffer I/O error on dev sdb, logical block 77653053, async page read
I was able to restart the disk simply by pulling out its USB cable and plugging it back in, so whatever's happening here certainly isn't as severe as the original bug.

@tobetter - I'm planning to spend my time tweaking the tablesize and max_sectors_kb values in the latest kernel+u-boot, but I'm happy to look into other things if anything catches your eye in the log above. Also, if I understand correctly, the new recommended workaround is:
  1. apt-get update && apt-get upgrade (ignoring the U-boot blob in the link above, which is no longer necessary)
  2. manually edit /media/boot/boot.ini as described above
  3. set max_sectors_kb as described in the top post
If that's all correct, I'll update the top post again.
elatllat wrote:
Thu Jul 04, 2019 8:56 pm
Anyone try this on the 5.2 kernel?
Is this an issue for non UAS devices? ( I have to add my devices to quirks to get smartctl to work anyway )
This is is definitely an issue for non-UAS devices. People have reported lots of USB devices with high enough sustained transfer speeds triggering the bug, it's just that UAS disk drives are the most common example (also, I didn't really understand the problem when I chose the thread title :oops:). I'll mention that in the next update to the top post.

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Sun Jul 07, 2019 5:46 am

tobetter wrote:
Fri Jul 05, 2019 2:17 am
Linux kernel and U-boot are updated with patches, please do the commands for HK's Ubuntu users.

Code: Select all

$ sudo apt update
$ sudo apt upgrade
Once you have done updating, please add one kernel argument usb-xhci.tablesize=2 to /media/boot/boot.ini. So it would be look like this.

Code: Select all

# Boot Args
setenv bootargs "root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait rw ${condev} ${amlogic} no_console_suspend fsck.repair=yes net.ifnames=0 levator=noop hdmimode=${hdmimode} cvbsmode=576cvbs max_freq_a53=${max_freq_a53} max_freq_a73=${max_freq_a73} maxcpus=${maxcpus} outmode=${voutmode} ${cmode} disablehpd=${disablehpd} cvbscable=${cvbscable} overscan=${overscan} ${hid_quirks} usb-xhci.tablesize=2"
Unfortunately for me, the uboot+kernel updates just made it worse. Now even with max_sectors_kb set to 10, I'm getting failures almost immediately after reboot - in fact now roughly ~5 minutes after booting, my storage dies, making it useless as a home media server.

This made me seriously reconsider sending back the whole shebang to HardKernel, and replace it with a H2... Bit more pricy, but less of these quirks.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Sun Jul 07, 2019 11:29 am

fonix232 wrote:
Sun Jul 07, 2019 5:46 am
tobetter wrote:
Fri Jul 05, 2019 2:17 am
Linux kernel and U-boot are updated with patches, please do the commands for HK's Ubuntu users.

Code: Select all

$ sudo apt update
$ sudo apt upgrade
Once you have done updating, please add one kernel argument usb-xhci.tablesize=2 to /media/boot/boot.ini. So it would be look like this.

Code: Select all

# Boot Args
setenv bootargs "root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait rw ${condev} ${amlogic} no_console_suspend fsck.repair=yes net.ifnames=0 levator=noop hdmimode=${hdmimode} cvbsmode=576cvbs max_freq_a53=${max_freq_a53} max_freq_a73=${max_freq_a73} maxcpus=${maxcpus} outmode=${voutmode} ${cmode} disablehpd=${disablehpd} cvbscable=${cvbscable} overscan=${overscan} ${hid_quirks} usb-xhci.tablesize=2"
Unfortunately for me, the uboot+kernel updates just made it worse. Now even with max_sectors_kb set to 10, I'm getting failures almost immediately after reboot - in fact now roughly ~5 minutes after booting, my storage dies, making it useless as a home media server.

This made me seriously reconsider sending back the whole shebang to HardKernel, and replace it with a H2... Bit more pricy, but less of these quirks.
I am sorry to hear it does work worse. Since I understand you have RAID setup, if remember correctly, would you be able to test once after removing usb-xhci.tablesize and not setting max_sectors_kb?

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Sun Jul 07, 2019 9:08 pm

tobetter wrote:
Sun Jul 07, 2019 11:29 am
I am sorry to hear it does work worse. Since I understand you have RAID setup, if remember correctly, would you be able to test once after removing usb-xhci.tablesize and not setting max_sectors_kb?
Yes, I'm running a RAID0 array on two USB3 HDDs (identical hard drives and USB adapters), and I also have an NVMe SSD with a single partition and some symlinks (my docker config folder, which I need if I reinstall the system, some private documents, and the rest is used for temp cache for torrents while they seed). The SSD seems to be doing fine, which is weird because it has probably the highest continuous throughput in reads and writes, and it does use UASP... Could it be that there's also some kind of overflow in relevance to transaction count? RAID0 needs a lot more syncing even on an idle system than a regular direct partition, thus resulting in more frequent, but considerably smaller messages, and in that case it would make sense why RAID arrays die sooner than direct mounts. Just a hunch.

I've tried removing the tablesize parameter and commented out the line responsible for setting max_sectors_kb, in the appropriate autostart file as well. It reset back to 512 on all drives, and after restart it started producing the errors immediately. After a power cycle, and removing the USB drives while running, and repeating the power cycle, I was able to have a working system for about 15 minutes, with only a Plex scan running that affects the disks in question.

This is also a change from previous behaviour, as before the update I never needed a power cycle to restore working conditions, but now I apparently do.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Sun Jul 07, 2019 10:49 pm

fonix232 wrote:
Sun Jul 07, 2019 9:08 pm
tobetter wrote:
Sun Jul 07, 2019 11:29 am
I am sorry to hear it does work worse. Since I understand you have RAID setup, if remember correctly, would you be able to test once after removing usb-xhci.tablesize and not setting max_sectors_kb?
Yes, I'm running a RAID0 array on two USB3 HDDs (identical hard drives and USB adapters), and I also have an NVMe SSD with a single partition and some symlinks (my docker config folder, which I need if I reinstall the system, some private documents, and the rest is used for temp cache for torrents while they seed). The SSD seems to be doing fine, which is weird because it has probably the highest continuous throughput in reads and writes, and it does use UASP... Could it be that there's also some kind of overflow in relevance to transaction count? RAID0 needs a lot more syncing even on an idle system than a regular direct partition, thus resulting in more frequent, but considerably smaller messages, and in that case it would make sense why RAID arrays die sooner than direct mounts. Just a hunch.

I've tried removing the tablesize parameter and commented out the line responsible for setting max_sectors_kb, in the appropriate autostart file as well. It reset back to 512 on all drives, and after restart it started producing the errors immediately. After a power cycle, and removing the USB drives while running, and repeating the power cycle, I was able to have a working system for about 15 minutes, with only a Plex scan running that affects the disks in question.

This is also a change from previous behaviour, as before the update I never needed a power cycle to restore working conditions, but now I apparently do.
Well...I am trying to be on the track of your problem. I and my staff had tested with two SSD disks or SSD+NVMe which are faster storage device than HDD, so I assumed that they could make more troubles than slow storage such as USB stick. We've tried the different parameter of uxb-xhci.tablesize and max_sectors_kb and better stability are given when usb-xhci.tablesize is low. Having test with SSD and NVMe was fine even USB stick, RAID0 might be a different story. What I don't understand is that you have to power cycle which you didn't before the update, which only changes U-boot.

nikodemus
Posts: 25
Joined: Fri Mar 08, 2019 2:15 pm
languages_spoken: german, english
ODROIDs: C1, C2, XU4, N2
Has thanked: 2 times
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nikodemus » Sun Jul 07, 2019 11:09 pm

@fonix: Are you sure your power supply can catch up with the demands of your Nas? The Rockpi community had ongoing prolems with usb3 and TKaiser was investigating this by using a raid. (raid is perfect for stress test systems).
All errors he found were underpowering events caused by heavy load from the raid:

https://github.com/ayufan-rock64/linux-build/issues/112
"The 'good' news: utilizing more than one USB3 disk at the same time is the same sh*t show as everywhere else in SBC land: https://www.cnx-software.com/2019/06/24 ... ent-564128"

I am convinced you checked this before, but as you said, its a little surprising that your fastest ssd seems not to have issues.
These users thanked the author nikodemus for the post:
xabolcs (Mon Jul 08, 2019 3:30 am)

rondi7
Posts: 5
Joined: Mon May 13, 2019 1:04 am
languages_spoken: english
ODROIDs: n2
Has thanked: 0
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by rondi7 » Mon Jul 08, 2019 12:04 am

@tobetter After reading the links provided above by nikodemus, it appears a problem (and it could easily be THE problem) is a drop in the 5 volts at the USB3 connector during heavy writes. If this is the case--in your lab you could isolate the 5volts on the board to the USB3 connectors (and or the hub) and use an external power supply to power the USB3 connectors. Another suggestion--add a a fairly big ( your engineers can decide what is big :) )capacitor to the 5 volts close to the USB3 connector --maybe even to the 5v pin on the header.

Earlier in the thread you kept asking about the power adapter, as if you suspected a problem there. I wonder--has anyone tried a 16v 4-6 amp supply (iirc your input power spec is up to 20v). Maybe the standard HK 12v @2amp is not enough?

This from one of the links above:
What looked like ‘UAS problems’ or ‘broken USB3 controller’ was simply underpowering. In the first try both SSDs were powered by the RPi and as such the current limitation on the USB ports kicked in (slightly above 1.1A available for all ports together). Now with one SSD powered externally everything is fine.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Mon Jul 08, 2019 12:18 am

rondi7 wrote:
Mon Jul 08, 2019 12:04 am
@tobetter After reading the links provided above by nikodemus, it appears a problem (and it could easily be THE problem) is a drop in the 5 volts at the USB3 connector during heavy writes. If this is the case--in your lab you could isolate the 5volts on the board to the USB3 connectors (and or the hub) and use an external power supply to power the USB3 connectors. Another suggestion--add a a fairly big ( your engineers can decide what is big :) )capacitor to the 5 volts close to the USB3 connector --maybe even to the 5v pin on the header.

Earlier in the thread you kept asking about the power adapter, as if you suspected a problem there. I wonder--has anyone tried a 16v 4-6 amp supply (iirc your input power spec is up to 20v). Maybe the standard HK 12v @2amp is not enough?

This from one of the links above:
What looked like ‘UAS problems’ or ‘broken USB3 controller’ was simply underpowering. In the first try both SSDs were powered by the RPi and as such the current limitation on the USB ports kicked in (slightly above 1.1A available for all ports together). Now with one SSD powered externally everything is fine.
Thanks, I was reading the same and thinking of the way to test. It would be worth to try run with other adaptors as well...I am not sure, maybe wire the power to HDD from somewhere else like PC power and just wire USB 3.0...since they are mentioning of power issue, maybe I would set up new test env with a new external power source for disk drives? :?:

elatllat
Posts: 1355
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1, N2
Has thanked: 5 times
Been thanked: 19 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by elatllat » Mon Jul 08, 2019 2:09 am

I use a piece of paper and some tape to cover the 5v on the USB to isolate the SBC from a powered hub, so anyone who is able to reproduce this issue could just do that in about a minute and know if it's a power issue.

Andrew Sayers
Posts: 31
Joined: Fri May 17, 2019 5:06 pm
languages_spoken: english
ODROIDs: N2
Has thanked: 6 times
Been thanked: 3 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by Andrew Sayers » Mon Jul 08, 2019 2:34 am

Some more tests from me:
  • with the latest kernel and tablesize=2, my system is stable but read speeds are extremely low
  • with the latest kernel and no tablesize, my sdb fails after about 700 seconds of continuous reading from both disks at once
  • with the kernel tobetter posted here on July 1st and no tablesize, my sdb fails after about 7,000 seconds of continuous reading
All these tests were done after apt-get update && apt-get upgrade, so the only variables are in Image.gz and meson64_odroidn2.dtb. I assume the first test is stable because of the low speeds, but the other two show the latest kernel (not u-boot etc.) crashes 10 times quicker than the one posted here.

rondi7
Posts: 5
Joined: Mon May 13, 2019 1:04 am
languages_spoken: english
ODROIDs: n2
Has thanked: 0
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by rondi7 » Mon Jul 08, 2019 8:17 am

elatllat wrote:
Mon Jul 08, 2019 2:09 am
I use a piece of paper and some tape to cover the 5v on the USB to isolate the SBC from a powered hub, so anyone who is able to reproduce this issue could just do that in about a minute and know if it's a power issue.
I have not gone back to read your previous posts on this thread---What was the result of your testing--are you able to have a Raid with 2 USB3 sticks and have it work for a long time? or perhaps as Andrews 1st post describes, his test case. If both of those work with an external 5 volts then there may be a solution. BUT--is it not true--IF you connect your HDD, SSD or?? to an external powered USB3 hub--you are now going thru 2 hubs and this will decrease your transfer rate because you are only connected to 1 USB3 on the N2.

IF it is a quick sharp drop in the 5v because of the increase in current required, then a correctly sized capacitor from 5v ( or maybe even from the `12v supply) to ground may contain enough storage to keep 5v from sagging. If it is a limitation of the voltage regulator--which changes the 12v to 5v, and 3.3 and ??--- then maybe it is also effecting other circuits--ie the SoC. Again adding a capacitor might fix it and it could easily be done on the GPIO header.

@tobetter--IF this is a problem with the 5 volts and maybe the SoC power (3.3v ?)--your analog (dare I say that here :) ) engineering folks should have a close look at not only the rails, but also at the USB3 connector, SOC etc when you are making it fail. They would know what size cap to try, and where is best place to put it.

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Mon Jul 08, 2019 9:11 am

tobetter wrote:
Sun Jul 07, 2019 10:49 pm
Well...I am trying to be on the track of your problem. I and my staff had tested with two SSD disks or SSD+NVMe which are faster storage device than HDD, so I assumed that they could make more troubles than slow storage such as USB stick. We've tried the different parameter of uxb-xhci.tablesize and max_sectors_kb and better stability are given when usb-xhci.tablesize is low. Having test with SSD and NVMe was fine even USB stick, RAID0 might be a different story. What I don't understand is that you have to power cycle which you didn't before the update, which only changes U-boot.
I was guessing the same, especially since the issue seems to be throughput-related. Not sure what's so different in my case, I'll try to reinstall the whole board some other day, see if recreating the RAID0 array helps. The immediate issue on boot might be related to that, the software RAID trying to repair all the fallen-out-of-sync blocks, and overloads the bus immediately.
nikodemus wrote:
Sun Jul 07, 2019 11:09 pm
@fonix: Are you sure your power supply can catch up with the demands of your Nas? The Rockpi community had ongoing prolems with usb3 and TKaiser was investigating this by using a raid. (raid is perfect for stress test systems).
All errors he found were underpowering events caused by heavy load from the raid:

https://github.com/ayufan-rock64/linux-build/issues/112
"The 'good' news: utilizing more than one USB3 disk at the same time is the same sh*t show as everywhere else in SBC land: https://www.cnx-software.com/2019/06/24 ... ent-564128"

I am convinced you checked this before, but as you said, its a little surprising that your fastest ssd seems not to have issues.

I'm using a 12V@5A power adapter, not the one supplied with the board. So unless the N2 itself limits the max USB power use to some ridiculously low value. All in all I doubt my disks use more than 5V@1A each.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Mon Jul 08, 2019 3:59 pm

fonix232 wrote:
Mon Jul 08, 2019 9:11 am
tobetter wrote:
Sun Jul 07, 2019 10:49 pm
Well...I am trying to be on the track of your problem. I and my staff had tested with two SSD disks or SSD+NVMe which are faster storage device than HDD, so I assumed that they could make more troubles than slow storage such as USB stick. We've tried the different parameter of uxb-xhci.tablesize and max_sectors_kb and better stability are given when usb-xhci.tablesize is low. Having test with SSD and NVMe was fine even USB stick, RAID0 might be a different story. What I don't understand is that you have to power cycle which you didn't before the update, which only changes U-boot.
I was guessing the same, especially since the issue seems to be throughput-related. Not sure what's so different in my case, I'll try to reinstall the whole board some other day, see if recreating the RAID0 array helps. The immediate issue on boot might be related to that, the software RAID trying to repair all the fallen-out-of-sync blocks, and overloads the bus immediately.
nikodemus wrote:
Sun Jul 07, 2019 11:09 pm
@fonix: Are you sure your power supply can catch up with the demands of your Nas? The Rockpi community had ongoing prolems with usb3 and TKaiser was investigating this by using a raid. (raid is perfect for stress test systems).
All errors he found were underpowering events caused by heavy load from the raid:

https://github.com/ayufan-rock64/linux-build/issues/112
"The 'good' news: utilizing more than one USB3 disk at the same time is the same sh*t show as everywhere else in SBC land: https://www.cnx-software.com/2019/06/24 ... ent-564128"

I am convinced you checked this before, but as you said, its a little surprising that your fastest ssd seems not to have issues.

I'm using a 12V@5A power adapter, not the one supplied with the board. So unless the N2 itself limits the max USB power use to some ridiculously low value. All in all I doubt my disks use more than 5V@1A each.
Can I have the model name of your disks for RAID?

nikodemus
Posts: 25
Joined: Fri Mar 08, 2019 2:15 pm
languages_spoken: german, english
ODROIDs: C1, C2, XU4, N2
Has thanked: 2 times
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nikodemus » Mon Jul 08, 2019 11:47 pm

Peak value can be even much higher, see above article by Tkaiser.
Some years ago i had a drive that made trouble no matter whic PC i used though everything was within specification. Only an active hub finally helped. An additional hub sometimes is not a good idea because it can make trouble too, but i had no other chance, because the drive had no connector for external supply.

nick793
Posts: 8
Joined: Mon Jun 24, 2019 9:35 pm
languages_spoken: english
ODROIDs: N2, GO
Has thanked: 0
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nick793 » Tue Jul 09, 2019 10:44 pm

Just want to echo that I'm having issues as well.

I'm running two RAID1 arrays off my N2 and I'm having reboots every 12 hours or so. The disks are connected as follows:

Array 1 (1TB RAID1):
  • 2x 1TB Seagate Barracuda HDD (5400rpm)
  • Startech.com 2-bay HDD enclosure (powered)
Array 2 (240GB RAID1):
  • 1x 240GB M.2 SSD (Kingston)
  • 1x 256GB M.2 SSD (Samsung PM851)
  • 2x Startech.com M.2 NGFF to USB3 enclosure
The extra 16GB of space on the Samsung SSD is being used as part of a GlusterFS volume that's replicated across the network. Every few hours the bus resets and the GlusterFS daemon crashes.

How big of a performance hit is the workaround? I'm about to set up a new RAID5 array but if it's back to USB2 then I'm not going to bother for a while until a fix comes along.

User avatar
tobetter
Posts: 3698
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 28 times
Been thanked: 117 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by tobetter » Tue Jul 09, 2019 11:58 pm

nick793 wrote:
Tue Jul 09, 2019 10:44 pm
Just want to echo that I'm having issues as well.

I'm running two RAID1 arrays off my N2 and I'm having reboots every 12 hours or so. The disks are connected as follows:

Array 1 (1TB RAID1):
  • 2x 1TB Seagate Barracuda HDD (5400rpm)
  • Startech.com 2-bay HDD enclosure (powered)
Array 2 (240GB RAID1):
  • 1x 240GB M.2 SSD (Kingston)
  • 1x 256GB M.2 SSD (Samsung PM851)
  • 2x Startech.com M.2 NGFF to USB3 enclosure
The extra 16GB of space on the Samsung SSD is being used as part of a GlusterFS volume that's replicated across the network. Every few hours the bus resets and the GlusterFS daemon crashes.

How big of a performance hit is the workaround? I'm about to set up a new RAID5 array but if it's back to USB2 then I'm not going to bother for a while until a fix comes along.
Does the error also happen when you connect the Array 1 only?

nick793
Posts: 8
Joined: Mon Jun 24, 2019 9:35 pm
languages_spoken: english
ODROIDs: N2, GO
Has thanked: 0
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nick793 » Wed Jul 10, 2019 8:43 pm

tobetter wrote:
Tue Jul 09, 2019 11:58 pm
nick793 wrote:
Tue Jul 09, 2019 10:44 pm
Just want to echo that I'm having issues as well.

I'm running two RAID1 arrays off my N2 and I'm having reboots every 12 hours or so. The disks are connected as follows:

Array 1 (1TB RAID1):
  • 2x 1TB Seagate Barracuda HDD (5400rpm)
  • Startech.com 2-bay HDD enclosure (powered)
Array 2 (240GB RAID1):
  • 1x 240GB M.2 SSD (Kingston)
  • 1x 256GB M.2 SSD (Samsung PM851)
  • 2x Startech.com M.2 NGFF to USB3 enclosure
The extra 16GB of space on the Samsung SSD is being used as part of a GlusterFS volume that's replicated across the network. Every few hours the bus resets and the GlusterFS daemon crashes.

How big of a performance hit is the workaround? I'm about to set up a new RAID5 array but if it's back to USB2 then I'm not going to bother for a while until a fix comes along.
Does the error also happen when you connect the Array 1 only?
Not sure haven't tried it yet. The errors only became noticeable after I moved all my media to array 1 and started downloading more media to it. Last night after streaming with Plex for a few hours it rebooted. The most direct way I found to trigger the bug is to verify torrents in transmission. If I verify one at a time I have no problems but more than one at a time and my system reboots in minutes.

I can try and see if that impacts it, but right now I'm using array 1 for media storage, and array 2 for most of my docker persistent storage (config files, databases, etc) and Plex transcoding.

I should add that I'm not using any updates/workarounds beyond those that come from apt. If there's something I should try please let me know, but I'm not that skilled with kernel parameters/u-boot so I've avoided touching them until now

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Wed Jul 10, 2019 9:20 pm

tobetter wrote:
Mon Jul 08, 2019 3:59 pm
Can I have the model name of your disks for RAID?
Sure. For the RAID0 array I'm using two HGST TravelStar 7 K1000 (HGST721010A9E630). The 250GB SSD is a Samsung PM961 (MZVLW256HEHP-000).
nikodemus wrote:
Mon Jul 08, 2019 11:47 pm
Peak value can be even much higher, see above article by Tkaiser.
Some years ago i had a drive that made trouble no matter whic PC i used though everything was within specification. Only an active hub finally helped. An additional hub sometimes is not a good idea because it can make trouble too, but i had no other chance, because the drive had no connector for external supply.
I've been using these disks for two years now, in the current enclosures. No issues so far, and I've ran them from a 5V@3A adapter and a RasPi, and even a 12V@2A adapter and a router (Mi Router 3G). No issues whatsoever. It's only the N2 that refuses to work properly with it.


@tobetter: Is it possible that because of my RAID0 array's default block size of 64kB, the patching does not work? I'd try to reformat and lower it but ATM have no other storage option and I have some documents on the array I need.

EDIT:

Weird thing happening. Yesterday I did some copy work and moved data to the SSD from the RAID array (preparing to recreate it, just not yet). I rebooted as I was getting the XHCI errors again, and since... Nothing. The server downloaded about 20 gigs since, directly written to the RAID array (I disabled pooling incomplete downloads to the cache drive), and still no crash. No patch/workaround applied, just latest updates from APT - except for kernel, I'm going to download and install 4.9.184-37 right now (currently on 182-35).


EDIT2:

Apparently sequential reading speeds up the process in my case. Downloads are fine, but I can barely stream 10-15 minutes via Plex (no transcoding) before it crashes the device.

EDIT3:

Definitely a sequential read thing. Stopped torrent after an hour of continuous, 7MB/s (not a typo, 7 megabytes per sec, roughly 24GB of download) download which went without problems, waited a bit, started a stream via Plex and it died in 5 minutes... Streaming a different file at each time, so its not a single file causing the issue.

Soleil
Posts: 35
Joined: Tue Apr 30, 2019 9:20 am
languages_spoken: english
Has thanked: 1 time
Been thanked: 2 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by Soleil » Thu Jul 11, 2019 4:24 am

On power supply side - when I've run into the issue had same thought process about voltage dropping.
Issue happened to me with externally powered USB hub too as well as power supply 12V/6A (72W) - tried with this one:
JOYLIT-1260 - https://www.amazon.fr/dp/B0716XX3S6
Whilst it says to power LED, it was the most powerful one I could quickly come up at the time of testing.
Neither didn't help.

rondi7
Posts: 5
Joined: Mon May 13, 2019 1:04 am
languages_spoken: english
ODROIDs: n2
Has thanked: 0
Been thanked: 0
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by rondi7 » Thu Jul 11, 2019 7:46 am

@Soleil Have you tried this==posted 10 or so posts before this one.
by elatllat » Sun Jul 07, 2019 10:09 am
"I use a piece of paper and some tape to cover the 5v on the USB to isolate the SBC from a powered hub, so anyone who is able to reproduce this issue could just do that in about a minute and know if it's a power issue. "

I'm not sure of his results--he didn't say, but perhaps isolating the 2 5v supplies might help. I have no idea if the N2 5v is connected to the your external powered hub--I would hope not.

nikodemus
Posts: 25
Joined: Fri Mar 08, 2019 2:15 pm
languages_spoken: german, english
ODROIDs: C1, C2, XU4, N2
Has thanked: 2 times
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nikodemus » Fri Jul 12, 2019 1:27 am

@fonic: Thank you for taking time testing that much!

@Soleil
The N2 power supply is not questionable. It provides reliable 900 mA for USB3. In addition with the external hub we have enough power to supply the disk (it will not help to cut 5V line between N2 and active hub). But the critical distance still is the cabling between USB and the device. USB provides 5V, the device will get maximum 4,9V or below. USB cables are for data connections, not power source. With some cables i measured a drop to 4,6V with current of 500 mA, with 1000 mA the drop will be even higher (keep in mind Ohms Law). So what we need is a short and thick wiring. Best would be to use y cables.
I am not quite aware, if we have underpowering issues, but it must be clear before futher investigations.

User avatar
mad_ady
Posts: 6178
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 123 times
Been thanked: 89 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by mad_ady » Fri Jul 12, 2019 2:44 am

It would be nice if there were some cheap device you could plug between the sata power interface and the disk to measure and display instant power and current - like some usb dongles do...

elatllat
Posts: 1355
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1, N2
Has thanked: 5 times
Been thanked: 19 times
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by elatllat » Fri Jul 12, 2019 4:15 am

You'd need a 5GHz oscilloscope for that, way easier to just cover the five bolt pin with paper and tape.

nikodemus
Posts: 25
Joined: Fri Mar 08, 2019 2:15 pm
languages_spoken: german, english
ODROIDs: C1, C2, XU4, N2
Has thanked: 2 times
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by nikodemus » Fri Jul 12, 2019 5:32 am

I must excuse, what would it bring to cover the 5V line? Where will the power come from?
Besides if you want to check signal quality of the data lines, you need way more than 5Ghz (sampling theoreme).

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Sat Jul 13, 2019 4:18 am

nikodemus wrote:
Fri Jul 12, 2019 1:27 am
@fonic: Thank you for taking time testing that much!

@Soleil
The N2 power supply is not questionable. It provides reliable 900 mA for USB3. In addition with the external hub we have enough power to supply the disk (it will not help to cut 5V line between N2 and active hub). But the critical distance still is the cabling between USB and the device. USB provides 5V, the device will get maximum 4,9V or below. USB cables are for data connections, not power source. With some cables i measured a drop to 4,6V with current of 500 mA, with 1000 mA the drop will be even higher (keep in mind Ohms Law). So what we need is a short and thick wiring. Best would be to use y cables.
I am not quite aware, if we have underpowering issues, but it must be clear before futher investigations.
I'm testing with extra short cables (15cm USB A to C, 3.1 gen2 certified, 10Gbps cables), while the NVMe SSD is connected on a longer, 1m cable of the same brand and make. I've checked official specs, and the NVMe SSD seems to be using 6.1W tops with idle being around ~0.5W, whereas the HDDs go around 4.5W at startup, idling at <1W, and continuous read-write uses around 2W. I can see how the SSD might be causing power issues, especially if it's accessed continuously (it is, since it's used as a cache+docker config storage). I'll try moving docker config to the eMMC, and remove the SSD completely, see if it at least partially fixes my issue.

I've also done some further testing, and it's NOT the reading - I managed to watch (and watch only) roughly 6 hours of stream without a reboot, but then I started a heavier download (multiple files, so multiple disk accesses during write), it died within 10 minutes.

fonix232
Posts: 16
Joined: Sun Nov 12, 2017 9:47 pm
languages_spoken: English
ODROIDs: N2
Has thanked: 0
Been thanked: 1 time
Contact:

Re: Syncing two UAS devices causes Ubuntu to hang

Unread post by fonix232 » Tue Jul 16, 2019 3:50 pm

Has anyone tried using MergerFS instead of a RAID array?

Post Reply

Return to “Issues”

Who is online

Users browsing this forum: No registered users and 0 guests