JMB585 5 port SATA controller

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

JMB585 5 port SATA controller

Post by zupet »

Image

I'm getting pcie error with JMB585 6 port SATA controller.
tobetter wrote:
Mon Apr 04, 2022 6:39 pm
EDIT: when you have a chance to build your custom kernel, would you try to remove the lines #193 to #197 and try again?
https://github.com/hardkernel/linux/blo ... 1.dts#L193
tested modified kernel, but still showing this message and controller is not recognized;
Last edited by zupet on Sat Apr 09, 2022 12:49 am, edited 1 time in total.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by zupet »

Image

Image

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by tobetter »

I can say that the error logs in the photo comes always, I used to have the same log regardless PCIe device is attached or not. Which is kind of bug but does not mean PCIe device does not work. Please capture the kernel log with dmesg and share with me.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by zupet »

Image

Code: Select all

odroid@server:~$ lspci
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
It's working? I dont know why but it suddenly working. :shock:

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by zupet »

Image

I can access from petitboot too.

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by tobetter »

zupet wrote:
Sat Apr 09, 2022 12:34 am

Code: Select all

odroid@server:~$ lspci
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
It's working? I dont know why but it suddenly working. :shock:
Awesome!!
I suspect the PCIe connector on your ODROID-N2 since your NVMe also had similar problem. Or PCIe card was not properly stting to its position.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by zupet »

tobetter wrote:
Sat Apr 09, 2022 12:37 am
zupet wrote:
Sat Apr 09, 2022 12:34 am

Code: Select all

odroid@server:~$ lspci
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
It's working? I dont know why but it suddenly working. :shock:
Awesome!!
I suspect the PCIe connector on your ODROID-N2 since your NVMe also had similar problem. Or PCIe card was not properly stting to its position.
https://github.com/hardkernel/linux/blo ... 1.dts#L193

this is the fix !!

that blue LED always turn on with petitboot, but it turns off after selecting boot partition.
using stock kernel, it does not come back, nor lit again with reboot command. I had to cold boot to enable JMB858 from petitboot.

with modified kernel, it turns off after selecting boot partition, and came back after few seconds. I'll report if something changed.
These users thanked the author zupet for the post:
odroid (Sat Apr 09, 2022 1:14 pm)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Image
Image

looks good !! let's go with six drives. :-)

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller problem

Post by tkaiser »

zupet wrote:
Sat Apr 09, 2022 12:27 am
Image
OMG, how flimsy the PCB looks. No wonder people are complaining about these M.2 cards everywhere since they look like they can be damaged simply by staring at them :)

Will some airflow be possible around the small heatsink on the controller?

Are you going with a RAID-0 over six 12TB WD drives now for testing purposes? Or another storage setup?

12TB in 3.5" format should result in really high sequential transfer speeds on the outer tracks. As we have confirmed 1600 MB/s with fio and SSDs this would result in a theoretical bottleneck of 320 MB/s for each HDD connected to the JMB585. Plenty of headroom I would say... or in other words: as long as the HDDs can't achieve +300 MB/s sequential transfer speeds they're the bottleneck and not M1 + JMB585.

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller

Post by tobetter »

@zupet, not sure if JMB858 is detected after the 'reserved-memory' is removed from the device tree. Would you be able to capture the kernel log with dmesg | grep pci when the reserved memory is in the device tree file and not? Also, highly suggest you to run SATA disk long enough to see if SATA disks are disappeared. Probably around 10 min is good enough to test like compiling the Linux kernel.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Image

Image

Code: Select all

odroid@server:~$ sudo dd if=/dev/zero of=/dev/md0 bs=512k count=200M oflag=direct status=progress
149386428416 bytes (149 GB, 139 GiB) copied, 214 s, 698 MB/s^C
286252+0 records in
286252+0 records out
150078488576 bytes (150 GB, 140 GiB) copied, 214.96 s, 698 MB/s
698 MB/s write with mdadm level0 array. FAST !!

I'll start test next week with dedicated ATX power.
I just used my PC as power supply and it makes me little bit nervous.
I could not find old atx power supply.

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller

Post by tkaiser »

zupet wrote:
Sat Apr 09, 2022 1:44 am
698 MB/s write with mdadm level0 array. FAST !!
IMO not really fast for this setup. I would've expected numbers twice as high. Can you repeat the test with these settings please and show 'lspci -vv' output to confirm a Gen3 x2 link has been established (also check dmesg after the test for anything strange):

Code: Select all

sudo dd if=/dev/zero of=/dev/md0 bs=16384k count=100 oflag=direct status=progress

domih
Posts: 583
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, N2+, H2, H2+, C4, HC4, M1 - 1GbE, 2.5GbE, 10GbE, 45+ GbE
Location: San Francisco Bay Area
Has thanked: 189 times
Been thanked: 188 times
Contact:

Re: JMB585 5 port SATA controller

Post by domih »

@zupet

Could you issue a:

Code: Select all

sudo lspci -vv -s <bus_addr> | grep Width
replace <bus_addr> with your card bus address.

The LnkCap line will tell you what the card "would love" to get in terms of # of lanes.
The LnkSta lines will tell you what it gets.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tobetter wrote:
Sat Apr 09, 2022 1:31 am
@zupet, not sure if JMB858 is detected after the 'reserved-memory' is removed from the device tree. Would you be able to capture the kernel log with dmesg | grep pci when the reserved memory is in the device tree file and not? Also, highly suggest you to run SATA disk long enough to see if SATA disks are disappeared. Probably around 10 min is good enough to test like compiling the Linux kernel.
stock kernel

Code: Select all

[    0.000000] Kernel command line: root=/dev/mmcblk1p2 rootwait ro quiet earlycon=uart8250,mmio32,0xfe660000 console=tty1 pci=nomsi fsck.mode=force fsck.repair=yes mtdparts=sfc_nor:0x20000@0xe0000(env),0x200000@0x100000(uboot),0x100000@0x300000(splash),0xc00000@0x400000(firmware) console=ttyFIQ0 
[    0.114561] reg-fixed-voltage pcie30-avdd0v9: Looking up vin-supply from device tree
[    0.114573] pcie30_avdd0v9: supplied by vcc3v3_sys
[    0.114637] pcie30_avdd0v9: 900 mV 
[    0.114890] reg-fixed-voltage pcie30-avdd0v9: pcie30_avdd0v9 supplying 900000uV
[    0.115035] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[    0.115049] pcie30_avdd1v8: supplied by vcc3v3_sys
[    0.115111] pcie30_avdd1v8: 1800 mV 
[    0.115375] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[    0.115544] vcc3v3_pcie: 3300 mV 
[    0.115742] vcc3v3_pcie: supplied by dc_12v
[    0.115882] reg-fixed-voltage gpio-regulator: vcc3v3_pcie supplying 3300000uV
[    0.589928] rk-pcie 3c0800000.pcie: Looking up vpcie3v3-supply from device tree
[    0.590057] rk-pcie 3c0800000.pcie: Linked as a consumer to regulator.9
[    0.595680] rk-pcie 3c0800000.pcie: missing legacy IRQ resource
[    0.595697] rk-pcie 3c0800000.pcie: Missing *config* reg space
[    0.595755] rk-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    0.595772] rk-pcie 3c0800000.pcie: Parsing ranges property...
[    0.595797] rk-pcie 3c0800000.pcie:   err 0x380000000..0x3807fffff -> 0x80000000
[    0.595815] rk-pcie 3c0800000.pcie:    IO 0x380800000..0x3808fffff -> 0x80800000
[    0.595832] rk-pcie 3c0800000.pcie:   MEM 0x380900000..0x3bfffffff -> 0x80900000
[    0.658760] ehci-pci: EHCI PCI platform driver
[    1.606220] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    2.619553] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[    3.632894] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x2
[    4.646417] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[    5.659562] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[    6.672906] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[    7.687192] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x2
[    8.699562] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[    9.712892] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[   10.726219] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[   11.739547] rk-pcie 3c0800000.pcie: PCIe Link Fail
[   11.739638] rk-pcie 3c0800000.pcie: failed to initialize host
[   32.408535] vcc3v3_pcie: disabling
modified kernel

Code: Select all

[    0.000000] Kernel command line: root=/dev/mmcblk1p2 rootwait ro quiet earlycon=uart8250,mmio32,0xfe660000 console=tty1 pci=nomsi fsck.mode=force fsck.repair=yes mtdparts=sfc_nor:0x20000@0xe0000(env),0x200000@0x100000(uboot),0x100000@0x300000(splash),0xc00000@0x400000(firmware) console=ttyFIQ0 
[    0.115043] reg-fixed-voltage pcie30-avdd0v9: Looking up vin-supply from device tree
[    0.115056] pcie30_avdd0v9: supplied by vcc3v3_sys
[    0.115119] pcie30_avdd0v9: 900 mV 
[    0.115372] reg-fixed-voltage pcie30-avdd0v9: pcie30_avdd0v9 supplying 900000uV
[    0.115517] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[    0.115531] pcie30_avdd1v8: supplied by vcc3v3_sys
[    0.115591] pcie30_avdd1v8: 1800 mV 
[    0.115856] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[    0.116027] vcc3v3_pcie: 3300 mV 
[    0.116265] vcc3v3_pcie: supplied by dc_12v
[    0.116401] reg-fixed-voltage gpio-regulator: vcc3v3_pcie supplying 3300000uV
[    4.658435] rk-pcie 3c0800000.pcie: Looking up vpcie3v3-supply from device tree
[    4.658565] rk-pcie 3c0800000.pcie: Linked as a consumer to regulator.9
[    4.664210] rk-pcie 3c0800000.pcie: missing legacy IRQ resource
[    4.664227] rk-pcie 3c0800000.pcie: Missing *config* reg space
[    4.664283] rk-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    4.664301] rk-pcie 3c0800000.pcie: Parsing ranges property...
[    4.664323] rk-pcie 3c0800000.pcie:   err 0x380000000..0x3807fffff -> 0x80000000
[    4.664342] rk-pcie 3c0800000.pcie:    IO 0x380800000..0x3808fffff -> 0x80800000
[    4.664359] rk-pcie 3c0800000.pcie:   MEM 0x380900000..0x3bfffffff -> 0x80900000
[    4.728297] ehci-pci: EHCI PCI platform driver
[    5.680080] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    6.800108] rk-pcie 3c0800000.pcie: PCIe Link up, LTSSM is 0x230011
[    6.800275] rk-pcie 3c0800000.pcie: PCI host bridge to bus 0002:20
[    6.800284] pci_bus 0002:20: root bus resource [bus 20-2f]
[    6.800289] pci_bus 0002:20: root bus resource [??? 0x380000000-0x3807fffff flags 0x0] (bus address [0x80000000-0x807fffff])
[    6.800294] pci_bus 0002:20: root bus resource [io  0x0000-0xfffff] (bus address [0x80800000-0x808fffff])
[    6.800299] pci_bus 0002:20: root bus resource [mem 0x380900000-0x3bfffffff] (bus address [0x80900000-0xbfffffff])
[    6.800304] pci_bus 0002:20: scanning bus
[    6.800328] pci 0002:20:00.0: [1d87:3566] type 01 class 0x060400
[    6.800352] pci 0002:20:00.0: reg 0x10: [mem 0x00000000-0x3fffffff 64bit]
[    6.800359] pci 0002:20:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    6.800402] pci 0002:20:00.0: supports D1 D2
[    6.800406] pci 0002:20:00.0: PME# supported from D0 D1 D3hot
[    6.800412] pci 0002:20:00.0: PME# disabled
[    6.800440] PCI: OF: /pcie@fe280000: msi-map, using mask ffffffff, rid-base: 00002000, out-base: 00002000, length: 00001000, rid: 00002000 -> 00002000
[    6.805129] pci_bus 0002:20: fixups for bus
[    6.805143] pci 0002:20:00.0: scanning [bus 01-ff] behind bridge, pass 0
[    6.805146] pci 0002:20:00.0: Primary bus is hard wired to 0
[    6.805150] pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
[    6.805160] pci 0002:20:00.0: scanning [bus 00-00] behind bridge, pass 1
[    6.805321] pci_bus 0002:21: scanning bus
[    6.805404] pci 0002:21:00.0: [197b:0585] type 00 class 0x010601
[    6.805527] pci 0002:21:00.0: reg 0x10: initial BAR value 0x00000000 invalid
[    6.805531] pci 0002:21:00.0: reg 0x10: [io  size 0x0080]
[    6.805563] pci 0002:21:00.0: reg 0x14: initial BAR value 0x00000000 invalid
[    6.805566] pci 0002:21:00.0: reg 0x14: [io  size 0x0080]
[    6.805597] pci 0002:21:00.0: reg 0x18: initial BAR value 0x00000000 invalid
[    6.805600] pci 0002:21:00.0: reg 0x18: [io  size 0x0080]
[    6.805630] pci 0002:21:00.0: reg 0x1c: initial BAR value 0x00000000 invalid
[    6.805633] pci 0002:21:00.0: reg 0x1c: [io  size 0x0080]
[    6.805664] pci 0002:21:00.0: reg 0x20: initial BAR value 0x00000000 invalid
[    6.805667] pci 0002:21:00.0: reg 0x20: [io  size 0x0080]
[    6.805698] pci 0002:21:00.0: reg 0x24: [mem 0x00000000-0x00001fff]
[    6.805729] pci 0002:21:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    6.806045] pci 0002:21:00.0: PME# supported from D3hot
[    6.806061] pci 0002:21:00.0: PME# disabled
[    6.806165] PCI: OF: /pcie@fe280000: msi-map, using mask ffffffff, rid-base: 00002000, out-base: 00002000, length: 00001000, rid: 00002100 -> 00002100
[    6.811775] pci_bus 0002:21: fixups for bus
[    6.811784] pci_bus 0002:21: bus scan returning with max=21
[    6.811795] pci_bus 0002:21: busn_res: [bus 21-2f] end is updated to 21
[    6.811804] pci_bus 0002:20: bus scan returning with max=21
[    6.811836] pci 0002:20:00.0: BAR 0: no space for [mem size 0x40000000 64bit]
[    6.811840] pci 0002:20:00.0: BAR 0: failed to assign [mem size 0x40000000 64bit]
[    6.811845] pci 0002:20:00.0: BAR 8: assigned [mem 0x380900000-0x3809fffff]
[    6.811850] pci 0002:20:00.0: BAR 6: assigned [mem 0x380a00000-0x380a0ffff pref]
[    6.811855] pci 0002:20:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[    6.811864] pci 0002:21:00.0: BAR 6: assigned [mem 0x380900000-0x38090ffff pref]
[    6.811868] pci 0002:21:00.0: BAR 5: assigned [mem 0x380910000-0x380911fff]
[    6.811886] pci 0002:21:00.0: BAR 0: assigned [io  0x1000-0x107f]
[    6.811902] pci 0002:21:00.0: BAR 1: assigned [io  0x1080-0x10ff]
[    6.811917] pci 0002:21:00.0: BAR 2: assigned [io  0x1100-0x117f]
[    6.811933] pci 0002:21:00.0: BAR 3: assigned [io  0x1180-0x11ff]
[    6.811948] pci 0002:21:00.0: BAR 4: assigned [io  0x1200-0x127f]
[    6.811965] pci 0002:20:00.0: PCI bridge to [bus 21]
[    6.811969] pci 0002:20:00.0:   bridge window [io  0x1000-0x1fff]
[    6.811974] pci 0002:20:00.0:   bridge window [mem 0x380900000-0x3809fffff]
[    6.812167] pcieport 0002:20:00.0: assign IRQ: got 98
[    6.812350] pcieport 0002:20:00.0: Signaling PME with IRQ 98
I'll test array at few days later.
I just ordered new power for my PC :-)
These users thanked the author zupet for the post:
tobetter (Sat Apr 09, 2022 11:49 am)

brad
Posts: 1595
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 C4 HC4 N1 N2 N2+ H2 H2+ M1 (64 bit ftw)
Location: Australia
Has thanked: 178 times
Been thanked: 251 times
Contact:

Re: JMB585 5 port SATA controller

Post by brad »

I've always been very suspect on those M.2 JMB585 cards mainly because they will struggle to dissipate any heat. I do wonder if they have some internal thermal control.

I don't think I would trust my data with them. I have a JMB363 on a full size pcie x1 card, I will do some tests with that at some stage. (It's a slow interface on the JMB363)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

brad wrote:
Sat Apr 09, 2022 9:51 am
I've always been very suspect on those M.2 JMB585 cards mainly because they will struggle to dissipate any heat. I do wonder if they have some internal thermal control.
thoses disk are stocked to replace my 4 years old array. M1 and JMB585 was not planed and this build is for test.

but if it works well, it will use as cold storage with old disks.
These users thanked the author zupet for the post:
brad (Sat Apr 09, 2022 10:53 am)

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller

Post by tkaiser »

brad wrote:
Sat Apr 09, 2022 9:51 am
I've always been very suspect on those M.2 JMB585 cards mainly because they will struggle to dissipate any heat. I do wonder if they have some internal thermal control.
I don't think so based on reviews of those cards reporting them as suddenly being dead and based on the fact that PCIe cards with this chip in countries where the seller actually has to deal with warranty handling usually look like this (most probably mounted on a PCB with copper ground plane to also dissipate the controller's heat into the PCB):

Image

I don't think @zupet's small heatsink will do the job. At least not without a fan nearby blowing a little air laterally through the heatsink fins :)
Last edited by tkaiser on Sat Apr 09, 2022 6:04 pm, edited 1 time in total.

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller

Post by tkaiser »

zupet wrote:
Sat Apr 09, 2022 10:20 am
if it works well, it will use as cold storage with old disks.
Then with a heatsink that small on the controller it might be an idea to artificially limit the controller's bandwidth by adjusting PCIe link width and/or speed (though the 700 MB/s you reported already smell like Gen2 x2 or Gen3 x1).

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

domih wrote:
Sat Apr 09, 2022 6:34 am
@zupet

Could you issue a:

Code: Select all

sudo lspci -vv -s <bus_addr> | grep Width
replace <bus_addr> with your card bus address.

The LnkCap line will tell you what the card "would love" to get in terms of # of lanes.
The LnkSta lines will tell you what it gets.

Code: Select all

odroid@server:~$ sudo lspci -vv -s 0002:21:00.0 | grep Width
lspci: Unable to load libkmod resources: error -12
		LnkCap:	Port #0, Speed 8GT/s, Width x2, ASPM not supported
		LnkSta:	Speed 8GT/s (ok), Width x2 (ok)
it looks good.
Last edited by zupet on Sat Apr 09, 2022 11:57 pm, edited 1 time in total.
These users thanked the author zupet for the post (total 2):
domih (Sun Apr 10, 2022 1:41 am) • m.one (Thu May 26, 2022 6:05 am)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tkaiser wrote:
Sat Apr 09, 2022 3:56 pm
brad wrote:
Sat Apr 09, 2022 9:51 am
I've always been very suspect on those M.2 JMB585 cards mainly because they will struggle to dissipate any heat. I do wonder if they have some internal thermal control.
I don't think so based on reviews of those cards reporting them as suddenly being dead and based on the fact that PCIe cards with this chip in countries where the seller actually has to deal with warranty handling usually look like this (most probably mounted on a PCB with copper ground plane to also dissipate the controller's heat into the PCB):

Image

I don't think @zupet's small heatsink will do the job. At least not without a fan nearby blowing a little air laterally through the heatsink fins :)
https://forums.unraid.net/topic/41340-s ... ity-check/

someone measured real power usage of SATA controllers.
if it's really 2w, it will work well with only heat sink.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

zupet wrote:
Sat Apr 09, 2022 11:56 pm
someone measured real power usage of SATA controllers.
if it's really 2w, it will work well with only heat sink.
damn it's hot with single disk write test. :-(
I can't measure temporature, but it's not possible to put finger more than 2 seconds.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

zupet wrote:
Sun Apr 10, 2022 12:03 am
zupet wrote:
Sat Apr 09, 2022 11:56 pm
someone measured real power usage of SATA controllers.
if it's really 2w, it will work well with only heat sink.
damn it's hot with single disk write test. :-(
I can't measure temporature, but it's not possible to put finger more than 2 seconds.

Image

got my thermal camera. it has focus issue but it still measure temperature well. controller close up.
These users thanked the author zupet for the post:
odroid (Sun Apr 10, 2022 10:04 am)

User avatar
rooted
Posts: 9447
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: JMB585 5 port SATA controller

Post by rooted »

Yeah that's quite hot at 60+

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

PigLover and tobetter made zfs run on M1. I could not wait to test it.

p.s.screenshot removed. comress was on, didn't written to disk much. I attached real uncompressed result.
Last edited by zupet on Tue Apr 12, 2022 4:40 pm, edited 6 times in total.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Image

Image

Image

it report max 68C and stable for now. (third image)
I've ordered larger heat sink from aliexpress.

p.s.rotated thermal images.
These users thanked the author zupet for the post:
odroid (Tue Apr 12, 2022 2:24 pm)

User avatar
rooted
Posts: 9447
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: JMB585 5 port SATA controller

Post by rooted »

It's good that you are finally getting closer to what you set out to put together.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        9.43M  65.5T      0    792     21  3.54M
Test        4.13M  65.5T      0    899      0  4.06M
Test        6.88M  65.5T      0    914      0  4.13M
I don't know why but it's getting faster. :roll:

User avatar
rooted
Posts: 9447
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: JMB585 5 port SATA controller

Post by rooted »

Cooled off perhaps?

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

rooted wrote:
Tue Apr 12, 2022 4:12 pm
Cooled off perhaps?
something wrong. it's look like compression is enabled.

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller

Post by tkaiser »

zupet wrote:
Tue Apr 12, 2022 12:24 pm
raidz is like raid5, using 5 data + 1 parity.
i know you're only testing but for anybody who comes accross this and thinks RAIDz (especially really RAIDz and not RAIDz2/RAIDz3) is a good idea: nope, it's not. It's a horribly stupid idea especially with large HDDs and here's why: https://jrs-s.net/2015/02/06/zfs-you-sh ... not-raidz/

At least when using a recent OpenZFS version a RAIDz resilver isn't pure random I/O any more (which could have and actually did kill many RAIDz HDD arrays on Linux while resilvering).

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tkaiser wrote:
Tue Apr 12, 2022 4:20 pm
zupet wrote:
Tue Apr 12, 2022 12:24 pm
raidz is like raid5, using 5 data + 1 parity.
i know you're only testing but for anybody who comes accross this and thinks RAIDz (especially really RAIDz and not RAIDz2/RAIDz3) is a good idea: nope, it's not. It's a horribly stupid idea especially with large HDDs and here's why: https://jrs-s.net/2015/02/06/zfs-you-sh ... not-raidz/

At least when using a recent OpenZFS version a RAIDz resilver isn't pure random I/O any more (which could have and actually did kill many RAIDz HDD arrays on Linux while resilvering).
thats why I have two mirrored raidz. zfs send / receive is really good feature.
looks stupid, but I had tons of raid broke, disk fail too. ;-)

to use fast resilver, you need to change to dRAID.
In my opinion, it's for large block storage, service for large vm things.
not for my usage.
Last edited by zupet on Tue Apr 12, 2022 4:38 pm, edited 1 time in total.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Image

Image

Image

Uncompressed ZFS write speed.

I was wondering why it's faster than my PC :-)

I'm still satisfied, it's fast enough to handle 2.5Gbps network speed.

I wonder zfs uses arm neon instructions.
These users thanked the author zupet for the post:
m.one (Thu May 26, 2022 6:11 am)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test         419G  65.1T      0  1.20K    488  29.1M
Test        1.84T  63.6T      0  6.91K  2.46K   418M
Test        3.26T  62.2T      0  7.63K  1.80K   414M
two hours passed.
2.9TB written, controller is 75~80C.

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller

Post by tobetter »

zupet wrote:
Tue Apr 12, 2022 6:41 pm

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test         419G  65.1T      0  1.20K    488  29.1M
Test        1.84T  63.6T      0  6.91K  2.46K   418M
Test        3.26T  62.2T      0  7.63K  1.80K   414M
two hours passed.
2.9TB written, controller is 75~80C.
Good stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.

Code: Select all

193 &reserved_memory {
194         pcie3x2@80900000{
195                 reg = <0x0 0x80900000 0x0 0x100000>;
196         };
197 };

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tobetter wrote:
Tue Apr 12, 2022 6:47 pm
Good stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.

Code: Select all

193 &reserved_memory {
194         pcie3x2@80900000{
195                 reg = <0x0 0x80900000 0x0 0x100000>;
196         };
197 };
yes. I'm using that modified kernel.
I can't enable controller with that reserved_memory.

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller

Post by tobetter »

zupet wrote:
Tue Apr 12, 2022 6:52 pm
tobetter wrote:
Tue Apr 12, 2022 6:47 pm
Good stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.

Code: Select all

193 &reserved_memory {
194         pcie3x2@80900000{
195                 reg = <0x0 0x80900000 0x0 0x100000>;
196         };
197 };
yes. I'm using that modified kernel.
I can't enable controller with that reserved_memory.
Thank you...and this makes me a homework. :lol:
Did you modify a kernel source file other than device tree for your PCIe?

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tobetter wrote:
Tue Apr 12, 2022 6:57 pm
Thank you...and this makes me a homework. :lol:
Did you modify a kernel source file other than device tree for your PCIe?
nop. only removed that few lines.

User avatar
tobetter
Posts: 9838
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 627 times
Been thanked: 1658 times
Contact:

Re: JMB585 5 port SATA controller

Post by tobetter »

zupet wrote:
Tue Apr 12, 2022 7:08 pm
tobetter wrote:
Tue Apr 12, 2022 6:57 pm
Thank you...and this makes me a homework. :lol:
Did you modify a kernel source file other than device tree for your PCIe?
nop. only removed that few lines.
Sure...thank you, glad to know it fixes... :D

brad
Posts: 1595
Joined: Tue Mar 29, 2016 1:22 pm
languages_spoken: english
ODROIDs: C2 C4 HC4 N1 N2 N2+ H2 H2+ M1 (64 bit ftw)
Location: Australia
Has thanked: 178 times
Been thanked: 251 times
Contact:

Re: JMB585 5 port SATA controller

Post by brad »

zupet wrote:
Tue Apr 12, 2022 4:37 pm
I wonder zfs uses arm neon instructions.
Yes I remember looking into that years ago on a C2 and it's likely improved since then viewtopic.php?p=171067#p171067

I recall one of the last optimizations to implement on arm64 was proper mirror load balancing, I can't find their old list of features for arm that are in progress.

PS: neon works

Code: Select all

odroid@gnome-desktop:~/linux$ cat /proc/spl/kstat/zfs/fletcher_4_bench 
0 0 0x01 -1 0 11231561409 5964552280832
implementation   native         byteswap       
scalar           1187840386     1037793919     
superscalar      1847439753     1507395640     
superscalar4     2536613447     1900288026     
aarch64_neon     1571184259     1376981993     
fastest          superscalar4   superscalar4   

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

brad wrote:
Tue Apr 12, 2022 8:17 pm
zupet wrote:
Tue Apr 12, 2022 4:37 pm
I wonder zfs uses arm neon instructions.
Yes I remember looking into that years ago on a C2 and it's likely improved since then viewtopic.php?p=171067#p171067

I recall one of the last optimizations to implement on arm64 was proper mirror load balancing, I can't find their old list of features for arm that are in progress.

PS: neon works

Code: Select all

odroid@gnome-desktop:~/linux$ cat /proc/spl/kstat/zfs/fletcher_4_bench 
0 0 0x01 -1 0 11231561409 5964552280832
implementation   native         byteswap       
scalar           1187840386     1037793919     
superscalar      1847439753     1507395640     
superscalar4     2536613447     1900288026     
aarch64_neon     1571184259     1376981993     
fastest          superscalar4   superscalar4   

Code: Select all

odroid@server:~$ cat /proc/spl/kstat/zfs/fletcher_4_bench
0 0 0x01 -1 0 132571607217 34004011043952
implementation   native         byteswap
scalar           1104830371     961325607
superscalar      1715842121     1401469063
superscalar4     2365597656     1764258478
aarch64_neon     1455955673     1282024681
fastest          superscalar4   superscalar4
odroid@server:~$
from M1. it's using SIMD !!
I need 8 core M2 :P
These users thanked the author zupet for the post:
brad (Tue Apr 12, 2022 10:42 pm)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test         419G  65.1T      0  1.20K    488  29.1M
Test        1.84T  63.6T      0  6.91K  2.46K   418M
Test        3.26T  62.2T      0  7.63K  1.80K   414M
Test        4.66T  60.8T      0  8.40K  1.42K   408M
Test        6.04T  59.4T      0  8.75K  1.37K   405M
Test        7.41T  58.1T      0  9.28K  1.70K   399M
Test        8.77T  56.7T      0  9.50K  1.62K   397M
Test        10.1T  55.3T      0  9.53K  1.25K   396M
Test        11.5T  54.0T      0  9.75K  1.45K   390M
Test        12.8T  52.7T      0  9.90K  1.52K   385M
Test        14.1T  51.4T      0  9.84K  1.34K   379M
Test        15.3T  50.1T      0  9.78K  1.26K   372M
Test        16.6T  48.9T      0  9.52K  1.16K   363M
Test        17.8T  47.7T      0  9.23K    946   353M
Test        19.0T  46.5T      0  8.95K  1.22K   344M
Test        20.1T  45.4T      0  8.47K  1.04K   328M
Test        21.3T  44.2T      0  8.66K  1.67K   347M
Test        22.6T  42.8T      0  8.47K  2.70K   392M

Code: Select all

odroid@server:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.7G     0  3.7G   0% /dev
tmpfs           747M  1.2M  746M   1% /run
/dev/mmcblk1p2  232G  3.5G  220G   2% /
tmpfs           3.7G     0  3.7G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           3.7G     0  3.7G   0% /sys/fs/cgroup
/dev/mmcblk1p1  248M   34M  202M  15% /boot
tmpfs           747M     0  747M   0% /run/user/1000
Test             53T   19T   34T  36% /Test
passed 17 hours, stable. filled 1/3 of array.

I made inch of air gap between HDDs and M1.
Hottest spot of HDD was around 60C and M1's heat sink got really hot too.
The gap mad M1 cooler and JMB585 's temperature drop to 68~70C.

I ordered two 80mm fans. it will cool HDDs and M1.
These users thanked the author zupet for the post:
odroid (Wed Apr 13, 2022 11:06 am)

User avatar
odroid
Site Admin
Posts: 39120
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2514 times
Been thanked: 1382 times
Contact:

Re: JMB585 5 port SATA controller

Post by odroid »

@zupet,

Where did you buy the 6 HDDs mountable plastic rack?
It looks like a customized laser cut acrylic thing.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

odroid wrote:
Wed Apr 13, 2022 11:08 am
@zupet,

Where did you buy the 6 HDDs mountable plastic rack?
It looks like a customized laser cut acrylic thing.
https://www.aliexpress.com/item/1005003818836045.html

6 bay at $9.24

User avatar
odroid
Site Admin
Posts: 39120
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2514 times
Been thanked: 1382 times
Contact:

Re: JMB585 5 port SATA controller

Post by odroid »

Thanks,
Quite affordable. :o

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Image

I got new power supply for PC and old ATX power assigned to M1 and array.
I put M1 on power supply's fan and it really cools well. JMB585 shows only 48C compare to HDD's 54C.

controller is less busy too.
I'm doing network zfs volume copy with send/recv command.
it sends zfs snapshot via ssh.

Code: Select all

zupet@KWONIL:/media/Backup$ sudo zfs send Backup@20220409 | ssh root@192.168.1.200 zfs recv -F Test
root@192.168.1.200's password: 
it shows 69.8MB/s of data write, real data transfer is about 5/6, 58.3 MB/s.

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        19.0G  65.5T     64     98   962K  7.56M
Test         261G  65.2T      0    425  1.28K  69.8M
sshd is using 97% of CPU. I should use less encryption.
Last edited by zupet on Wed Apr 13, 2022 7:05 pm, edited 1 time in total.
These users thanked the author zupet for the post:
tobetter (Wed Apr 13, 2022 5:52 pm)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

zupet wrote:
Wed Apr 13, 2022 5:51 pm
sshd is using 97% of CPU. I should use less encryption.

Code: Select all

zupet@KWONIL:/media/Backup$ dd if=/dev/zero count=10000 bs=1M | ssh -c aes128-gcm@openssh.com root@192.168.1.200 "cat >/dev/null"
root@192.168.1.200's password: 
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 102.828 s, 102 MB/s
zupet@KWONIL:/media/Backup$ dd if=/dev/zero count=10000 bs=1M | ssh root@192.168.1.200 "cat >/dev/null"
root@192.168.1.200's password: 
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 149.304 s, 70.2 MB/s
zupet@KWONIL:/media/Backup$ 
cipher aes128-gcm@openssh.com works. great.

tkaiser
Posts: 752
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1
Has thanked: 0
Been thanked: 20 times
Contact:

Re: JMB585 5 port SATA controller

Post by tkaiser »

zupet wrote:
Wed Apr 13, 2022 5:51 pm
sshd is using 97% of CPU. I should use less encryption.
Or even none? Netcat to the rescue! Or at least bbcp (combined with mbuffer).

The acrylic HDD mount... does this result in significant drive vibration? Have you checked whether SMART output of your HDDs contains any attributes hinting to vibration related problems?

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

tkaiser wrote:
Wed Apr 13, 2022 6:38 pm
zupet wrote:
Wed Apr 13, 2022 5:51 pm
sshd is using 97% of CPU. I should use less encryption.
Or even none? Netcat to the rescue! Or at least bbcp (combined with mbuffer).

Code: Select all

odroid@server:~$ zpool iostat 60
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        4.92G  65.5T     18    242   272K  37.4M
Test        10.4G  65.5T      0    608  1.60K  95.4M
Test        15.9G  65.5T      0    580  3.00K  94.8M
Test        21.4G  65.4T      0    618  3.00K  95.4M
I got better result.
I'll post same 3600 second iostat result.
tkaiser wrote:
Wed Apr 13, 2022 6:38 pm
The acrylic HDD mount... does this result in significant drive vibration? Have you checked whether SMART output of your HDDs contains any attributes hinting to vibration related problems?

Code: Select all

odroid@server:~$ sudo sma -a /dev/sdc
smartctl 7.1 2019-12-30 r5022 [aarch64-linux-4.19.219-odroid-arm64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD120EDBZ-11B1HA0
Serial Number:    5QGUKWVF
LU WWN Device Id: 5 000cca 2b0cba077
Firmware Version: 85.00A85
User Capacity:    12,000,138,625,024 bytes [12.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 13 09:54:19 2022 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (   87) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1335) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   016    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0004   126   126   054    Old_age   Offline      -       116
  3 Spin_Up_Time            0x0007   184   184   024    Pre-fail  Always       -       364 (Average 350)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       12
  5 Reallocated_Sector_Ct   0x0033   100   100   005    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000a   100   100   067    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0004   128   128   020    Old_age   Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       75
 10 Spin_Retry_Count        0x0012   100   100   060    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       11
 22 Unknown_Attribute       0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       18
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       18
194 Temperature_Celsius     0x0002   110   110   000    Old_age   Always       -       59 (Min/Max 24/62)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
no vibration data.
this disks are low rpm helium sealed model. less vibration and noise, I think.

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Code: Select all

odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        23.1G  65.4T     18    249   267K  38.5M
Test         365G  65.1T      0    624  1.82K  98.5M
Test         691G  64.8T      0    583    362  94.0M
Test        1011G  64.5T      0    543    278  92.3M
Test        1.29T  64.2T      0    520    104  88.6M
Test        1.60T  63.9T      0    559     17  91.5M
Test        1.91T  63.6T      0    513     17  91.7M
Test        2.21T  63.3T      0    501      0  90.3M
Test        2.53T  62.9T      0    561      0  92.9M
Test        2.85T  62.6T      0    592      0  92.9M
Test        3.16T  62.3T      0    603      0  94.1M
Test        3.48T  62.0T      0    589      0  93.1M
Test        3.80T  61.7T      0    610      0  94.1M
Test        4.12T  61.4T      0    608      0  93.3M
Test        4.43T  61.0T      0    591      0  92.3M
Test        4.74T  60.7T      0    570      0  92.7M
Test        5.05T  60.4T      0    570      0  91.8M
Test        5.37T  60.1T      0    566      0  92.1M
Test        5.68T  59.8T      0    574      0  92.3M
Test        5.99T  59.5T      0    549      0  92.1M
Test        6.30T  59.2T      0    553      0  92.2M
Test        6.63T  58.8T      0    576      0  95.2M
Test        6.96T  58.5T      0    624      0  97.2M
Test        7.29T  58.2T      0    628      0  97.5M
Test        7.61T  57.9T      0    534      0  94.2M
Test        7.94T  57.5T      0    558      0  97.6M
Test        8.27T  57.2T      0    551      0  97.2M
Test        8.59T  56.9T      0    541      0  95.4M
Test        8.92T  56.6T      0    548      0  96.5M
Test        9.24T  56.2T      0    545    169  96.2M
Test        9.56T  55.9T      0    530    497  93.9M
Test        9.88T  55.6T      0    535    384  93.6M
Test        10.2T  55.3T      0    531    137  93.3M
Test        10.5T  55.0T      0    528    180  93.4M
Test        10.8T  54.6T      0    532    135  93.8M
Test        11.1T  54.3T      0    527    141  92.7M
Test        11.5T  54.0T      0    547    160  91.9M
Test        11.8T  53.7T      0    546    127  91.7M
Test        12.1T  53.4T      0    540    128  92.1M
Test        12.4T  53.1T      0    544    134  92.0M
Test        12.7T  52.8T      0    570    114  93.1M
Test        13.0T  52.4T      0    572    105  93.4M
Test        13.3T  52.1T      0    568     94  93.3M
Test        13.7T  51.8T      0    562     94  91.7M
^C // accidentally pressed Ctrl+C
odroid@server:~$ zpool iostat 3600
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        13.9T  51.6T      1    543  15.7K  90.3M
Test        14.2T  51.3T      0    570     87  93.5M
Test        14.5T  51.0T      0    572     69  93.2M
Test        14.8T  50.7T      0    572     70  92.9M
Test        15.1T  50.3T      0    573     92  93.5M
Test        15.4T  50.0T      0    566     88  92.8M
Test        15.8T  49.7T      0    568     96  93.4M
Test        16.1T  49.4T      0    570     61  92.9M
Test        16.4T  49.1T      0    577     81  93.6M
Test        16.7T  48.8T      0    565     67  92.9M
Test        17.0T  48.4T      0    556     68  92.7M
Test        17.3T  48.1T      0    550     54  92.6M                                                                                
Test        17.7T  47.8T      0    559     64  93.0M                                                                                
Test        18.0T  47.5T      0    555     71  92.4M                                                                                
Test        18.3T  47.2T      0    591     76  97.3M                                                                                
Test        18.6T  46.8T      0    588     56  97.6M
Test        19.0T  46.5T      0    576     89  97.3M
Test        19.3T  46.2T      0    586     79  97.5M
Test        19.6T  45.9T      0    596     62  97.8M
Test        19.9T  45.5T      0    587     61  97.6M
Test        20.3T  45.2T      0    595     70  97.2M
Test        20.6T  44.9T      0    598     58  97.7M
Test        20.9T  44.5T      0    595     79  97.5M                                                                                
Test        21.3T  44.2T      0    578    547  94.5M                                                                                
Test        21.6T  43.9T      0    588    552  94.8M                                                                                
Test        21.9T  43.6T      0    579    425  93.9M                                                                                
Test        22.2T  43.3T      0    567     98  94.2M                                                                                
Test        22.5T  42.9T      0    563    220  94.4M                                                                                
Test        22.9T  42.6T      0    564     76  92.7M                                                                                
Test        23.2T  42.3T      0    568     54  94.9M                                                                                
Test        23.5T  42.0T      0    568     38  95.0M                                                                                
Test        23.8T  41.7T      0    569     28  95.1M                                                                                
Test        24.1T  41.3T      0    572     43  94.6M                                                                                
Test        24.5T  41.0T      0    556     63  94.2M                                                                                
Test        24.8T  40.7T      0    548     72  93.2M                                                                                
Test        25.1T  40.4T      0    539     91  92.3M                                                                                
Test        25.4T  40.1T      0    539    117  92.4M                                                                                
Test        25.7T  39.8T      0    547    176  93.6M                                                                                
Test        26.0T  39.4T      0    553    145  93.8M                                                                                
Test        26.3T  39.1T      0    558    122  93.8M                                                                                
Test        26.7T  38.8T      0    557     88  94.1M                                                                                
Test        27.0T  38.5T      0    553    109  93.9M                                                                                
Test        27.3T  38.2T      0    540     70  93.0M                                                                                
Test        27.6T  37.9T      0    549     75  94.0M                                                                                
Test        27.9T  37.5T      0    539     75  93.3M                                                                                
Test        28.3T  37.2T      0    556     76  93.7M                                                                                
Test        28.6T  36.9T      0    547    126  92.8M                                                                                
Test        28.9T  36.6T      0    551     43  93.1M                                                                                
Test        29.2T  36.3T      0    559     51  93.6M                                                                                
Test        29.5T  36.0T      0    577     29  93.1M                                                                                
Test        29.8T  35.6T      0    580     35  93.2M                                                                                
Test        30.2T  35.3T      0    563     23  97.1M                                                                                
Test        30.5T  35.0T      0    551     21  95.3M                                                                                
Test        30.8T  34.7T      0    572     29  97.0M                                                                                
Test        31.1T  34.3T      0    574     48  97.4M                                                                                
Test        31.5T  34.0T      0    572     26  98.0M                                                                                
Test        31.8T  33.7T      0    595     31  98.2M      
Test        32.1T  33.3T      0    604     29  98.3M
Test        32.5T  33.0T      0    601     17  98.3M
Test        32.8T  32.7T      0    585     14  94.6M
Test        33.1T  32.4T      0    570    155  93.0M
Test        33.4T  32.1T      0    571    465  92.1M
Test        33.7T  31.7T      0    577    369  93.6M
Test        34.0T  31.4T      0    572    221  93.6M
Test        34.4T  31.1T      0    572    194  93.9M
Test        34.7T  30.8T      0    579     93  94.1M
Test        35.0T  30.5T      0    580    110  94.1M
Test        35.3T  30.1T      0    586    118  94.2M
Test        35.6T  29.8T      0    590    172  94.9M
Test        36.0T  29.5T      0    582     62  94.3M
Test        36.3T  29.2T      0    591     77  94.9M
Test        36.6T  28.9T      0    581     69  93.7M
Test        36.9T  28.5T      0    577     53  93.7M
Test        37.2T  28.2T      0    576     47  93.2M
Test        37.6T  27.9T      0    578     56  93.2M
Test        37.9T  27.6T      0    580     62  94.0M
Test        38.2T  27.3T      0    576     48  94.5M
Test        38.5T  27.0T      0    603    262  95.3M
Test        38.8T  26.6T      0    598     68  95.5M
Test        39.2T  26.3T      0    580    220  94.8M
Test        39.5T  26.0T      0    594    108  95.5M
Test        39.8T  25.7T      0    565    106  93.3M
Test        40.1T  25.4T      0    580    111  93.5M
Test        40.4T  25.0T      0    569    141  93.7M
Test        40.7T  24.7T      0    584    127  94.3M
Test        41.1T  24.4T      0    587    185  94.3M
Test        41.4T  24.1T      0    574    175  94.0M
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        41.7T  23.8T      0    593    166  96.5M
Test        42.0T  23.4T      0    606    174  98.0M
Test        42.4T  23.1T      0    586    131  98.0M
Test        42.7T  22.8T      0    600    138  97.9M
              capacity     operations     bandwidth 
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Test        43.0T  22.4T      0    618    105  98.1M
Test        43.4T  22.1T      0    592     85  97.7M
Test        43.7T  21.8T      0    609     84  97.9M
Test        44.0T  21.4T      0    617     83  97.8M
Test        44.4T  21.1T      0    601     85  94.5M
Test        44.6T  20.9T      0    460    600  73.2M
it's getting slow little bit, but works very well.
controller is stable at 49~50C.
Last edited by zupet on Tue Apr 19, 2022 2:27 pm, edited 1 time in total.
These users thanked the author zupet for the post:
odroid (Thu Apr 14, 2022 1:06 pm)

zupet
Posts: 173
Joined: Tue Dec 26, 2017 11:13 pm
languages_spoken: korean,english
ODROIDs: HC1
Has thanked: 5 times
Been thanked: 30 times
Contact:

Re: JMB585 5 port SATA controller

Post by zupet »

Code: Select all

zupet@KWONIL:/media/Backup$ sudo zfs send Backup@20220409 | ssh -c aes128-gcm@openssh.com root@192.168.1.200 zfs recv -F Test
root@192.168.1.200's password: 
zupet@KWONIL:/media/Backup$ 

Code: Select all

odroid@server:/media/Test/Anime/2022Q2$ df 
Filesystem       1K-blocks        Used   Available Use% Mounted on
udev               3808640           0     3808640   0% /dev
tmpfs               764500        1188      763312   1% /run
/dev/mmcblk1p2   243185620     3854492   229443816   2% /
tmpfs              3822488           0     3822488   0% /dev/shm
tmpfs                 5120           4        5116   1% /run/lock
tmpfs              3822488           0     3822488   0% /sys/fs/cgroup
/dev/mmcblk1p1      253920       34076      206740  15% /boot
tmpfs               764496           0      764496   0% /run/user/1000
Test           56020397440 38258230912 17762166528  69% /media/Test
I can't believe it succeeded.

zfs send/recv copied 35.63 TB of data using ssh, over 93MB/s, six days.
zfs wrote 44.6TB to array. This is my first time sending this large snapshot through ssh.
I'm going to start full hash check.

Image

Image

last night, I fixed my thermal camera.
There was a hole in center of the nvme, now I can confirm it was heat sink.
heat sink is slight cooler than surroundings.
These users thanked the author zupet for the post (total 2):
tobetter (Tue Apr 19, 2022 2:33 pm) • odroid (Tue Apr 19, 2022 2:35 pm)

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 2 guests