tested modified kernel, but still showing this message and controller is not recognized;tobetter wrote: ↑Mon Apr 04, 2022 6:39 pmEDIT: when you have a chance to build your custom kernel, would you try to remove the lines #193 to #197 and try again?
https://github.com/hardkernel/linux/blo ... 1.dts#L193
Code: Select all
odroid@server:~$ lspci
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01)
0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
Awesome!!zupet wrote: ↑Sat Apr 09, 2022 12:34 amIt's working? I dont know why but it suddenly working.Code: Select all
odroid@server:~$ lspci 0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01) 0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
https://github.com/hardkernel/linux/blo ... 1.dts#L193tobetter wrote: ↑Sat Apr 09, 2022 12:37 amAwesome!!zupet wrote: ↑Sat Apr 09, 2022 12:34 amIt's working? I dont know why but it suddenly working.Code: Select all
odroid@server:~$ lspci 0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3566 (rev 01) 0002:21:00.0 SATA controller: JMicron Technology Corp. Device 0585
I suspect the PCIe connector on your ODROID-N2 since your NVMe also had similar problem. Or PCIe card was not properly stting to its position.
OMG, how flimsy the PCB looks. No wonder people are complaining about these M.2 cards everywhere since they look like they can be damaged simply by staring at them
Code: Select all
odroid@server:~$ sudo dd if=/dev/zero of=/dev/md0 bs=512k count=200M oflag=direct status=progress
149386428416 bytes (149 GB, 139 GiB) copied, 214 s, 698 MB/s^C
286252+0 records in
286252+0 records out
150078488576 bytes (150 GB, 140 GiB) copied, 214.96 s, 698 MB/s
IMO not really fast for this setup. I would've expected numbers twice as high. Can you repeat the test with these settings please and show 'lspci -vv' output to confirm a Gen3 x2 link has been established (also check dmesg after the test for anything strange):
Code: Select all
sudo dd if=/dev/zero of=/dev/md0 bs=16384k count=100 oflag=direct status=progressCode: Select all
sudo lspci -vv -s <bus_addr> | grep Widthstock kerneltobetter wrote: ↑Sat Apr 09, 2022 1:31 am@zupet, not sure if JMB858 is detected after the 'reserved-memory' is removed from the device tree. Would you be able to capture the kernel log with dmesg | grep pci when the reserved memory is in the device tree file and not? Also, highly suggest you to run SATA disk long enough to see if SATA disks are disappeared. Probably around 10 min is good enough to test like compiling the Linux kernel.
Code: Select all
[ 0.000000] Kernel command line: root=/dev/mmcblk1p2 rootwait ro quiet earlycon=uart8250,mmio32,0xfe660000 console=tty1 pci=nomsi fsck.mode=force fsck.repair=yes mtdparts=sfc_nor:0x20000@0xe0000(env),0x200000@0x100000(uboot),0x100000@0x300000(splash),0xc00000@0x400000(firmware) console=ttyFIQ0
[ 0.114561] reg-fixed-voltage pcie30-avdd0v9: Looking up vin-supply from device tree
[ 0.114573] pcie30_avdd0v9: supplied by vcc3v3_sys
[ 0.114637] pcie30_avdd0v9: 900 mV
[ 0.114890] reg-fixed-voltage pcie30-avdd0v9: pcie30_avdd0v9 supplying 900000uV
[ 0.115035] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[ 0.115049] pcie30_avdd1v8: supplied by vcc3v3_sys
[ 0.115111] pcie30_avdd1v8: 1800 mV
[ 0.115375] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[ 0.115544] vcc3v3_pcie: 3300 mV
[ 0.115742] vcc3v3_pcie: supplied by dc_12v
[ 0.115882] reg-fixed-voltage gpio-regulator: vcc3v3_pcie supplying 3300000uV
[ 0.589928] rk-pcie 3c0800000.pcie: Looking up vpcie3v3-supply from device tree
[ 0.590057] rk-pcie 3c0800000.pcie: Linked as a consumer to regulator.9
[ 0.595680] rk-pcie 3c0800000.pcie: missing legacy IRQ resource
[ 0.595697] rk-pcie 3c0800000.pcie: Missing *config* reg space
[ 0.595755] rk-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[ 0.595772] rk-pcie 3c0800000.pcie: Parsing ranges property...
[ 0.595797] rk-pcie 3c0800000.pcie: err 0x380000000..0x3807fffff -> 0x80000000
[ 0.595815] rk-pcie 3c0800000.pcie: IO 0x380800000..0x3808fffff -> 0x80800000
[ 0.595832] rk-pcie 3c0800000.pcie: MEM 0x380900000..0x3bfffffff -> 0x80900000
[ 0.658760] ehci-pci: EHCI PCI platform driver
[ 1.606220] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[ 2.619553] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[ 3.632894] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x2
[ 4.646417] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[ 5.659562] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[ 6.672906] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[ 7.687192] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x2
[ 8.699562] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x10007
[ 9.712892] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[ 10.726219] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1000f
[ 11.739547] rk-pcie 3c0800000.pcie: PCIe Link Fail
[ 11.739638] rk-pcie 3c0800000.pcie: failed to initialize host
[ 32.408535] vcc3v3_pcie: disablingCode: Select all
[ 0.000000] Kernel command line: root=/dev/mmcblk1p2 rootwait ro quiet earlycon=uart8250,mmio32,0xfe660000 console=tty1 pci=nomsi fsck.mode=force fsck.repair=yes mtdparts=sfc_nor:0x20000@0xe0000(env),0x200000@0x100000(uboot),0x100000@0x300000(splash),0xc00000@0x400000(firmware) console=ttyFIQ0
[ 0.115043] reg-fixed-voltage pcie30-avdd0v9: Looking up vin-supply from device tree
[ 0.115056] pcie30_avdd0v9: supplied by vcc3v3_sys
[ 0.115119] pcie30_avdd0v9: 900 mV
[ 0.115372] reg-fixed-voltage pcie30-avdd0v9: pcie30_avdd0v9 supplying 900000uV
[ 0.115517] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[ 0.115531] pcie30_avdd1v8: supplied by vcc3v3_sys
[ 0.115591] pcie30_avdd1v8: 1800 mV
[ 0.115856] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[ 0.116027] vcc3v3_pcie: 3300 mV
[ 0.116265] vcc3v3_pcie: supplied by dc_12v
[ 0.116401] reg-fixed-voltage gpio-regulator: vcc3v3_pcie supplying 3300000uV
[ 4.658435] rk-pcie 3c0800000.pcie: Looking up vpcie3v3-supply from device tree
[ 4.658565] rk-pcie 3c0800000.pcie: Linked as a consumer to regulator.9
[ 4.664210] rk-pcie 3c0800000.pcie: missing legacy IRQ resource
[ 4.664227] rk-pcie 3c0800000.pcie: Missing *config* reg space
[ 4.664283] rk-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[ 4.664301] rk-pcie 3c0800000.pcie: Parsing ranges property...
[ 4.664323] rk-pcie 3c0800000.pcie: err 0x380000000..0x3807fffff -> 0x80000000
[ 4.664342] rk-pcie 3c0800000.pcie: IO 0x380800000..0x3808fffff -> 0x80800000
[ 4.664359] rk-pcie 3c0800000.pcie: MEM 0x380900000..0x3bfffffff -> 0x80900000
[ 4.728297] ehci-pci: EHCI PCI platform driver
[ 5.680080] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[ 6.800108] rk-pcie 3c0800000.pcie: PCIe Link up, LTSSM is 0x230011
[ 6.800275] rk-pcie 3c0800000.pcie: PCI host bridge to bus 0002:20
[ 6.800284] pci_bus 0002:20: root bus resource [bus 20-2f]
[ 6.800289] pci_bus 0002:20: root bus resource [??? 0x380000000-0x3807fffff flags 0x0] (bus address [0x80000000-0x807fffff])
[ 6.800294] pci_bus 0002:20: root bus resource [io 0x0000-0xfffff] (bus address [0x80800000-0x808fffff])
[ 6.800299] pci_bus 0002:20: root bus resource [mem 0x380900000-0x3bfffffff] (bus address [0x80900000-0xbfffffff])
[ 6.800304] pci_bus 0002:20: scanning bus
[ 6.800328] pci 0002:20:00.0: [1d87:3566] type 01 class 0x060400
[ 6.800352] pci 0002:20:00.0: reg 0x10: [mem 0x00000000-0x3fffffff 64bit]
[ 6.800359] pci 0002:20:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 6.800402] pci 0002:20:00.0: supports D1 D2
[ 6.800406] pci 0002:20:00.0: PME# supported from D0 D1 D3hot
[ 6.800412] pci 0002:20:00.0: PME# disabled
[ 6.800440] PCI: OF: /pcie@fe280000: msi-map, using mask ffffffff, rid-base: 00002000, out-base: 00002000, length: 00001000, rid: 00002000 -> 00002000
[ 6.805129] pci_bus 0002:20: fixups for bus
[ 6.805143] pci 0002:20:00.0: scanning [bus 01-ff] behind bridge, pass 0
[ 6.805146] pci 0002:20:00.0: Primary bus is hard wired to 0
[ 6.805150] pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
[ 6.805160] pci 0002:20:00.0: scanning [bus 00-00] behind bridge, pass 1
[ 6.805321] pci_bus 0002:21: scanning bus
[ 6.805404] pci 0002:21:00.0: [197b:0585] type 00 class 0x010601
[ 6.805527] pci 0002:21:00.0: reg 0x10: initial BAR value 0x00000000 invalid
[ 6.805531] pci 0002:21:00.0: reg 0x10: [io size 0x0080]
[ 6.805563] pci 0002:21:00.0: reg 0x14: initial BAR value 0x00000000 invalid
[ 6.805566] pci 0002:21:00.0: reg 0x14: [io size 0x0080]
[ 6.805597] pci 0002:21:00.0: reg 0x18: initial BAR value 0x00000000 invalid
[ 6.805600] pci 0002:21:00.0: reg 0x18: [io size 0x0080]
[ 6.805630] pci 0002:21:00.0: reg 0x1c: initial BAR value 0x00000000 invalid
[ 6.805633] pci 0002:21:00.0: reg 0x1c: [io size 0x0080]
[ 6.805664] pci 0002:21:00.0: reg 0x20: initial BAR value 0x00000000 invalid
[ 6.805667] pci 0002:21:00.0: reg 0x20: [io size 0x0080]
[ 6.805698] pci 0002:21:00.0: reg 0x24: [mem 0x00000000-0x00001fff]
[ 6.805729] pci 0002:21:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[ 6.806045] pci 0002:21:00.0: PME# supported from D3hot
[ 6.806061] pci 0002:21:00.0: PME# disabled
[ 6.806165] PCI: OF: /pcie@fe280000: msi-map, using mask ffffffff, rid-base: 00002000, out-base: 00002000, length: 00001000, rid: 00002100 -> 00002100
[ 6.811775] pci_bus 0002:21: fixups for bus
[ 6.811784] pci_bus 0002:21: bus scan returning with max=21
[ 6.811795] pci_bus 0002:21: busn_res: [bus 21-2f] end is updated to 21
[ 6.811804] pci_bus 0002:20: bus scan returning with max=21
[ 6.811836] pci 0002:20:00.0: BAR 0: no space for [mem size 0x40000000 64bit]
[ 6.811840] pci 0002:20:00.0: BAR 0: failed to assign [mem size 0x40000000 64bit]
[ 6.811845] pci 0002:20:00.0: BAR 8: assigned [mem 0x380900000-0x3809fffff]
[ 6.811850] pci 0002:20:00.0: BAR 6: assigned [mem 0x380a00000-0x380a0ffff pref]
[ 6.811855] pci 0002:20:00.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 6.811864] pci 0002:21:00.0: BAR 6: assigned [mem 0x380900000-0x38090ffff pref]
[ 6.811868] pci 0002:21:00.0: BAR 5: assigned [mem 0x380910000-0x380911fff]
[ 6.811886] pci 0002:21:00.0: BAR 0: assigned [io 0x1000-0x107f]
[ 6.811902] pci 0002:21:00.0: BAR 1: assigned [io 0x1080-0x10ff]
[ 6.811917] pci 0002:21:00.0: BAR 2: assigned [io 0x1100-0x117f]
[ 6.811933] pci 0002:21:00.0: BAR 3: assigned [io 0x1180-0x11ff]
[ 6.811948] pci 0002:21:00.0: BAR 4: assigned [io 0x1200-0x127f]
[ 6.811965] pci 0002:20:00.0: PCI bridge to [bus 21]
[ 6.811969] pci 0002:20:00.0: bridge window [io 0x1000-0x1fff]
[ 6.811974] pci 0002:20:00.0: bridge window [mem 0x380900000-0x3809fffff]
[ 6.812167] pcieport 0002:20:00.0: assign IRQ: got 98
[ 6.812350] pcieport 0002:20:00.0: Signaling PME with IRQ 98thoses disk are stocked to replace my 4 years old array. M1 and JMB585 was not planed and this build is for test.
I don't think so based on reviews of those cards reporting them as suddenly being dead and based on the fact that PCIe cards with this chip in countries where the seller actually has to deal with warranty handling usually look like this (most probably mounted on a PCB with copper ground plane to also dissipate the controller's heat into the PCB):
Then with a heatsink that small on the controller it might be an idea to artificially limit the controller's bandwidth by adjusting PCIe link width and/or speed (though the 700 MB/s you reported already smell like Gen2 x2 or Gen3 x1).
domih wrote: ↑Sat Apr 09, 2022 6:34 am@zupet
Could you issue a:
replace <bus_addr> with your card bus address.Code: Select all
sudo lspci -vv -s <bus_addr> | grep Width
The LnkCap line will tell you what the card "would love" to get in terms of # of lanes.
The LnkSta lines will tell you what it gets.
Code: Select all
odroid@server:~$ sudo lspci -vv -s 0002:21:00.0 | grep Width
lspci: Unable to load libkmod resources: error -12
LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM not supported
LnkSta: Speed 8GT/s (ok), Width x2 (ok)
https://forums.unraid.net/topic/41340-s ... ity-check/tkaiser wrote: ↑Sat Apr 09, 2022 3:56 pmI don't think so based on reviews of those cards reporting them as suddenly being dead and based on the fact that PCIe cards with this chip in countries where the seller actually has to deal with warranty handling usually look like this (most probably mounted on a PCB with copper ground plane to also dissipate the controller's heat into the PCB):
I don't think @zupet's small heatsink will do the job. At least not without a fan nearby blowing a little air laterally through the heatsink fins
damn it's hot with single disk write test.
Code: Select all
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 9.43M 65.5T 0 792 21 3.54M
Test 4.13M 65.5T 0 899 0 4.06M
Test 6.88M 65.5T 0 914 0 4.13M
i know you're only testing but for anybody who comes accross this and thinks RAIDz (especially really RAIDz and not RAIDz2/RAIDz3) is a good idea: nope, it's not. It's a horribly stupid idea especially with large HDDs and here's why: https://jrs-s.net/2015/02/06/zfs-you-sh ... not-raidz/
thats why I have two mirrored raidz. zfs send / receive is really good feature.tkaiser wrote: ↑Tue Apr 12, 2022 4:20 pmi know you're only testing but for anybody who comes accross this and thinks RAIDz (especially really RAIDz and not RAIDz2/RAIDz3) is a good idea: nope, it's not. It's a horribly stupid idea especially with large HDDs and here's why: https://jrs-s.net/2015/02/06/zfs-you-sh ... not-raidz/
At least when using a recent OpenZFS version a RAIDz resilver isn't pure random I/O any more (which could have and actually did kill many RAIDz HDD arrays on Linux while resilvering).
Code: Select all
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 419G 65.1T 0 1.20K 488 29.1M
Test 1.84T 63.6T 0 6.91K 2.46K 418M
Test 3.26T 62.2T 0 7.63K 1.80K 414M
Good stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.zupet wrote: ↑Tue Apr 12, 2022 6:41 pmtwo hours passed.Code: Select all
odroid@server:~$ zpool iostat 3600 capacity operations bandwidth pool alloc free read write read write ---------- ----- ----- ----- ----- ----- ----- Test 419G 65.1T 0 1.20K 488 29.1M Test 1.84T 63.6T 0 6.91K 2.46K 418M Test 3.26T 62.2T 0 7.63K 1.80K 414M
2.9TB written, controller is 75~80C.
Code: Select all
193 &reserved_memory {
194 pcie3x2@80900000{
195 reg = <0x0 0x80900000 0x0 0x100000>;
196 };
197 };
yes. I'm using that modified kernel.tobetter wrote: ↑Tue Apr 12, 2022 6:47 pmGood stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.Code: Select all
193 &reserved_memory { 194 pcie3x2@80900000{ 195 reg = <0x0 0x80900000 0x0 0x100000>; 196 }; 197 };
Thank you...and this makes me a homework.zupet wrote: ↑Tue Apr 12, 2022 6:52 pmyes. I'm using that modified kernel.tobetter wrote: ↑Tue Apr 12, 2022 6:47 pmGood stuff...I wonder if you removed this node in rk3568-odroid-m1.dts.Code: Select all
193 &reserved_memory { 194 pcie3x2@80900000{ 195 reg = <0x0 0x80900000 0x0 0x100000>; 196 }; 197 };
I can't enable controller with that reserved_memory.
Yes I remember looking into that years ago on a C2 and it's likely improved since then viewtopic.php?p=171067#p171067
Code: Select all
odroid@gnome-desktop:~/linux$ cat /proc/spl/kstat/zfs/fletcher_4_bench
0 0 0x01 -1 0 11231561409 5964552280832
implementation native byteswap
scalar 1187840386 1037793919
superscalar 1847439753 1507395640
superscalar4 2536613447 1900288026
aarch64_neon 1571184259 1376981993
fastest superscalar4 superscalar4
brad wrote: ↑Tue Apr 12, 2022 8:17 pmYes I remember looking into that years ago on a C2 and it's likely improved since then viewtopic.php?p=171067#p171067
I recall one of the last optimizations to implement on arm64 was proper mirror load balancing, I can't find their old list of features for arm that are in progress.
PS: neon works
Code: Select all
odroid@gnome-desktop:~/linux$ cat /proc/spl/kstat/zfs/fletcher_4_bench 0 0 0x01 -1 0 11231561409 5964552280832 implementation native byteswap scalar 1187840386 1037793919 superscalar 1847439753 1507395640 superscalar4 2536613447 1900288026 aarch64_neon 1571184259 1376981993 fastest superscalar4 superscalar4
Code: Select all
odroid@server:~$ cat /proc/spl/kstat/zfs/fletcher_4_bench
0 0 0x01 -1 0 132571607217 34004011043952
implementation native byteswap
scalar 1104830371 961325607
superscalar 1715842121 1401469063
superscalar4 2365597656 1764258478
aarch64_neon 1455955673 1282024681
fastest superscalar4 superscalar4
odroid@server:~$
Code: Select all
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 419G 65.1T 0 1.20K 488 29.1M
Test 1.84T 63.6T 0 6.91K 2.46K 418M
Test 3.26T 62.2T 0 7.63K 1.80K 414M
Test 4.66T 60.8T 0 8.40K 1.42K 408M
Test 6.04T 59.4T 0 8.75K 1.37K 405M
Test 7.41T 58.1T 0 9.28K 1.70K 399M
Test 8.77T 56.7T 0 9.50K 1.62K 397M
Test 10.1T 55.3T 0 9.53K 1.25K 396M
Test 11.5T 54.0T 0 9.75K 1.45K 390M
Test 12.8T 52.7T 0 9.90K 1.52K 385M
Test 14.1T 51.4T 0 9.84K 1.34K 379M
Test 15.3T 50.1T 0 9.78K 1.26K 372M
Test 16.6T 48.9T 0 9.52K 1.16K 363M
Test 17.8T 47.7T 0 9.23K 946 353M
Test 19.0T 46.5T 0 8.95K 1.22K 344M
Test 20.1T 45.4T 0 8.47K 1.04K 328M
Test 21.3T 44.2T 0 8.66K 1.67K 347M
Test 22.6T 42.8T 0 8.47K 2.70K 392M
Code: Select all
odroid@server:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.7G 0 3.7G 0% /dev
tmpfs 747M 1.2M 746M 1% /run
/dev/mmcblk1p2 232G 3.5G 220G 2% /
tmpfs 3.7G 0 3.7G 0% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 3.7G 0 3.7G 0% /sys/fs/cgroup
/dev/mmcblk1p1 248M 34M 202M 15% /boot
tmpfs 747M 0 747M 0% /run/user/1000
Test 53T 19T 34T 36% /Test
Code: Select all
zupet@KWONIL:/media/Backup$ sudo zfs send Backup@20220409 | ssh root@192.168.1.200 zfs recv -F Test
root@192.168.1.200's password:
Code: Select all
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 19.0G 65.5T 64 98 962K 7.56M
Test 261G 65.2T 0 425 1.28K 69.8M
Code: Select all
zupet@KWONIL:/media/Backup$ dd if=/dev/zero count=10000 bs=1M | ssh -c aes128-gcm@openssh.com root@192.168.1.200 "cat >/dev/null"
root@192.168.1.200's password:
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 102.828 s, 102 MB/s
zupet@KWONIL:/media/Backup$ dd if=/dev/zero count=10000 bs=1M | ssh root@192.168.1.200 "cat >/dev/null"
root@192.168.1.200's password:
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 149.304 s, 70.2 MB/s
zupet@KWONIL:/media/Backup$
Or even none? Netcat to the rescue! Or at least bbcp (combined with mbuffer).
Code: Select all
odroid@server:~$ zpool iostat 60
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 4.92G 65.5T 18 242 272K 37.4M
Test 10.4G 65.5T 0 608 1.60K 95.4M
Test 15.9G 65.5T 0 580 3.00K 94.8M
Test 21.4G 65.4T 0 618 3.00K 95.4M
Code: Select all
odroid@server:~$ sudo sma -a /dev/sdc
smartctl 7.1 2019-12-30 r5022 [aarch64-linux-4.19.219-odroid-arm64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Device Model: WDC WD120EDBZ-11B1HA0
Serial Number: 5QGUKWVF
LU WWN Device Id: 5 000cca 2b0cba077
Firmware Version: 85.00A85
User Capacity: 12,000,138,625,024 bytes [12.0 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 7200 rpm
Form Factor: 3.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Wed Apr 13 09:54:19 2022 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 87) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: (1335) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000b 100 100 016 Pre-fail Always - 0
2 Throughput_Performance 0x0004 126 126 054 Old_age Offline - 116
3 Spin_Up_Time 0x0007 184 184 024 Pre-fail Always - 364 (Average 350)
4 Start_Stop_Count 0x0012 100 100 000 Old_age Always - 12
5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0
7 Seek_Error_Rate 0x000a 100 100 067 Old_age Always - 0
8 Seek_Time_Performance 0x0004 128 128 020 Old_age Offline - 18
9 Power_On_Hours 0x0012 100 100 000 Old_age Always - 75
10 Spin_Retry_Count 0x0012 100 100 060 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 11
22 Unknown_Attribute 0x0023 100 100 025 Pre-fail Always - 100
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 18
193 Load_Cycle_Count 0x0012 100 100 000 Old_age Always - 18
194 Temperature_Celsius 0x0002 110 110 000 Old_age Always - 59 (Min/Max 24/62)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age Always - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Code: Select all
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 23.1G 65.4T 18 249 267K 38.5M
Test 365G 65.1T 0 624 1.82K 98.5M
Test 691G 64.8T 0 583 362 94.0M
Test 1011G 64.5T 0 543 278 92.3M
Test 1.29T 64.2T 0 520 104 88.6M
Test 1.60T 63.9T 0 559 17 91.5M
Test 1.91T 63.6T 0 513 17 91.7M
Test 2.21T 63.3T 0 501 0 90.3M
Test 2.53T 62.9T 0 561 0 92.9M
Test 2.85T 62.6T 0 592 0 92.9M
Test 3.16T 62.3T 0 603 0 94.1M
Test 3.48T 62.0T 0 589 0 93.1M
Test 3.80T 61.7T 0 610 0 94.1M
Test 4.12T 61.4T 0 608 0 93.3M
Test 4.43T 61.0T 0 591 0 92.3M
Test 4.74T 60.7T 0 570 0 92.7M
Test 5.05T 60.4T 0 570 0 91.8M
Test 5.37T 60.1T 0 566 0 92.1M
Test 5.68T 59.8T 0 574 0 92.3M
Test 5.99T 59.5T 0 549 0 92.1M
Test 6.30T 59.2T 0 553 0 92.2M
Test 6.63T 58.8T 0 576 0 95.2M
Test 6.96T 58.5T 0 624 0 97.2M
Test 7.29T 58.2T 0 628 0 97.5M
Test 7.61T 57.9T 0 534 0 94.2M
Test 7.94T 57.5T 0 558 0 97.6M
Test 8.27T 57.2T 0 551 0 97.2M
Test 8.59T 56.9T 0 541 0 95.4M
Test 8.92T 56.6T 0 548 0 96.5M
Test 9.24T 56.2T 0 545 169 96.2M
Test 9.56T 55.9T 0 530 497 93.9M
Test 9.88T 55.6T 0 535 384 93.6M
Test 10.2T 55.3T 0 531 137 93.3M
Test 10.5T 55.0T 0 528 180 93.4M
Test 10.8T 54.6T 0 532 135 93.8M
Test 11.1T 54.3T 0 527 141 92.7M
Test 11.5T 54.0T 0 547 160 91.9M
Test 11.8T 53.7T 0 546 127 91.7M
Test 12.1T 53.4T 0 540 128 92.1M
Test 12.4T 53.1T 0 544 134 92.0M
Test 12.7T 52.8T 0 570 114 93.1M
Test 13.0T 52.4T 0 572 105 93.4M
Test 13.3T 52.1T 0 568 94 93.3M
Test 13.7T 51.8T 0 562 94 91.7M
^C // accidentally pressed Ctrl+C
odroid@server:~$ zpool iostat 3600
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 13.9T 51.6T 1 543 15.7K 90.3M
Test 14.2T 51.3T 0 570 87 93.5M
Test 14.5T 51.0T 0 572 69 93.2M
Test 14.8T 50.7T 0 572 70 92.9M
Test 15.1T 50.3T 0 573 92 93.5M
Test 15.4T 50.0T 0 566 88 92.8M
Test 15.8T 49.7T 0 568 96 93.4M
Test 16.1T 49.4T 0 570 61 92.9M
Test 16.4T 49.1T 0 577 81 93.6M
Test 16.7T 48.8T 0 565 67 92.9M
Test 17.0T 48.4T 0 556 68 92.7M
Test 17.3T 48.1T 0 550 54 92.6M
Test 17.7T 47.8T 0 559 64 93.0M
Test 18.0T 47.5T 0 555 71 92.4M
Test 18.3T 47.2T 0 591 76 97.3M
Test 18.6T 46.8T 0 588 56 97.6M
Test 19.0T 46.5T 0 576 89 97.3M
Test 19.3T 46.2T 0 586 79 97.5M
Test 19.6T 45.9T 0 596 62 97.8M
Test 19.9T 45.5T 0 587 61 97.6M
Test 20.3T 45.2T 0 595 70 97.2M
Test 20.6T 44.9T 0 598 58 97.7M
Test 20.9T 44.5T 0 595 79 97.5M
Test 21.3T 44.2T 0 578 547 94.5M
Test 21.6T 43.9T 0 588 552 94.8M
Test 21.9T 43.6T 0 579 425 93.9M
Test 22.2T 43.3T 0 567 98 94.2M
Test 22.5T 42.9T 0 563 220 94.4M
Test 22.9T 42.6T 0 564 76 92.7M
Test 23.2T 42.3T 0 568 54 94.9M
Test 23.5T 42.0T 0 568 38 95.0M
Test 23.8T 41.7T 0 569 28 95.1M
Test 24.1T 41.3T 0 572 43 94.6M
Test 24.5T 41.0T 0 556 63 94.2M
Test 24.8T 40.7T 0 548 72 93.2M
Test 25.1T 40.4T 0 539 91 92.3M
Test 25.4T 40.1T 0 539 117 92.4M
Test 25.7T 39.8T 0 547 176 93.6M
Test 26.0T 39.4T 0 553 145 93.8M
Test 26.3T 39.1T 0 558 122 93.8M
Test 26.7T 38.8T 0 557 88 94.1M
Test 27.0T 38.5T 0 553 109 93.9M
Test 27.3T 38.2T 0 540 70 93.0M
Test 27.6T 37.9T 0 549 75 94.0M
Test 27.9T 37.5T 0 539 75 93.3M
Test 28.3T 37.2T 0 556 76 93.7M
Test 28.6T 36.9T 0 547 126 92.8M
Test 28.9T 36.6T 0 551 43 93.1M
Test 29.2T 36.3T 0 559 51 93.6M
Test 29.5T 36.0T 0 577 29 93.1M
Test 29.8T 35.6T 0 580 35 93.2M
Test 30.2T 35.3T 0 563 23 97.1M
Test 30.5T 35.0T 0 551 21 95.3M
Test 30.8T 34.7T 0 572 29 97.0M
Test 31.1T 34.3T 0 574 48 97.4M
Test 31.5T 34.0T 0 572 26 98.0M
Test 31.8T 33.7T 0 595 31 98.2M
Test 32.1T 33.3T 0 604 29 98.3M
Test 32.5T 33.0T 0 601 17 98.3M
Test 32.8T 32.7T 0 585 14 94.6M
Test 33.1T 32.4T 0 570 155 93.0M
Test 33.4T 32.1T 0 571 465 92.1M
Test 33.7T 31.7T 0 577 369 93.6M
Test 34.0T 31.4T 0 572 221 93.6M
Test 34.4T 31.1T 0 572 194 93.9M
Test 34.7T 30.8T 0 579 93 94.1M
Test 35.0T 30.5T 0 580 110 94.1M
Test 35.3T 30.1T 0 586 118 94.2M
Test 35.6T 29.8T 0 590 172 94.9M
Test 36.0T 29.5T 0 582 62 94.3M
Test 36.3T 29.2T 0 591 77 94.9M
Test 36.6T 28.9T 0 581 69 93.7M
Test 36.9T 28.5T 0 577 53 93.7M
Test 37.2T 28.2T 0 576 47 93.2M
Test 37.6T 27.9T 0 578 56 93.2M
Test 37.9T 27.6T 0 580 62 94.0M
Test 38.2T 27.3T 0 576 48 94.5M
Test 38.5T 27.0T 0 603 262 95.3M
Test 38.8T 26.6T 0 598 68 95.5M
Test 39.2T 26.3T 0 580 220 94.8M
Test 39.5T 26.0T 0 594 108 95.5M
Test 39.8T 25.7T 0 565 106 93.3M
Test 40.1T 25.4T 0 580 111 93.5M
Test 40.4T 25.0T 0 569 141 93.7M
Test 40.7T 24.7T 0 584 127 94.3M
Test 41.1T 24.4T 0 587 185 94.3M
Test 41.4T 24.1T 0 574 175 94.0M
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 41.7T 23.8T 0 593 166 96.5M
Test 42.0T 23.4T 0 606 174 98.0M
Test 42.4T 23.1T 0 586 131 98.0M
Test 42.7T 22.8T 0 600 138 97.9M
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
Test 43.0T 22.4T 0 618 105 98.1M
Test 43.4T 22.1T 0 592 85 97.7M
Test 43.7T 21.8T 0 609 84 97.9M
Test 44.0T 21.4T 0 617 83 97.8M
Test 44.4T 21.1T 0 601 85 94.5M
Test 44.6T 20.9T 0 460 600 73.2M
Code: Select all
zupet@KWONIL:/media/Backup$ sudo zfs send Backup@20220409 | ssh -c aes128-gcm@openssh.com root@192.168.1.200 zfs recv -F Test
root@192.168.1.200's password:
zupet@KWONIL:/media/Backup$
Code: Select all
odroid@server:/media/Test/Anime/2022Q2$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 3808640 0 3808640 0% /dev
tmpfs 764500 1188 763312 1% /run
/dev/mmcblk1p2 243185620 3854492 229443816 2% /
tmpfs 3822488 0 3822488 0% /dev/shm
tmpfs 5120 4 5116 1% /run/lock
tmpfs 3822488 0 3822488 0% /sys/fs/cgroup
/dev/mmcblk1p1 253920 34076 206740 15% /boot
tmpfs 764496 0 764496 0% /run/user/1000
Test 56020397440 38258230912 17762166528 69% /media/Test
Users browsing this forum: No registered users and 2 guests