Kernel 4.14 debugging party

Test and fix the Kernel 4.14 features
phaseshifter
Posts: 3014
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: N-1..c1+ ..c-2..xu3 lite,xu4...u-3..
u-2...other odroid acc`s as well
Contact:

Re: Kernel 4.14 debugging party

Unread post by phaseshifter » Thu Dec 14, 2017 12:08 pm

yes it takes about 20 seconds with the Ethernet connected and a live connection..as for a networked unit i cannot say...

having a hdd in the usb3 port makes no difference at all..

p.s..i have not reflashed the chip as yet..

interesting enough i think it was caused by this..
1: my rtc was not connected properly

2 it was waiting to update the time via the network/ethernet/web ..etc..but on inital cold boot`s it often takes it`s time to reboot via command line...

i think this is fixed..only assumption i can come upo with so prolly not a problem if your rtc battery is good..no i have no idea..
Last edited by phaseshifter on Thu Dec 14, 2017 12:27 pm, edited 1 time in total.
Build It And They Will Come...Be Bold And Mighty Forces Will Come To Your Aid..!!!

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Thu Dec 14, 2017 12:25 pm

Once you meet the same issue, make a new thread in the same sub-forum.
viewforum.php?f=146

derko
Posts: 58
Joined: Tue Jul 14, 2015 2:16 am
languages_spoken: englisch, german
ODROIDs: XU4
Contact:

Re: Kernel 4.14 debugging party

Unread post by derko » Fri Dec 29, 2017 4:37 am

Hi unfortunately Kernel 4.14 is very unstable for me. If I attach my USB 3 HDD, UAS driver is loaded and sometimes ist works, sometimes kernel crashes. This can also happen after several hours /days - odroid xu4 is dead then.

How can I boot my 4.9 kernel again ? Can I revert somehow ? Thanks !

Trace:
Dec 28 20:32:14 Homeserver kernel: [ 60.498341] usb 4-1.1: new SuperSpeed USB device number 3 using xhci-hcd
Dec 28 20:32:15 Homeserver kernel: [ 60.520351] usb 4-1.1: New USB device found, idVendor=174c, idProduct=5106
Dec 28 20:32:15 Homeserver kernel: [ 60.520372] usb 4-1.1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
Dec 28 20:32:15 Homeserver kernel: [ 60.520388] usb 4-1.1: Product: AS2105
Dec 28 20:32:15 Homeserver kernel: [ 60.520404] usb 4-1.1: Manufacturer: ASMedia
Dec 28 20:32:15 Homeserver kernel: [ 60.520418] usb 4-1.1: SerialNumber: 13VB0EAGS
Dec 28 20:32:15 Homeserver kernel: [ 60.521895] usb-storage 4-1.1:1.0: USB Mass Storage device detected
Dec 28 20:32:15 Homeserver kernel: [ 60.529986] scsi host0: usb-storage 4-1.1:1.0
Dec 28 20:32:15 Homeserver mtp-probe: checking bus 4, device 3: "/sys/devices/platform/soc/soc:usb3-0/12000000.dwc3/xhci-hcd.3.auto/usb4/4-1/4-1.1"
Dec 28 20:32:15 Homeserver mtp-probe: bus: 4, device: 3 was not an MTP device
Dec 28 20:32:15 Homeserver kernel: [ 61.121625] usbcore: registered new interface driver uas
Dec 28 20:32:16 Homeserver kernel: [ 61.540655] scsi 0:0:0:0: Direct-Access TOSHIBA DT01ABA300 MZ6O PQ: 0 ANSI: 5
Dec 28 20:32:16 Homeserver kernel: [ 61.542849] sd 0:0:0:0: Attached scsi generic sg0 type 0
Dec 28 20:32:16 Homeserver kernel: [ 61.542910] sd 0:0:0:0: [sda] 732566646 4096-byte logical blocks: (3.00 TB/2.73 TiB)
Dec 28 20:32:16 Homeserver kernel: [ 61.543864] sd 0:0:0:0: [sda] Write Protect is off
Dec 28 20:32:16 Homeserver kernel: [ 61.543893] sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
Dec 28 20:32:16 Homeserver kernel: [ 61.544622] sd 0:0:0:0: [sda] No Caching mode page found
Dec 28 20:32:16 Homeserver kernel: [ 61.548609] sd 0:0:0:0: [sda] Assuming drive cache: write through
Dec 28 20:32:16 Homeserver kernel: [ 61.622000] sda: sda1 sda2 sda3
Dec 28 20:32:16 Homeserver kernel: [ 61.626467] sd 0:0:0:0: [sda] Attached SCSI disk
Dec 28 20:32:16 Homeserver kernel: [ 61.840911] Unable to handle kernel NULL pointer dereference at virtual address 00000004
Dec 28 20:32:16 Homeserver kernel: [ 61.847703] pgd = ed1cc000
Dec 28 20:32:16 Homeserver kernel: [ 61.850376] [00000004] *pgd=b5863835
Dec 28 20:32:16 Homeserver systemd-udevd[3853]: failed to poll: Invalid argument
Dec 28 20:32:16 Homeserver systemd-udevd[349]: worker [3853] exited with return code 1
Dec 28 20:32:16 Homeserver kernel: [ 61.853805] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
Dec 28 20:32:16 Homeserver kernel: [ 61.859145] Modules linked in: uas fuse ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_REDIRECT nf_nat_redirect xt_nat iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 binfmt_misc iptable_filter xt_conntrack nf_conntrack libcrc32c ip_tables cfg80211 rfkill cpufreq_userspace cpufreq_powersave cpufreq_conservative tun lzo lzo_compress zram spidev spi_s3c64xx exynos_gpiomem nfsd ipv6
Dec 28 20:32:16 Homeserver kernel: [ 61.893557] CPU: 3 PID: 2952 Comm: probing-thread Not tainted 4.14.9-94 #1
Dec 28 20:32:16 Homeserver kernel: [ 61.900391] Hardware name: ODROID-XU4
Dec 28 20:32:16 Homeserver kernel: [ 61.904030] task: ed00a1c0 task.stack: ed606000
Dec 28 20:32:16 Homeserver kernel: [ 61.908548] PC is at bio_uncopy_user+0xbc/0x130
Dec 28 20:32:16 Homeserver kernel: [ 61.913041] LR is at 0x4
Dec 28 20:32:16 Homeserver kernel: [ 61.915555] pc : [<c041c718>] lr : [<00000004>] psr: 60000013
Dec 28 20:32:16 Homeserver kernel: [ 61.921793] sp : ed607d78 ip : ed607d78 fp : ed4758e0
Dec 28 20:32:16 Homeserver kernel: [ 61.926992] r10: 00000010 r9 : ffff176b r8 : 00000000
Dec 28 20:32:16 Homeserver kernel: [ 61.932194] r7 : 00000000 r6 : ea08f900 r5 : 00000000 r4 : 00000000
Dec 28 20:32:16 Homeserver kernel: [ 61.938691] r3 : c2159500 r2 : 00000001 r1 : 00000000 r0 : ea08f900
Dec 28 20:32:16 Homeserver kernel: [ 61.945192] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Dec 28 20:32:16 Homeserver kernel: [ 61.952296] Control: 10c5387d Table: 6d1cc06a DAC: 00000051
Dec 28 20:32:16 Homeserver kernel: [ 61.958017] Process probing-thread (pid: 2952, stack limit = 0xed606210)
Dec 28 20:32:16 Homeserver kernel: [ 61.964687] Stack: (0xed607d78 to 0xed608000)
Dec 28 20:32:16 Homeserver kernel: [ 61.969021] 7d60: ed4758e0 c026ef6c
Dec 28 20:32:16 Homeserver kernel: [ 61.977170] 7d80: 00000001 c042a4e4 ea08f240 00000000 ea08f900 00000000 ea08f900 00000051
Dec 28 20:32:16 Homeserver kernel: [ 61.985321] 7da0: ea08f240 c042a4c0 ed607e10 ed475800 00000016 c043e134 00000200 014000c0
Dec 28 20:32:16 Homeserver kernel: [ 61.993460] 7dc0: 80808080 ed161400 00010003 00000000 ee441000 ea08f540 c02c043c 0800005d
Dec 28 20:32:16 Homeserver kernel: [ 62.001605] 7de0: 00000000 00020800 00000000 c0d03d48 ed161400 b4c9db98 c2a2bac0 0802005d
Dec 28 20:32:16 Homeserver kernel: [ 62.009751] 7e00: ffffe000 ed607e10 00000051 c043ecd8 00000053 fffffffd 00002010 00000200
Dec 28 20:32:16 Homeserver kernel: [ 62.017896] 7e20: b5705a00 b4c9dc7c b4c9dc8c 00001388 00000000 00000000 00000000 16000102
Dec 28 20:32:16 Homeserver kernel: [ 62.026041] 7e40: 00080000 00000000 00000000 00000001 00000001 c23a8d80 00000000 00000004
Dec 28 20:32:16 Homeserver kernel: [ 62.034186] 7e60: ea08f540 00000041 ed606000 ed607e6c ed607e6c ed00a1c0 00000000 7d4c37fa
Dec 28 20:32:16 Homeserver kernel: [ 62.042333] 7e80: c2159500 00002285 b4c9db98 0802005d ee441000 ed3b3800 ed606000 00000000
Dec 28 20:32:16 Homeserver kernel: [ 62.050477] 7ea0: b4c9db98 c043eec8 b4c9db98 0000000d ed607f74 00002285 ee441000 b4c9db98
Dec 28 20:32:16 Homeserver kernel: [ 62.058623] 7ec0: 0802005d c05cc1d0 ee441000 00002285 b4c9db98 0802005d c05cc15c c0436278
Dec 28 20:32:16 Homeserver kernel: [ 62.066768] 7ee0: c2242015 b4c9dd00 00000000 c27984c8 ed4a9c58 00000101 00000002 00001eba
Dec 28 20:32:16 Homeserver kernel: [ 62.074914] 7f00: 00000000 00000000 00000000 b4c9db98 ed4a9c58 ea08f540 0000000d b4c9db98
Dec 28 20:32:16 Homeserver kernel: [ 62.083059] 7f20: ed606000 00000000 b4c9db98 c0295ef8 00000000 00000000 00000400 c02a1cb4
Dec 28 20:32:16 Homeserver kernel: [ 62.091205] 7f40: c2242000 0000000d 0000000d ea08f540 c2242000 00000020 ea08f548 ed4a9c58
Dec 28 20:32:16 Homeserver kernel: [ 62.099351] 7f60: 00000000 c0281528 ea08f540 c02a1714 ea08f541 0000000d ea08f540 00002285
Dec 28 20:32:16 Homeserver kernel: [ 62.107496] 7f80: b4c9db98 ed606000 00000000 c0296770 b4c9dcfc b4c9dcec 00001388 00000036
Dec 28 20:32:16 Homeserver kernel: [ 62.115652] 7fa0: c01089a4 c01087c0 b4c9dcfc b4c9dcec 0000000d 00002285 b4c9db98 b4c9e8bc
Dec 28 20:32:16 Homeserver kernel: [ 62.123787] 7fc0: b4c9dcfc b4c9dcec 00001388 00000036 0000000d b4c9dd48 b4c9dc7c b4c9db98
Dec 28 20:32:16 Homeserver kernel: [ 62.131932] 7fe0: 000594c0 b4c9db74 0003bd7b b6b3eda6 00040030 0000000d 00000000 00000000
Dec 28 20:32:16 Homeserver kernel: [ 62.137738] VFS: Close: file count is 0
Dec 28 20:32:16 Homeserver kernel: [ 62.143914] [<c041c718>] (bio_uncopy_user) from [<c042a4c0>] (blk_rq_unmap_user+0x24/0x64)
Dec 28 20:32:16 Homeserver kernel: [ 62.152130] [<c042a4c0>] (blk_rq_unmap_user) from [<c043e134>] (sg_io+0x280/0x3f0)
Dec 28 20:32:16 Homeserver kernel: [ 62.159664] [<c043e134>] (sg_io) from [<c043ecd8>] (scsi_cmd_ioctl+0x42c/0x5dc)
Dec 28 20:32:16 Homeserver kernel: [ 62.166942] [<c043ecd8>] (scsi_cmd_ioctl) from [<c043eec8>] (scsi_cmd_blk_ioctl+0x40/0x48)
Dec 28 20:32:16 Homeserver kernel: [ 62.175187] [<c043eec8>] (scsi_cmd_blk_ioctl) from [<c05cc1d0>] (sd_ioctl+0x74/0x90)
Dec 28 20:32:16 Homeserver kernel: [ 62.182889] [<c05cc1d0>] (sd_ioctl) from [<c0436278>] (blkdev_ioctl+0x5e4/0xc1c)
Dec 28 20:32:16 Homeserver kernel: [ 62.190254] [<c0436278>] (blkdev_ioctl) from [<c0295ef8>] (do_vfs_ioctl+0x9c/0x8e0)
Dec 28 20:32:16 Homeserver kernel: [ 62.197879] [<c0295ef8>] (do_vfs_ioctl) from [<c0296770>] (SyS_ioctl+0x34/0x5c)
Dec 28 20:32:16 Homeserver kernel: [ 62.205162] [<c0296770>] (SyS_ioctl) from [<c01087c0>] (ret_fast_syscall+0x0/0x54)
Dec 28 20:32:16 Homeserver kernel: [ 62.212697] Code: e287e004 e1d085b0 e1a0c00d e5964058 (e8be000f)
Dec 28 20:32:16 Homeserver kernel: [ 62.219106] ---[ end trace dee8b75a3d7830f1 ]---

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Fri Dec 29, 2017 9:23 am

@derko
I have no idea how to downgrade the kernel to 4.9 except for cleanly installing an old image.

BTW, what happens if you disable UAS mode?

elatllat
Posts: 1110
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1
Contact:

Re: Kernel 4.14 debugging party

Unread post by elatllat » Fri Dec 29, 2017 10:55 am

You can switch to 4.9.y by building your own kernel.
Which is what I am doing due to usb (non-uas) crashes on 4.14.y

daviessm
Posts: 110
Joined: Thu Jul 25, 2013 2:14 am
languages_spoken: English, German
ODROIDs: X2, XU4
Location: Belfast, UK
Contact:

Re: Kernel 4.14 debugging party

Unread post by daviessm » Tue Jan 02, 2018 12:13 am

I get USB crashes too - more with UAS than without it loaded - but I decided to stick to 4.14 because downgrading would be a pain in the rear.

alexknuckles
Posts: 1
Joined: Sat Jan 06, 2018 11:16 am
languages_spoken: english
ODROIDs: xu4q
Contact:

Re: Kernel 4.14 debugging party

Unread post by alexknuckles » Sat Jan 06, 2018 11:19 am

Way more UAS USB crashes for me too. Especially with the newest update. I reverted back to a couple versions ago instead of going all the way back to 4.9 and that seems a bit better but still nowhere near stable.

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Sat Jan 06, 2018 3:21 pm

@daviessm & @alexknuckles
We've tried a few different Jmicron UAS capable bridges and there was no crash issue yet.
Which USB-to-SATA bridge chipset do you use?

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Sat Jan 06, 2018 3:26 pm

elatllat wrote:I tried the hardkernel repo with a lightening config command (that I use on 4.9)

Code: Select all

perl -pi -e 's/(.*)(WIRELESS|WIFI|TOUCHSCREEN|DEBUG)(.*=)[ym]/#$1$2$3n/g' .config
We've fixed the SPI driver and released 4.14.12 update today.
https://github.com/hardkernel/linux/com ... 38e3eb1130
https://github.com/hardkernel/linux/com ... 9cb2bd0b61
https://github.com/hardkernel/linux/com ... 1e7ce9336e
https://github.com/hardkernel/linux/com ... 3f7cc2245b

I've tried your lightening config and it seems to boot well.
Please confirm it if you have spare time.

daviessm
Posts: 110
Joined: Thu Jul 25, 2013 2:14 am
languages_spoken: English, German
ODROIDs: X2, XU4
Location: Belfast, UK
Contact:

Re: Kernel 4.14 debugging party

Unread post by daviessm » Sat Jan 06, 2018 7:18 pm

odroid wrote:@daviessm & @alexknuckles
We've tried a few different Jmicron UAS capable bridges and there was no crash issue yet.
Which USB-to-SATA bridge chipset do you use?
I recently had to replace one of mine because it was overheating so I'll re-enable UAS and re-test with the new one. It'll take a few days because the crash generally happens during my nightly backup/btrfs cleanup run.

I have had one crash with UAS disabled, but nothing was logged that time so I can't confirm if it was related, although it is likely (it was at the same time of night) and would suggest a general USB3 problem rather than specifically UAS.

derko
Posts: 58
Joined: Tue Jul 14, 2015 2:16 am
languages_spoken: englisch, german
ODROIDs: XU4
Contact:

Re: Kernel 4.14 debugging party

Unread post by derko » Sat Jan 06, 2018 9:26 pm

Can´t you provide an installable alternative kernel (like linux-image-xu3-4.9) via the repository, for those who suffer from problems like USB,UAS,SPI ?

Thx !

daviessm
Posts: 110
Joined: Thu Jul 25, 2013 2:14 am
languages_spoken: English, German
ODROIDs: X2, XU4
Location: Belfast, UK
Contact:

Re: Kernel 4.14 debugging party

Unread post by daviessm » Sun Jan 07, 2018 5:56 am

@odroid it happened again this evening, a few hours after re-enabling UAS. On the lsusb output below, bus 4 devices 7, 8 and 9 are all UAS-capable SATA adapters from Sabrent but apparently the newest one uses a different controller from the older ones. I can't confirm it 100% but it appears that is the one that's having the most problems.

Code: Select all

bang:~# lsusb
Bus 006 Device 002: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 003 Device 005: ID 16c0:05df Van Ooijen Technische Informatica HID device except mice, keyboards, and joysticks
Bus 004 Device 006: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 004 Device 007: ID 152d:1561 JMicron Technology Corp. / JMicron USA Technology Corp.
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 006: ID 16c0:05df Van Ooijen Technische Informatica HID device except mice, keyboards, and joysticks
Bus 004 Device 008: ID 152d:1561 JMicron Technology Corp. / JMicron USA Technology Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0a5c:21e8 Broadcom Corp. BCM20702A0 Bluetooth 4.0
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 003: ID 05e3:0735 Genesys Logic, Inc.
Bus 004 Device 009: ID 2109:0715 VIA Labs, Inc.
Bus 004 Device 005: ID 2109:0813 VIA Labs, Inc.
Bus 004 Device 004: ID 2109:0813 VIA Labs, Inc.
Bus 004 Device 002: ID 05e3:0616 Genesys Logic, Inc. hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 004: ID 2109:2813 VIA Labs, Inc.
Bus 003 Device 003: ID 2109:2813 VIA Labs, Inc.
Bus 003 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
bang:~# lsusb -t
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
        |__ Port 2: Dev 4, If 0, Class=Hub, Driver=hub/4p, 5000M
            |__ Port 1: Dev 5, If 0, Class=Hub, Driver=hub/4p, 5000M
                |__ Port 4: Dev 9, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 2: Dev 8, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 1: Dev 7, If 0, Class=Mass Storage, Driver=uas, 5000M
            |__ Port 2: Dev 6, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 480M
        |__ Port 2: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 1: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 3: Dev 5, If 0, Class=Human Interface Device, Driver=usbfs, 1.5M
            |__ Port 4: Dev 6, If 0, Class=Human Interface Device, Driver=usbfs, 1.5M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=exynos-ohci/3p, 12M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=btusb, 12M
    |__ Port 1: Dev 2, If 1, Class=Vendor Specific Class, Driver=btusb, 12M
    |__ Port 1: Dev 2, If 2, Class=Vendor Specific Class, Driver=btusb, 12M
    |__ Port 1: Dev 2, If 3, Class=Application Specific Interface, Driver=, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=exynos-ehci/3p, 480M
Anything else you need?

Edit: this is on hardkernel 4.14.11 kernel, commit ce2e9895cc4e2942b1397ce80684d25f1e7cda42.

moon.linux
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by moon.linux » Sun Jan 07, 2018 11:27 pm

derko wrote:Can´t you provide an installable alternative kernel (like linux-image-xu3-4.9) via the repository, for those who suffer from problems like USB,UAS,SPI ?

Thx !
@derko. Could you try to adding blacklist spi and spi_s3c64xx in /etc/modprobe.d/blacklist-odroid.conf just to isolate the issue.

derko
Posts: 58
Joined: Tue Jul 14, 2015 2:16 am
languages_spoken: englisch, german
ODROIDs: XU4
Contact:

Re: Kernel 4.14 debugging party

Unread post by derko » Mon Jan 08, 2018 1:40 am

I blacklisted my USB HDDs now via bootargs, as mentioned in the FAQs:

usb-storage.quirks=0x174c:0x5106:u usb-storage.quirks=0x1058:0x107c:u

--> Stable since yesterday !

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Mon Jan 08, 2018 9:46 am

daviessm wrote:@odroid it happened again this evening, a few hours after re-enabling UAS. On the lsusb output below, bus 4 devices 7, 8 and 9 are all UAS-capable SATA adapters from Sabrent but apparently the newest one uses a different controller from the older ones. I can't confirm it 100% but it appears that is the one that's having the most problems.
Can you find USB VID/PID of devices 7,8 and 9?
If yes, add them in the bootargs of boot.ini file to disable UAS mode.
Even my Intel desktop PC(with Kernel 4.14) has a stability issue with some UAS capable storages.

daviessm
Posts: 110
Joined: Thu Jul 25, 2013 2:14 am
languages_spoken: English, German
ODROIDs: X2, XU4
Location: Belfast, UK
Contact:

Re: Kernel 4.14 debugging party

Unread post by daviessm » Mon Jan 08, 2018 11:45 am

Can you find USB VID/PID of devices 7,8 and 9?
If yes, add them in the bootargs of boot.ini file to disable UAS mode.
Even my Intel desktop PC(with Kernel 4.14) has a stability issue with some UAS capable storages.
Yeah I can, but they worked fine under 4.9. As soon as I updated to 4.14 the system became unstable. Currently I'm running with UAS disabled globally.

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Mon Jan 08, 2018 12:26 pm

@daviessm
I know Kernel 4.9 had slightly better UAS compatibility with some specific USB-SATA controllers.
But I really don't have any idea which part in Kernel 4.14 make it worse.

BTW, I think the UAS seems to be still enabled according to your "lsusb -t" output.

Code: Select all

                |__ Port 4: Dev 9, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 2: Dev 8, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 1: Dev 7, If 0, Class=Mass Storage, Driver=uas, 5000M

User avatar
memeka
Posts: 4072
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Contact:

Re: Kernel 4.14 debugging party

Unread post by memeka » Mon Jan 08, 2018 12:42 pm

I don't think the issue is UAS
I am still having issues with cloudshell 1, which is not UAS compatible... :(
So I think there are still some USB3 issues :(

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Mon Jan 08, 2018 1:16 pm

CloudShell1 USB-SATA controller(GL3321G) needs a smaller block queue size on Kernel 4.14 as well as 4.9.
viewtopic.php?f=146&t=28692&p=207966#p207948

User avatar
memeka
Posts: 4072
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Contact:

Re: Kernel 4.14 debugging party

Unread post by memeka » Mon Jan 08, 2018 1:18 pm

odroid wrote:CloudShell1 USB-SATA controller(GL3321G) needs a smaller block queue size on Kernel 4.14 as well as 4.9.
viewtopic.php?f=146&t=28692&p=207966#p207948
yes i already use that.

daviessm
Posts: 110
Joined: Thu Jul 25, 2013 2:14 am
languages_spoken: English, German
ODROIDs: X2, XU4
Location: Belfast, UK
Contact:

Re: Kernel 4.14 debugging party

Unread post by daviessm » Tue Jan 09, 2018 7:25 pm

odroid wrote:@daviessm
I know Kernel 4.9 had slightly better UAS compatibility with some specific USB-SATA controllers.
But I really don't have any idea which part in Kernel 4.14 make it worse.

BTW, I think the UAS seems to be still enabled according to your "lsusb -t" output.

Code: Select all

                |__ Port 4: Dev 9, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 2: Dev 8, If 0, Class=Mass Storage, Driver=uas, 5000M
                |__ Port 1: Dev 7, If 0, Class=Mass Storage, Driver=uas, 5000M
Yeah, I enabled UAS and rebooted to capture that. Once a device failed again I disabled it and rebooted. The devices now show up as usb-storage.

kidovate
Posts: 49
Joined: Fri Jun 21, 2013 8:26 am
languages_spoken: english
ODROIDs: ODroid X2
Contact:

Re: Kernel 4.14 debugging party

Unread post by kidovate » Mon Feb 19, 2018 1:29 pm

At the moment, I cannot get 4.14.x to boot, but I can get 4.9.x to boot.

I am cross-compiling with buildroot, using https://github.com/paralin/SkiffOS

I suspected the toolchain may be the issue, so I did some tests:
  • GCC 7.x with 4.9.x - works
    GCC 7.x with 4.14.x - doesn't boot
    GCC 6.x with 4.14.x - doesn't boot
    GCC 6.x with 4.9.x - works
Building with HEAD of the 4.9 tree works:

Code: Select all

-BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/1ebaaab89354d604066acb505f6574d0b926b3ea/linux-xu4-4.14.18-r1.tar.gz"
-BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_14=y

+BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION="https://github.com/hardkernel/linux/archive/be592282a08a2493692448e365e46b52ac715b3f/linux-xu4-4.9.61-r2.tar.gz"
+BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_9=y
Seems the toolchain is not the issue. I suspected the kernel had grown large, and might be overwriting the dtb in memory while being loaded by u-boot. But, the boot script allows for 67Mb of kernel, while the actual kernel loaded is 4.9-6Mb:

Code: Select all

reading zImage
4948704 bytes read in 334 ms (14.1 MiB/s)
So... What should I try next?

moon.linux
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by moon.linux » Tue Feb 20, 2018 2:29 am

@kidovate can you try to convert some config option from module =y to module=m to reduce the size, just enough to get this working.

MastaG
Posts: 226
Joined: Mon Aug 26, 2013 6:05 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by MastaG » Tue Feb 20, 2018 11:39 pm

@kidovate: are you using the odroidxu4_defconfig ? or your own?

MastaG
Posts: 226
Joined: Mon Aug 26, 2013 6:05 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by MastaG » Tue Feb 20, 2018 11:47 pm

Perhaps the latest commits have broken the 4.14 kernel for XU4.
I'll have to test it myself, but the main developer of batocera.linux reports the same issue: https://batocera-linux.xorhub.com/forum ... el-4-x/126

Could it be that the last two commits made the kernel too big to boot?
https://github.com/hardkernel/linux/com ... _defconfig

moon.linux
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by moon.linux » Wed Feb 21, 2018 4:49 am

@MastaG
Actually I tried to sync with X86 config to improve BLK, do you suggest revert this patch.

kidovate
Posts: 49
Joined: Fri Jun 21, 2013 8:26 am
languages_spoken: english
ODROIDs: ODroid X2
Contact:

Re: Kernel 4.14 debugging party

Unread post by kidovate » Wed Feb 21, 2018 10:13 am

@MastaG I am using my own, which is derived from the official one - here is the defconfig:
https://gist.github.com/paralin/bbe578b ... 8fa37de4f8

I do enable some additional modules. I'll try to bisect the issue to see what commit broke it, and also try setting some more "y -> m" in order to reduce zImage size.

neal
Posts: 160
Joined: Fri Apr 14, 2017 10:02 am
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by neal » Wed Feb 21, 2018 6:49 pm

kidovate wrote:GCC 7.x with 4.9.x - works
GCC 7.x with 4.14.x - doesn't boot
GCC 6.x with 4.14.x - doesn't boot
GCC 6.x with 4.9.x - works
When I using this https://github.com/bkrepo/buildroot.git, and I tried the below all of them work.
GCC 7.x with 4.9.x - works
GCC 7.x with 4.14.x - works
GCC 6.x with 4.14.x - works
GCC 6.x with 4.9.x - works

Code: Select all

$> git clone https://github.com/bkrepo/buildroot.git
$> cd buildroot
$> make odroidxu4_defconfig
$> make menuconfig -> select cross compiler
$> make
$> cd output/image
Write the "sdcard.img" to SD Card and booting

MastaG
Posts: 226
Joined: Mon Aug 26, 2013 6:05 pm
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by MastaG » Wed Feb 21, 2018 9:31 pm

@moon.linux
I still have to try it myself.
Will probably be able to test it next week at earliest.

@neal
Could you also try the vanilla odroidxu4_defconfig without any changes?

kidovate
Posts: 49
Joined: Fri Jun 21, 2013 8:26 am
languages_spoken: english
ODROIDs: ODroid X2
Contact:

Re: Kernel 4.14 debugging party

Unread post by kidovate » Thu Feb 22, 2018 5:26 am

@neal Okay so toolchain is not the issue.

I suspect maybe the kernel is now just too large to boot.

You are not using the latest, by the way, from the 4.14.x branch.

neal
Posts: 160
Joined: Fri Apr 14, 2017 10:02 am
languages_spoken: english
Contact:

Re: Kernel 4.14 debugging party

Unread post by neal » Thu Feb 22, 2018 6:55 pm

MastaG wrote:@neal
Could you also try the vanilla odroidxu4_defconfig without any changes?
kidovate wrote:@neal Okay so toolchain is not the issue.

I suspect maybe the kernel is now just too large to boot.

You are not using the latest, by the way, from the 4.14.x branch.
I just tried that commands as I mentioned before. and also there were No any changes with odroidxu4_defconfig(If that is the meaning vanilla, I'd say yes).

Code: Select all

$> git clone https://github.com/bkrepo/buildroot.git
$> cd buildroot
$> make odroidxu4_defconfig
$> make menuconfig -> select cross compiler
$> make
$> cd output/image
And Let me show you Kernel section of my buildroot .config where it is buildroot root directory.
I think that when I tried the build yesterday, kernel git ID is "1ebaaab89354d604066acb505f6574d0b926b3ea" the latest.

Code: Select all

#
# Kernel
#
BR2_LINUX_KERNEL=y
# BR2_LINUX_KERNEL_LATEST_VERSION is not set
# BR2_LINUX_KERNEL_LATEST_CIP_VERSION is not set
# BR2_LINUX_KERNEL_CUSTOM_VERSION is not set
# BR2_LINUX_KERNEL_CUSTOM_TARBALL is not set
BR2_LINUX_KERNEL_CUSTOM_GIT=y
# BR2_LINUX_KERNEL_CUSTOM_HG is not set
# BR2_LINUX_KERNEL_CUSTOM_SVN is not set
BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/hardkernel/linux.git"
BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="1ebaaab89354d604066acb505f6574d0b926b3ea"
BR2_LINUX_KERNEL_VERSION="1ebaaab89354d604066acb505f6574d0b926b3ea"
BR2_LINUX_KERNEL_PATCH=""
BR2_LINUX_KERNEL_USE_DEFCONFIG=y
# BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG is not set
# BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG is not set
BR2_LINUX_KERNEL_DEFCONFIG="odroidxu4"
BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES=""
# BR2_LINUX_KERNEL_UIMAGE is not set
# BR2_LINUX_KERNEL_APPENDED_UIMAGE is not set
BR2_LINUX_KERNEL_ZIMAGE=y
# BR2_LINUX_KERNEL_APPENDED_ZIMAGE is not set
# BR2_LINUX_KERNEL_VMLINUX is not set
# BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM is not set
BR2_LINUX_KERNEL_GZIP=y
# BR2_LINUX_KERNEL_LZ4 is not set
# BR2_LINUX_KERNEL_LZMA is not set
# BR2_LINUX_KERNEL_LZO is not set
# BR2_LINUX_KERNEL_XZ is not set
BR2_LINUX_KERNEL_DTS_SUPPORT=y
BR2_LINUX_KERNEL_USE_INTREE_DTS=y
# BR2_LINUX_KERNEL_USE_CUSTOM_DTS is not set
BR2_LINUX_KERNEL_INTREE_DTS_NAME="exynos5422-odroidxu4"
# BR2_LINUX_KERNEL_INSTALL_TARGET is not set

kidovate
Posts: 49
Joined: Fri Jun 21, 2013 8:26 am
languages_spoken: english
ODROIDs: ODroid X2
Contact:

Re: Kernel 4.14 debugging party

Unread post by kidovate » Sun Feb 25, 2018 10:26 am

@neal That is not the latest. The latest is:

- a93fc4b63a1e184b2ac2c9b5c935f4ad4f06fe85

https://github.com/hardkernel/linux/com ... ad4f06fe85

However, if that version works, that would sure be nice.

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Sat Mar 03, 2018 10:42 am

Incomplete power-off and reboot issue!

A few people have reported the Kernel 4.14 has a critical issue. This issue seems not to exist in Kernel 4.9/3.10.
When they ran "poweroff" command, the system looks shutting down well. The cooling fan turned off, the blue LED was off...
But the system still consumed the power obviously. This problem appeared in the reboot process too.

It means the shutdown process didn't complete well and a remained process or driver might block the system turn-off process.
So we've tried to reproduce the issue over a month. But it was a very rare problem like one time per a few hundred of power-off cycles.
Therefore, it was extremely hard to find a root cause and we spent a lot of time.

Anyway, we added a few debug messages in the Kernel to find a clue. And we modified some kernel sources to make a more reliable power-off process too.
https://github.com/hardkernel/linux/com ... 2bb85ec73c
https://github.com/hardkernel/linux/com ... 1d72af39ce
https://github.com/hardkernel/linux/com ... bd7b47d4be

Before the update, you could see an incomplete power-down message even though the system turned off correctly.

Code: Select all

odroid@odroid:~$ uname -a
Linux odroid 4.14.23-110 #1 SMP PREEMPT Wed Feb 28 13:34:26 UTC 2018 armv7l armv7l armv7l GNU/Linux
odroid@odroid:~$ sudo poweroff
[sudo] password for odroid: 
[  OK  ] Stopped target Timers.
         Stopping Authenticate and Authorize Users to Run Privileged Tasks...
         Stopping TLP system startup/shutdown...
         Stopping Session c1 of user odroid.
[  OK  ] Stopped Stop ureadahead data col         Stopping Daemon for power management...
         Stopping Accounts Service...
[  OK  ] Stopped target Sound Card.
         Stopping Disk Manager...
         Stopping RealtimeKit Scheduling Policy Service...
         Stopping Save/Restore Sound Card State...
[  OK  ] Unmounted /run/user/1000/gvfs.
[  OK  ] Stopped Accounts Service.
[  OK  ] Stopped Authenticate and Authorize Users to Run Privileged Tasks.
[  OK  ] Stopped User Manager for UID 1000.
[  OK  ] Stopped RealtimeKit Scheduling Policy Service.
[  OK  ] Stopped Disk Manager.
[  OK  ] Stopped Daemon for power management.
[  OK  ] Stopped Serial Getty on ttySAC2.                                                                      
[  OK  ] Stopped TLP system startup/shutdown.                                                                  
[  OK  ] Stopped Session 1 of user odroid.                                                                     
[  OK  ] Stopped Session c1 of user odroid.                                                                    
[  OK  ] Stopped Save/Restore Sound Card State.                                                                
[  OK  ] Stopped Light Display Manager.                                                                        
[  OK  ] Stopped target Multi-User System.                                                                     
         Stopping LSB: automatic crash report generation...                                                    
[  OK  ] Stopped Initialize hardware monitoring sensors.                                                       
         Stopping crash report submission daemon...                                                            
         Stopping Snappy daemon...                                                                             
         Stopping An object/document-oriented database...                                                      
[  OK  ] Stopped ODROID Specific System Tweaks.                                                                
         Stopping Make remote CUPS printers available locally...                                               
         Stopping LSB: Speech Dispatcher...                                                                    
[  OK  ] Stopped Timer to automatically refresh installed snaps.                                               
[  OK  ] Stopped target System Time Synchronized.                                                              
         Stopping Modem Manager...                                                                             
         Stopping Unattended Upgrades Shutdown...                                                              
         Stopping Regular background program processing daemon...                                              
         Stopping LSB: disk temperature monitoring daemon...                                                   
         Stopping LSB: Start NTP daemon...                                                                     
         Stopping LSB: set CPUFreq kernel parameters...                                                        
         Stopping System Logging Service...                                                                    
         Stopping OpenBSD Secure Shell server...                                                               
[  OK  ] Stopped target Login Prompts.                                                                         
         Stopping Getty on tty1...                                                                             
[  OK  ] Removed slice system-serial\x2dgetty.slice.                                                           
[  OK  ] Removed slice User Slice of odroid.                                                                   
         Stopping Login Service...                                                                             
[  OK  ] Stopped Modem Manager.                                                                                
[  OK  ] Stopped An object/document-oriented database.                                                         
[  OK  ] Stopped Login Service.                                                                                
[  OK  ] Stopped Regular background program processing daemon.                                                 
[  OK  ] Stopped System Logging Service.                                                                       
[  OK  ] Stopped Snappy daemon.                                                                                
[  OK  ] Stopped OpenBSD Secure Shell server.                                                                  
[  OK  ] Stopped crash report submission daemon.                                                               
[  OK  ] Stopped Getty on tty1.                                                                                
[  OK  ] Stopped Make remote CUPS printers available locally.                                                  
[  OK  ] Stopped Unattended Upgrades Shutdown.                                                                 
[  OK  ] Stopped LSB: automatic crash report generation.                                                       
[  OK  ] Stopped LSB: Speech Dispatcher.                                                                       
[  OK  ] Stopped LSB: disk temperature monitoring daemon.                                                      
[  OK  ] Stopped LSB: set CPUFreq kernel parameters.                                                           
[  OK  ] Stopped LSB: Start NTP daemon.                                                                        
         Stopping LSB: Load kernel modules needed to enable cpufreq scaling...                                 
         Stopping CUPS Scheduler...                                                                            
         Stopping Avahi mDNS/DNS-SD Stack...                                                                   
[  OK  ] Stopped /etc/rc.local Compatibility.                                                                  
         Stopping Permit User Sessions...                                                                      
[  OK  ] Removed slice system-getty.slice.                                                                     
[  OK  ] Stopped target Network is Online.                                                                     
[  OK  ] Stopped Network Manager Wait Online.                                                                  
[  OK  ] Stopped target Network.                                                                               
         Stopping Raise network interfaces...                                                                  
         Stopping Network Manager...                                                                           
[  OK  ] Stopped Avahi mDNS/DNS-SD Stack.                                                                      
[  OK  ] Stopped Network Manager.                                                                              
[  OK  ] Stopped CUPS Scheduler.                                                                               
[  OK  ] Stopped Permit User Sessions.                                                                         
[  OK  ] Stopped LSB: Load kernel modules needed to enable cpufreq scaling.                                    
[  OK  ] Stopped target Remote File Systems.                                                                   
[  OK  ] Stopped target Remote File Systems (Pre).                                                             
[  OK  ] Stopped target User and Group Name Lookups.                                                           
         Stopping D-Bus System Message Bus...                                                                  
[  OK  ] Stopped Raise network interfaces.                                                                     
[  OK  ] Stopped D-Bus System Message Bus.                                                                     
[  OK  ] Stopped target Basic System.                                                                          
[  OK  ] Stopped target Slices.                                                                                
[  OK  ] Removed slice User and Session Slice.                                                                 
[  OK  ] Stopped target Sockets.                                                                               
[  OK  ] Closed Socket activation for snappy daemon.                                                           
[  OK  ] Closed Avahi mDNS/DNS-SD Stack Activation Socket.                                                     
[  OK  ] Closed Syslog Socket.                                                                                 
[  OK  ] Closed CUPS Scheduler.                                                                                
[  OK  ] Stopped target Paths.                                                                                 
[  OK  ] Stopped Dispatch Password Requests to Console Directory Watch.                                        
[  OK  ] Stopped CUPS Scheduler.                                                                               
[  OK  ] Stopped Forward Password Requests to Wall Directory Watch.                                            
[  OK  ] Closed D-Bus System Message Bus Socket.                                                               
[  OK  ] Stopped target System Initialization.                                                                 
[  OK  ] Stopped Create Volatile Files and Directories.                                                        
         Stopping Load/Save Random Seed...                                                                     
[  OK  ] Stopped target Encrypted Volumes.                                                                     
[  OK  ] Stopped target Swap.                                                                                  
[  OK  ] Stopped target Network (Pre).                                                                         
[  OK  ] Stopped target Local File Systems.                                                                    
         Unmounting /run/user/1000...                                                                          
         Unmounting /media/boot...                                                                             
[  OK  ] Stopped Apply Kernel Variables.                                                                       
[  OK  ] Stopped Load/Save Random Seed.                                                                        
[  OK  ] Unmounted /run/user/1000.                                                                             
[  OK  ] Unmounted /media/boot.                                                                                
[  OK  ] Stopped File System Check on /dev/disk/by-label/boot.                                                 
[  OK  ] Removed slice system-systemd\x2dfsck.slice.                                                           
[  OK  ] Reached target Unmount All Filesystems.                                                               
[  OK  ] Stopped target Local File Systems (Pre).                                                              
[  OK  ] Stopped Create Static Device Nodes in /dev.                                                           
[  OK  ] Stopped Remount Root and Kernel File Systems.                                                         
[  OK  ] Reached target Shutdown.                                                                              
[  964.763663] rebo�
After the update, we can see syscon_poweroff message from the PMIC driver now. I believe we can see what's wrong from the USB-UART serial console at least.

Code: Select all

odroid@odroid:~$ uname -a                                                                                      
Linux odroid 4.14.23-112 #1 SMP PREEMPT Fri Mar 2 22:54:30 UTC 2018 armv7l armv7l armv7l GNU/Linux             
odroid@odroid:~$ sudo poweroff                                                                                 
[sudo] password for odroid:                                                                                    
         Stopping Disk Manager...                                                                              
[  OK  ] Stopped target S         Stopping TLP system startup/shutdown...                                      
[  OK  ] Stopped target Timers.                                                                                
         Stopping Accounts Service...                                                                          
         Stopping Hostname Service...                                                                          
[  OK  ] Stopped Daily apt upgrade and clean activities.                                                       
[  OK  ] Stopped Daily apt download activities.                                                                
         Stopping Light Display Manager...                                                                     
         Stopping Session c1 of user odroid.                                                                   
         Stopping RealtimeKit Scheduling Policy Service...                                                     
[  OK  ] Stopped Daily Cleanup of Temporary Directories.                                                       
         Stopping Save/Restore Sound Card State...                                                             
[  OK  ] Unmounted /run/user/1000/gvfs.                                                                        
[  OK  ] Stopped Accounts Service.                                                                             
[  OK  ] Stopped Authenticate and Authorize Users to Run Privileged Tasks.                                     
[  OK  ] Stopped Hostname Service.                                                                             
[  OK  ] Stopped User Manager for UID 1000.                                                                    
[  OK  ] Stopped RealtimeKit Scheduling Policy Service.                                                        
[  OK  ] Stopped Disk Manager.                                                                                 
[  OK  ] Stopped Daemon for power management.                                                                  
[  OK  ] Stopped Serial Getty on ttySAC2.                                                                      
[  OK  ] Stopped TLP system startup/shutdown.                                                                  
[  OK  ] Stopped Session 1 of user odroid.                                                                     
[  OK  ] Stopped Session c1 of user odroid.                                                                    
[  OK  ] Stopped Save/Restore Sound Card State.                                                                
[  OK  ] Stopped target Multi-User System.                                                                     
         Stopping LSB: Speech Dispatcher...                                                                    
         Stopping LSB: set CPUFreq kernel parameters...                                                        
         Stopping Modem Manager...                                                                             
         Stopping LSB: Start NTP daemon...                                                                     
         Stopping System Logging Service...                                                                    
[  OK  ] Stopped ODROID Specific System Tweaks.                                                                
         Stopping Make remote CUPS printers available locally...                                               
         Stopping crash report submission daemon...                                                            
         Stopping OpenBSD Secure Shell server...                                                               
         Stopping LSB: automatic crash report generation...                                                    
         Stopping Regular background program processing daemon...                                              
[  OK  ] Stopped Timer to automatically refresh installed snaps.                                               
[  OK  ] Stopped target System Time Synchronized.                                                              
         Stopping Unattended Upgrades Shutdown...                                                              
         Stopping An object/document-oriented database...                                                      
         Stopping Snappy daemon...                                                                             
         Stopping LSB: disk temperature monitoring daemon...                                                   
[  OK  ] Stopped Initialize hardware monitoring sensors.                                                       
[  OK  ] Stopped target Login Prompts.                                                                         
         Stopping Getty on tty1...                                                                             
[  OK  ] Removed slice system-serial\x2dgetty.slice.                                                           
[  OK  ] Removed slice User Slice of odroid.                                                                   
         Stopping Login Service...                                                                             
[  OK  ] Stopped Regular background program processing daemon.                                                 
[  OK  ] Stopped An object/document-oriented database.                                                         
[  OK  ] Stopped Snappy daemon.                                                                                
[  OK  ] Stopped System Logging Service.                                                                       
[  OK  ] Stopped Login Service.                                                                                
[  OK  ] Stopped Modem Manager.                                                                                
[  OK  ] Stopped Make remote CUPS printers available locally.                                                  
[  OK  ] Stopped OpenBSD Secure Shell server.                                                                  
[  OK  ] Stopped Light Display Manager.                                                                        
[  OK  ] Stopped crash report submission daemon.                                                               
[  OK  ] Stopped Getty on tty1.                                                                                
[  OK  ] Stopped Unattended Upgrades Shutdown.                                                                 
[  OK  ] Stopped LSB: Speech Dispatcher.                                                                       
[  OK  ] Stopped LSB: set CPUFreq kernel parameters.                                                           
[  OK  ] Stopped LSB: Start NTP daemon.                                                                        
[  OK  ] Stopped LSB: automatic crash report generation.                                                       
[  OK  ] Stopped LSB: disk temperature monitoring daemon.                                                      
         Stopping LSB: Load kernel modules needed to enable cpufreq scaling...                                 
[  OK  ] Stopped /etc/rc.local Compatibility.                                                                  
[  OK  ] Removed slice system-getty.slice.                                                                     
[  OK  ] Stopped target Network is Online.                                                                     
[  OK  ] Stopped Network Manager Wait Online.                                                                  
         Stopping Permit User Sessions...                                                                      
[  OK  ] Stopped target Network.                                                                               
         Stopping Network Manager...                                                                           
         Stopping Raise network interfaces...                                                                  
         Stopping Avahi mDNS/DNS-SD Stack...                                                                   
         Stopping CUPS Scheduler...                                                                            
[  OK  ] Stopped CUPS Scheduler.                                                                               
[  OK  ] Stopped Network Manager.                                                                              
[  OK  ] Stopped Avahi mDNS/DNS-SD Stack.                                                                      
[  OK  ] Stopped Permit User Sessions.                                                                         
[  OK  ] Stopped LSB: Load kernel modules needed to enable cpufreq scaling.                                    
[  OK  ] Stopped target Remote File Systems.                                                                   
[  OK  ] Stopped target Remote File Systems (Pre).                                                             
[  OK  ] Stopped target User and Group Name Lookups.                                                           
         Stopping D-Bus System Message Bus...                                                                  
[  OK  ] Stopped Raise network interfaces.                                                                     
[  OK  ] Stopped D-Bus System Message Bus.                                                                     
[  OK  ] Stopped target Basic System.                                                                          
[  OK  ] Stopped target Slices.                                                                                
[  OK  ] Removed slice User and Session Slice.                                                                 
[  OK  ] Stopped target Paths.                                                                                 
[  OK  ] Stopped Dispatch Password Requests to Console Directory Watch.                                        
[  OK  ] Stopped Forward Password Requests to Wall Directory Watch.                                            
[  OK  ] Stopped CUPS Scheduler.                                                                               
[  OK  ] Stopped target Sockets.                                                                               
[  OK  ] Closed Syslog Socket.                                                                                 
[  OK  ] Closed CUPS Scheduler.                                                                                
[  OK  ] Closed Socket activation for snappy daemon.                                                           
[  OK  ] Closed Avahi mDNS/DNS-SD Stack Activation Socket.                                                     
[  OK  ] Closed D-Bus System Message Bus Socket.                                                               
[  OK  ] Stopped target System Initialization.                                                                 
[  OK  ] Stopped target Encrypted Volumes.                                                                     
[  OK  ] Stopped target Swap.                                                                                  
         Stopping Load/Save Random Seed...                                                                     
[  OK  ] Stopped Create Volatile Files and Directories.                                                        
[  OK  ] Stopped target Network (Pre).                                                                         
[  OK  ] Stopped target Local File Systems.                                                                    
         Unmounting /sys/kernel/debug/tracing...                                                               
         Unmounting /media/boot...                                                                             
         Unmounting /run/user/1000...                                                                          
[  OK  ] Stopped Apply Kernel Variables.                                                                       
[  OK  ] Stopped Load/Save Random Seed.                                                                        
[  OK  ] Unmounted /sys/kernel/debug/tracing.                                                                  
[  OK  ] Unmounted /media/boot.                                                                                
[  OK  ] Unmounted /run/user/1000.                                                                             
[  OK  ] Reached target Unmount All Filesystems.                                                               
[  OK  ] Stopped File System Check on /dev/disk/by-label/boot.                                                 
[  OK  ] Removed slice system-systemd\x2dfsck.slice.                                                           
[  OK  ] Stopped target Local File Systems (Pre).                                                              
[  OK  ] Stopped Create Static Device Nodes in /dev.                                                           
[  OK  ] Stopped Remount Root and Kernel File Systems.                                                         
[  OK  ] Reached target Shutdown.                                                                              
[   36.704019] reboot: Power down                                                                              
[   36.705675] syscon_poweroff : offset = 0x330c, value = 0x5200, mask = 0xffffffff
I hope the Kernel changes solve the improper power-off issue.

kidovate
Posts: 49
Joined: Fri Jun 21, 2013 8:26 am
languages_spoken: english
ODROIDs: ODroid X2
Contact:

Re: Kernel 4.14 debugging party

Unread post by kidovate » Wed Dec 19, 2018 5:57 pm

I can confirm that fixing the issue I reported previously in this thread is possible by disabling the kernel option CONFIG_CGROUP_PIDS.

bumbum
Posts: 2
Joined: Wed Jan 09, 2019 6:41 pm
languages_spoken: english
ODROIDs: odroid XU4
Contact:

Re: Kernel 4.14 debugging party

Unread post by bumbum » Wed Jan 09, 2019 6:44 pm

im getting this error any idea how to fix it ? i have already reflashed 4.14.87-153

Code: Select all

39.377392] vidioc_try_fmt:401: Unsupported format for destination.
[45039.382319] vidioc_try_fmt:401: Unsupported format for destination.
[45039.845721] vidioc_try_fmt:401: Unsupported format for destination.
[45039.850806] vidioc_try_fmt:401: Unsupported format for destination.
[45070.725017] ksoftirqd/6: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[45070.725049] ksoftirqd/6 cpuset=/ mems_allowed=0
[45070.725071] CPU: 6 PID: 40 Comm: ksoftirqd/6 Tainted: G        W       4.14.87-153 #1
[45070.725077] Hardware name: ODROID-XU4
[45070.725103] [<c0110884>] (unwind_backtrace) from [<c010cad4>] (show_stack+0x10/0x14)
[45070.725114] [<c010cad4>] (show_stack) from [<c08ab7b4>] (dump_stack+0x88/0x9c)
[45070.725128] [<c08ab7b4>] (dump_stack) from [<c0227904>] (warn_alloc+0xc4/0x16c)
[45070.725140] [<c0227904>] (warn_alloc) from [<c0228bd4>] (__alloc_pages_nodemask+0x118c/0x1290)
[45070.725151] [<c0228bd4>] (__alloc_pages_nodemask) from [<c0228e70>] (page_frag_alloc+0x138/0x14c)
[45070.725165] [<c0228e70>] (page_frag_alloc) from [<c07a0fb8>] (__napi_alloc_skb+0x5c/0xf4)
[45070.725182] [<c07a0fb8>] (__napi_alloc_skb) from [<c05e3ebc>] (r8152_poll+0x214/0xf4c)
[45070.725197] [<c05e3ebc>] (r8152_poll) from [<c07b81f8>] (net_rx_action+0x178/0x474)
[45070.725211] [<c07b81f8>] (net_rx_action) from [<c0101578>] (__do_softirq+0x108/0x3e4)
[45070.725224] [<c0101578>] (__do_softirq) from [<c012a5f8>] (run_ksoftirqd+0x3c/0x64)
[45070.725238] [<c012a5f8>] (run_ksoftirqd) from [<c0147394>] (smpboot_thread_fn+0x188/0x290)
[45070.725252] [<c0147394>] (smpboot_thread_fn) from [<c0143638>] (kthread+0x150/0x158)
[45070.725261] [<c0143638>] (kthread) from [<c0108768>] (ret_from_fork+0x14/0x2c)
[45070.725268] Mem-Info:
[45070.725280] active_anon:216901 inactive_anon:897 isolated_anon:0
                active_file:51345 inactive_file:196720 isolated_file:0
                unevictable:1 dirty:5314 writeback:14800 unstable:0
                slab_reclaimable:10604 slab_unreclaimable:7452
                mapped:19157 shmem:1781 pagetables:1603 bounce:5300
                free:3811 free_pcp:1082 free_cma:1544
[45070.725289] Node 0 active_anon:867604kB inactive_anon:3588kB active_file:205380kB inactive_file:786880kB unevictable:4kB isolated(anon):0kB isolated(file):0kB mapped:76628kB dirty:21256kB writeback:59200kB shmem:7124kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[45070.725299] Normal free:4124kB min:3452kB low:4312kB high:5172kB active_anon:193268kB inactive_anon:100kB active_file:75236kB inactive_file:344448kB unevictable:0kB writepending:38456kB present:786432kB managed:754248kB mlocked:0kB kernel_stack:2896kB pagetables:1488kB bounce:21200kB free_pcp:2828kB local_pcp:120kB free_cma:0kB
[45070.725303] lowmem_reserve[]: 0 9040 9040
[45070.725325] HighMem free:11288kB min:512kB low:1852kB high:3192kB active_anon:674336kB inactive_anon:3488kB active_file:130124kB inactive_file:442424kB unevictable:4kB writepending:42000kB present:1288192kB managed:1288192kB mlocked:4kB kernel_stack:0kB pagetables:4924kB bounce:0kB free_pcp:1504kB local_pcp:140kB free_cma:6176kB
[45070.725330] lowmem_reserve[]: 0 0 0
[45070.725379] Normal: 0*4kB 1*8kB (H) 1*16kB (H) 1*32kB (H) 1*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 1*1024kB (H) 1*2048kB (H) 0*4096kB = 4088kB
[45070.725460] HighMem: 1020*4kB (UMC) 702*8kB (UMC) 80*16kB (UMC) 13*32kB (UC) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 11392kB
[45070.725523] 249797 total pagecache pages
[45070.725528] 0 pages in swap cache
[45070.725534] Swap cache stats: add 0, delete 0, find 0/0
[45070.725539] Free swap  = 0kB
[45070.725544] Total swap = 0kB
[45070.725550] 518656 pages RAM
[45070.725555] 322048 pages HighMem/MovableOnly
[45070.725559] 8046 pages reserved
[45070.725565] 32768 pages cma reserved
[45070.725703] swapper/6: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[45070.725726] swapper/6 cpuset=/ mems_allowed=0
[45070.725742] CPU: 6 PID: 0 Comm: swapper/6 Tainted: G        W       4.14.87-153 #1

User avatar
odroid
Site Admin
Posts: 29325
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: Kernel 4.14 debugging party

Unread post by odroid » Thu Jan 10, 2019 9:22 am

@bumbum,
Did you flash this image?
ubuntu-18.04.1-4.14-mate-odroid-xu4-20181203.img.xz
https://wiki.odroid.com/odroid-xu4/os_i ... 4/20181203

bumbum
Posts: 2
Joined: Wed Jan 09, 2019 6:41 pm
languages_spoken: english
ODROIDs: odroid XU4
Contact:

Re: Kernel 4.14 debugging party

Unread post by bumbum » Fri Jan 11, 2019 6:52 pm

@odroid

Yes i did

User avatar
rooted
Posts: 6061
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Contact:

Re: Kernel 4.14 debugging party

Unread post by rooted » Fri Jan 11, 2019 8:41 pm

bumbum wrote:@odroid

Yes i did
What are you doing on the device when you get that crash?

phaseshifter
Posts: 3014
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: N-1..c1+ ..c-2..xu3 lite,xu4...u-3..
u-2...other odroid acc`s as well
Contact:

Re: Kernel 4.14 debugging party

Unread post by phaseshifter » Sun Jan 13, 2019 3:10 pm

look`s like bad media.my 20 cent`s worth....what do you use ...pls try sandisk ultra or similar .or orange emmc chip.

check supply for a steady voltage approx 5.0 vdc ...pin one of the 30 pin header con10 ..
if it falls below 4.8 volts then try swap-out your supply..should be 4 amp 5 vdc
Build It And They Will Come...Be Bold And Mighty Forces Will Come To Your Aid..!!!

ard
Posts: 72
Joined: Tue Jul 09, 2013 2:12 am
languages_spoken: english, dutch, german
ODROIDs: ODROID-U2
Contact:

Re: Kernel 4.14 debugging party

Unread post by ard » Mon Jan 14, 2019 9:59 pm

bumbum wrote:
Wed Jan 09, 2019 6:44 pm
im getting this error any idea how to fix it ? i have already reflashed 4.14.87-153

[45070.725017] ksoftirqd/6: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[45070.725049] ksoftirqd/6 cpuset=/ mems_allowed=0
[45070.725071] CPU: 6 PID: 40 Comm: ksoftirqd/6 Tainted: G W 4.14.87-153 #1
[45070.725077] Hardware name: ODROID-XU4
[45070.725103] [<c0110884>] (unwind_backtrace) from [<c010cad4>] (show_stack+0x10/0x14)
[45070.725114] [<c010cad4>] (show_stack) from [<c08ab7b4>] (dump_stack+0x88/0x9c)
[45070.725128] [<c08ab7b4>] (dump_stack) from [<c0227904>] (warn_alloc+0xc4/0x16c)
[45070.725140] [<c0227904>] (warn_alloc) from [<c0228bd4>] (__alloc_pages_nodemask+0x118c/0x1290)
[45070.725151] [<c0228bd4>] (__alloc_pages_nodemask) from [<c0228e70>] (page_frag_alloc+0x138/0x14c)
[45070.725165] [<c0228e70>] (page_frag_alloc) from [<c07a0fb8>] (__napi_alloc_skb+0x5c/0xf4)
[45070.725182] [<c07a0fb8>] (__napi_alloc_skb) from [<c05e3ebc>] (r8152_poll+0x214/0xf4c)
[45070.725197] [<c05e3ebc>] (r8152_poll) from [<c07b81f8>] (net_rx_action+0x178/0x474)
[45070.725211] [<c07b81f8>] (net_rx_action) from [<c0101578>] (__do_softirq+0x108/0x3e4)
[45070.725224] [<c0101578>] (__do_softirq) from [<c012a5f8>] (run_ksoftirqd+0x3c/0x64)
[45070.725238] [<c012a5f8>] (run_ksoftirqd) from [<c0147394>] (smpboot_thread_fn+0x188/0x290)
[45070.725252] [<c0147394>] (smpboot_thread_fn) from [<c0143638>] (kthread+0x150/0x158)
[45070.725261] [<c0143638>] (kthread) from [<c0108768>] (ret_from_fork+0x14/0x2c)
[45070.725268] Mem-Info:
[45070.725280] active_anon:216901 inactive_anon:897 isolated_anon:0
active_file:51345 inactive_file:196720 isolated_file:0
unevictable:1 dirty:5314 writeback:14800 unstable:0
slab_reclaimable:10604 slab_unreclaimable:7452
mapped:19157 shmem:1781 pagetables:1603 bounce:5300
free:3811 free_pcp:1082 free_cma:1544
[45070.725289] Node 0 active_anon:867604kB inactive_anon:3588kB active_file:205380kB inactive_file:786880kB unevictable:4kB isolated(anon):0kB isolated(file):0kB mapped:76628kB dirty:21256kB writeback:59200kB shmem:7124kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
[45070.725299] Normal free:4124kB min:3452kB low:4312kB high:5172kB active_anon:193268kB inactive_anon:100kB active_file:75236kB inactive_file:344448kB unevictable:0kB writepending:38456kB present:786432kB managed:754248kB mlocked:0kB kernel_stack:2896kB pagetables:1488kB bounce:21200kB free_pcp:2828kB local_pcp:120kB free_cma:0kB
[45070.725303] lowmem_reserve[]: 0 9040 9040
[45070.725325] HighMem free:11288kB min:512kB low:1852kB high:3192kB active_anon:674336kB inactive_anon:3488kB active_file:130124kB inactive_file:442424kB unevictable:4kB writepending:42000kB present:1288192kB managed:1288192kB mlocked:4kB kernel_stack:0kB pagetables:4924kB bounce:0kB free_pcp:1504kB local_pcp:140kB free_cma:6176kB
[45070.725330] lowmem_reserve[]: 0 0 0
[45070.725379] Normal: 0*4kB 1*8kB (H) 1*16kB (H) 1*32kB (H) 1*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 1*1024kB (H) 1*2048kB (H) 0*4096kB = 4088kB
[45070.725460] HighMem: 1020*4kB (UMC) 702*8kB (UMC) 80*16kB (UMC) 13*32kB (UC) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 11392kB
[45070.725523] 249797 total pagecache pages
[45070.725528] 0 pages in swap cache
[45070.725534] Swap cache stats: add 0, delete 0, find 0/0
[45070.725539] Free swap = 0kB
[45070.725544] Total swap = 0kB
[45070.725550] 518656 pages RAM
[45070.725555] 322048 pages HighMem/MovableOnly
[45070.725559] 8046 pages reserved
[45070.725565] 32768 pages cma reserved
[45070.725703] swapper/6: : order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[45070.725726] swapper/6 cpuset=/ mems_allowed=0
[45070.725742] CPU: 6 PID: 0 Comm: swapper/6 Tainted: G W 4.14.87-153 #1[/code]
This sounds like an ordinary out-of-memory during interrupt time (GGP_ATOMIC). It usually means the kernel has not enough free resources due to a very big burst of incoming network traffic and/or the memory was in another way already exhausted.
You can up that number to see if it works: sysctl -w vm.min_free_kbytes=32768 or higher...
I doubt it is a real problem, because it should just drop packets in that moment.

Post Reply

Return to “Linux Kernel 4.14 Debugging Party”

Who is online

Users browsing this forum: No registered users and 0 guests