kworker error on boot

Moderators: mdrjr, odroid

kworker error on boot

Unread postby mbonamusa » Fri Aug 03, 2018 9:14 pm

Sometimes when my XU4 boots, after some seconds (40-60 seconds) system crashes and blue led blinks very fast.

On syslog appears:
Code: Select all
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.122855] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129564] pgd = c0004000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129592] [0000000c] *pgd=00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129638] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129795] Process kworker/6:0 (pid: 1395, stack limit = 0xebc82210)
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129801] Stack: (0xebc83db8 to 0xebc84000)
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129808] 3da0:                                                       c090503c ffffffff
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129818] 3dc0: c0d978bf c0a51ee0 c0d0f5cc ebca5bc0 c0d03e44 ffffb43f c0ddc7e0 014080c0
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129826] 3de0: 00000005 c0d978bf ebca5000 c0a51ee0 d1696944 ebcd0400 001583a6 00000001
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129834] 3e00: 00000001 ebca5080 ececb000 ebca5080 c0db3ac0 ffffffff 0000007f c0167b2c
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129842] 3e20: ebc83e44 ebca5080 ebca5000 00000000 00000001 ebca5000 00000001 c0db3e80
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129850] 3e40: c0d03e44 00000000 ee801e40 ebca50c0 ebc60e00 00000001 c0d13d4c 00000001
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129858] 3e60: c0dd9898 c0d13c18 c0d13ca0 c0d13c18 c0d12ef4 c01b5b88 001ed73b 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129866] 3e80: 00000001 ebc60e00 ebc60d80 ebca5000 ebc60d80 eedd3780 00000000 ebca5080
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129874] 3ea0: ebc61900 200f0093 00000006 c0d13c18 c0d13d4c 00000001 c0dd9898 c0d13c18
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129883] 3ec0: c0d13ca0 ebc82000 c0d12ef4 c01b79e0 00000000 c01b7ab4 ebc83f14 00000040
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129910] 3ee0: ebc83ed8 c0d13c18 00000001 200f0093 ebc83f14 00000000 00000003 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129927] 3f00: eedd33c8 c0d13d58 ecf1d280 eedd3380 00000000 eedd6400 eedd3398 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129942] 3f20: c0d97d74 c013d8bc ecf1d280 eedd3380 eedd3380 ecf1d280 eedd3380 ecf1d298
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129958] 3f40: 00000108 c0d02d00 eedd3398 ebc82000 eedd3380 c013dd38 00000000 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129967] 3f60: ffffe000 ecde7c80 00000000 c2285780 ebc82000 ecf1d280 c013dd00 ecde7c9c
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129975] 3f80: eced9d68 c0143444 00000000 c2285780 c014331c 00000000 00000000 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.129982] 3fa0: 00000000 00000000 00000000 c01088a8 00000000 00000000 00000000 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.130000] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.130015] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
Aug  3 13:06:29 pc4d_CF345 kernel: [   61.130205] Code: e5824068 0a00000c e592c008 e1a0300c (e593100c)


And then also appears:
Code: Select all
Aug  3 13:13:38 pc4d_CF345 kernel: [  490.451443] INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P1395 } 5272 jiffies s: 21 root: 0x0/T
Aug  3 13:13:38 pc4d_CF345 kernel: [  490.460153] blocking rcu_node structures:
Aug  3 13:14:11 pc4d_CF345 kernel: [  523.258387] INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug  3 13:14:11 pc4d_CF345 kernel: [  523.262598]       Tasks blocked on level-0 rcu_node (CPUs 0-7): P1395
Aug  3 13:14:11 pc4d_CF345 kernel: [  523.268564]       (detected by 3, t=115539 jiffies, g=1430, c=1429, q=105509)
Aug  3 13:14:42 pc4d_CF345 kernel: [  553.937517] INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P1395 } 21144 jiffies s: 21 root: 0x0/T
Aug  3 13:14:42 pc4d_CF345 kernel: [  553.946320] blocking rcu_node structures:
Aug  3 13:15:14 pc4d_CF345 kernel: [  586.276488] INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug  3 13:15:14 pc4d_CF345 kernel: [  586.280695]       Tasks blocked on level-0 rcu_node (CPUs 0-7): P1395
Aug  3 13:15:14 pc4d_CF345 kernel: [  586.286666]       (detected by 4, t=131292 jiffies, g=1430, c=1429, q=110844)
Aug  3 13:15:45 pc4d_CF345 kernel: [  617.423597] INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P1395 } 37016 jiffies s: 21 root: 0x0/T
Aug  3 13:15:45 pc4d_CF345 kernel: [  617.432350] blocking rcu_node structures:
Aug  3 13:16:17 pc4d_CF345 kernel: [  649.294564] INFO: rcu_preempt detected stalls on CPUs/tasks:
Aug  3 13:16:17 pc4d_CF345 kernel: [  649.298777]       Tasks blocked on level-0 rcu_node (CPUs 0-7): P1395
Aug  3 13:16:17 pc4d_CF345 kernel: [  649.304742]       (detected by 3, t=147049 jiffies, g=1430, c=1429, q=122604)
Aug  3 13:16:49 pc4d_CF345 kernel: [  680.909683] INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P1395 } 52888 jiffies s: 21 root: 0x0/T


When it happens some processes works and some other stops working: My custom machine vision process is working and sending data via ethernet to my server and I can access the board webpage (apache2), but any process configured in crontab is executed and I can't access the board via ssh and tty

I'm using ubuntu 16.04.3 LTS and kernel 4.14.10-96
In the boot.ini I've configured maxcpus=5 (4 x arm-A7 + 1 x arm A15) to ensure a low power boot. Then, from rc.local an script is executed to bring up all A15 cpus (echo 1 > /sys/devices/system/cpuX/online). I don't know if it can produce the error.
mbonamusa
 
Posts: 175
Joined: Mon Feb 25, 2013 9:08 pm
languages_spoken: catalan, spanish, english
ODROIDs: ODROID-X2, U2, U3

Re: kworker error on boot

Unread postby phaseshifter » Sat Aug 04, 2018 1:44 am

pls refer to this post..make shure the primary factors are correct,,it looks like a power problem to me..i may be wrong..


viewtopic.php?f=93&t=31729&p=230693#p230693
odroid N-1 16Gb emmc orange..
phaseshifter
 
Posts: 2879
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: N-1..c1+ ..c-2..xu3 lite,xu4...u-3..
u-2...other odroid acc`s as well

Re: kworker error on boot

Unread postby phaseshifter » Sat Aug 04, 2018 1:47 am

0nly 4.14 kernels work on orange emmc`s if this is the issue...

sd-cards can be finicky,,if using sd-card ..try a sandisk..or another sd-card..that is proven reliable
odroid N-1 16Gb emmc orange..
phaseshifter
 
Posts: 2879
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: N-1..c1+ ..c-2..xu3 lite,xu4...u-3..
u-2...other odroid acc`s as well

Re: kworker error on boot

Unread postby phaseshifter » Sat Aug 04, 2018 1:49 am

also if power is an issue just try to boot the op sys as normal to rule out errors in you cpu cores issue ..make shure the board can actually boot...


but check that the boards functions as normal..before you go modding image`s and swapping out kernels...
odroid N-1 16Gb emmc orange..
phaseshifter
 
Posts: 2879
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: N-1..c1+ ..c-2..xu3 lite,xu4...u-3..
u-2...other odroid acc`s as well

Re: kworker error on boot

Unread postby odroid » Sat Aug 04, 2018 9:57 am

Try to update the Kernel to 4.14.55 or higher.
Or try our latest Ubuntu 18.04 image.
https://wiki.odroid.com/odroid-xu4/os_i ... 4/20180501
User avatar
odroid
Site Admin
 
Posts: 28521
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: kworker error on boot

Unread postby mbonamusa » Sat Aug 04, 2018 6:01 pm

phaseshifter wrote:pls refer to this post..make shure the primary factors are correct,,it looks like a power problem to me..i may be wrong..


viewtopic.php?f=93&t=31729&p=230693#p230693


My board is working well with any power supply problem. I'm using the hardkernel 5V 4A power adapter.
I've tried the emmc in different XU4 boards with different power adapters and I've found this issue in all boards, so seems something related to software (kernel, etc), not hardware.

phaseshifter wrote:0nly 4.14 kernels work on orange emmc`s if this is the issue...

sd-cards can be finicky,,if using sd-card ..try a sandisk..or another sd-card..that is proven reliable


I'm using an orange emmc with the 4.14 kernel. I started from 4.9 kernel image and then I upgraded to 4.14. I installed some packages and compiled my c++ program, and added maxcpus=5 in boot.ini. That's all.

I'm going to make some tests with the maxcpus boot option and with the set cpu online sequence and I tell you.
mbonamusa
 
Posts: 175
Joined: Mon Feb 25, 2013 9:08 pm
languages_spoken: catalan, spanish, english
ODROIDs: ODROID-X2, U2, U3

Re: kworker error on boot

Unread postby mbonamusa » Sat Aug 04, 2018 6:04 pm

odroid wrote:Try to update the Kernel to 4.14.55 or higher.
Or try our latest Ubuntu 18.04 image.
https://wiki.odroid.com/odroid-xu4/os_i ... 4/20180501


When I have a moment I'll try the latest image with the same configuration.
mbonamusa
 
Posts: 175
Joined: Mon Feb 25, 2013 9:08 pm
languages_spoken: catalan, spanish, english
ODROIDs: ODROID-X2, U2, U3

Re: kworker error on boot

Unread postby mbonamusa » Sun Aug 05, 2018 11:34 pm

I've made some tests and the problem only appears when I set online any of the offline cpus. If XU4 boot with maxcpus=5 (6,7 or 8) and it doesn't set online any cpu then the error never appears.
I've tried setting the cpus online with a time delay between each without any difference:
...
sleep XX
echo 1 > /sys/devices/system/cpu/cpu5 online
sleep XX
echo 1 > /sys/devices/system/cpu/cpu6 online
...
(I've tried with sleep times between 1 and 60 seconds)

So it seems this kworker kernel error is related to the set online cpus.

I'll try the same with the new ubuntu 18.04 image and I tell you.
mbonamusa
 
Posts: 175
Joined: Mon Feb 25, 2013 9:08 pm
languages_spoken: catalan, spanish, english
ODROIDs: ODROID-X2, U2, U3

Re: kworker error on boot

Unread postby odroid » Mon Aug 06, 2018 10:20 am

Update the kernel first once you install 18.04 image.
https://wiki.odroid.com/odroid-xu4/os_i ... st-upgrade
User avatar
odroid
Site Admin
 
Posts: 28521
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID


Return to Issues

Who is online

Users browsing this forum: No registered users and 1 guest