ODroid HC2 becomes unresponsive

Post Reply
nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

ODroid HC2 becomes unresponsive

Unread post by nz_will » Sat Jun 01, 2019 12:05 pm

I bought an ODroid HC2 about 2 weeks ago (#HK1905186380). I've noticed that after about 24 hours, the board becomes unresponsive. It doesn't respond to ping requests like usual, nor does it respond to HTTP requests or SSH.

System configuration:
  • I'm running Ubuntu 18.04.2 installed from the Hardkernel website.
  • I'm using a class 10 Samsung 32GB micro SD card
  • The connected HDD is a 500GB 3.5" Western Digital WD5000AAJS-22A8B0
Troubleshooting:
  • I've looked through the kernel logs and system logs. I can't see anything interesting, they just stop suddenly, and then show me rebooting.
  • I've cloned the 32GB micro SD card to a 64GB micro SD card (both class 10) in case that was the issue. Despite this, it happened again.
Can anyone suggest what might be wrong here? It's frustrating to have to manually toggle power every day or so.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Sun Jun 02, 2019 6:08 am

Additional troubleshooting:

I wrote a script that logs the time and pings my router every 10 seconds. When the ODroid becomes unresponsive, it continues to log the time without getting ping responses. Other devices on the network are fine. Could be something to do with the network adapter :/

User avatar
tobetter
Posts: 4156
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 50 times
Been thanked: 215 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by tobetter » Sun Jun 02, 2019 1:00 pm

nz_will wrote:
Sun Jun 02, 2019 6:08 am
Additional troubleshooting:

I wrote a script that logs the time and pings my router every 10 seconds. When the ODroid becomes unresponsive, it continues to log the time without getting ping responses. Other devices on the network are fine. Could be something to do with the network adapter :/
Do you know what color of LEDs are lit and blink when HC2 is running?
Are you able to see the IP for onboard NIC and how its name is configured?
These users thanked the author tobetter for the post:
nz_will (Tue Jun 04, 2019 5:35 pm)

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Jun 04, 2019 5:34 pm

tobetter:
As you can see in this video, the red power light is steady, the green light is steady, the blue light is flashing in a regular pattern of 2 flashes, and the network light is orange and steady.

The network card is configured to use the same IP every time by its MAC address from the DHCP server running on my router.
  • When I try to establish an SSH connection I get connection refused.
  • During this period I get connection refused for SSH and unable to connect for the web server.
  • When I ping it, I get destination host unreachable.
I'm going to mail the board back to Korea tomorrow for Hardkernel to take a look.

User avatar
tobetter
Posts: 4156
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 50 times
Been thanked: 215 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by tobetter » Tue Jun 04, 2019 5:41 pm

nz_will wrote:
Tue Jun 04, 2019 5:34 pm
tobetter:
As you can see in this video, the red power light is steady, the green light is steady, the blue light is flashing in a regular pattern of 2 flashes, and the network light is orange and steady.

The network card is configured to use the same IP every time by its MAC address from the DHCP server running on my router.
  • When I try to establish an SSH connection I get connection refused.
  • During this period I get connection refused for SSH and unable to connect for the web server.
  • When I ping it, I get destination host unreachable.
I'm going to mail the board back to Korea tomorrow for Hardkernel to take a look.
Before you mail the board, can you please make a tarball of the directory /etc and /var in your SD card? Then let me look into the configuration and run on my HC2.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Jun 04, 2019 5:57 pm

Here's the tarball of /etc, but the /var one is 1GB :shock: Do you still want?
Attachments
etc.tar.gz
(568.34 KiB) Downloaded 17 times

User avatar
tobetter
Posts: 4156
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 50 times
Been thanked: 215 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by tobetter » Tue Jun 04, 2019 5:58 pm

nz_will wrote:
Tue Jun 04, 2019 5:57 pm
Here's the tarball of /etc, but the /var one is 1GB :shock: Do you still want?
Let me see /etc first... 1GB is too huge.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Jun 04, 2019 6:13 pm

I uploaded my whole /var for you to download in case you decide that you want to. I think the reason it is 1GB is because /var/lib/docker is 2.4GB uncompressed.

Otherwise there's a much less intimidating 6MB tarball of /var/log as well.

P.S.
In case anyone is looking - no there isn't any useful key material in any of these tarballs. You can try find some though, I promise you disappointment.

User avatar
tobetter
Posts: 4156
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 50 times
Been thanked: 215 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by tobetter » Wed Jun 05, 2019 9:04 am

@nz_will, I've tried to run my HC2 with your files but having no issue. I suspected somehow system configuration becomes wrong, I would be wrong. Do you happen to have the USB-to-Serial debugging board? If so it would be very helpful to ping from HC2 to somewhere to verify the network. One doubt in my mind is that some service running on HC2 change the network configuration, so I would suggest to boot with on clean OS plus new SD card.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Fri Aug 02, 2019 3:41 am

So here's an update on this. Hardkernel got me to ship my HC2 back to them, they did some testing and said they think that something was wrong with it. They then sent me a new one. I put the same SD card in the new one, and it went offline about a day later. I then tried disabling everything but my NGINX serving a static site, but it still went offline a day later. Next thing I'm going to try is a clean install of Ubuntu Minimal 18.04.1 LTS (v1.1) adding only NGNIX and seeing if it will stay online,

User avatar
mad_ady
Posts: 6805
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 218 times
Been thanked: 167 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by mad_ady » Fri Aug 02, 2019 3:56 am

You can try to enable netconsole to log messages over the network, but since network fails...

Here is a script that runs as a watchdog (via cron) and reboots if it looses network. It collects some data before reboot. Needs fping, add some ips that are bound to be up, change the logdir and uncomment the reboot: http://paste.ubuntu.com/p/bDKTJVfZBf/

Once it reboots, let us see the log.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Fri Aug 02, 2019 2:55 pm

Mulțumesc Adrian! I've got the following script running under the cron "* * * * * /home/will/odroid-debug-tool.sh":

Code: Select all

#!/bin/bash
wall -n "FYI, cron is running the odroid debug tool"
down=`fping -c 1 -s 192.168.1.1 >&1 | grep unreachable | awk '{ print $1; }'`
total=1
logdir=/home/will
if [[ $down -lt $total ]]; then
   exit 0;
else
   logger -s -t $0 "Network is down!"
   timestamp=`date +%Y%m%d-%H%M`
   uname -a >> $logdir/watchdog-$timestamp
   date >> $logdir/watchdog-$timestamp
   dmesg >> $logdir/watchdog-$timestamp
   /sbin/ifconfig -a >> $logdir/watchdog-$timestamp
   date >> $logdir/watchdog-$timestamp
   /sbin/route -n >> $logdir/watchdog-$timestamp
   lsusb -t >> $logdir/watchdog-$timestamp
   lsusb >> $logdir/watchdog-$timestamp
   date >> $logdir/watchdog-$timestamp
   /sbin/ethtool eth0 >> $logdir/watchdog-$timestamp
   tail -100 /var/log/syslog >> $logdir/watchdog-$timestamp
   date >> $logdir/watchdog-$timestamp
   sync
   sync
   /sbin/reboot
   exit 255
fi
I'll let you know when I get a reboot!

User avatar
mad_ady
Posts: 6805
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 218 times
Been thanked: 167 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by mad_ady » Fri Aug 02, 2019 4:20 pm

Great! By the way, I'm running this script as part of the watchdog - that's why it exits with 255, to let the watchdog handle the reboot.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Sun Aug 04, 2019 5:02 pm

Latest update. Most recently I did the following:
  1. Flashed my SD card with the Ubuntu 18.04.1 (20181203) (MINIMAL, BARE OS) from Hardkernel using Balena Etcher as recommended by Hardkernel.
  2. Booted up my Odroid HC2 and did some basic setup.
    1. Created a user and added them to the sudoers group.
    2. Ran apt-get update && apt-get upgrade --assume-yes && apt-get dist-upgrade --assume-yes as root.
    3. Changed some settings in /etc/ssh/sshd.conf to be more secure, added some ~/.ssh/authorized_keys.
    4. Disabled the root account.
  3. Added the above script to my home directory, made is executable, and usedcrontab -e to schedule it to run every minute.
I then left for the weekend. When I got back, the Odroid was not responding to attempts to SSH into it, nor was it responding to pings. I pulled the power and plugged it in again. When it booted and I logged in, the script hadn't output anything. You can see it crashing based on the following output from /var/log/syslog:

Code: Select all

Aug  4 07:00:01 odroid CRON[27647]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:00:02 odroid CRON[27646]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:01:01 odroid CRON[27655]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:01:02 odroid CRON[27654]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:02:01 odroid CRON[27663]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:02:02 odroid CRON[27662]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:03:01 odroid CRON[27671]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:03:02 odroid CRON[27670]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:04:01 odroid CRON[27680]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:04:02 odroid CRON[27679]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:05:01 odroid CRON[27688]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:05:02 odroid CRON[27687]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:06:01 odroid CRON[27699]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:06:02 odroid CRON[27698]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:07:01 odroid CRON[27707]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:07:02 odroid CRON[27706]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:08:01 odroid CRON[27715]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:08:02 odroid CRON[27714]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:09:01 odroid CRON[27724]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:09:02 odroid CRON[27723]: (CRON) info (No MTA installed, discarding output)
Aug  4 07:10:01 odroid CRON[27732]: (root) CMD (/home/will/odroid-debug-tool.sh)
Aug  4 07:10:03 odroid CRON[27731]: (CRON) info (No MTA installed, discarding output)
Aug  3 05:36:35 odroid systemd[1]: Started Create list of required static device nodes for the current kernel.
Aug  3 05:36:35 odroid systemd[1]: Started Set the console keyboard layout.
Aug  3 05:36:35 odroid systemd[1]: Started Remount Root and Kernel File Systems.
Aug  3 05:36:35 odroid systemd[1]: Started Nameserver information manager.
Aug  3 05:36:35 odroid kernel: [    0.000000] Booting Linux on physical CPU 0x100
Aug  3 05:36:35 odroid kernel: [    0.000000] Linux version 4.14.133-165 (root@1604_builder_armhf) (gcc version 7.3.0 (Ubuntu/Linaro 7.3.0-16ubuntu3)) #1 SMP PREEMPT Mon Jul 15 22:14:09 -03 2019
Aug  3 05:36:35 odroid kernel: [    0.000000] CPU: ARMv7 Processor [410fc073] revision 3 (ARMv7), cr=10c5387d
Aug  3 05:36:35 odroid systemd[1]: Reached target Network (Pre).
Aug  3 05:36:35 odroid kernel: [    0.000000] CPU: div instructions available: patching division code
Aug  3 05:36:35 odroid kernel: [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Aug  3 05:36:35 odroid kernel: [    0.000000] OF: fdt: Machine model: Hardkernel Odroid XU4
Aug  3 05:36:35 odroid kernel: [    0.000000] Memory policy: Data cache writealloc
Aug  3 05:36:35 odroid kernel: [    0.000000] cma: Reserved 128 MiB at 0xb6800000
Aug  3 05:36:35 odroid kernel: [    0.000000] Samsung CPU ID: 0xe5422001
Aug  3 05:36:35 odroid kernel: [    0.000000] On node 0 totalpages: 518656
Aug  3 05:36:35 odroid kernel: [    0.000000] free_area_init_node: node 0, pgdat c0d8ec80, node_mem_map eee25000
Aug  3 05:36:35 odroid kernel: [    0.000000]   Normal zone: 1728 pages used for memmap
Aug  3 05:36:35 odroid kernel: [    0.000000]   Normal zone: 0 pages reserved
From Aug 4 07:00:01 you can see the script being run every minute by cron. This stops abruptly at 7:10:03, and then some time later we can see the kernel rebooting. This is starting to look like an entire system crash rather than loss of network 🤔

Here's the last kernel log from /var/log/kern.log.1

Code: Select all

Aug  2 05:42:07 odroid kernel: [    0.000000] Booting Linux on physical CPU 0x100
Aug  2 05:42:07 odroid kernel: [    0.000000] Linux version 4.14.133-165 (root@1604_builder_armhf) (gcc version 7.3.0 (Ubuntu/Linaro 7.3.0-16ubuntu3)) #1 SMP PREEMPT Mon Jul 15 22:14:09 -03 2019
Aug  2 05:42:07 odroid kernel: [    0.000000] CPU: ARMv7 Processor [410fc073] revision 3 (ARMv7), cr=10c5387d
Aug  2 05:42:07 odroid kernel: [    0.000000] CPU: div instructions available: patching division code
Aug  2 05:42:07 odroid kernel: [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Aug  2 05:42:07 odroid kernel: [    0.000000] OF: fdt: Machine model: Hardkernel Odroid XU4
Aug  2 05:42:07 odroid kernel: [    0.000000] Memory policy: Data cache writealloc
Aug  2 05:42:07 odroid kernel: [    0.000000] cma: Reserved 128 MiB at 0xb6800000
Aug  2 05:42:07 odroid kernel: [    0.000000] Samsung CPU ID: 0xe5422001
Aug  2 05:42:07 odroid kernel: [    0.000000] On node 0 totalpages: 518656
Aug  2 05:42:07 odroid kernel: [    0.000000] free_area_init_node: node 0, pgdat c0d8ec80, node_mem_map eee25000
Aug  2 05:42:07 odroid kernel: [    0.000000]   Normal zone: 1728 pages used for memmap
Aug  2 05:42:07 odroid kernel: [    0.000000]   Normal zone: 0 pages reserved
Aug  2 05:42:07 odroid kernel: [    0.000000]   Normal zone: 196608 pages, LIFO batch:31
Aug  2 05:42:07 odroid kernel: [    0.000000]   HighMem zone: 322048 pages, LIFO batch:31
Aug  2 05:42:07 odroid kernel: [    0.000000] Running under secure firmware.
Aug  2 05:42:07 odroid kernel: [    0.000000] random: get_random_bytes called from start_kernel+0x90/0x420 with crng_init=0
Aug  2 05:42:07 odroid kernel: [    0.000000] percpu: Embedded 17 pages/cpu s37516 r8192 d23924 u69632
Aug  2 05:42:07 odroid kernel: [    0.000000] pcpu-alloc: s37516 r8192 d23924 u69632 alloc=17*4096
Aug  2 05:42:07 odroid kernel: [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7
Aug  2 05:42:07 odroid kernel: [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516928
Aug  2 05:42:07 odroid kernel: [    0.000000] Kernel command line: console=tty1 console=ttySAC2,115200n8 root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait ro fsck.repair=yes net.ifnames=0  HPD=true vout=hdmi usbhid.quirks=0x0eef:0x
0005:0x0004 smsc95xx.macaddr=00:1e:06:61:7a:39 false s5p_mfc.mem=16M
Aug  2 05:42:07 odroid kernel: [    0.000000] hdmi: using HDMI mode
Aug  2 05:42:07 odroid kernel: [    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
Aug  2 05:42:07 odroid kernel: [    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Aug  2 05:42:07 odroid kernel: [    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Aug  2 05:42:07 odroid kernel: [    0.000000] Memory: 1902220K/2074624K available (8192K kernel code, 656K rwdata, 2352K rodata, 1024K init, 318K bss, 41332K reserved, 131072K cma-reserved, 1157120K highmem)
Aug  2 05:42:07 odroid kernel: [    0.000000] Virtual kernel memory layout:
Aug  2 05:42:07 odroid kernel: [    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000]     vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
Aug  2 05:42:07 odroid kernel: [    0.000000]     lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
Aug  2 05:42:07 odroid kernel: [    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
Aug  2 05:42:07 odroid kernel: [    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
Aug  2 05:42:07 odroid kernel: [    0.000000]       .text : 0xc0008000 - 0xc0900000   (9184 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000]       .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000]       .data : 0xc0d00000 - 0xc0da42a0   ( 657 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000]        .bss : 0xc0dac3a8 - 0xc0dfbee0   ( 319 kB)
Aug  2 05:42:07 odroid kernel: [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
Aug  2 05:42:07 odroid kernel: [    0.000000] hperf_hmp: fast CPUs mask: 000000F0
Aug  2 05:42:07 odroid kernel: [    0.000000] hperf_hmp: slow CPUs mask: 0000000F
Aug  2 05:42:07 odroid kernel: [    0.000000] Preemptible hierarchical RCU implementation.
Aug  2 05:42:07 odroid kernel: [    0.000000]   Tasks RCU enabled.
Aug  2 05:42:07 odroid kernel: [    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
Aug  2 05:42:07 odroid kernel: [    0.000000] GIC: Using split EOI/Deactivate mode
Aug  2 05:42:07 odroid kernel: [    0.000000] Switching to timer-based delay loop, resolution 41ns
Aug  2 05:42:07 odroid kernel: [    0.000000] clocksource: mct-frc: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
Aug  2 05:42:07 odroid kernel: [    0.000007] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
Aug  2 05:42:07 odroid kernel: [    0.000030] genirq: irq_chip COMBINER did not update eff. affinity mask of irq 49
Aug  2 05:42:07 odroid kernel: [    0.001613] Console: colour dummy device 80x30
Aug  2 05:42:07 odroid kernel: [    0.002287] console [tty1] enabled
Aug  2 05:42:07 odroid kernel: [    0.002329] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=96000)
Aug  2 05:42:07 odroid kernel: [    0.002370] pid_max: default: 32768 minimum: 301
Aug  2 05:42:07 odroid kernel: [    0.002570] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
Aug  2 05:42:07 odroid kernel: [    0.002611] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
Aug  2 05:42:07 odroid kernel: [    0.003474] CPU: Testing write buffer coherency: ok
Aug  2 05:42:07 odroid kernel: [    0.004395] CPU0: thread -1, cpu 0, socket 1, mpidr 80000100
Aug  2 05:42:07 odroid kernel: [    0.020050] Setting up static identity map for 0x40100000 - 0x40100060
Aug  2 05:42:07 odroid kernel: [    0.020671] ARM CCI driver probed
Aug  2 05:42:07 odroid kernel: [    0.020908] Exynos MCPM support installed
Aug  2 05:42:07 odroid kernel: [    0.027989] Hierarchical SRCU implementation.
Aug  2 05:42:07 odroid kernel: [    0.043990] smp: Bringing up secondary CPUs ...
Aug  2 05:42:07 odroid kernel: [    0.072375] CPU1: thread -1, cpu 1, socket 1, mpidr 80000101
Aug  2 05:42:07 odroid kernel: [    0.096330] CPU2: thread -1, cpu 2, socket 1, mpidr 80000102
Aug  2 05:42:07 odroid kernel: [    0.124325] CPU3: thread -1, cpu 3, socket 1, mpidr 80000103
Aug  2 05:42:07 odroid kernel: [    0.152383] CPU4: thread -1, cpu 0, socket 0, mpidr 80000000
Aug  2 05:42:07 odroid kernel: [    0.152391] CPU4: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable
Aug  2 05:42:07 odroid kernel: [    0.172361] CPU5: thread -1, cpu 1, socket 0, mpidr 80000001
Aug  2 05:42:07 odroid kernel: [    0.172368] CPU5: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable
Aug  2 05:42:07 odroid kernel: [    0.180684] CPU6: thread -1, cpu 2, socket 0, mpidr 80000002
Aug  2 05:42:07 odroid kernel: [    0.180691] CPU6: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable
Aug  2 05:42:07 odroid kernel: [    0.188671] CPU7: thread -1, cpu 3, socket 0, mpidr 80000003
Aug  2 05:42:07 odroid kernel: [    0.188678] CPU7: Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable
Aug  2 05:42:07 odroid kernel: [    0.188931] smp: Brought up 1 node, 8 CPUs
Aug  2 05:42:07 odroid kernel: [    0.188979] SMP: Total of 8 processors activated (384.00 BogoMIPS).
Aug  2 05:42:07 odroid kernel: [    0.189001] CPU: All CPU(s) started in HYP mode.
Aug  2 05:42:07 odroid kernel: [    0.189020] CPU: Virtualization extensions available.
Aug  2 05:42:07 odroid kernel: [    0.190842] devtmpfs: initialized
Aug  2 05:42:07 odroid kernel: [    0.207528] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0
Aug  2 05:42:07 odroid kernel: [    0.207684] hperf_hmp: registered cpufreq transition notifier
Aug  2 05:42:07 odroid kernel: [    0.207869] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
Aug  2 05:42:07 odroid kernel: [    0.207904] futex hash table entries: 2048 (order: 5, 131072 bytes)
Aug  2 05:42:07 odroid kernel: [    0.210699] pinctrl core: initialized pinctrl subsystem
Aug  2 05:42:07 odroid kernel: [    0.212854] NET: Registered protocol family 16
Aug  2 05:42:07 odroid kernel: [    0.218402] DMA: preallocated 1024 KiB pool for atomic coherent allocations
Aug  2 05:42:07 odroid kernel: [    0.224447] cpuidle: using governor menu
Aug  2 05:42:07 odroid kernel: [    0.225264] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
Aug  2 05:42:07 odroid kernel: [    0.225291] hw-breakpoint: maximum watchpoint size is 8 bytes.
Aug  2 05:42:07 odroid kernel: [    0.233595] exynos-audss-clk 3810000.audss-clock-controller: epll 180633609
Aug  2 05:42:07 odroid kernel: [    0.255940] EXYNOS5420 PMU initialized
Aug  2 05:42:07 odroid kernel: [    0.299320] usbcore: registered new interface driver usbfs
Aug  2 05:42:07 odroid kernel: [    0.299389] usbcore: registered new interface driver hub
Aug  2 05:42:07 odroid kernel: [    0.299538] usbcore: registered new device driver usb
Aug  2 05:42:07 odroid kernel: [    0.299983] i2c-gpio soc:i2c-hdmi: using pins 177 (SDA) and 178 (SCL)
Aug  2 05:42:07 odroid kernel: [    0.300575] s3c-i2c 12c70000.i2c: slave address 0x00
Aug  2 05:42:07 odroid kernel: [    0.300602] s3c-i2c 12c70000.i2c: bus frequency set to 378 KHz
Aug  2 05:42:07 odroid kernel: [    0.300828] s3c-i2c 12c70000.i2c: i2c-1: S3C I2C adapter
Aug  2 05:42:07 odroid kernel: [    0.301351] media: Linux media interface: v0.10
Aug  2 05:42:07 odroid kernel: [    0.301399] Linux video capture interface: v2.00
Aug  2 05:42:07 odroid kernel: [    0.301506] pps_core: LinuxPPS API ver. 1 registered
Aug  2 05:42:07 odroid kernel: [    0.301526] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
Aug  2 05:42:07 odroid kernel: [    0.301792] s3c2410-wdt 101d0000.watchdog: watchdog inactive, reset disabled, irq disabled
Aug  2 05:42:07 odroid kernel: [    0.302641] Advanced Linux Sound Architecture Driver Initialized.
Aug  2 05:42:07 odroid kernel: [    0.303853] clocksource: Switched to clocksource mct-frc
Aug  2 05:42:07 odroid kernel: [    0.363944] VFS: Disk quotas dquot_6.6.0
Aug  2 05:42:07 odroid kernel: [    0.364032] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Aug  2 05:42:07 odroid kernel: [    0.377777] NET: Registered protocol family 2
Aug  2 05:42:07 odroid kernel: [    0.378361] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
Aug  2 05:42:07 odroid kernel: [    0.378458] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
Aug  2 05:42:07 odroid kernel: [    0.378626] TCP: Hash tables configured (established 8192 bind 8192)
Aug  2 05:42:07 odroid kernel: [    0.378741] UDP hash table entries: 512 (order: 2, 16384 bytes)
Aug  2 05:42:07 odroid kernel: [    0.378793] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
Aug  2 05:42:07 odroid kernel: [    0.379004] NET: Registered protocol family 1
Aug  2 05:42:07 odroid kernel: [    0.379692] RPC: Registered named UNIX socket transport module.
Aug  2 05:42:07 odroid kernel: [    0.379715] RPC: Registered udp transport module.
Aug  2 05:42:07 odroid kernel: [    0.379733] RPC: Registered tcp transport module.
Aug  2 05:42:07 odroid kernel: [    0.379750] RPC: Registered tcp NFSv4.1 backchannel transport module.
Aug  2 05:42:07 odroid kernel: [    0.380018] Trying to unpack rootfs image as initramfs...
Aug  2 05:42:07 odroid kernel: [    0.671876] random: fast init done
Aug  2 05:42:07 odroid kernel: [    0.833903] Freeing initrd memory: 8120K
Aug  2 05:42:07 odroid kernel: [    0.835369] hw perfevents: enabled with armv7_cortex_a7 PMU driver, 5 counters available
Aug  2 05:42:07 odroid kernel: [    0.836617] hw perfevents: enabled with armv7_cortex_a15 PMU driver, 7 counters available
Aug  2 05:42:07 odroid kernel: [    0.840073] audit: initializing netlink subsys (disabled)
Aug  2 05:42:07 odroid kernel: [    0.840272] audit: type=2000 audit(0.836:1): state=initialized audit_enabled=0 res=1
Aug  2 05:42:07 odroid kernel: [    0.840825] Initialise system trusted keyrings
Aug  2 05:42:07 odroid kernel: [    0.841038] workingset: timestamp_bits=14 max_order=19 bucket_order=5
Aug  2 05:42:07 odroid kernel: [    0.848008] squashfs: version 4.0 (2009/01/31) Phillip Lougher
Aug  2 05:42:07 odroid kernel: [    0.848853] NFS: Registering the id_resolver key type
Aug  2 05:42:07 odroid kernel: [    0.848886] Key type id_resolver registered
Aug  2 05:42:07 odroid kernel: [    0.848905] Key type id_legacy registered
Aug  2 05:42:07 odroid kernel: [    0.848933] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
Aug  2 05:42:07 odroid kernel: [    0.848984] romfs: ROMFS MTD (C) 2007 Red Hat, Inc.
Aug  2 05:42:07 odroid kernel: [    0.853687] Key type asymmetric registered
Aug  2 05:42:07 odroid kernel: [    0.853711] Asymmetric key parser 'x509' registered
Aug  2 05:42:07 odroid kernel: [    0.853776] bounce: pool size: 64 pages
Aug  2 05:42:07 odroid kernel: [    0.853840] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
Aug  2 05:42:07 odroid kernel: [    0.854059] io scheduler noop registered
Aug  2 05:42:07 odroid kernel: [    0.854082] io scheduler deadline registered
Aug  2 05:42:07 odroid kernel: [    0.854235] io scheduler cfq registered (default)
Aug  2 05:42:07 odroid kernel: [    0.854257] io scheduler mq-deadline registered
Aug  2 05:42:07 odroid kernel: [    0.854276] io scheduler kyber registered
Aug  2 05:42:07 odroid kernel: [    0.854510] io scheduler bfq registered
Aug  2 05:42:07 odroid kernel: [    0.856948] samsung-usb2-phy 12130000.phy: 12130000.phy supply vbus not found, using dummy regulator
Aug  2 05:42:07 odroid kernel: [    0.857955] exynos5_usb3drd_phy 12100000.phy: 12100000.phy supply vbus not found, using dummy regulator
Aug  2 05:42:07 odroid kernel: [    0.858040] exynos5_usb3drd_phy 12100000.phy: 12100000.phy supply vbus-boost not found, using dummy regulator
Aug  2 05:42:07 odroid kernel: [    0.858406] exynos5_usb3drd_phy 12500000.phy: 12500000.phy supply vbus not found, using dummy regulator
Aug  2 05:42:07 odroid kernel: [    0.858489] exynos5_usb3drd_phy 12500000.phy: 12500000.phy supply vbus-boost not found, using dummy regulator
Aug  2 05:42:07 odroid kernel: [    0.863369] dma-pl330 3880000.adma: Loaded driver for PL330 DMAC-241330
Aug  2 05:42:07 odroid kernel: [    0.863397] dma-pl330 3880000.adma:   DBUFF-4x8bytes Num_Chans-6 Num_Peri-16 Num_Events-6
Aug  2 05:42:07 odroid kernel: [    0.865928] dma-pl330 121a0000.pdma: Loaded driver for PL330 DMAC-241330
Aug  2 05:42:07 odroid kernel: [    0.865955] dma-pl330 121a0000.pdma:  DBUFF-32x4bytes Num_Chans-8 Num_Peri-32 Num_Events-32
Aug  2 05:42:07 odroid kernel: [    0.868518] dma-pl330 121b0000.pdma: Loaded driver for PL330 DMAC-241330
Aug  2 05:42:07 odroid kernel: [    0.868545] dma-pl330 121b0000.pdma:  DBUFF-32x4bytes Num_Chans-8 Num_Peri-32 Num_Events-32
Aug  2 05:42:07 odroid kernel: [    0.869367] dma-pl330 10800000.mdma: Loaded driver for PL330 DMAC-241330
Aug  2 05:42:07 odroid kernel: [    0.869393] dma-pl330 10800000.mdma:  DBUFF-64x8bytes Num_Chans-8 Num_Peri-1 Num_Events-32
Aug  2 05:42:07 odroid kernel: [    0.935724] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
Aug  2 05:42:07 odroid kernel: [    0.938082] 12c00000.serial: ttySAC0 at MMIO 0x12c00000 (irq = 58, base_baud = 0) is a S3C6400/10
Aug  2 05:42:07 odroid kernel: [    0.938486] 12c10000.serial: ttySAC1 at MMIO 0x12c10000 (irq = 59, base_baud = 0) is a S3C6400/10
Aug  2 05:42:07 odroid kernel: [    0.938875] 12c20000.serial: ttySAC2 at MMIO 0x12c20000 (irq = 60, base_baud = 0) is a S3C6400/10
Aug  2 05:42:07 odroid kernel: [    1.875394] console [ttySAC2] enabled
Aug  2 05:42:07 odroid kernel: [    1.879387] 12c30000.serial: ttySAC3 at MMIO 0x12c30000 (irq = 61, base_baud = 0) is a S3C6400/10
Aug  2 05:42:07 odroid kernel: [    1.890042] exynos-mixer 14450000.mixer: Linked as a consumer to 14650000.sysmmu
Aug  2 05:42:07 odroid kernel: [    1.896068] iommu: Adding device 14450000.mixer to group 0
Aug  2 05:42:07 odroid kernel: [    1.903151] exynos-hdmi 14530000.hdmi: Failed to get supply 'vdd': -517
Aug  2 05:42:07 odroid kernel: [    1.909034] exynos-drm-g2d 10850000.g2d: Linked as a consumer to 10a60000.sysmmu
Aug  2 05:42:07 odroid kernel: [    1.915806] exynos-drm-g2d 10850000.g2d: Linked as a consumer to 10a70000.sysmmu
Aug  2 05:42:07 odroid kernel: [    1.923112] iommu: Adding device 10850000.g2d to group 1
Aug  2 05:42:07 odroid kernel: [    1.928722] exynos-drm-g2d 10850000.g2d: The Exynos G2D (ver 4.1) successfully probed.
Aug  2 05:42:07 odroid kernel: [    1.937320] exynos-rot 11c00000.rotator: Linked as a consumer to 11d40000.sysmmu
Aug  2 05:42:07 odroid kernel: [    1.943642] iommu: Adding device 11c00000.rotator to group 2
Aug  2 05:42:07 odroid kernel: [    1.949597] exynos-rot 11c00000.rotator: The exynos rotator is probed successfully
Aug  2 05:42:07 odroid kernel: [    1.957594] exynos-drm-ipp exynos-drm-ipp: drm ipp registered successfully.
Aug  2 05:42:07 odroid kernel: [    1.968128] mali 11800000.mali: Continuing without Mali regulator control
Aug  2 05:42:07 odroid kernel: [    1.974256] mali 11800000.mali: GPU identified as 0x0620 r0p1 status 0
Aug  2 05:42:07 odroid kernel: [    1.980473] mali 11800000.mali: Protected mode not available
Aug  2 05:42:07 odroid kernel: [    1.986037] devfreq devfreq0: Couldn't update frequency transition information.
Aug  2 05:42:07 odroid kernel: [    1.993577] mali 11800000.mali: Probed as mali0
Aug  2 05:42:07 odroid kernel: [    2.006965] brd: module loaded
Aug  2 05:42:07 odroid kernel: [    2.009224] libphy: Fixed MDIO Bus: probed
Aug  2 05:42:07 odroid kernel: [    2.013015] usbcore: registered new interface driver r8152
Aug  2 05:42:07 odroid kernel: [    2.018175] usbcore: registered new interface driver cdc_ether
Aug  2 05:42:07 odroid kernel: [    2.023955] usbcore: registered new interface driver cdc_subset
Aug  2 05:42:07 odroid kernel: [    2.031964] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Aug  2 05:42:07 odroid kernel: [    2.037068] ehci-exynos: EHCI EXYNOS driver
Aug  2 05:42:07 odroid kernel: [    2.041547] exynos-ehci 12110000.usb: EHCI Host Controller
Aug  2 05:42:07 odroid kernel: [    2.046692] exynos-ehci 12110000.usb: new USB bus registered, assigned bus number 1
Aug  2 05:42:07 odroid kernel: [    2.054613] exynos-ehci 12110000.usb: irq 80, io mem 0x12110000
Aug  2 05:42:07 odroid kernel: [    2.075909] exynos-ehci 12110000.usb: USB 2.0 started, EHCI 1.00
Aug  2 05:42:07 odroid kernel: [    2.080750] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
Aug  2 05:42:07 odroid kernel: [    2.087243] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [    2.094429] usb usb1: Product: EHCI Host Controller
Aug  2 05:42:07 odroid kernel: [    2.099306] usb usb1: Manufacturer: Linux 4.14.133-165 ehci_hcd
Aug  2 05:42:07 odroid kernel: [    2.105172] usb usb1: SerialNumber: 12110000.usb
Aug  2 05:42:07 odroid kernel: [    2.110315] hub 1-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [    2.113535] hub 1-0:1.0: 3 ports detected
Aug  2 05:42:07 odroid kernel: [    2.118396] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Aug  2 05:42:07 odroid kernel: [    2.123639] ohci-exynos: OHCI EXYNOS driver
Aug  2 05:42:07 odroid kernel: [    2.128025] exynos-ohci 12120000.usb: USB Host Controller
Aug  2 05:42:07 odroid kernel: [    2.133180] exynos-ohci 12120000.usb: new USB bus registered, assigned bus number 2
Aug  2 05:42:07 odroid kernel: [    2.140900] exynos-ohci 12120000.usb: irq 80, io mem 0x12120000
Aug  2 05:42:07 odroid kernel: [    2.208142] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
Aug  2 05:42:07 odroid kernel: [    2.213496] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [    2.221227] usb usb2: Product: USB Host Controller
Aug  2 05:42:07 odroid kernel: [    2.225448] usb usb2: Manufacturer: Linux 4.14.133-165 ohci_hcd
Aug  2 05:42:07 odroid kernel: [    2.231338] usb usb2: SerialNumber: 12120000.usb
Aug  2 05:42:07 odroid kernel: [    2.236444] hub 2-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [    2.239660] hub 2-0:1.0: 3 ports detected
Aug  2 05:42:07 odroid kernel: [    2.245886] mousedev: PS/2 mouse device common for all mice
Aug  2 05:42:07 odroid kernel: [    2.251291] i2c /dev entries driver
Aug  2 05:42:07 odroid kernel: [    2.269272] vdd_ldo9: Bringing 3300000uV into 3000000-3000000uV
Aug  2 05:42:07 odroid kernel: [    2.291590] vdd_sd: Bringing 3300000uV into 2800000-2800000uV
Aug  2 05:42:07 odroid kernel: [    2.339303] vdd_1.8v_ldo: Bringing 1850000uV into 1500000-1500000uV
Aug  2 05:42:07 odroid kernel: [    3.385735] s5m-rtc s2mps14-rtc: rtc core: registered s5m-rtc as rtc0
Aug  2 05:42:07 odroid kernel: [    3.394722] s5p-jpeg 11f50000.jpeg: Linked as a consumer to 11f10000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.400312] iommu: Adding device 11f50000.jpeg to group 3
Aug  2 05:42:07 odroid kernel: [    3.405996] s5p-jpeg 11f50000.jpeg: encoder device registered as /dev/video30
Aug  2 05:42:07 odroid kernel: [    3.412900] s5p-jpeg 11f50000.jpeg: decoder device registered as /dev/video31
Aug  2 05:42:07 odroid kernel: [    3.419831] s5p-jpeg 11f50000.jpeg: Samsung S5P JPEG codec
Aug  2 05:42:07 odroid kernel: [    3.425503] s5p-jpeg 11f60000.jpeg: Linked as a consumer to 11f20000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.432267] iommu: Adding device 11f60000.jpeg to group 4
Aug  2 05:42:07 odroid kernel: [    3.437958] s5p-jpeg 11f60000.jpeg: encoder device registered as /dev/video32
Aug  2 05:42:07 odroid kernel: [    3.444870] s5p-jpeg 11f60000.jpeg: decoder device registered as /dev/video33
Aug  2 05:42:07 odroid kernel: [    3.451806] s5p-jpeg 11f60000.jpeg: Samsung S5P JPEG codec
Aug  2 05:42:07 odroid kernel: [    3.458381] s5p-mfc 11000000.codec: Linked as a consumer to 11200000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.464333] s5p-mfc 11000000.codec: Linked as a consumer to 11210000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.471188] iommu: Adding device 11000000.codec to group 5
Aug  2 05:42:07 odroid kernel: [    3.494409] s5p-mfc 11000000.codec: preallocated 16 MiB buffer for the firmware and context buffers
Aug  2 05:42:07 odroid kernel: [    3.502124] (NULL device *): Direct firmware load for s5p-mfc-v8.fw failed with error -2
Aug  2 05:42:07 odroid kernel: [    3.510091] s5p_mfc_load_firmware:73: Firmware is not present in the /lib/firmware directory nor compiled in kernel
Aug  2 05:42:07 odroid kernel: [    3.520652] s5p-mfc 11000000.codec: decoder registered as /dev/video10
Aug  2 05:42:07 odroid kernel: [    3.527142] s5p-mfc 11000000.codec: encoder registered as /dev/video11
Aug  2 05:42:07 odroid kernel: [    3.535194] exynos-gsc 13e00000.video-scaler: Linked as a consumer to 13e80000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.541640] iommu: Adding device 13e00000.video-scaler to group 6
Aug  2 05:42:07 odroid kernel: [    3.548974] exynos-gsc 13e10000.video-scaler: Linked as a consumer to 13e90000.sysmmu
Aug  2 05:42:07 odroid kernel: [    3.555497] iommu: Adding device 13e10000.video-scaler to group 7
Aug  2 05:42:07 odroid kernel: [   15.210592] thermal thermal_zone0: failed to read out thermal zone (-22)
Aug  2 05:42:07 odroid kernel: [   15.216761] thermal thermal_zone1: failed to read out thermal zone (-22)
Aug  2 05:42:07 odroid kernel: [   15.223458] thermal thermal_zone2: failed to read out thermal zone (-22)
Aug  2 05:42:07 odroid kernel: [   15.230151] thermal thermal_zone3: failed to read out thermal zone (-22)
Aug  2 05:42:07 odroid kernel: [   15.236843] thermal thermal_zone4: failed to read out thermal zone (-22)
Aug  2 05:42:07 odroid kernel: [   15.243294] device-mapper: uevent: version 1.0.3
Aug  2 05:42:07 odroid kernel: [   15.247469] device-mapper: ioctl: 4.37.0-ioctl (2017-09-20) initialised: dm-devel@redhat.com
Aug  2 05:42:07 odroid kernel: [   15.269857] sdhci: Secure Digital Host Controller Interface driver
Aug  2 05:42:07 odroid kernel: [   15.274580] sdhci: Copyright(c) Pierre Ossman
Aug  2 05:42:07 odroid kernel: [   15.279111] Synopsys Designware Multimedia Card Interface Driver
Aug  2 05:42:07 odroid kernel: [   15.285497] dwmmc_exynos 12200000.mmc: IDMAC supports 32-bit address mode.
Aug  2 05:42:07 odroid kernel: [   15.291805] dwmmc_exynos 12200000.mmc: Using internal DMA controller.
Aug  2 05:42:07 odroid kernel: [   15.298148] dwmmc_exynos 12200000.mmc: Version ID is 250a
Aug  2 05:42:07 odroid kernel: [   15.303539] dwmmc_exynos 12200000.mmc: DW MMC controller at irq 83,64 bit host data width,64 deep fifo
Aug  2 05:42:07 odroid kernel: [   15.312944] dwmmc_exynos 12200000.mmc: allocated mmc-pwrseq
Aug  2 05:42:07 odroid kernel: [   15.341103] mmc_host mmc0: Bus speed (slot 0) = 50000000Hz (slot req 400000Hz, actual 396825HZ div = 63)
Aug  2 05:42:07 odroid kernel: [   15.368242] dwmmc_exynos 12220000.mmc: IDMAC supports 32-bit address mode.
Aug  2 05:42:07 odroid kernel: [   15.373699] dwmmc_exynos 12220000.mmc: Using internal DMA controller.
Aug  2 05:42:07 odroid kernel: [   15.380097] dwmmc_exynos 12220000.mmc: Version ID is 250a
Aug  2 05:42:07 odroid kernel: [   15.385475] dwmmc_exynos 12220000.mmc: DW MMC controller at irq 84,64 bit host data width,64 deep fifo
Aug  2 05:42:07 odroid kernel: [   15.416336] mmc_host mmc1: Bus speed (slot 0) = 50000000Hz (slot req 400000Hz, actual 396825HZ div = 63)
Aug  2 05:42:07 odroid kernel: [   15.444930] ledtrig-cpu: registered to indicate activity on CPUs
Aug  2 05:42:07 odroid kernel: [   15.453276] s5p-secss 10830000.sss: s5p-sss driver registered
Aug  2 05:42:07 odroid kernel: [   15.457956] hidraw: raw HID events driver (C) Jiri Kosina
Aug  2 05:42:07 odroid kernel: [   15.471091] exynos-nocp: new NoC Probe device registered: 10ca1000.nocp
Aug  2 05:42:07 odroid kernel: [   15.476373] exynos-nocp: new NoC Probe device registered: 10ca1400.nocp
Aug  2 05:42:07 odroid kernel: [   15.482948] exynos-nocp: new NoC Probe device registered: 10ca1800.nocp
Aug  2 05:42:07 odroid kernel: [   15.489538] exynos-nocp: new NoC Probe device registered: 10ca1c00.nocp
Aug  2 05:42:07 odroid kernel: [   15.502213] NET: Registered protocol family 17
Aug  2 05:42:07 odroid kernel: [   15.505424] NET: Registered protocol family 15
Aug  2 05:42:07 odroid kernel: [   15.509661] Key type dns_resolver registered
Aug  2 05:42:07 odroid kernel: [   15.514235] Registering SWP/SWPB emulation handler
Aug  2 05:42:07 odroid kernel: [   15.518949] registered taskstats version 1
Aug  2 05:42:07 odroid kernel: [   15.523754] Loading compiled-in X.509 certificates
Aug  2 05:42:07 odroid kernel: [   15.533000] Key type encrypted registered
Aug  2 05:42:07 odroid kernel: [   15.534706] mmc_host mmc1: Bus speed (slot 0) = 200000000Hz (slot req 200000000Hz, actual 200000000HZ div = 0)
Aug  2 05:42:07 odroid kernel: [   15.546204] mmc1: new ultra high speed SDR104 SDXC card at address 0001
Aug  2 05:42:07 odroid kernel: [   15.552477] mmcblk1: mmc1:0001 EC2QT 59.6 GiB
Aug  2 05:42:07 odroid kernel: [   15.557487]  mmcblk1: p1 p2
Aug  2 05:42:07 odroid kernel: [   15.571348] OF: graph: no port node found in /soc/hdmi@14530000
Aug  2 05:42:07 odroid kernel: [   15.576711] [drm] Exynos DRM: using 14450000.mixer device for DMA mapping operations
Aug  2 05:42:07 odroid kernel: [   15.583665] exynos-drm exynos-drm: bound 14450000.mixer (ops 0xc0966304)
Aug  2 05:42:07 odroid kernel: [   15.590202] exynos-drm exynos-drm: bound 14530000.hdmi (ops 0xc09669a4)
Aug  2 05:42:07 odroid kernel: [   15.596783] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
Aug  2 05:42:07 odroid kernel: [   15.603362] [drm] No driver support for vblank timestamp query.
Aug  2 05:42:07 odroid kernel: [   15.609361] [drm] Cannot find any crtc or sizes
Aug  2 05:42:07 odroid kernel: [   15.614199] [drm] Initialized exynos 1.0.0 20110530 for exynos-drm on minor 0
Aug  2 05:42:07 odroid kernel: [   15.621124] [drm] Cannot find any crtc or sizes
Aug  2 05:42:07 odroid kernel: [   15.626887] xhci-hcd xhci-hcd.3.auto: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.630922] xhci-hcd xhci-hcd.3.auto: new USB bus registered, assigned bus number 3
Aug  2 05:42:07 odroid kernel: [   15.638767] xhci-hcd xhci-hcd.3.auto: hcc params 0x0220f04c hci version 0x100 quirks 0x0000000002010010
Aug  2 05:42:07 odroid kernel: [   15.647931] xhci-hcd xhci-hcd.3.auto: irq 149, io mem 0x12000000
Aug  2 05:42:07 odroid kernel: [   15.654007] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
Aug  2 05:42:07 odroid kernel: [   15.660633] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [   15.667822] usb usb3: Product: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.672676] usb usb3: Manufacturer: Linux 4.14.133-165 xhci-hcd
Aug  2 05:42:07 odroid kernel: [   15.678568] usb usb3: SerialNumber: xhci-hcd.3.auto
Aug  2 05:42:07 odroid kernel: [   15.683724] hub 3-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [   15.687171] hub 3-0:1.0: 1 port detected
Aug  2 05:42:07 odroid kernel: [   15.691237] xhci-hcd xhci-hcd.3.auto: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.696511] xhci-hcd xhci-hcd.3.auto: new USB bus registered, assigned bus number 4
Aug  2 05:42:07 odroid kernel: [   15.704145] xhci-hcd xhci-hcd.3.auto: Host supports USB 3.0  SuperSpeed
Aug  2 05:42:07 odroid kernel: [   15.710769] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
Aug  2 05:42:07 odroid kernel: [   15.718872] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
Aug  2 05:42:07 odroid kernel: [   15.725537] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [   15.732726] usb usb4: Product: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.737580] usb usb4: Manufacturer: Linux 4.14.133-165 xhci-hcd
Aug  2 05:42:07 odroid kernel: [   15.743472] usb usb4: SerialNumber: xhci-hcd.3.auto
Aug  2 05:42:07 odroid kernel: [   15.748618] hub 4-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [   15.752070] hub 4-0:1.0: 1 port detected
Aug  2 05:42:07 odroid kernel: [   15.757971] xhci-hcd xhci-hcd.6.auto: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.762001] xhci-hcd xhci-hcd.6.auto: new USB bus registered, assigned bus number 5
Aug  2 05:42:07 odroid kernel: [   15.769828] xhci-hcd xhci-hcd.6.auto: hcc params 0x0220f04c hci version 0x100 quirks 0x0000000002010010
Aug  2 05:42:07 odroid kernel: [   15.779007] xhci-hcd xhci-hcd.6.auto: irq 150, io mem 0x12400000
Aug  2 05:42:07 odroid kernel: [   15.785082] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
Aug  2 05:42:07 odroid kernel: [   15.791714] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [   15.798908] usb usb5: Product: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.803757] usb usb5: Manufacturer: Linux 4.14.133-165 xhci-hcd
Aug  2 05:42:07 odroid kernel: [   15.809648] usb usb5: SerialNumber: xhci-hcd.6.auto
Aug  2 05:42:07 odroid kernel: [   15.814794] hub 5-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [   15.818251] hub 5-0:1.0: 1 port detected
Aug  2 05:42:07 odroid kernel: [   15.822320] xhci-hcd xhci-hcd.6.auto: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.827595] xhci-hcd xhci-hcd.6.auto: new USB bus registered, assigned bus number 6
Aug  2 05:42:07 odroid kernel: [   15.835219] xhci-hcd xhci-hcd.6.auto: Host supports USB 3.0  SuperSpeed
Aug  2 05:42:07 odroid kernel: [   15.841854] usb usb6: We don't know the algorithms for LPM for this host, disabling LPM.
Aug  2 05:42:07 odroid kernel: [   15.849953] usb usb6: New USB device found, idVendor=1d6b, idProduct=0003
Aug  2 05:42:07 odroid kernel: [   15.856618] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Aug  2 05:42:07 odroid kernel: [   15.863811] usb usb6: Product: xHCI Host Controller
Aug  2 05:42:07 odroid kernel: [   15.868661] usb usb6: Manufacturer: Linux 4.14.133-165 xhci-hcd
Aug  2 05:42:07 odroid kernel: [   15.874553] usb usb6: SerialNumber: xhci-hcd.6.auto
Aug  2 05:42:07 odroid kernel: [   15.879732] hub 6-0:1.0: USB hub found
Aug  2 05:42:07 odroid kernel: [   15.883153] hub 6-0:1.0: 1 port detected
Aug  2 05:42:07 odroid kernel: [   15.888009] rtc rtc1: invalid alarm value: 1900-1-1 0:0:0
Aug  2 05:42:07 odroid kernel: [   15.892507] s3c-rtc 101e0000.rtc: rtc core: registered s3c as rtc1
Aug  2 05:42:07 odroid kernel: [   15.899715] exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.907215] exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.915149] exynos-bus: new bus device registered: soc:bus_fsys_apb (200000 KHz ~ 200000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.923588] exynos-bus: new bus device registered: soc:bus_fsys (200000 KHz ~ 200000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.931880] exynos-bus: new bus device registered: soc:bus_fsys2 ( 75000 KHz ~ 150000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.940180] exynos-bus: new bus device registered: soc:bus_mfc ( 96000 KHz ~ 333000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.948210] exynos-bus: new bus device registered: soc:bus_gen ( 89000 KHz ~ 267000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.956178] exynos-bus: new bus device registered: soc:bus_peri ( 67000 KHz ~  67000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.964434] exynos-bus: new bus device registered: soc:bus_g2d ( 84000 KHz ~ 333000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.972444] exynos-bus: new bus device registered: soc:bus_g2d_acp ( 67000 KHz ~ 267000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.980856] exynos-bus: new bus device registered: soc:bus_jpeg ( 75000 KHz ~ 300000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.989002] exynos-bus: new bus device registered: soc:bus_jpeg_apb ( 84000 KHz ~ 167000 KHz)
Aug  2 05:42:07 odroid kernel: [   15.997422] exynos-bus: new bus device registered: soc:bus_disp1_fimd (120000 KHz ~ 200000 KHz)
Aug  2 05:42:07 odroid kernel: [   16.006119] exynos-bus: new bus device registered: soc:bus_disp1 (120000 KHz ~ 300000 KHz)
Aug  2 05:42:07 odroid kernel: [   16.014350] exynos-bus: new bus device registered: soc:bus_gscl_scaler (150000 KHz ~ 300000 KHz)
Aug  2 05:42:07 odroid kernel: [   16.023183] exynos-bus: new bus device registered: soc:bus_mscl ( 84000 KHz ~ 400000 KHz)
Aug  2 05:42:07 odroid kernel: [   16.031226] samsung-i2s 3830000.i2s: ASoC: Failed to create component debugfs directory
Aug  2 05:42:07 odroid kernel: [   16.038882] samsung-i2s 3830000.i2s: ASoC: Failed to create component debugfs directory
Aug  2 05:42:07 odroid kernel: [   16.047388] odroid-audio sound: multicodec <-> 3830000.i2s mapping ok
Aug  2 05:42:07 odroid kernel: [   16.056169] s5m-rtc s2mps14-rtc: setting system clock to 2019-08-02 05:42:02 UTC (1564724522)
Aug  2 05:42:07 odroid kernel: [   16.068137] ALSA device list:
Aug  2 05:42:07 odroid kernel: [   16.069625]   #0: Odroid-XU4
Aug  2 05:42:07 odroid kernel: [   16.073589] Freeing unused kernel memory: 1024K
Aug  2 05:42:07 odroid kernel: [   17.051971] usb 6-1: new SuperSpeed USB device number 2 using xhci-hcd
Aug  2 05:42:07 odroid kernel: [   17.076674] usb 6-1: New USB device found, idVendor=0bda, idProduct=8153
Aug  2 05:42:07 odroid kernel: [   17.081934] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=6
Aug  2 05:42:07 odroid kernel: [   17.089033] usb 6-1: Product: USB 10/100/1000 LAN
Aug  2 05:42:07 odroid kernel: [   17.093715] usb 6-1: Manufacturer: Realtek
Aug  2 05:42:07 odroid kernel: [   17.097781] usb 6-1: SerialNumber: 000001000000
Aug  2 05:42:07 odroid kernel: [   17.265957] usb 6-1: reset SuperSpeed USB device number 2 using xhci-hcd
Aug  2 05:42:07 odroid kernel: [   17.332828] r8152 6-1:1.0 eth0: v1.09.9
Aug  2 05:42:07 odroid kernel: [   17.472247] EXT4-fs (mmcblk1p2): mounted filesystem without journal. Opts: (null)
Aug  2 05:42:07 odroid kernel: [   17.784537] NET: Registered protocol family 10
Aug  2 05:42:07 odroid kernel: [   17.788285] Segment Routing with IPv6
Aug  2 05:42:07 odroid kernel: [   17.793500] ip_tables: (C) 2000-2006 Netfilter Core Team
Aug  2 05:42:07 odroid kernel: [   18.106090] random: systemd: uninitialized urandom read (16 bytes read)
Aug  2 05:42:07 odroid kernel: [   18.131971] random: systemd: uninitialized urandom read (16 bytes read)
Aug  2 05:42:07 odroid kernel: [   18.155945] random: systemd: uninitialized urandom read (16 bytes read)
Aug  2 05:42:07 odroid kernel: [   18.330655] EXT4-fs (mmcblk1p2): re-mounted. Opts: errors=remount-ro
Aug  2 05:42:07 odroid kernel: [   19.122172] random: crng init done
Aug  2 05:42:07 odroid kernel: [   19.122186] random: 7 urandom warning(s) missed due to ratelimiting
Aug  2 05:42:07 odroid kernel: [   19.339194] input: gpio_keys as /devices/platform/gpio_keys/input/input0
Aug  2 05:42:07 odroid kernel: [   19.342865] gpiomem-exynos 13400000.gpiomem: Initialised: GPIO register area is 2
Aug  2 05:42:07 odroid kernel: [   19.343107] gpiomem-exynos 13400000.gpiomem: Initialised: Registers at 0x13400000
Aug  2 05:42:07 odroid kernel: [   19.343115] gpiomem-exynos 13400000.gpiomem: Initialised: Registers at 0x14010000
Aug  2 05:42:07 odroid kernel: [   21.489763] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Aug  2 05:42:07 odroid kernel: [   21.494434] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Aug  2 05:42:07 odroid kernel: [   21.522819] r8152 6-1:1.0 eth0: carrier on
Aug  2 05:42:07 odroid kernel: [   21.522968] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Aug  2 05:42:12 odroid kernel: [   25.740240] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
Aug  2 05:42:12 odroid kernel: [   25.761499] usb 4-1: New USB device found, idVendor=152d, idProduct=0578
Aug  2 05:42:12 odroid kernel: [   25.761524] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Aug  2 05:42:12 odroid kernel: [   25.761543] usb 4-1: Product: USB to SATA bridge
Aug  2 05:42:12 odroid kernel: [   25.761561] usb 4-1: Manufacturer: JMicron
Aug  2 05:42:12 odroid kernel: [   25.761579] usb 4-1: SerialNumber: DB00000000013B
Aug  2 05:42:12 odroid kernel: [   25.797442] SCSI subsystem initialized
Aug  2 05:42:12 odroid kernel: [   25.812183] usbcore: registered new interface driver usb-storage
Aug  2 05:42:12 odroid kernel: [   25.822957] scsi host0: uas
Aug  2 05:42:12 odroid kernel: [   25.823298] usbcore: registered new interface driver uas
Aug  2 05:42:12 odroid kernel: [   25.824119] scsi 0:0:0:0: Direct-Access     JMicron                   3101 PQ: 0 ANSI: 6
Aug  2 05:42:12 odroid kernel: [   25.849119] scsi 0:0:0:0: Attached scsi generic sg0 type 0
Aug  2 05:42:12 odroid kernel: [   25.852854] sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
Aug  2 05:42:12 odroid kernel: [   25.852863] sd 0:0:0:0: [sda] 4096-byte physical blocks
Aug  2 05:42:12 odroid kernel: [   25.853048] sd 0:0:0:0: [sda] Write Protect is off
Aug  2 05:42:12 odroid kernel: [   25.853058] sd 0:0:0:0: [sda] Mode Sense: 53 00 00 08
Aug  2 05:42:12 odroid kernel: [   25.853450] sd 0:0:0:0: [sda] Disabling FUA
Aug  2 05:42:12 odroid kernel: [   25.853458] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Aug  2 05:42:12 odroid kernel: [   25.853823] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes)
Aug  2 05:42:12 odroid kernel: [   25.863969]  sda: sda1
Aug  2 05:42:12 odroid kernel: [   25.866035] sd 0:0:0:0: [sda] Attached SCSI disk
Aug  2 15:00:25 odroid kernel: [33520.824612] r8152 6-1:1.0 eth0: carrier off
Aug  2 15:00:28 odroid kernel: [33523.444227] r8152 6-1:1.0 eth0: carrier on
Aug  2 15:01:17 odroid kernel: [33573.400529] r8152 6-1:1.0 eth0: carrier off
Aug  2 15:01:20 odroid kernel: [33576.052720] r8152 6-1:1.0 eth0: carrier on
Aug  3 15:00:25 odroid kernel: [119920.856150] r8152 6-1:1.0 eth0: carrier off
Aug  3 15:00:27 odroid kernel: [119923.475970] r8152 6-1:1.0 eth0: carrier on
Aug  3 15:01:17 odroid kernel: [119973.440413] r8152 6-1:1.0 eth0: carrier off
Aug  3 15:01:20 odroid kernel: [119976.148142] r8152 6-1:1.0 eth0: carrier on
  1. Are there any other logs I can post to shed some more light on the root cause?
  2. Any chance this is related to this XU4 thread?
  3. For the next step I'm going to swap out the HDD (it's the only thing aside from Ubuntu and the LAN that hasn't changed) and install linux-crashdump as recommended by Ubuntu for debugging crashes

User avatar
mad_ady
Posts: 6805
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 218 times
Been thanked: 167 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by mad_ady » Sun Aug 04, 2019 6:45 pm

Hmm. Either it could still reach other hosts with ping, or something (oom-killer?) killed cron, or something happened to the disk and it couldn't save the logs. Where is your rootfs? On the sd card or on the hdd?
I'm guessing you don't have a serial adapter, right? In this case you should enable netconsole (look for the thread/odroid-magazine article) and export kernel logs over the network to a collector. Hopefully, what triggers the problem generates some logs before killing the system.
Are you using dhcp? You should also try with a static ip. Maybe the dhcp process fails to renew your lease...
Also, look into monitoring system resources - with something like munin or netdata. In this case you should be able to see system usage values before the crash - maybe it will expose some low ram or too many processes...
From your last log I can see that the network is flapping. I'd recheck network cables/connectors, since it doesn't seem related with the usb bus.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Aug 06, 2019 5:29 am

Hey @mad_ady, thanks again for continuing to help me diagnose.

My root file system is on an SD card. As mentioned in my original post,
nz_will wrote:
Sat Jun 01, 2019 12:05 pm
I've cloned the 32GB micro SD card to a 64GB micro SD card (both class 10) in case that was the issue. Despite this, it happened again.
This makes me think it is unlikely that the SD card is the issue.

I don't have a serial adapter, so I'll look into netconsole. Having said that, I don't think that this is a kernel crash because I see a steady blinking blue LED after it becomes unresponsive. Like x..x........x..x........x..x........x..x. As mentioned by DarkBahamut:
DarkBahamut wrote:
Sun Feb 25, 2018 8:53 am
The speed of the blue 'heartbeat' LED is linked to CPU usage. A slow pulse with a long gap indicates low usage, and it speeds up to a very fast blink with almost no gap at full load.
Everything is fine unless it either stops blinking completely or goes solid. This means the kernel has crashed so it also serves as a handy indicator for checking the system is live when running headless :)
Yes I'm using DHCP. My Odroid is allocated a reserved IP address (192.168.1.2) from the pool (192.168.1.0/24) based on its MAC address. When I looked at my router config, I noticed that the DHCP lease time is set to 24 hours. I've now set that to 15 minutes. If DHCP was the cause, I'll expect to see a decrease in the time between failures.

As for monitoring system resources... I'll look into that.

As for network cables and connectors, I'll swap out the cable and the port on the router that it is connected to so that we can eliminate this. I don't think that any hardware on the Odroid is at fault here, because this is my second new Odroid with this problem and I have swapped out the SD card and HDD for different ones. I'll let you know if I have more data on the next crash, or if they start happening more regularly (suggesting DHCP).

Thanks again! I'll keep you posted.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Aug 06, 2019 9:34 am

Another update: it crashed again today about 2 hours after I changed the DHCP settings, router port, and cable. Here's the interesting thing: I unplugged the network cable, plugged it in again, and the Odroid started responding to pings again. What?!

Here are the relevant syslogs from the time. There is definitely something funky happening with DHCP:

Code: Select all

Aug  6 00:20:50 odroid NetworkManager[395]: <info>  [1565050850.2428] manager: NetworkManager state is now CONNECTED_SITE
Aug  6 00:20:50 odroid dbus-daemon[391]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.7' (uid=0 pid=395 comm="/usr/sbin/NetworkManager --no-daemon ")
Aug  6 00:20:50 odroid systemd[1]: Starting Network Manager Script Dispatcher Service...
Aug  6 00:20:50 odroid dbus-daemon[391]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Aug  6 00:20:50 odroid systemd[1]: Started Network Manager Script Dispatcher Service.
Aug  6 00:20:50 odroid nm-dispatcher: req:1 'connectivity-change': new request (1 scripts)
Aug  6 00:20:50 odroid nm-dispatcher: req:1 'connectivity-change': start running ordered scripts...
Aug  6 00:22:05 odroid kernel: [11928.499411] r8152 6-1:1.0 eth0: carrier off
Aug  6 00:22:11 odroid NetworkManager[395]: <info>  [1565050931.7746] device (eth0): state change: activated -> unavailable (reason 'carrier-changed', sys-iface-state: 'managed')
Aug  6 00:22:11 odroid NetworkManager[395]: <info>  [1565050931.7965] manager: NetworkManager state is now CONNECTED_LOCAL
Aug  6 00:22:11 odroid systemd[1]: Starting resolvconf-pull-resolved.service...
Aug  6 00:22:11 odroid dbus-daemon[391]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.7' (uid=0 pid=395 comm="/usr/sbin/NetworkManager --no-daemon ")
Aug  6 00:22:11 odroid systemd[1]: Starting Network Manager Script Dispatcher Service...
Aug  6 00:22:11 odroid systemd[1]: Started resolvconf-pull-resolved.service.
Aug  6 00:22:11 odroid dbus-daemon[391]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Aug  6 00:22:11 odroid systemd[1]: Started Network Manager Script Dispatcher Service.
Aug  6 00:22:11 odroid nm-dispatcher: req:1 'down' [eth0]: new request (1 scripts)
Aug  6 00:22:11 odroid nm-dispatcher: req:1 'down' [eth0]: start running ordered scripts...
Aug  6 00:22:14 odroid kernel: [11936.745943] r8152 6-1:1.0 eth0: carrier on
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0180] device (eth0): carrier: link connected
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0233] device (eth0): state change: unavailable -> disconnected (reason 'carrier-changed', sys-iface-state: 'managed')
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0290] policy: auto-activating connection 'Wired connection 1'
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0366] device (eth0): Activation: starting connection 'Wired connection 1' (0e5d5e2c-3c1e-37f0-9993-9f2383399891)
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0375] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0383] manager: NetworkManager state is now CONNECTING
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0401] device (eth0): state change: prepare -> config(reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0420] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0433] dhcp4 (eth0): activation: beginning transaction (timeout in 45 seconds)
Aug  6 00:22:14 odroid NetworkManager[395]: <info>  [1565050934.0679] dhcp4 (eth0): dhclient started with pid 6211
Aug  6 00:22:14 odroid dhclient[6211]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0xce38ef2e)
Aug  6 00:22:17 odroid dhclient[6211]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0xce38ef2e)
Aug  6 00:22:18 odroid dhclient[6211]: DHCPREQUEST of 192.168.1.2 on eth0 to 255.255.255.255 port 67 (xid=0x2eef38ce)
Aug  6 00:22:18 odroid dhclient[6211]: DHCPOFFER of 192.168.1.2 from 192.168.1.1
Aug  6 00:22:18 odroid dhclient[6211]: DHCPACK of 192.168.1.2 from 192.168.1.1
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1731] dhcp4 (eth0):   address 192.168.1.2
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1731] dhcp4 (eth0):   plen 24 (255.255.255.0)
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1732] dhcp4 (eth0):   gateway 192.168.1.1
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1733] dhcp4 (eth0):   lease time 900
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1734] dhcp4 (eth0):   nameserver '121.98.0.2'
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1734] dhcp4 (eth0):   nameserver '121.98.0.1'
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1735] dhcp4 (eth0): state changed unknown -> bound
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1789] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1826] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1841] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.1848] manager: NetworkManager state is now CONNECTED_LOCAL
Aug  6 00:22:18 odroid dhclient[6211]: bound to 192.168.1.2 -- renewal in 408 seconds.
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.2022] manager: NetworkManager state is now CONNECTED_SITE
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.2027] policy: set 'Wired connection 1' (eth0) as default for IPv4 routing and DNS
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.2058] device (eth0): Activation: successful, device activated.
Aug  6 00:22:18 odroid NetworkManager[395]: <info>  [1565050938.2122] manager: NetworkManager state is now CONNECTED_GLOBAL
Aug  6 00:22:18 odroid systemd[1]: Starting resolvconf-pull-resolved.service...
Aug  6 00:22:18 odroid nm-dispatcher: req:2 'up' [eth0]: new request (1 scripts)
Aug  6 00:22:18 odroid nm-dispatcher: req:2 'up' [eth0]: start running ordered scripts...
Aug  6 00:22:18 odroid nm-dispatcher: req:3 'connectivity-change': new request (1 scripts)
Aug  6 00:22:18 odroid systemd[1]: Started resolvconf-pull-resolved.service.
Aug  6 00:22:18 odroid systemd[1]: Reloading OpenBSD Secure Shell server.
Aug  6 00:22:18 odroid systemd[1]: Reloaded OpenBSD Secure Shell server.
Aug  6 00:22:18 odroid nm-dispatcher: req:3 'connectivity-change': start running ordered scripts...
Aug  6 00:22:41 odroid systemd[1]: Started Session 9 of user will.

User avatar
mad_ady
Posts: 6805
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 218 times
Been thanked: 167 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by mad_ady » Tue Aug 06, 2019 1:23 pm

Based on your log I see that:
* a carrier off causes the disconnection. To me this indicates a faulty cable/faulty port on the odroid/router (or you pulling/replugging the cable)
* NM stops the connection, but restores it correctly once carrier is on again. DHCP works. It should work automatically without the need to reboot. Is the odroid not responding to ping/ssh once you do that?

How about the watchdog script? Is it not collecting logs/rebooting the system? Does the script run ok when run manually?

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Wed Aug 07, 2019 3:45 am

I've got another syslog for you. Uptime Robot reported my web server went down somewhere between 2019-08-06T15:54:11Z and 2019-08-06T15:59:11Z. Here is the syslog from the time:

Here's the timeline
  • From 15:00:38 - 15:01:34 (I think) we can see a DHCP lease being renewed
  • At 15:40:23 Network Manager starts running some scripts
  • Around 15:54:00, Uptime Robot reports that my site is down
  • Afterward, we see Docker failing to hit internet services
  • At 18:06:57 we see me disconnecting the network cable
  • At 18:07:03 we see me reconnecting the network cable having tried my best to count to 10 (but having actually counted to 8).

Code: Select all

Aug  6 15:00:38 odroid kernel: [64641.358747] r8152 6-1:1.0 eth0: carrier off
Aug  6 15:00:41 odroid kernel: [64644.042854] r8152 6-1:1.0 eth0: carrier on
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.3743] device (eth0): carrier: link connected
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.3778] device (eth0): DHCPv4 lease renewal requested
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.4358] dhcp4 (eth0): canceled DHCP transaction, DHCP client pid 10044
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.4359] dhcp4 (eth0): state changed bound -> done
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.4390] dhcp4 (eth0): activation: beginning transaction (timeout in 45 seconds)
Aug  6 15:00:41 odroid NetworkManager[395]: <info>  [1565103641.4823] dhcp4 (eth0): dhclient started with pid 19038
Aug  6 15:00:41 odroid dhclient[19038]: DHCPREQUEST of 192.168.1.2 on eth0 to 255.255.255.255 port 67 (xid=0x359fa458)
Aug  6 15:00:51 odroid dhclient[19038]: message repeated 3 times: [ DHCPREQUEST of 192.168.1.2 on eth0 to 255.255.255.255 port 67 (xid=0x359fa458)]
Aug  6 15:01:01 odroid dhclient[19038]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0x182b124a)
Aug  6 15:01:04 odroid dhclient[19038]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5 (xid=0x182b124a)
Aug  6 15:01:09 odroid dhclient[19038]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 11 (xid=0x182b124a)
Aug  6 15:01:20 odroid dhclient[19038]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13 (xid=0x182b124a)
Aug  6 15:01:26 odroid NetworkManager[395]: <warn>  [1565103686.4996] dhcp4 (eth0): request timed out
Aug  6 15:01:26 odroid NetworkManager[395]: <info>  [1565103686.4999] dhcp4 (eth0): state changed unknown -> timeout
Aug  6 15:01:26 odroid NetworkManager[395]: <info>  [1565103686.5661] dhcp4 (eth0): canceled DHCP transaction, DHCP client pid 19038
Aug  6 15:01:26 odroid NetworkManager[395]: <info>  [1565103686.5663] dhcp4 (eth0): state changed timeout -> done
Aug  6 15:01:31 odroid kernel: [64694.029114] r8152 6-1:1.0 eth0: carrier off
Aug  6 15:01:34 odroid kernel: [64696.681717] r8152 6-1:1.0 eth0: carrier on
Aug  6 15:01:34 odroid NetworkManager[395]: <info>  [1565103694.0118] device (eth0): carrier: link connected
Aug  6 15:17:01 odroid CRON[19333]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Aug  6 15:40:23 odroid NetworkManager[395]: <info>  [1565106023.3738] manager: NetworkManager state is now CONNECTED_SITE
Aug  6 15:40:23 odroid systemd[1]: Starting Network Manager Script Dispatcher Service...
Aug  6 15:40:23 odroid dbus-daemon[391]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.7' (uid=0 pid=395 comm="/usr/sbin/NetworkManager --no-daemon ")
Aug  6 15:40:23 odroid dbus-daemon[391]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Aug  6 15:40:23 odroid systemd[1]: Started Network Manager Script Dispatcher Service.
Aug  6 15:40:23 odroid nm-dispatcher: req:1 'connectivity-change': new request (1 scripts)
Aug  6 15:40:23 odroid nm-dispatcher: req:1 'connectivity-change': start running ordered scripts...
Aug  6 16:17:01 odroid CRON[20356]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Aug  6 17:17:01 odroid CRON[21375]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Aug  6 17:24:56 odroid systemd[1]: Starting Message of the Day...
Aug  6 17:25:07 odroid systemd[1]: Started Message of the Day.
Aug  6 17:41:22 odroid dockerd[504]: time="2019-08-06T17:41:22.517405489Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55757->127.0.0.53:53: i/o timeout"
Aug  6 17:41:22 odroid dockerd[504]: time="2019-08-06T17:41:22.520386697Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/:dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55757->127.0.0.53:53: i/o timeout"
Aug  6 17:41:22 odroid dockerd[504]: time="2019-08-06T17:41:22.525515821Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55757->127.0.0.53:53: i/o timeout"
Aug  6 17:41:32 odroid dockerd[504]: time="2019-08-06T17:41:32.541632151Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:56250->127.0.0.53:53: i/o timeout"
Aug  6 17:41:32 odroid dockerd[504]: time="2019-08-06T17:41:32.541985276Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:56250->127.0.0.53:53: i/o timeout"
Aug  6 17:41:32 odroid dockerd[504]: time="2019-08-06T17:41:32.542250235Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:56250->127.0.0.53:53: i/o timeout"
Aug  6 17:41:42 odroid dockerd[504]: time="2019-08-06T17:41:42.554902690Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:44078->127.0.0.53:53: i/o timeout"
Aug  6 17:41:42 odroid dockerd[504]: time="2019-08-06T17:41:42.555645440Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:44078->127.0.0.53:53: i/o timeout"
Aug  6 17:41:42 odroid dockerd[504]: time="2019-08-06T17:41:42.556850065Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:44078->127.0.0.53:53: i/o timeout"
Aug  6 17:41:52 odroid dockerd[504]: time="2019-08-06T17:41:52.564620812Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55704->127.0.0.53:53: i/o timeout"
Aug  6 17:41:52 odroid dockerd[504]: time="2019-08-06T17:41:52.565376645Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55704->127.0.0.53:53: i/o timeout"
Aug  6 17:41:52 odroid dockerd[504]: time="2019-08-06T17:41:52.566250645Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:55704->127.0.0.53:53: i/o timeout"
Aug  6 17:42:02 odroid dockerd[504]: time="2019-08-06T17:42:02.576200851Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:54819->127.0.0.53:53: i/o timeout"
Aug  6 17:42:02 odroid dockerd[504]: time="2019-08-06T17:42:02.576806184Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:54819->127.0.0.53:53: i/o timeout"
Aug  6 17:42:02 odroid dockerd[504]: time="2019-08-06T17:42:02.578316767Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:54819->127.0.0.53:53: i/o timeout"
Aug  6 17:42:12 odroid dockerd[504]: time="2019-08-06T17:42:12.592607681Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:42881->127.0.0.53:53: i/o timeout"
Aug  6 17:42:12 odroid dockerd[504]: time="2019-08-06T17:42:12.592840264Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:42881->127.0.0.53:53: i/o timeout"
Aug  6 17:42:12 odroid dockerd[504]: time="2019-08-06T17:42:12.593187097Z" level=error msg="Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.53:53: read udp 127.0.0.1:42881->127.0.0.53:53: i/o timeout"
Aug  6 18:06:57 odroid kernel: [75820.397100] r8152 6-1:1.0 eth0: carrier off
Aug  6 18:07:03 odroid NetworkManager[395]: <info>  [1565114823.7353] device (eth0): state change: activated -> unavailable (reason 'carrier-changed', sys-iface-state: 'managed')
Aug  6 18:07:03 odroid NetworkManager[395]: <info>  [1565114823.7576] manager: NetworkManager state is now CONNECTED_LOCAL
Aug  6 18:07:03 odroid systemd[1]: Starting resolvconf-pull-resolved.service...
Aug  6 18:07:03 odroid dbus-daemon[391]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.7' (uid=0 pid=395 comm="/usr/sbin/NetworkManager --no-daemon ")
Aug  6 18:07:03 odroid systemd[1]: Starting Network Manager Script Dispatcher Service...
Aug  6 18:07:03 odroid systemd[1]: Started resolvconf-pull-resolved.service.
Aug  6 18:07:03 odroid systemd[1]: Started Network Manager Script Dispatcher Service.
Aug  6 18:07:03 odroid dbus-daemon[391]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Aug  6 18:07:03 odroid nm-dispatcher: req:1 'down' [eth0]: new request (1 scripts)
Aug  6 18:07:03 odroid nm-dispatcher: req:1 'down' [eth0]: start running ordered scripts...
Aug  6 18:07:08 odroid kernel: [75831.305576] r8152 6-1:1.0 eth0: carrier on
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6355] device (eth0): carrier: link connected
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6409] device (eth0): state change: unavailable -> disconnected (reason 'carrier-changed', sys-iface-state: 'managed')
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6472] policy: auto-activating connection 'Wired connection 1'
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6541] device (eth0): Activation: starting connection 'Wired connection 1' (0e5d5e2c-3c1e-37f0-9993-9f2383399891)
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6553] device (eth0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6561] manager: NetworkManager state is now CONNECTING
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6580] device (eth0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6598] device (eth0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6610] dhcp4 (eth0): activation: beginning transaction (timeout in 45 seconds)
Aug  6 18:07:08 odroid NetworkManager[395]: <info>  [1565114828.6905] dhcp4 (eth0): dhclient started with pid 22269
Aug  6 18:07:08 odroid dhclient[22269]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0x7fd0da44)
Aug  6 18:07:08 odroid dhclient[22269]: DHCPREQUEST of 192.168.1.2 on eth0 to 255.255.255.255 port 67 (xid=0x44dad07f)
Aug  6 18:07:08 odroid dhclient[22269]: DHCPOFFER of 192.168.1.2 from 192.168.1.1
Aug  6 18:07:08 odroid dhclient[22269]: DHCPACK of 192.168.1.2 from 192.168.1.1
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0060] dhcp4 (eth0):   address 192.168.1.2
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0060] dhcp4 (eth0):   plen 24 (255.255.255.0)
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0061] dhcp4 (eth0):   gateway 192.168.1.1
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0062] dhcp4 (eth0):   lease time 3600
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0064] dhcp4 (eth0):   nameserver '121.98.0.2'
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0065] dhcp4 (eth0):   nameserver '121.98.0.1'
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0065] dhcp4 (eth0): state changed unknown -> bound
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0126] device (eth0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0176] device (eth0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0191] device (eth0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0200] manager: NetworkManager state is now CONNECTED_LOCAL
Aug  6 18:07:09 odroid dhclient[22269]: bound to 192.168.1.2 -- renewal in 1555 seconds.
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0665] manager: NetworkManager state is now CONNECTED_SITE
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0670] policy: set 'Wired connection 1' (eth0) as default for IPv4 routing and DNS
Aug  6 18:07:09 odroid nm-dispatcher: req:2 'up' [eth0]: new request (1 scripts)
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0698] device (eth0): Activation: successful, device activated.
Aug  6 18:07:09 odroid nm-dispatcher: req:2 'up' [eth0]: start running ordered scripts...
Aug  6 18:07:09 odroid NetworkManager[395]: <info>  [1565114829.0778] manager: NetworkManager state is now CONNECTED_GLOBAL
Aug  6 18:07:09 odroid nm-dispatcher: req:3 'connectivity-change': new request (1 scripts)
Aug  6 18:07:09 odroid systemd[1]: Starting resolvconf-pull-resolved.service...
Aug  6 18:07:09 odroid systemd[1]: Started resolvconf-pull-resolved.service.
Aug  6 18:07:09 odroid systemd[1]: Reloading OpenBSD Secure Shell server.
Aug  6 18:07:09 odroid systemd[1]: Reloaded OpenBSD Secure Shell server.
Aug  6 18:07:09 odroid systemd[1]: Reloading OpenBSD Secure Shell server.
Aug  6 18:07:09 odroid nm-dispatcher: req:3 'connectivity-change': start running ordered scripts...
Aug  6 18:07:09 odroid systemd[1]: Reloaded OpenBSD Secure Shell server.
Aug  6 18:08:26 odroid systemd[1]: Created slice User Slice of will.
Aug  6 18:08:26 odroid systemd[1]: Starting User Manager for UID 1000...
Aug  6 18:08:26 odroid systemd[1]: Started Session 31 of user will.
Aug  6 18:08:27 odroid systemd[22369]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Aug  6 18:08:27 odroid systemd[22369]: Reached target Paths.
Aug  6 18:08:27 odroid systemd[22369]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Aug  6 18:08:27 odroid systemd[22369]: Listening on GnuPG cryptographic agent and passphrase cache.
Aug  6 18:08:27 odroid systemd[22369]: Starting D-Bus User Message Bus Socket.
Aug  6 18:08:27 odroid systemd[22369]: Listening on GnuPG network certificate management daemon.
Aug  6 18:08:27 odroid systemd[22369]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Aug  6 18:08:27 odroid systemd[22369]: Reached target Timers.
Aug  6 18:08:27 odroid systemd[22369]: Listening on D-Bus User Message Bus Socket.
Aug  6 18:08:27 odroid systemd[22369]: Reached target Sockets.
Aug  6 18:08:27 odroid systemd[22369]: Reached target Basic System.
Aug  6 18:08:27 odroid systemd[22369]: Reached target Default.
Aug  6 18:08:27 odroid systemd[22369]: Startup finished in 349ms.
Aug  6 18:08:27 odroid systemd[1]: Started User Manager for UID 1000.
Aug  6 18:08:33 odroid systemd-timesyncd[348]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
As for your question about the watchdog script, it wasn't working so I disabled it. When I took a closer look just now I saw that I had made some mistakes in editing it. Namely if [[ $down -lt $total ]]; rather than if [[ "$down" -lt "$total" ]]; and fping -c 1 -s 192.168.1.1 >&1 rather than fping -c 1 -s 192.168.1.1 2>&1. I've fixed these issues and tested them with the classic echos in ifs, so I'm more confident that this will work next time.

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Wed Aug 07, 2019 5:59 am

I had a crash, but the Odroid didn't come back online. The blue "heartbeat" LED was not blinking. When I power cycled it I found that there was a watchdog log. It is attached.
Attachments
watchdog-20190806-1910.txt
(141.02 KiB) Downloaded 14 times

User avatar
mad_ady
Posts: 6805
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 218 times
Been thanked: 167 times
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by mad_ady » Wed Aug 07, 2019 3:14 pm

From the log I see that before the attempted reboot eth0 lost its link, dhcp failed and link is still down (electrically) when collecting logs. Not sure why reboot didn't work, but you can enable hardware watchdog and crash the kernel instead (if really needed).

Now, based on the logs the problem doesn't seem software related. It's either the hc2 port, the cable or the switch port. I recommend trying a different switch port and a different cable. If you have a usb-ethernet adapter you can plug that in too and try to use it (even if it's not gigabit).

nz_will
Posts: 14
Joined: Sat Jun 01, 2019 11:44 am
languages_spoken: english
ODROIDs: HC2
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: ODroid HC2 becomes unresponsive

Unread post by nz_will » Tue Aug 13, 2019 4:32 am

So I have since switched from DHCP to static IP, and the odroid hasn't crashed in over 24 hours... I think this could be it.
These users thanked the author nz_will for the post:
mad_ady (Tue Aug 13, 2019 5:29 am)

Post Reply

Return to “Issues”

Who is online

Users browsing this forum: No registered users and 8 guests