Stability and cleaning up boot sequence

Post Reply
ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Stability and cleaning up boot sequence

Unread post by ASword » Sat Oct 05, 2019 1:29 pm

I've noticed that my N2 has been occasionally rebooting spontaneously. It most sits untouched, running the Ubuntu 18.04 minimal distro (from eMMC) and a few daemons active (postgresql, mosquitto, collectd, apcupsd) and with an SSD (via USB3 / SATA3 bridge), an additional USB3 Ethernet adapter, and 2 UPS plugged in via USB2 (on a small hub). I'm planning to move my main workloads over to this machine eventually, so this instability is a significant concern. My current SBC runs for months at a time without rebooting.

Looking at the journalctl output, I don't see any messages indicating the cause of the failure leading to reboot. It looks like it can take as much as half an hour for the system to come back up again.

I do see a large number of messages like this:
alsactl[2272]: /usr/sbin/alsactl: set_control:1325: failed to obtain info for control #45 (No such file or directory)
There are also a lot of these:
Oct 01 19:17:01 n2c CRON[28899]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Oct 01 19:17:01 n2c CRON[28898]: pam_unix(cron:session): session opened for user root by (uid=0)
Oct 01 18:17:01 n2c CRON[28599]: pam_unix(cron:session): session closed for user root
I'd like to clean up my boot process as much as possible. What are these messages and how to I fix/remove them?

ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Re: Stability and cleaning up boot sequence

Unread post by ASword » Sat Oct 12, 2019 2:03 am

Nobody else having any spontaneous reboot issues while running Ubuntu, eh? I've updated to the latest version, and am waiting to see if the failure recurs.

User avatar
odroid
Site Admin
Posts: 32355
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 145 times
Been thanked: 325 times
Contact:

Re: Stability and cleaning up boot sequence

Unread post by odroid » Sat Oct 12, 2019 10:08 am

It seems to be a PSU issue.
Do you have a DMM to measure the system power stability?
If yes, keep measuring the voltage level on 5.0V rail on the 40pin header.

ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Re: Stability and cleaning up boot sequence

Unread post by ASword » Sat Oct 12, 2019 12:24 pm

My PSU is a triply redundant set of 3 24v 600W PSUs, from which the N2 is powered via a 24->12v DC-DC converter. The maximum load is only <1000W, and the other devices on the circuit aren’t having any stability issues.

ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Re: Stability and cleaning up boot sequence

Unread post by ASword » Sun Oct 13, 2019 3:29 am

I just discovered that one of my systemd units isn't starting properly -- it loads and then fails:

Code: Select all

andrew@n2c:~$ sudo service systemd-modules-load status
● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2018-01-28 07:58:19 PST; 1 years 8 months ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
  Process: 2163 ExecStart=/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 2163 (code=exited, status=1/FAILURE)
Looking in the journal:

Code: Select all

Oct 12 11:24:18 n2c systemd-modules-load[3257]: Failed to find module 'amvdec_h264_4k2k'
I don't need an h264 decoder. How can I just disable/remove that module?


UPDATE: actually google showed me the way to /etc/modules-load.d/modules.conf wherein I found a list of modules I don't need... rebooting without them now...

User avatar
odroid
Site Admin
Posts: 32355
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 145 times
Been thanked: 325 times
Contact:

Re: Stability and cleaning up boot sequence

Unread post by odroid » Mon Oct 14, 2019 9:45 am

ASword wrote:
Sat Oct 12, 2019 12:24 pm
My PSU is a triply redundant set of 3 24v 600W PSUs, from which the N2 is powered via a 24->12v DC-DC converter. The maximum load is only <1000W, and the other devices on the circuit aren’t having any stability issues.
If the DC cable wire thickness is 18 gauge or better, there should be no power issue.
viewtopic.php?f=95&t=29158

ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Re: Stability and cleaning up boot sequence

Unread post by ASword » Mon Oct 14, 2019 11:30 pm

odroid wrote:
Mon Oct 14, 2019 9:45 am
ASword wrote:
Sat Oct 12, 2019 12:24 pm
My PSU is a triply redundant set of 3 24v 600W PSUs, from which the N2 is powered via a 24->12v DC-DC converter. The maximum load is only <1000W, and the other devices on the circuit aren’t having any stability issues.
If the DC cable wire thickness is 18 gauge or better, there should be no power issue.
viewtopic.php?f=95&t=29158
I’m using the power lead I bought directly from you. The DC-DC converter supports 10A. Power isn’t the issue,

User avatar
odroid
Site Admin
Posts: 32355
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Has thanked: 145 times
Been thanked: 325 times
Contact:

Re: Stability and cleaning up boot sequence

Unread post by odroid » Tue Oct 15, 2019 9:48 am

Okay. Let's believe there is no PSU issue.

BTW, did you freshly flash the latest 18.04.3 image?

ASword
Posts: 188
Joined: Fri Aug 04, 2017 12:48 pm
languages_spoken: english
ODROIDs: XU4, HC1, 2x N2
Has thanked: 5 times
Been thanked: 1 time
Contact:

Re: Stability and cleaning up boot sequence

Unread post by ASword » Tue Oct 15, 2019 1:18 pm

No, I started with the minimal image from back in March/April timeframe. This past week I updated to the latest using the update-in-place mechanism. It has been running for nearly 3 days so far... unfortunately the time between failures is painfully long.

What is the best system log(s) to look in for finding potential causes of failure?

Currently:

Code: Select all

$ uname -a
Linux n2c 4.9.190-62 #1 SMP PREEMPT Tue Sep 10 01:00:59 -03 2019 aarch64 aarch64 aarch64 GNU/Linux

User avatar
tobetter
Posts: 4036
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1
Location: Paju, South Korea
Has thanked: 43 times
Been thanked: 188 times
Contact:

Re: Stability and cleaning up boot sequence

Unread post by tobetter » Tue Oct 15, 2019 1:26 pm

ASword wrote:
Tue Oct 15, 2019 1:18 pm
No, I started with the minimal image from back in March/April timeframe. This past week I updated to the latest using the update-in-place mechanism. It has been running for nearly 3 days so far... unfortunately the time between failures is painfully long.

What is the best system log(s) to look in for finding potential causes of failure?

Currently:

Code: Select all

$ uname -a
Linux n2c 4.9.190-62 #1 SMP PREEMPT Tue Sep 10 01:00:59 -03 2019 aarch64 aarch64 aarch64 GNU/Linux
Can you collect all files in /var/log and share it with me?

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 1 guest