[Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post Reply
mad_ady
Posts: 8300
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 427 times
Contact:

[Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by mad_ady »

I rebooted a C1 and it failed to come up. I hooked up a serial console and this is what I get:

Code: Select all

...
[    9.942695@1] hdmitx: cec: CEC init
[    9.946192@3] hdmitx: cec: CEC task process
[    9.946321@1] input: cec_input as /devices/virtual/input/input0
[    9.946473@1] hdmitx: cec: hdmitx_device->cec_init_ready:0x1
[    9.946473@1] ALSA device list:
[  �[   10.006761@1] Freeing unused kernel memory: 224K (c0935000 - c096d000)
Loading, please wait...
starting version 229
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... [   10.821583@2] tvmode set to 1080p
[   10.821583@2] 
[   10.831232@2] don't set the same mode as current.
[   10.889215@0] tvmode set to 1080p
[   10.889215@0] 
[   10.889313@0] don't set the same mode as current.
Tue Jan  1 00:00:00 UTC 1980
done.
Begin: Running /scripts/local-premount ... done.
Begin: Will now check root file system ... fsck from util-linux 2.27.1
[/sbin/fsck.ext4 (1) -- /dev/mmcblk0p2] fsck.ext4 -y -C0 /dev/mmcblk0p2 
e2fsck 1.42.13 (17-May-2015)
rootfs: clean, 217896/950272 files, 1787414/3783808 blocks
done.
[   11.120373@0] force enable DISCARD here for ext4 fs
[   11.123199@0] checked enable EXT4 DISCARD here
[   11.124113@0] EXT4-fs (mmcblk0p2): mounted filesystem without journal. Opts: (null)
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[   11.617300@0] systemd[1]: System time before build time, advancing clock.
[   11.701848@0] ip_tables: (C) 2000-2006 Netfilter Core Team
[   11.747999@0] systemd[1]: systemd 237 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
[   11.764531@0] systemd[1]: De


Welcome to Ubuntu 18.04.3 LTS!

[   11.791996@0] systemd[1]: Set hostname to <spica>.
[   11.820941@1] systemd[174]: Failed to open serialization file: Read-only file system
[   11.823714@1] systemd[1]: (sd-executor) failed with exit status 1.
[   11.829395@1] system!!!!!!] Failed to start up manager, freezing.
[   11.856622@1] systemd[1]: Freezing execution.
[   15.607589@0] sd: mmc_rescan_try_freq: trying to init card at 300000 Hz
[   18.696572@3] Connect Timeout
[   18.696634@3] Device Not Connected/Responding

I'm using the black emmc (16G), and did fsck on both partitions on a separate system without any errors. Not sure what's wrong. Any ideas? The system has been rock solid for a couple of years (migrated to 18.04 about a year ago).
Last edited by mad_ady on Sun Dec 29, 2019 5:02 am, edited 1 time in total.

mad_ady
Posts: 8300
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 427 times
Contact:

Re: C1 kiled after reboot, though filesystem is fine

Post by mad_ady »

Might be related to this snapd update, will try to remove it:
https://www.digitalocean.com/community/ ... ile-system

mad_ady
Posts: 8300
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 427 times
Contact:

Re: C1 kiled after reboot, though filesystem is fine

Post by mad_ady »

Ok, these are the steps I took to downgrade snapd. I ran them on a Odroid C2, though they could have run on a C1, XU4, N2, whatever can run armhf code:

Code: Select all

root@bellatrix:/media# mount /dev/mmcblk1p2 /media/usb
root@bellatrix:/media# mount -o bind /dev /media/usb/dev 
root@bellatrix:/media# mount -o bind /proc /media/usb/proc
root@bellatrix:/media# mount -o bind /sys /media/usb/sys
root@bellatrix:/media# chroot /media/usb bash
root@bellatrix:/# apt-cache policy snapd                                       
snapd:                                 
  Installed: 2.42.1+18.04                
  Candidate: 2.42.1+18.04                 
  Version table:
 *** 2.42.1+18.04 500                                                       
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main armhf Packages
        100 /var/lib/dpkg/status                      
     2.37.4+18.04.1 500         
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main armhf Packages           
     2.32.5+18.04 500             
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main armhf Packages              
root@bellatrix:/# sudo apt-get install snapd=2.32.5+18.04         
Reading package lists... Done                                           
Building dependency tree                                
Reading state information... Done                     
The following packages were automatically installed and are no longer required:
  libdrm-freedreno1 libllvm6.0 libllvm7                                                  
Use 'sudo apt autoremove' to remove them.                                
The following packages will be DOWNGRADED:                                               
  snapd                                                     
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 8 not upgraded.
Need to get 10.0 MB of archives.                                      
After this operation, 705 kB disk space will be freed.                  
Do you want to continue? [Y/n] y                  
Err:1 http://ports.ubuntu.com/ubuntu-ports bionic/main armhf snapd armhf 2.32.5+18.04
  Could not resolve 'ports.ubuntu.com'
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/s/snapd/snapd_2.32.5+18.04_armhf.deb  Could not resolve 'ports.ubuntu.com'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
root@bellatrix:/# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=55 time=15.9 ms
^C
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 15.997/15.997/15.997/0.000 ms
root@bellatrix:/# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1
search mdrjr.local
root@bellatrix:/# vi /etc/resolv.conf
root@bellatrix:/# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
search mdrjr.local
root@bellatrix:/# sudo apt-get install snapd=2.32.5+18.04
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libdrm-freedreno1 libllvm6.0 libllvm7
Use 'sudo apt autoremove' to remove them.
The following packages will be DOWNGRADED:
  snapd
0 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 8 not upgraded.
Need to get 10.0 MB of archives.
After this operation, 705 kB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic/main armhf snapd armhf 2.32.5+18.04 [10.0 MB]
Fetched 10.0 MB in 2s (6,030 kB/s)
E: Can not write log (Is /dev/pts mounted?) - posix_openpt (2: No such file or directory)
dpkg: warning: downgrading snapd from 2.42.1+18.04 to 2.32.5+18.04
(Reading database ... 190421 files and directories currently installed.)
Preparing to unpack .../snapd_2.32.5+18.04_armhf.deb ...
Unpacking snapd (2.32.5+18.04) over (2.42.1+18.04) ...
Setting up snapd (2.32.5+18.04) ...
Installing new version of config file /etc/apparmor.d/usr.lib.snapd.snap-confine.real ...
Installing new version of config file /etc/profile.d/apps-bin-path.sh ...
Installing new version of config file /etc/xdg/autostart/snap-userd-autostart.desktop ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for bamfdaemon (0.5.3+18.04.20180207.2-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
root@bellatrix:/# sync
root@bellatrix:/# exit
exit
root@bellatrix:/media#

I can't test whether this fixed it or not (yet) because junior is asleep in the room where C1 lies. Will give it a try tomorrow.
Hope this is helpful for others facing this issue in the future.
These users thanked the author mad_ady for the post (total 3):
danr677 (Thu Jan 02, 2020 6:44 am) • joy (Thu Jan 02, 2020 11:45 am) • safeman (Mon Jun 15, 2020 11:45 pm)

mad_ady
Posts: 8300
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 427 times
Contact:

Re: systemd[174]: Failed to open serialization file: Read-only file system

Post by mad_ady »

Ok, reverting snapd did the trick. I also ran sudo apt-mark hold snapd
These users thanked the author mad_ady for the post:
danr677 (Thu Jan 02, 2020 6:44 am)

danr677
Posts: 1
Joined: Fri Dec 20, 2019 4:50 am
languages_spoken: english
ODROIDs: Odroid-C1
Has thanked: 2 times
Been thanked: 0
Contact:

Re: [Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by danr677 »

Very useful, it saved me. The only difference is that I used a freshly written eMMC, with updated with the latest packages. However, I couldn't use Kodi (the main purpose for this Odroid-C1) with the latest image (v3.1), so I used v3.0 and updated it with apt dist-upgrade and it worked.

mad_ady
Posts: 8300
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 427 times
Contact:

Re: [Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by mad_ady »

It's strange that so far it only affects C1. Must be related to the old kernel version...

Nuems
Posts: 162
Joined: Thu Sep 19, 2013 3:50 am
languages_spoken: english, german
ODROIDs: xu, c1, n2
Has thanked: 1 time
Been thanked: 10 times
Contact:

Re: [Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by Nuems »

I encountered the same (or a very similar) problem recently (no serial-to-usb adapter on my C1, so it was hard to debug with just HDMI output). I solved it with a possibly dirty workaround: In the kernel commandline in boot.ini, I changed "ro" (for the rootfs) to "rw". It might cause all kinds of problems, but it seems to work for the moment.
These users thanked the author Nuems for the post:
mad_ady (Fri Jan 10, 2020 3:45 pm)

mdrjr
Site Admin
Posts: 11756
Joined: Fri Feb 22, 2013 11:34 pm
languages_spoken: english, portuguese
ODROIDs: -
Location: Brazil
Has thanked: 1 time
Been thanked: 28 times
Contact:

Re: [Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by mdrjr »

Hello Guys,

I've issued a boot.ini update today to fix it.

In case your board is not working, just replace the boot.ini with this file -> https://github.com/mdrjr/c1_bootini/blo ... r/boot.ini
These users thanked the author mdrjr for the post:
odroid (Wed Feb 12, 2020 10:13 am)

safeman
Posts: 53
Joined: Fri Jun 27, 2014 11:04 pm
languages_spoken: english
ODROIDs: Odroid U3 + 64G eMMC
Has thanked: 1 time
Been thanked: 1 time
Contact:

Re: [Solved] systemd[174]: Failed to open serialization file: Read-only file system

Post by safeman »

mad_ady wrote:
Sun Dec 29, 2019 5:01 am
Ok, these are the steps I took to downgrade snapd. I ran them on a Odroid C2, though they could have run on a C1, XU4, N2, whatever can run armhf code:
I can confirm this also works with Odroid-U3+ just now (using the 16.04 LTS system to downgrade snapd on the non-booting 18.04 LTS after system upgrade).

Thanks!

EDIT: I then discovered I needed to put upgrade of snapd on hold, which I did as follows:

Code: Select all

root@odroid:/# sudo apt-mark hold snapd
snapd set on hold.
root@odroid:/# sudo apt-mark showhold
snapd
root@odroid:/# apt-cache policy snapd
snapd:

  Installed: 2.32.5+18.04
  Candidate: 2.42.1+18.04
  Version table:
     2.42.1+18.04 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main armhf Packages
     2.37.4+18.04.1 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic-security/main armhf Packages
 *** 2.32.5+18.04 500
        500 http://ports.ubuntu.com/ubuntu-ports bionic/main armhf Packages
        100 /var/lib/dpkg/status
These users thanked the author safeman for the post:
odroid (Tue Jun 16, 2020 8:40 am)

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 0 guests