No login prompt after changing rootfs UUID

Post Reply
ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

No login prompt after changing rootfs UUID

Post by ntarcds »

Hello,

I would like to change the UUID of the rootfs partition of the default Ubuntu server image (ubuntu-20.04-server-odroidm1-20220531.img).
First I generate a random UUID (say, "7dded70e-47db-11ee-a647-001e06510b32").
Then here are my steps:

Code: Select all

sudo tune2fs -O metadata_csum_seed /dev/mmcblk0p2
sudo tune2fs -U 7dded70e-47db-11ee-a647-001e06510b32 /dev/mmcblk0p2
sudo vi /etc/fstab
sudo vi /etc/default/flash-kernel
sudo flash-kernel
sudo reboot
I modify the UUID of the rootfs partition in "/etc/fstab" and "/etc/default/flash-kernel".
The command "sudo blkid /dev/mmcblk0p2" shows that the UUID is correctly updated.
Also, the UUID in "/boot/boot.scr" is correctly updated by "flash-kernel".

The kernel boots fine but the login prompt never appears.
I compared the boot log before and after the modification and nothing shows up, log is the same.
I suspect that the kernel is not able the rootfs after the UUID change for some reason.

Is there anything else I should do? Any ideas?
Thanks in advance!

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

You would need to update initramfs before rebooting when you change the UUID.
Can you open and modify your ramdisk in the first partition?

Also, I am wondering what makes you try to change the UUID?

ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by ntarcds »

I am trying to automate the creation of a custom Ubuntu server image in the context of our continuous integration.
Starting from the base ubuntu-20.04-server-odroidm1-20220531.img image, I need to update, remove, install packages, create users+groups etc and repackage a new image.
Then we deploy this custom image onto several ODROID boards and install dedicated applications.

My process is to use a real ODROID-M1 board running the base ubuntu-20.04-server-odroidm1-20220531.img as a slave node.
The continuous integration shall download a file ubuntu-20.04-server-odroidm1-20220531.img image file to the node, resize and mount partitions ("/boot" and "/") as loopback devices, chroot into the rootfs partition and apply the changes.
However I am facing the issue that the real "/" and the "/" mounted as loopback have by default the same UUID.
In this case, I am unable to resize the partition containing the "/" mounted as loopback because the kernel believes the partition is mounted even though it is not.

Hence my need to change the UUID of the real "/" partition.
Maybe there's a better way?

Anyway looking into the initramfs is a good idea.
Here are my steps:

Code: Select all

mkdir /tmp/initrd
cd /tmp/initrd
zcat /boot/initrd.img | cpio --extract --make-directories
vi conf/param.conf
The file conf/param.conf indeed contains the UUID of the rootfs partition.

Then if I do:

Code: Select all

cd /boot
sudo mkinitramfs -o new-initrd.img
sudo rm initrd.img
sudo ln -s new-initrd.img initrd.img
The initrd is updated, the kernel still boots fine but the login prompt never appears.
Same behaviour as before.
Do you think I am missing yet anything else?

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

ntarcds wrote:
Thu Aug 31, 2023 10:04 pm
I am trying to automate the creation of a custom Ubuntu server image in the context of our continuous integration.
Starting from the base ubuntu-20.04-server-odroidm1-20220531.img image, I need to update, remove, install packages, create users+groups etc and repackage a new image.
Then we deploy this custom image onto several ODROID boards and install dedicated applications.

My process is to use a real ODROID-M1 board running the base ubuntu-20.04-server-odroidm1-20220531.img as a slave node.
The continuous integration shall download a file ubuntu-20.04-server-odroidm1-20220531.img image file to the node, resize and mount partitions ("/boot" and "/") as loopback devices, chroot into the rootfs partition and apply the changes.
However I am facing the issue that the real "/" and the "/" mounted as loopback have by default the same UUID.
In this case, I am unable to resize the partition containing the "/" mounted as loopback because the kernel believes the partition is mounted even though it is not.

Hence my need to change the UUID of the real "/" partition.
Maybe there's a better way?

Anyway looking into the initramfs is a good idea.
Here are my steps:

Code: Select all

mkdir /tmp/initrd
cd /tmp/initrd
zcat /boot/initrd.img | cpio --extract --make-directories
vi conf/param.conf
The file conf/param.conf indeed contains the UUID of the rootfs partition.

Then if I do:

Code: Select all

cd /boot
sudo mkinitramfs -o new-initrd.img
sudo rm initrd.img
sudo ln -s new-initrd.img initrd.img
The initrd is updated, the kernel still boots fine but the login prompt never appears.
Same behaviour as before.
Do you think I am missing yet anything else?
Can you please share there screen where the booting is stuck?

For your real purpose, the image ubuntu-20.04-server-odroidm1-20220531.img is built by a tool ODROID-STAMPER that you can add your custom build.
viewtopic.php?f=55&t=37197

This tool could help you to add/modify the file system and install certain packages easily. If you want to try, please share the changes what you want to make for your projects so I can help you to achieve it.

ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by ntarcds »

Please find the complete boot log attached (debug uart output) from boot rom to the point where the kernel gets stuck.
Nothing else happens past this point, the serial terminal is unresponsive, no echo when I type characters.

Thanks for the link!
Before I dig into the repo though (https://github.com/tobetter/odroid-stamper), could you please clarify whether this also applies to ODROID-M1?
Which version of this script has been used to generate the image at https://dn.odroid.com/RK3568/ODROID-M1/ ... 531.img.xz? (the one I'm using)
Sorry, but this is not so clear from the README.

Thanks in advance!
Attachments
boot-log.txt
(15.22 KiB) Downloaded 10 times

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

ntarcds wrote:
Fri Sep 01, 2023 12:00 am
Please find the complete boot log attached (debug uart output) from boot rom to the point where the kernel gets stuck.
Nothing else happens past this point, the serial terminal is unresponsive, no echo when I type characters.
Sorry, the log does not help. Please share the screen where you expect to get log in prompt.
Thanks for the link!
Before I dig into the repo though (https://github.com/tobetter/odroid-stamper), could you please clarify whether this also applies to ODROID-M1?
Absolutely, but you have to build after switching to the branch next in the repository.
Which version of this script has been used to generate the image at https://dn.odroid.com/RK3568/ODROID-M1/ ... 531.img.xz? (the one I'm using)
Sorry, but this is not so clear from the README.

Thanks in advance!
When you start to build, run this command at the top directory of the source tree, then select the board ODROID-M1 and OS version forcal in the dialog.

Code: Select all

sudo ODROID_STAMPER_CHECKOUT=$PWD ./odroid-stamper
스크린샷 2023-09-01 00-11-20.png
스크린샷 2023-09-01 00-11-20.png (41.7 KiB) Viewed 767 times
Note, you will need to install packages in order to proceed properly.

Code: Select all

sudo apt update
sudo apt install git wget dialog pv lynx qemu-user-static dialog rsync squashfs-tools uuid-runtime

ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by ntarcds »

Thanks for the additional information.

Here is the screen :
Image

What kind of debug logs could I enable to help troubleshoot the problem?
Thanks in advance.

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

ntarcds wrote:
Fri Sep 01, 2023 12:41 am
Thanks for the additional information.

Here is the screen :
Image

What kind of debug logs could I enable to help troubleshoot the problem?
Thanks in advance.
I don't think you are booting your Ubuntu 20.04 OS or logging is disabled? Since I can see the prompt / # in the middle of screen.
Kernel debugging log does not help at all, since your problem is you are not able to get login prompt. And syslog cannot be captured since you are not able to access.
If your problem is only not able to get login prompt but you are able to give Linux command line, somehow your Linux kernel is in the maintenance mode by pressing Ctrl+D in the screen?

ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by ntarcds »

The "#" that you can see is the root shell prompt from PetitBoot. It also happens with the stock Ubuntu server image.

Until now I tried to login from the debug UART serial console.
This one is unresponsive when the kernel appears stuck.

I think you suggest trying to connect an HDMI screen and USB keyboard, I'll try that to see if any luck.
Maybe enabling kernel debug output from the kernel command line could provide more info?

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

ntarcds wrote:
Fri Sep 01, 2023 1:24 am
The "#" that you can see is the root shell prompt from PetitBoot. It also happens with the stock Ubuntu server image.

Until now I tried to login from the debug UART serial console.
This one is unresponsive when the kernel appears stuck.

I think you suggest trying to connect an HDMI screen and USB keyboard, I'll try that to see if any luck.
Maybe enabling kernel debug output from the kernel command line could provide more info?
It's not enough to judge or help your problem unless you provide the screen. The screen what I meant is the HDMI display, not logs through UART in terminal.
If you mean the prompt # is from Petitboot, then your kernel is not started yet which I am not able to judge with the logs what you provide since all about it is kernel log, it's not able to judge it's from Petitboot or your kernel. If this is your situation, the problem is why your change does not make booting from Petitboot, not why log in prompt does not appear. Also Petitboot does not provide the login prompt.
So...you have to probe if your OS image is started after Petitboot or not. That's why I am asking screen/video, not kernel log.
These users thanked the author tobetter for the post:
ntarcds (Fri Sep 01, 2023 5:37 pm)

ntarcds
Posts: 22
Joined: Mon Aug 14, 2023 11:19 pm
languages_spoken: english
Has thanked: 4 times
Been thanked: 3 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by ntarcds »

Thank you for your explanation, now I understand better.
I am used to tighter embedded environments where there is typically no multiboot stage (like PetitBoot) nor initramfs.

Turns out plugging in screen+keyboard was a very good suggestion and provides a very useful insight.

Image

Indeed, the initramfs contains a reference to the outdated UUID:

Image

I thought I had updated the initrd correctly in the steps I mentionned previously but I was wrong.
Adding an updated initrd (here, "new-initrd.img") and linking "initrd.img" to it is not the way to go.

First, the initrd needs to be programmed in memory, it is not read directly from the filesystem.
If my understanding is correct, the /boot directory is just a transient space to prepare files that will be flashed at specific locations (at least for vmlinuz and initrd). Am I correct here?

Second, instead of "mkinitramfs" you just need to use "update-initramfs -u" that does the steps of updating the current initramfs and triggering a flashing operation (of initrd and also boot.scr).

So to sum up here is the full list of commands required to update the UUID of the "/" partition:

Code: Select all

sudo apt install -y uuid
OLD_UUID=$(blkid --match-tag UUID --output value /dev/mmcblk0p2)
NEW_UUID=$(uuid)
sudo tune2fs -O metadata_csum_seed /dev/mmcblk0p2
sudo tune2fs -U $NEW_UUID /dev/mmcblk0p2
sudo sed -i "s/$OLD_UUID/$NEW_UUID/" /etc/fstab
sudo sed -i "s/$OLD_UUID/$NEW_UUID/" /etc/default/flash-kernel
sudo update-initramfs -u
sudo reboot
Success!
Thank you very much for your support.
These users thanked the author ntarcds for the post:
tobetter (Fri Sep 01, 2023 5:37 pm)

User avatar
tobetter
Posts: 12585
Joined: Mon Feb 25, 2013 10:55 am
languages_spoken: Korean, English
ODROIDs: Many
Location: Paju, South Korea
Has thanked: 931 times
Been thanked: 2155 times
Contact:

Re: No login prompt after changing rootfs UUID

Post by tobetter »

Thank you for the update and that's nice finding!!

Just in case, if you need to use the image and customise it for your project, please do not hesitate to use ODROID-STAMPER that will be useful to install and modify the root file system for your purpose. Also I am grateful if I can help you with it.

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 0 guests