[Deprecated][Howto] Persistent boot.ini settings

Moderators: odroid, meveric, mdrjr

[Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Wed Nov 16, 2016 5:45 pm

Picture this setting: you come home, late at night, accompanied by a beautiful lady and you want to impress her by viewing a movie or listening to music on your fancy Odroid setup. You turn on the Odroid and the TV and notice that it keeps saying "No signal". The blue led blinks as it should, but you feel a cold sweat when your date asks you what's wrong. You politely excuse yourself and hide for a second in the bathroom, to "freshen up". You use your phone to ssh into the Odroid to investigate the problem. You quickly realize that boot.ini had been overwritten by an update and that the resolution is not compatible with the TV. After fiddling with vi on your phone and rebooting the Odroid, you return to the living room. Now the room is filled with Kodi's blueish glow and you are relieved. However, your date has felt something was off and excuses herself for the evening.
I wonder if that could have been avoided…


The problem is that when the bootini package updates, it overwrites /media/boot/boot.ini, and removes any customizations you had done on it, such as setting the resolution, enabling DAC support or even choosing a different rootfs. Every new user stumbles over this problem and, so far, had to either give up updates or learn to live with it.


I'm now proposing a simple solution that allows you to update without risk, while keeping your custom settings. I've put together a simple perl script that is run on system shutdown and does the following:
  • Checks to see if boot.ini has been changed since last seen
  • If it has, read the configuration file and rewrite the desired settings
This allows you to have an updated boot.ini (which may contain new options, such as overclocking or default timers), but at the same time preserves the options that matter to you.


As you may know, the boot.ini file consists of comments, variables set with "setenv" and boot directives (figure 1). You can get a better understanding of its syntax by reading the u-boot article http://magazine.odroid.com/wp-content/u ... pdf#page=6.


Image
Figure 1 - boot.ini example


The bootini-persistence script has a configuration file (/etc/bootini-persistence.ini) that has several sections. Each section has a match and a value directive.


Code: Select all
#C2 resolution                                                                                                                                                                                                 
[m]                                                                                                                                                                                                           
match=^\s*setenv\s+m\s+"([^\"]+)"                                                                                                                                                                             
value=1080p60hz



The match directive uses a regular expression to match a line from boot.ini while the value directive causes the matched line to be overwritten with the line setenv $section $value. The regular expressions are written as lax as possible, but still look too scary (figure 2). However, they are all very similar, and can be easily decoded.

Image
Figure 2. A sample of bootini-persistence.ini (all settings commented)

If we were to decode that regular expression (the thing that follows the equal sign), we would read it like this:
^ - line starts with
\s* - any number of white spaces, including no whitespace, followed by
setenv - the string "setenv", followed by
\s+ - one or more white spaces, followed by
m - the string "m" (which in this case is the uboot variable name for display mode)
\s+ - followed by one or more white spaces
" - followed by the string called "quote"
() - group the contents of this and save it as an internal variable
[^\"]+ - one or more non-quote symbols. This means one or more characters that are strictly not a quote
" - ends with a quote


This line will match the line setenv m "1080p60hz" from boot.ini, because 1080p60hz fits the description "one or more characters that are not a quote". The expression will not match any commented lines from boot.ini, because all other lines have a '#' sign before setenv and invalidate the expression. Similar regular expressions are already defined in /etc/bootini-persistence.ini for all configuration items and only need to be uncommented to be used.


In order to use this method, you will need to install the odroid-bootini-persistence deb package (works on all odroid platforms):


Code: Select all
$ wget https://github.com/mad-ady/odroid-bootini-persistence/raw/master/odroid-bootini-persistence_0.1-1_all.deb
$ sudo apt-get install libconfig-inifiles-perl
$ sudo dpkg -i odroid-bootini-persistence_0.1-1_all.deb


Next, you should edit the configuration file (located at /etc/bootini-persistence.ini) and uncomment and tweak it to fit your needs. By default the configuration file is commented out, and it won't make any changes. Figure 3 shows an example configuration for C2 which overrides the resolution, sets video output to dvi and sets max frequency to 1.75Ghz. When you are done, you need to enable and start the service:
Code: Select all

$ sudo systemctl enable bootini-persistence.service
$ sudo systemctl start bootini-persistence.service



Image
Figure 3 - bootini-persistence.ini example for C2


On the next shutdown/clean reboot, boot.ini will be parsed and the values you set in bootini-persistence.ini will be written in /media/boot/boot.ini automatically. They will only be written as long as there is no change on boot.ini (there is a md5sum saved in /media/boot/.boot.ini.md5sum, as well as a backup of the original file in /media/boot/boot.ini.original). If you want to manually apply the changes, you can run


Code: Select all
$ sudo /usr/local/bin/bootini-persistence.pl



If you ever want to disable automatic updates of boot.ini you need to:

Code: Select all
$ sudo systemctl disable bootini-persistence.service
$ sudo systemctl stop bootini-persistence.service



And if you ever want to restore the distribution's boot.ini, you can do:
Code: Select all
$ sudo dpkg-reconfigure bootini



The current script only replaces "setenv" lines, which is what most people need. It could be extended in the future to support other lines as well, through the powerful regular expression engine. Let me know what you think in this support thread.
Last edited by mad_ady on Thu Nov 17, 2016 10:16 pm, edited 1 time in total.
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Howto] Persistent boot.ini settings

Unread postby rooted » Wed Nov 16, 2016 6:07 pm

This is a great bandaid, but the boot.ini deb needs to be fixed so this wouldn't happen.

Once the package is fixed this makes for a nice countermeasure also.
User avatar
rooted
 
Posts: 4479
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1, C1+, C2
XU3 Lite, XU4
N1
VU7+
HiFi Shield 2
Smart Power (original)

Re: [Howto] Persistent boot.ini settings

Unread postby mad_ady » Wed Nov 16, 2016 6:53 pm

I agree it's a dirty hack and the best fix is to mark /media/boot/boot.ini as a config file, or maybe integrate a similar fix to copy the values and push them on the new boot.ini. But it's not my package so I couldn't fix it :)
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Howto] Persistent boot.ini settings

Unread postby rooted » Wed Nov 16, 2016 7:01 pm

It's not dirty at all that I see.

It's been requested multiple times for this to be fixed :/
User avatar
rooted
 
Posts: 4479
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1, C1+, C2
XU3 Lite, XU4
N1
VU7+
HiFi Shield 2
Smart Power (original)

Re: [Howto] Persistent boot.ini settings

Unread postby meveric » Wed Nov 16, 2016 11:05 pm

On my Debian images I use patches for the boot.ini file instead of replacing the entire file.
If patches do not work (for sections that may be changed by users for example the rootfs device) I use sed calls instead of patches.
It's a little more work then simply replacing the boot.ini file, but it allows for user to change the boot.ini and the update process does not break the changes.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 8535
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: X2, U2, U3, XU-Lite, XU3, XU3-Lite, C1, XU4, C2, C1+, XU4Q, HC1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Thu Nov 17, 2016 10:16 pm

Note that I've marked this thread as deprecated since I've been working with mdrjr to integrate this mechanism directly into boot.ini.
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mdrjr » Tue Dec 06, 2016 6:57 am

Well, it is integrated now :) Hoping to hear feedback on it..
mdrjr
Site Admin
 
Posts: 11662
Joined: Fri Feb 22, 2013 11:34 pm
Location: Brazil
languages_spoken: english, portuguese
ODROIDs: -

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby elatllat » Mon Jul 03, 2017 10:51 pm

mdrjr wrote:Well, it is integrated now :) Hoping to hear feedback on it..

It's Ubuntu/Debian policy that "the user is prompted" when config changes conflict. Because using /etc/bootini-persistence.ini is not non-standard please note it with the upgrade warning and at the top of the config itself (that's also a standard policy). 3rd ploicy that should be observed and is not is a sample (commented) /etc/bootini-persistence.ini file.

Using /etc/bootini-persistence.ini is not as rhobust as the standard becuse conflicts can still arise without a user prompt.
Like if I do this:
Code: Select all
    #
    #config
    #
 
    # Enable/Disable ODROID-VU7 Touchsreen
    [disable_vu7]
    match=^\s*setenv\s+disable_vu7\s+(.*)$
    value="true"
 
    # --- CPU Governor Setup ---
    [governor]
    match=^\s*setenv\s+governor\s+(.*)$
    value="interactive"

    # Load kernel, initrd and dtb in that sequence
    [fatload]
    match=^\s*fatload\s+mmc\s+0:1\s+0x44000000\s+(.*)$
    value=exynos5422-odroidxu4.dtb

and the offset changes it will break when the persistence file gave me false conference that it would not.
elatllat
 
Posts: 936
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Tue Jul 04, 2017 2:33 am

@elatllat: This persistence attempt has been deprecated and has been merged in the bootini package (for c2 at least, I need to check for xu4).
Now it works differently than in this thread. The configuration is in /media/boot/boot.ini.default and is never overwritten. The settings are changed and parsed on a bootini package update.
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby elatllat » Tue Jul 04, 2017 8:23 am

maybe mdrjr will enlighten us. Or I could try finding the source code...
elatllat
 
Posts: 936
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Tue Jul 04, 2017 2:23 pm

See if you have this file on your xu4: /usr/share/bootini/bootini-persistence.pl. My xu4 doesn't have an updated bootini, so I'm not sure it's there, but it exists on my C2. It gets called after the bootini.dpkg has been installed.
I sent patches for @mdrjr for c1/c2/xu4, but I think he only merged the c2 branch. https://github.com/mdrjr/5422_bootini/pull/2
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby odroid » Tue Jul 04, 2017 4:39 pm

@mdrjr will check it soon.
User avatar
odroid
Site Admin
 
Posts: 27375
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Tue Jul 04, 2017 4:47 pm

@elatillat: Regarding the dtb change that you pointed to - The advantage of this original method was that the user was free to write their own regular expressions to match the configuration. The problem is that most users don't have the ability to write those regular expressions, so instead of helping them it would confuse them further. The new method uses more generic regular expression inside the script, but is not as flexible as the old method.

In your case, if you want to keep using the bootini-persistence package, you could write your dtb regex as follows:
Code: Select all
# Load kernel, initrd and dtb in that sequence
    [fatload]
    match=^\s*fatload\s+mmc\s+[0-9]:[0-9]\s+0x[0-9a-f]+\s+(.*)$
    value=exynos5422-odroidxu4.dtb

This will match independent of what load addresses are used. (Warning - not actually tested!)
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby odroid » Wed Jul 05, 2017 6:54 pm

We've released new boot.ini package for C1.
We will release one for XU4 soon.
User avatar
odroid
Site Admin
 
Posts: 27375
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Wed Jul 05, 2017 7:02 pm

@odroid: My original push requests contained a boot.ini.default with commented-out defaults. You should update those, since it will likely be missing some newly added boot parameters.
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby elatllat » Thu Jul 06, 2017 2:05 am

mad_ady wrote:...
Code: Select all
^\s*fatload\s+mmc\s+[0-9]:[0-9]\s+0x[0-9a-f]+\s+(.*)$
...

Yah that would also break because it matches 3 lines,
Code: Select all
^\s*fatload\s+mmc\s+[0-9]:[0-9]\s+0x[0-9a-f]+\s+(exynos5422.*)$

might work but my point is there can still be conflicts so user interaction is still the best solution (and Debian policy)

odroid wrote:...We will release one for XU4 soon.


Please just prompt the user (Replace/Keep/Diff) if the config conflicts from the default on upgrade.

Code: Select all
cd /media/boot/
diff -q boot.ini.default boot.ini
if [ $? == 0 ] ; then
    cp boot.ini.new boot.ini.default
    cp boot.ini.new boot.ini
else
    echo "There is a new version of boot.ini would you like to Keep the current version, use the Maintainers version or edit a Diff [K/M/D]?"
    read KMD
    if [ "$KMD" == "M" ] ; then
        cp boot.ini.new boot.ini.default
        cp boot.ini.new boot.ini
    elif [ "$KMD" == "D" ] ; then
        cp boot.ini boot.ini.old
        vimdiff -f -d -c '4wincmd w | wincmd J' boot.ini.old boot.ini.default boot.ini.new boot.ini
        cp boot.ini.new boot.ini.default
    fi
fi
elatllat
 
Posts: 936
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby meveric » Thu Jul 06, 2017 2:46 am

elatllat wrote:Please just prompt the user (Replace/Keep/Diff) if the config conflicts from the default on upgrade.

Code: Select all
cd /media/boot/
diff -q boot.ini.default boot.ini
if [ $? == 0 ] ; then
    cp boot.ini.new boot.ini.default
    cp boot.ini.new boot.ini
else
    echo "There is a new version of boot.ini would you like to Keep the current version, use the Maintainers version or edit a Diff [K/M/D]?"
    read KMD
    if [ "$KMD" == "M" ] ; then
        cp boot.ini.new boot.ini.default
        cp boot.ini.new boot.ini
    elif [ "$KMD" == "D" ] ; then
        cp boot.ini boot.ini.old
        vimdiff -f -d -c '4wincmd w | wincmd J' boot.ini.old boot.ini.default boot.ini.new boot.ini
        cp boot.ini.new boot.ini.default
    fi
fi

If you want something like this, you should rather let apt/dpkg handle it completely and just mark the file as a config file.
That way you don't need to write these handles yourself, but apt/dpkg will bring up the dialog for keep, diff and overwrite.
It will also automatically create a .dpkg-old or .dpkg-new file depending on your choice.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 8535
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: X2, U2, U3, XU-Lite, XU3, XU3-Lite, C1, XU4, C2, C1+, XU4Q, HC1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby elatllat » Thu Jul 06, 2017 3:52 am

meveric wrote:...you don't need to write these handles yourself...


Yah what you said would be the correct way to do it, I was just explicitly stating the logic hoping to convince HK to use the Debian way.
elatllat
 
Posts: 936
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby mad_ady » Thu Jul 06, 2017 1:24 pm

@meveric: what happens during unattended upgrades? Are the changes accepted automatically?

Normally, I would agree with you, that boot.ini is a config file and should be treated the same way as other config files. However, since it is responsible with booting it may have greater impact than other packages. For example, I kept my old boot.ini on my C2 about a year ago when the timer option was introduced, but updated the kernel. After booting the kernel behaved in a very strange way and the system was not usable. It was my fault, but a mechanism like bootini persistence would have saved me the trouble.
I agree, it's not foolproof, but it's better than nothing...
User avatar
mad_ady
 
Posts: 3517
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: [Deprecated][Howto] Persistent boot.ini settings

Unread postby meveric » Thu Jul 06, 2017 1:52 pm

mad_ady wrote:@meveric: what happens during unattended upgrades? Are the changes accepted automatically?

Anything that requires user interaction will not be installed automatically.
Also since the repository from HardKernel is not part of the official Ubuntu repositories, unattended upgrades will ignore it, unless you explicitly configure unattended upgrades to install updates from the repository, and I doubt that HardKernel is doing that on their images.

mad_ady wrote:Normally, I would agree with you, that boot.ini is a config file and should be treated the same way as other config files. However, since it is responsible with booting it may have greater impact than other packages. For example, I kept my old boot.ini on my C2 about a year ago when the timer option was introduced, but updated the kernel. After booting the kernel behaved in a very strange way and the system was not usable. It was my fault, but a mechanism like bootini persistence would have saved me the trouble.
I agree, it's not foolproof, but it's better than nothing...

One of the many reasons why I use patches rather than overwriting the boot.ini files on my images.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 8535
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: X2, U2, U3, XU-Lite, XU3, XU3-Lite, C1, XU4, C2, C1+, XU4Q, HC1


Return to Ubuntu (All Linux'es)

Who is online

Users browsing this forum: No registered users and 1 guest