Compiling the android kernel for n2

[mad_ady]

I'm trying to add some extra configuration to voodik's ATV and I want to add one-wire temperature sensor support to the kernel. So, here I go on a new journey and I need some advice...

I'm guessing the kernel for N2 comes from here:
https://github.com/hardkernel/linux/tre ... .y-android. Though this branch doesn't have all the patches from the non-android one.
Do I need to use a cross compiler as stated here - https://wiki.odroid.com/odroid-n2/softw ... ng_android, or can I compile it on ubuntu 18.04 arm64 with gcc 7.5.0? I see the original one was built with gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05).

I can get the default configuration from /proc/config.gz, so no problem there.
What commands do I need to build the kernel? I'm thinking of:

Code: Select all

make -j 4 Image dtbs modules

The modules should go to /vendor/lib/modules, I see.
Kernel and dtb seem to be flashed in two partitions:

Code: Select all

[    1.269566] meson-mmc: [mmc_read_partition_tbl] mmc read partition OK!
[    1.269568] meson-mmc: add_emmc_partition
[    1.269661] meson-mmc: [mmcblk0p01]           bootloader  offset 0x000000000001, size 0x00000000077f     
[    1.269738] meson-mmc: [mmcblk0p02]                  env  offset 0x000000000780, size 0x000000000080     
[    1.269817] meson-mmc: [mmcblk0p03]               ptable  offset 0x000000000800, size 0x000000000008     
[    1.269891] meson-mmc: [mmcblk0p04]                 misc  offset 0x000000000808, size 0x000000000008     
[    1.269974] meson-mmc: [mmcblk0p05]                 logo  offset 0x000000000810, size 0x000000001000     
[    1.270054] meson-mmc: [mmcblk0p06]                 dtbs  offset 0x000000001810, size 0x000000000100     
[    1.270138] meson-mmc: [mmcblk0p07]                 boot  offset 0x000000001910, size 0x000000008000     
[    1.270231] meson-mmc: [mmcblk0p08]             recovery  offset 0x000000009910, size 0x00000000c000     
[    1.270312] meson-mmc: [mmcblk0p09]                cache  offset 0x000000015910, size 0x000000200000     
[    1.270394] meson-mmc: [mmcblk0p10]                  odm  offset 0x000000215910, size 0x000000010000     
[    1.270475] meson-mmc: [mmcblk0p11]               system  offset 0x000000225910, size 0x000000380000
[    1.270552] meson-mmc: [mmcblk0p12]               vendor  offset 0x0000005a5910, size 0x000000100000     
[    1.270633] meson-mmc: [mmcblk0p13]              product  offset 0x0000006a5910, size 0x000000010000    

I'm not sure what size/offset are measured in. Bytes is too small, so it's probably blocks. I'm guessing 512 bytes long.

So my plan is - compile the new kernel (ideally on a c2, but I can do it on a pc if I must), dd the new kernel (is it in Image format?) and dtb to their partitions (e.g. with dd from android directly or offline with dd + offset) and copy over the modules.

Am I missing something?

Edit:
I extracted the kernel, dtb from their partitions and they look ok:

Code: Select all

root@localhost:~# file /kernel.img
/kernel.img: Android bootimg, kernel (0x1080000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)
root@localhost:~# file /dtb.img
/dtb.img: Device Tree Blob version 17, size=74368, boot CPU=0, string block size=7632, DT structure block size=66680
root@localhost:~#

[secuflag]

Download toolchain from here: https://releases.linaro.org/components/ ... gnu.tar.xz

Code: Select all

$ git clone --depth 1 https://github.com/hardkernel/linux.git -b odroidn2-4.9.y-android
$ cd linux
$ export ARCH=arm64
$ export CROSS_COMPILE=aarch64-linux-gnu-
$ export PATH=/PATH/TO/gcc-linaro-6.3.1-2017.02-x86_64_aarch64-linux-gnu/bin/:$PATH
$ make odroidn2_android_defconfig
$ make -j$(nproc)

Replace /PATH/TO/ with the correct path.

BUT, to build a flashable boot image you also need the rootFS and ramdisk.img so I suggest you to download the whole android BSP and compile the boot image from there.

[mad_ady]

Thanks. If I replace only the kernel, wouldn't it boot an existing android image? If not, why not? What's the problem?
I don't want to build the whole android for a few modules. It doesn't sound normal...

[secuflag]

On Android the kernel it's packed inside boot.img file with rootFS, ramdisk and/or other tools so if you add or remove features you must recompile it and rebuild the boot.img.

[mad_ady]

Oh, ok, forgot about that... Can I cheat and extract the rootfs from the existing boot partition and glue it back to the new kernel with some dd trickery?

If I build the boot.img from the android bsp - does that involve building the android framework/apps as well, or is it a relatively fast build, with just the cost of sources/disk space?

Thanks for enlightning me.

[secuflag]

You can try with a tool like this: https://forum.xda-developers.com/showth ... ?t=2319018

If you download the BSP you can build only the boot image by using: make -j$(nproc) bootimage
It just take some minutes, depending on your hardware of course.

[mad_ady]

Thanks, I'm in your debt!

[mad_ady]

Ok, made a little time at work to play with this and I feel I'm close to success (but not there yet!).
So - I compiled the android kernel as instructed. All went well there. Next I'm trying to rebuild the bootimg:

Code: Select all

cd ~/development/bootimg_tools
chmod a+x *
cd ..
PATH=~/development/bootimg_tools/:$PATH
mkdir bootimg
cd bootimg

adb shell
odroidn2:/ # dd if=/dev/block/boot of=/dev/shm/boot.img
odroidn2:/ # dd if=/dev/block/dtbs of=/dev/shm/dtbs.img
odroidn2:/ # exit
adb pull /dev/shm/boot.img
adb pull /dev/shm/dtbs.img

Code: Select all

$ file boot.img 
boot.img: Android bootimg, kernel (0x1080000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)
$ file dtbs.img
dtbs.img: Device Tree Blob version 17, size=74368, boot CPU=0, string block size=7632, DT structure block size=66680

$ boot_info 
PAGE SIZE: 2048
BASE ADDRESS: 0x01080000
RAMDISK ADDRESS: 0x01000000
CMDLINE: 'otg_device=1 buildvariant=userdebug'

$ unpack
Boot = boot.img
unmkbootimg version 1.2 - Mikael Q Kuisma <kuisma@ping.se>
Kernel size 9643391
Kernel address 0x1080000
Ramdisk size 1631542
Ramdisk address 0x1000000
Secondary size 0
Secondary address 0xf00000
Kernel tags address 0x100
Flash page size 2048
Board name is ""
Command line "otg_device=1 buildvariant=userdebug"

*** WARNING ****
This image is built using NON-standard mkbootimg!
OFF_KERNEL_ADDR is 0x01080000
Please modify mkbootimg.c using the above values to build your image.
****************

Extracting kernel to file zImage ...
Extracting root filesystem to file initramfs.cpio.gz ...
All done.
---------------
To recompile this image, use:
  mkbootimg --kernel zImage --ramdisk initramfs.cpio.gz --base 0x0 --cmdline 'otg_device=1 buildvariant=userdebug' -o new_boot.img
---------------
7472 blocks

To build the new boot img I tried this:

Code: Select all

mkbootimg --kernel ~/development/linux/arch/arm64/boot/Image.gz --ramdisk boot/ramdisk.cpio.gz --base 0x1000000 --ramdiskaddr 0x1000000 --cmdline 'otg_device=1 buildvariant=userdebug' -o new_boot.img

However, the new bootimg has a different kernel load address than the original one:

Code: Select all

$ file boot.img
boot.img: Android bootimg, kernel (0x1080000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)
$ file new_boot.img 
new_boot.img: Android bootimg, kernel (0x1008000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)

The message above suggests changing mkbootimg OFF_KERNEL_ADDR, but the one that came from xda_developers has only the binary file. Any suggestion on where I should look for a compatible mkbootimg (or source) that can build on N2?
Thanks!

[mad_ady]

Ok, I think I moved forward. I got a different mkbootimg:

Code: Select all

cd ~/development
git clone  https://github.com/osm0sis/mkbootimg.git
cd mkbootimg
make
cd ../bootimg_tools/
mv mkbootimg mkbootimg.ol
cp ../mkbootimg/mkbootimg .
cd ../android-development/
mkbootimg --kernel ~/development/linux/arch/arm64/boot/Image.gz --ramdisk boot/ramdisk.cpio.gz --base 0x1000000 --kernel_offset 0x80000 --ramdisk_offset 0x0 --cmdline 'otg_device=1 buildvariant=userdebug' -o new_boot.img

And now before and after:

Code: Select all

$ file boot.img 
boot.img: Android bootimg, kernel (0x1080000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)
$ file new_boot.img 
new_boot.img: Android bootimg, kernel (0x1080000), ramdisk (0x1000000), page size: 2048, cmdline (otg_device=1 buildvariant=userdebug)

So, it looks good.

Old (original) modules have this vermagic:

Code: Select all

odroidn2:/ # modinfo /vendor/lib/modules/lcd.ko                                
filename:       /vendor/lib/modules/lcd.ko
description:    LCD Lowlevel Control Abstraction
author:         Jamey Hicks <jamey.hicks@hp.com>, Andrew Zabolotny <zap@homelink.ru>
license:        GPL
depends:        
intree:         Y
vermagic:       4.9.y SMP preempt mod_unload modversions aarch64

New drivers have the same vermagic, so I guess I don't need to overwrite them if I just compiled things into the kernel, right?:

Code: Select all

$ modinfo ./drivers/video/backlight/lcd.ko
filename:       /home/adrianp/development/linux/./drivers/video/backlight/lcd.ko
description:    LCD Lowlevel Control Abstraction
author:         Jamey Hicks <jamey.hicks@hp.com>, Andrew Zabolotny <zap@homelink.ru>
license:        GPL
depends:        
intree:         Y
vermagic:       4.9.y SMP preempt mod_unload modversions aarch64

If I were to compile things as modules, how would I copy them to one directory without keeping the directory structure (like make modules_install does)?
Also, when using modules, how do I load them on Android startup?

Now - for my use case I think I only need to do the following (when I have time to unbrick it):

Code: Select all

adb push new_boot.img /sdcard/
adb push ~/development/linux/arch/arm64/boot/dts/amlogic/meson64_odroidn2_android.dtb /sdcard/
adb shell
# dd if=/sdcard/new_boot.img /dev/block/boot
# dd if=/sdcard/meson64_odroidn2_android.dtb /dev/block/dtbs

Is my assumption correct and it should work?

Now I need to find 30 minutes to try it out... Maybe next week...
Thanks!

[tobetter]

@mad_ady, I hope I am not late to help you out.

I suggest using another tool abootimg instead of mkbootimg in case if you like to replace Kernel for the running Android system. For example, if you have Android Pie 64bit (v2020010), https://wiki.odroid.com/odroid-n2/os_im ... 4_20200110, you can download a file Fastboot update that has many files can be flashed to Android partitions in a memory card. From the file, take boot.img which is compress Linux kernel and ramdisk combo.

abootimg -x boot.img will extract the boot image.

Code: Select all

$ abootimg -x boot.img
writing boot image config in bootimg.cfg
extracting kernel in zImage
extracting ramdisk in initrd.img
extracting second stage image in stage2.img

After compiling the kernel for Android, you must be having arch/arm64/boot/zImage in your kernel tree. abootimg is your friend to merge your zImage to the original boot.img. Suppose that you run the command in the same directory where you extract the original boot.img, you will have new boot image boot_new.img that has your custom kernel.

Code: Select all

$ abootimg --create boot_new.img -f bootimg.cfg -k <path/of/your/kernel/arch/arm64/boot/zImage> -r initrd.img

Please do use fastboot instead of dd since fastboot is supporting by U-boot, as long as you do not break U-boot you can recover bricked Android due to invalid boot image.
https://wiki.odroid.com/getting_started/adb_fastboot

Android does not use the module directory like /lib/modules/4.9.y... like ordinary Android. But you can copy your .ko to anywhere and load them. Hardkernel's Android stores .ko in /vendor/lib/modules and a kernel module can be loaded by an init script.
https://github.com/codewalkerster/andro ... tem.rc#L96

Hope this could help you.

[mad_ady]

Thanks, I'll try your suggestions and see how it goes. Regarding dtb - is it read from the dtbs partition or from the meson64_odroidn2_android.dtb file in /odm?

If I manage to brick things, how can I get in fastboot mode? I know I can via serial (and interrupting uboot), but I should be able to run the fastboot command in boot.ini as well, right - if I don't have serial?

[tobetter]

Thanks, I'll try your suggestions and see how it goes. Regarding dtb - is it read from the dtbs partition or from the meson64_odroidn2_android.dtb file in /odm?

boot.ini in /odm has a higher priority than others flashed with fastboot.
https://github.com/hardkernel/u-boot/bl ... mmon.h#L97

If I manage to brick things, how can I get in fastboot mode? I know I can via serial (and interrupting uboot), but I should be able to run the fastboot command in boot.ini as well, right - if I don't have serial?

Absolutely, also you can use reboot fastboot in the Android command shell.

[mad_ady]

Ok, I managed to try out the kernel I compiled. I tried two things:
- one kernel with only 1-wire support
- one kernel with the necessary stuff built-in to support docker (I like to live dangerously!)

The one-wire kernel booted fine, though I'm not sure (yet) that one-wire works. I enabled it in the dtb before flashing:

Code: Select all

    onewire {
        compatible = "w1-gpio";
        gpios = <0x0000001a 0x0000003f 0x00000000>;
        status = "disabled";
        phandle = <0x000000e4>;
    };

However, under /sys/bus/w1/devices/ I don't get the w1_bus_master1 entry - so I may be missing something, not sure what...
The wire, w1_gpio and w1_therm modules are built-in, not modules - maybe that's an issue?

The second kernel with the docker stuff boots and reaches userspace for ~30s but causes a panic and triggers a reboot. I need to capture logs and see what may trigger it.

[tobetter]

Ok, I managed to try out the kernel I compiled. I tried two things:
- one kernel with only 1-wire support
- one kernel with the necessary stuff built-in to support docker (I like to live dangerously!)

The one-wire kernel booted fine, though I'm not sure (yet) that one-wire works. I enabled it in the dtb before flashing:

Code: Select all

    onewire {
        compatible = "w1-gpio";
        gpios = <0x0000001a 0x0000003f 0x00000000>;
        status = "disabled";
        phandle = <0x000000e4>;
    };

However, under /sys/bus/w1/devices/ I don't get the w1_bus_master1 entry - so I may be missing something, not sure what...
The wire, w1_gpio and w1_therm modules are built-in, not modules - maybe that's an issue?

The second kernel with the docker stuff boots and reaches userspace for ~30s but causes a panic and triggers a reboot. I need to capture logs and see what may trigger it.

For Docker, did you check your config with this config?
https://github.com/moby/moby/blob/maste ... -config.sh

[mad_ady]

Yes, and enabled critical docker modules for cgroups, namespaces, network

Edit: This is what I enabled for docker:

Code: Select all

 adrianp  BR odroidn2-4.9.y-android  ~ | development | linux  1  0  diff -u .config-with-temperature .config | grep '^\+' | grep -v '^\+#'
+++ .config	2020-01-17 12:00:35.347262575 +0200
+CONFIG_CGROUP_PIDS=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_PAGE_COUNTER=y
+CONFIG_MEMCG=y
+CONFIG_MEMCG_SWAP=y
+CONFIG_MEMCG_SWAP_ENABLED=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CGROUP_WRITEBACK=y
+CONFIG_CFS_BANDWIDTH=y
+CONFIG_CGROUP_PERF=y
+CONFIG_NAMESPACES=y
+CONFIG_UTS_NS=y
+CONFIG_IPC_NS=y
+CONFIG_USER_NS=y
+CONFIG_PID_NS=y
+CONFIG_NET_NS=y
+CONFIG_BLK_DEV_THROTTLING=y
+CONFIG_CFQ_GROUP_IOSCHED=y
+CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
+CONFIG_NETFILTER_XT_MATCH_IPCOMP=y
+CONFIG_NETFILTER_XT_MATCH_IPVS=y
+CONFIG_IP_VS=y
+CONFIG_IP_VS_TAB_BITS=12
+
+CONFIG_IP_VS_PROTO_TCP=y
+CONFIG_IP_VS_PROTO_UDP=y
+
+CONFIG_IP_VS_RR=y
+
+CONFIG_IP_VS_SH_TAB_BITS=8
+
+CONFIG_IP_VS_NFCT=y
+CONFIG_BRIDGE_VLAN_FILTERING=y
+CONFIG_VLAN_8021Q=y
+CONFIG_NET_CLS_CGROUP=y
+CONFIG_NET_L3_MASTER_DEV=y
+CONFIG_CGROUP_NET_PRIO=y
+CONFIG_CGROUP_NET_CLASSID=y
+CONFIG_BONDING=y
+CONFIG_DUMMY=y
+CONFIG_MACVLAN=y
+CONFIG_MACVTAP=y
+CONFIG_IPVLAN=y
+CONFIG_VXLAN=y
+CONFIG_NETCONSOLE=y
+CONFIG_NETPOLL=y
+CONFIG_NET_POLL_CONTROLLER=y
+CONFIG_VETH=y
+CONFIG_OVERLAY_FS=y

[ali2019]

Hello, @tobetter.

Can you please share link for download abootimg tool?

[tobetter]

Hello, @tobetter.

Can you please share link for download abootimg tool?

If you use Debian/Ubuntu, you can simply install it with sudo apt install abootimg. Done.

[mad_ady]

Hmm, I'm tinkering again with my custom android kernel - and I'm unable to get onewire working. Most likely I'm missing something (something not enabled) in the dtb...
This time I compiled W1 as kernel built-in, but w1-gpio and w1_therm as modules so I can insert them manually.

When the kernel boots, it says:

Code: Select all

[    0.864943] Driver for 1-wire Dallas network protocol.

When I insmod w1-gpio.ko, I get this in dmesg:

Code: Select all

[  233.351466] type=1400 audit(1579703801.146:172): avc: denied { open } for pid=5877 comm="getprop" path="/dev/__properties__/u:object_r:net_dns_prop:s0" dev="tmpfs" ino=3258 scontext=u:r:sudaemon:s0 tcontext=u:object_r:net_dns_prop:s0 tclass=file permissive=1
[  233.351518] type=1400 audit(1579703977.370:177): avc: denied { module_load } for pid=6053 comm="insmod" path="/vendor/lib/modules/w1-gpio.ko" dev="mmcblk0p12" ino=5516 scontext=u:r:sudaemon:s0 tcontext=u:object_r:vendor_file:s0 tclass=system permissive=1

When I insmod w1_therm.ko I don't get anything in the dmesg log.
The problem is - I never get a /sys/bus/w1/devices/w1_bus_master, let alone any slaves devices...
In the dtb I have enabled onewire, but not sure if I need to enable other things as well. The gpio pins are the same as in the linux dtb, but there is a different phandle (whatever that is):

Code: Select all

$ fdtput -t s meson64_odroidn2_android.dtb /onewire status enabled

Code: Select all

N2 linux dtb:
    onewire {
        compatible = "w1-gpio";
        gpios = <0x0000001a 0x0000003f 0x00000000>;
        status = "disabled";
        phandle = <0x000000e4>;
    };

N2 android dtb:
    onewire {
        compatible = "w1-gpio";
        gpios = <0x0000001a 0x0000003f 0x00000000>;
        status = "enabled";
        phandle = <0x000000f0>;
    };

I'm stuck. Any ideas on how to get onewire working?