dqbydt wrote: ↑
Thu Nov 24, 2022 7:41 am
I was able to revive my system. A summary in case it is helpful for anyone:
- System was stuck in boot in zfs import. Booted into live image via petitboot by running netboot_default. chrooted into the rootfs and disabled zfs targets/services. This allowed the system to boot.
- zpool status was showing “no pools available”. Tried zpool import and that resulted in a kernel oops. After that, all zpool commands would just hang. This is what was causing the boot hang.
- `grep upgrade /var/log/dpkg.log` showed that kernel got upgraded from 5.10.23 to 5.10.105. (It was after this uprade that it had stopped booting.)
- dkms reported "Module zfs/2.0.3 already built for kernel 5.10.0-odroid-arm64/4." This seems reasonable: only the patchlevel changed; this should not have necessitated rebuild of any modules.
- Forced recompile of the zfs modules anyway, via dkms remove and dkms install.
- Then did a zpool import. That worked! Then restored the zfs target/services and rebooted. Everything now works fine.
Glad to hear you can finally recover your storage and happy to know Live System helps. It sounds like the kernel with Live System would need an update.
So somehow rebuild of the zfs modules worked, even though that shouldn't have been necessary. I would indeed be happier with a mainline kernel/latest ZFS, and will try installing Armbian. I have a couple of questions:
- If it is possible to run mainline kernel, why does HardKernel maintain a private kernel? I assume it is because not all parts of the SOC are supported in mainline? If so, what support is missing in mainline?
Critical element with mainline kernel is multimedia. Other features become way better stable than years ago by the genius kernel maintainers & contributors, also Panfrost team amazing that offers better GPU features. IMO, no matter mainline kernel is supported or not, having private kernel and supporting with it is also important.
IMO many reasons:
1) It's true that new SBC support is always behind than current kernel version/release and specifically when its SOC is new. We have this issue with ODROID-M1, it works with the upstream kernel after applying patches which are not merged - fortunately many of them are already merged into the next version.
2) Many demand to use recent kernel with new features, current upstream kernel version is 6.x while official Ubuntu 22.04 kernel version is 15.x.
3) Vanilla upstream kernel has lack support for the peripherals for industry customers. It's not a device driver issue at all, just it's a bit hard to manage the kernel for their product. So private kernel for ODROID have changes to help using GPIO/SPI/I2C for their field application and adding more peripherals.
4) Some hardware features could be supported with a hack due to the lack of hardware capability or the fix is only for a specific board in certain case, this change won't be accepted by kernel maintainers.
Any of them convince me to maintain a private upstream kernel for ODROID, one could be supported by tested packages with a features/hack not in upstream kernel and one can save time to huddle the issue and maintain a kernel by oneself using tested upstream kernel. This is a service, I believe.
[*] Being able to boot into a live image was critical to rescuing my system. Is this possible via Uboot?
If you mean to boot though the network, it's not possible with the stock U-Boot from Hardkernel since it does not support stable network access.
If you mean to boot from a SD on your HC4, I think this image could but not really since it's ISO format...not sure if U-Boot can recognize ISO image. Otherwise, the image must be converted into raw image format with regular file system types like FAT|EXT2/3/4.
http://ppa.linuxfactory.or.kr/installer ... 210331.iso
AFAIK, no SBC other than ODROID have such Live System image.