[OS] Debian Bullseye

[meveric]

Exagear from what I know is a dead project (one I was also using for many years) and therefore hasn't been updated in a very long time, Debian 9 guest is also out of support I would use neither nowadays.
It's also highly likely that Debian 9 -> Debian 11 is not really compatible when it comes to syscalls.

If you need to run an x86 application I would rather use box86 instead as it should be OS independent and is in active development.
If something is broken or not working it's easier to contact box86 developer and ask to try to find a solution.

[Danielsan]

Dear meveric

any plan for Bookworm?

Thanks

[meveric]

Currently working on arm64 but later armhf will be a topic again.
I encountered some issues with hardware video decoding for H265 under arm64 (amlogic devices) which works fine under Debian Bullseye but has issues with Bookworm.
Already found this is an issue in ffmpeg in combination with mpv as a player, but even after a couple weeks of experimentation only workaround I found so far is installing Debian Bullseye packages under Debian Bookworm for it to work properly.

I also look into updating the Kernel and try to see if I can incorporate HardKernel's new 6.1 Kernel for XU4 into my Kernel for the XU4 minus the GPU stack as I want to keep upstream panfrost drivers instead.
I also need to fix a mixup between armmp-lpae and just armmp for armhf boards.

Aside from that I backport LLVM 16 for a couple of days now which is required for latest MESA 23.2+ GPU drivers, which I also want to backport to Debian Bullseye and Bookworm.

So yes lots of things to do and everything sadly takes a lot of time to do so.

[AnotherLinuxUser]

If I understand this correctly, the limited OpenGL and OpenGL ES support for the XU4 GPU is due to a lack of proper MRT support aside from that of a hacky workaround by ARM in their blobby driver in order for MRT to exist on the GPU (by means of emulation). It looks like such a workaround is unmaintainable. You do get some benefits in Panfrost such as (at least a few) EGL 1.5 symbols that the blob never had.

https://oftc.irclog.whitequark.org/panfrost/2022-10-31

Regarding Panfrost, I thought that older yet somewhat modern Firefox builds that supported the pre-Webrender renderer were going to work correctly but it doesn’t appear to be the case. Chromium used to work. Some build around 105 I believe is where it worked but I believe it got progressively less stable with every new release. Something broke in Chromium somewhat recently and now GPU acceleration with this GPU is practically impossible as it is under Panfrost.

The only known build of Palemoon that’s still maintained with armhf support (Arctic Fox) should still have GPU-acceleration as the rendering mechanism is old enough to support OpenGL 2. Palemoon 26 does work with acceleration with Panfrost so long as you force it to run with acceleration from my testing.

Web browsers based on older Chromium builds such as Falkon (after manually enabling full acceleration apart from out-of-process rasterization as it is broken) and Puffin (with partial acceleration) do work with GPU acceleration though however. Epiphany does run with GPU acceleration (though it has to be enabled manually). I believe this applies to X11 as well and not just Wayland (at least for Puffin and Falkon). I noticed Wayland is generally a lot better on Panfrost and the blobs than X11. I think Panfrost is most performant under Wayland.

Also, the newer Mesa should be more performant as there’s now on-disk cache support in Panfrost. The Panfrost patches shouldn’t be very difficult to add to the latest HardKernel 6.1 LTS kernel after some kernel-related experimentation. Mali kbase doesn’t seem to like LLVM or at least some LLVM compiler optimizations (after it failed to build with O3 and LTO Thin on LLVM Clang 17) but no issues without it whatsoever.

Overall though, sounds like exciting stuff for sure! Great work on the Debian image by the way. You did great with it! Cheers!

[Danielsan]

Currently working on arm64 but later armhf will be a topic again.
[...]
So yes lots of things to do and everything sadly takes a lot of time to do so.

Wow, thank you very much for your great job! I look forward for your next release!

[meveric]

Regarding Panfrost, I thought that older yet somewhat modern Firefox builds that supported the pre-Webrender renderer were going to work correctly but it doesn’t appear to be the case. Chromium used to work. Some build around 105 I believe is where it worked but I believe it got progressively less stable with every new release. Something broke in Chromium somewhat recently and now GPU acceleration with this GPU is practically impossible as it is under Panfrost.

The only known build of Palemoon that’s still maintained with armhf support (Arctic Fox) should still have GPU-acceleration as the rendering mechanism is old enough to support OpenGL 2. Palemoon 26 does work with acceleration with Panfrost so long as you force it to run with acceleration from my testing.

Web browsers based on older Chromium builds such as Falkon (after manually enabling full acceleration apart from out-of-process rasterization as it is broken) and Puffin (with partial acceleration) do work with GPU acceleration though however. Epiphany does run with GPU acceleration (though it has to be enabled manually). I believe this applies to X11 as well and not just Wayland (at least for Puffin and Falkon). I noticed Wayland is generally a lot better on Panfrost and the blobs than X11. I think Panfrost is most performant under Wayland.

Yes sadly that is true.
In the past Chromium would just run fine starting with --use-gl=egl which forced it to run in OpenGL ES 2.0 compatibility mode, but it seems this is no longer the case and you need at least OpenGL 3.1 to work both FireFox and Chromium with GPU acceleration. Both are available on other boards with Panfrost drivers but not on the XU4.

Overall I still prefer X11 over Wayland, the lack of supported desktops makes it hard to switch. Gnome 3 is very sluggish on all systems and since most applications still run on X11 it's constantly using XWayland anyway which causes a bunch of other issues. While X11 works mostly fine on all boards with Panfrost.

[AnotherLinuxUser]

I'm very curious in seeing how difficult it would be to essentially "reverse backport" the older renderer from Chromium into the newer releases. Also, if you use Gnome 40+ with triple buffering enabled on Wayland, I'll say that it's very smooth. Google hasn't updated their ANGLE page yet and still claims OpenGL 2.0 support which is technically only if your GPU is OpenGL 3.1/OpenGL ES 3.1 or newer. To my knowledge and understanding regarding the Mali blobs is that they lack a few necessary EGL 1.5 symbols that Panfrost does have which also means newer Chromium and Firefox for a matter of fact will not work whatsoever (both on Wayland and on X11) with those Mali blobs. Both ways, there's no way anyone is going to get modern Chromium or Firefox to run without extensive modification as it currently stands. Someone who wishes to bring modern OpenGL and OpenGL ES to Midgard v4 on Panfrost would need to figure out how to emulate MRT just as ARM did with their blobby driver as the T760 (Midgard v5) is the first GPU that supports hardware MRT. No one really seems to really want to do that though understandably. Panfrost intends to avoid as many hacks as they possibly can which is perfectly understandable and it seems like that doing MRT maintenance is just a bit too much for what they aim for.

I'll say, for what we have now with Panfrost on Midgard v4, it's pretty good though a bit of a shame that Chromium and Firefox can't run with GPU acceleration anymore on this hardware.

[AnotherLinuxUser]

I would like to think that the legacy APIs just simply broke unintentionally in Chromium as you can see that ANGLE ends up intervening and making Chromium run at GL level OpenGL ES 2.0 anyways on hardware capable of newer APIs that do run with GPU acceleration with Chromium. I've noticed that even on hardware that supports OpenGL ES 3.1 under Panfrost such as the T760, the once-working flag doesn't work at all to run Chromium using a GLES context -> Another GLES context, so it looks like using native OpenGL ES is broken entirely, at least under Panfrost. I've noticed the same thing is true on a ChromeBook I have that runs Linux as well; it has a Mali G72 GPU. Another ChromeBook that I have with OpenGL/OpenGL ES 3.1 capability is a ChromeBook with a Mali T760 and the only way Chromium under Panfrost gets acceleration (just like with the G72) is by desktop OpenGL 3.1 -> OpenGL ES 2.0 via ANGLE. Perhaps the reason why this is the case is the lack of full API conformance on GPUs that are not the Mali G57 and G52. I can't say that the Mali blobs have went along very well with newer Chromium builds either.

I have decided to venture onto the #chromium-support (unofficial) IRC to ask about if there's an actual drop of support fo those APIs. It might be an undetected bug. I was hoping that those who use the Lima driver would have noticed this (as they too are limited OpenGL and OpenGL ES-wise) but it seems like they either don't or Chromium still works for them with GPU acceleration. I'm not entirely sure if Google would be willing to fix this though as they've refused to fix GPU acceleration-related issues in the past with GPUs about as old as Midgard v4. I'm no expert, but I might take a look at the backend and see how much has the renderer changed and see what was the last Chromium build that ran on Midgard v4 with Panfrost.

https://www.chromium.org/developers/irc/

[meveric]

I'm pretty sure most lima based boards would be too slow to handle accelerated Chromium anyway.
The Mali 400 was rather weak and even the Mali 450 has many bugs which prevents it to be used in a meaningful way.
The ODROID N1 (RK3399) runs on a Mali T860 which is a fully supported in Panfrost, except for Vulkan support which was dropped completely for Midgard GPUs by MESA developers.

I also don't think Google is eager to fix an issue that probably only affects legacy GPUs which are no longer in production. Even if some of their own older Chromebooks basically run on the same hardware than the XU3/XU4 I doubt they have any interest in continued support for it.

[AnotherLinuxUser]

Whoops! I should've mentioned that all Midgard GPUs in ChromeBooks are probably quite a bit past EOL. I took a look at the MESA GitLab and it seems like Midgard's Vulkan implementation never really worked to begin with under Panfrost according to the merge request that removed Midgard's Vulkan support under Panfrost. The ARM blobby Vulkan implementation on Midgard is there but virtually no applications support Vulkan 1.0 at all so there's kind of no point.

I'll note this however, if this is an ANGLE bug that also affects more modern and more relevant GPUs running Panfrost (such as Bifrost Mali G72, G76, G31, etc etc) when trying to use--use-gl=egl may Google fix it then. It may be ANGLE bug that affects most GPUs that run on Panfrost; I couldn't get my Mali Bifrost G72 device to run with a 100% native GLES-rendered context, but according to an “Issue” opened on MESA’s GitLab repo, 100% GLES-rendered Chromium was once a thing (previously) under Panfrost and it was faster too, so that could indicate something perhaps? So far, desktop OpenGL 3.1 being converted down to OpenGL ES 2.0 was the only setup that has worked with that G72 GPU with Panfrost. Bifrost Mali G72 isn't an EOL GPU either as far as Google is concerned, many very well currently supported CrOS devices (and you do have the non-Google Hikey 970 board, but none of their users can access blobs, only Panfrost) that use it, though they run with a 100% EGL-rendered context via the Mali blobs that are shipped wth CrOS even if it costs an aarch64 userspace.

Who knows?

[AnotherLinuxUser]

I forgot about this: https://bugs.chromium.org/p/chromium/is ... Lima&can=2

Someone launched a Chromium bug report about Chromium launching under the Lima driver.

EDIT (UPDATE): Our good friend JeffyCN has a repo containing a ton of Chromium 114 patches, much of which are EGL-related, which might indicate something. He still works with Mali blobs it seems like as he has his Mali blob injectors appear to be maintained frequently.

I’ll compile this sometime with these patches and see if it works on Panfrost and the blobs. If it runs on Panfrost, it might be via desktop OpenGL -> ANGLE -> OpenGL ES, but that shouldn’t be horrible.

https://github.com/JeffyCN/meta-rockchi ... 114.0.5735

[AnotherLinuxUser]

Update and big admission: I don't actually own an XU4 (yet) but I recently have bought one, so hopefully I'll be able to contribute a bit more around here; it won't be here for a couple of days though. Currently, I do own a ChromeBook that has almost identical specs to the XU4 (such as an Exynos 5420 GPU and a Mali T628 GPU). In fact, I have it running the Odroid XU4 6.1 LTS kernel with a small DTS patch to enable the Mali GPU on the laptop (authored by HexDump0815). I have actually went ahead and emailed JeffyCN to ask him my questions about building Chromium with his GPU acceleration patches. Currently, his repo is for 114 (there's a PR for 116 last I checked) but after a bit of work with trying to build it, I can successfully say that I have it running on the Mali T628 MP6 GPU (though with Mali blobs under Wayland) with GPU acceleration (well, mostly).

I used JeffyCN's kirkstone-chromium-next repo. Previously in Chromium 101 (by HardKernel) could you simply use --in-process-gpu but unfortunately, this causes Chromium to crash early on in newer releases and here is this still the case. I'll test this with Panfrost and see if this is different in a few hours.

Link: https://github.com/JeffyCN/yocto-manifests

(Had to go a GIt checkout on meta-browser and downgrade it as JeffyCN's patches are meant for 114 at the moment).

Code: Select all

Error: EGL version (1.4) must be at least 1.5
    at Create (../../third_party/dawn/src/dawn/native/opengl/ContextEGL.cpp:42)

Error: EGL version (1.4) must be at least 1.5
    at Create (../../third_party/dawn/src/dawn/native/opengl/ContextEGL.cpp:42)

This might be related to why --in-process-gpu doens't work? Panfrost to my knowledge has a number of EGL 1.5 symbols that make Chromium and Firefox (on GLES 3.0/3.1) happy. I would package up the Chromium build and send it here but the way I built it was kind of odd (and probably not the right way) so I'll have to actually figure out how to properly build it and build it on a Debian Bullseye system (my Chromium build used dependencies only available on newer distros as I built it on Fedora 39). I'll rebuild it later and send it here if you're interested.

I'll note, stock Chromium built by Debian seems to not include these patches at all, so having at least almost entirely GPU accelerated Chromium is better than none at least.

I actually did a bit of research; this patched Chromium build is targeted to run on an iMX8 board of some kind, which to my knowledge only support OpenGL ES 2.something, so theoretically this should work on Panfrost with OpenGL ES. I don't see how it couldn't realistically.



Looking forward to trying this on Panfrost.

UPDATE: The OpenGL ES implementation that Panfrost offers doesn't seem to be enough and I've been unable to get any actual hardware rendering backend to work on Chromium (yet). This could actually be in part to not having '--in-process-gpu' actually.
I took a look at this: https://github.com/OSSystems/meta-browser/issues/711
I spoke with JeffyCN about the non-working flag. From from what I've read from the GitHub issue, it looks like Chromium since version 110 on ARM needs Google's own Wayland fork in order for it to actually run... Fantastic! I'll give it a go and report back on what I can figure out.
Theoretically, chromium-ozone-x11 should work perfectly fine as these issues are Wayland-specific, though since Chromium creates its own minigbm, I can't say that for sure. I'll say, no X11 users have reported this problem (at least yet).

[marcelser]

Hi @meveric,

Is it normal that when it boots that you get an U-Boot warning?

Code: Select all

*** Warning - bad CRC, using default environment

You can also see that it doesn't find any boot.ini either, I think although it start loading some bytes later in the process,I don't know if this is connected to above warning, but here's whats printed after autboot prompt:

Code: Select all

Press quickly 'Enter' twice to stop autoboot:  0
** File not found /boot.ini **
## Executing script at 43e00000
Wrong image format for "source" command
** File not found /boot/boot.ini **
## Executing script at 43e00000
Wrong image format for "source" command
3664 bytes read in 7 ms (510.7 KiB/s)
## Executing script at 43e00000
4844 bytes read in 9 ms (525.4 KiB/s)
## Info: input data size = 4845 = 0x12ED
88751 bytes read in 66 ms (1.3 MiB/s)
loaded FDT from /usr/lib/linux-image-6.3.1-armmp-lpae/exynos5422-odroidxu4.dtb
11088928 bytes read in 494 ms (21.4 MiB/s)
7818828 bytes read in 344 ms (21.7 MiB/s)
Booting Debian 6.3.1-armmp-lpae from mmc 0:1...
Kernel image @ 0x42000000 [ 0x000000 - 0xa93420 ]
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
   Loading Ramdisk to 4f88b000, end 4ffffe4c ... OK
   Loading Device Tree to 4f872000, end 4f88aaae ... OK

And after updating to new kernel I get also this strange error below in boot log, any ideas how to fix it?

Code: Select all

    3.691815] cpu cpu0: EM: created perf domain
[    3.701985]
[    3.702049] ======================================================
[    3.708172] WARNING: possible circular locking dependency detected
[    3.714325] 6.3.1-armmp-lpae #exynos5 Not tainted
[    3.719003] ------------------------------------------------------
[    3.725155] swapper/0/1 is trying to acquire lock:
[    3.729922] c3e7a8b0 (&data->lock#2){+.+.}-{3:3}, at: exynos_get_temp+0x3c/0xc8
[    3.737200]
[    3.737200] but task is already holding lock:
[    3.743006] c422db98 (&tz->lock){+.+.}-{3:3}, at: __thermal_cooling_device_register.part.0+0x354/0x3a0
[    3.752278]
[    3.752278] which lock already depends on the new lock.
[    3.752278]
[    3.760423]
[    3.760423] the existing dependency chain (in reverse order) is:
[    3.767876]
[    3.767876] -> #1 (&tz->lock){+.+.}-{3:3}:
[    3.773422]        __mutex_lock+0xa0/0x95c
[    3.777494]        mutex_lock_nested+0x1c/0x24
[    3.781914]        thermal_zone_get_trip+0x20/0x44
[    3.786680]        exynos_tmu_initialize+0x144/0x1ec
[    3.791619]        exynos_tmu_probe+0x2a8/0x784
[    3.796125]        platform_probe+0x5c/0xb8
[    3.800284]        really_probe+0xe0/0x400
[    3.804357]        __driver_probe_device+0x9c/0x1f0
[    3.809210]        driver_probe_device+0x30/0xc0
[    3.813802]        __driver_attach+0x124/0x1d4
[    3.818222]        bus_for_each_dev+0x70/0xc4
[    3.822554]        bus_add_driver+0xe0/0x200
[    3.826801]        driver_register+0x7c/0x114
[    3.831133]        do_one_initcall+0x70/0x310
[    3.835466]        kernel_init_freeable+0x2b0/0x314
[    3.840318]        kernel_init+0x18/0x12c
[    3.844305]        ret_from_fork+0x14/0x2c
[    3.848377]
[    3.848377] -> #0 (&data->lock#2){+.+.}-{3:3}:
[    3.854270]        __lock_acquire+0x15e4/0x2a4c
[    3.858776]        lock_acquire+0x130/0x388
[    3.862935]        __mutex_lock+0xa0/0x95c
[    3.867008]        mutex_lock_nested+0x1c/0x24
[    3.871427]        exynos_get_temp+0x3c/0xc8
[    3.875674]        __thermal_zone_get_temp+0x48/0x11c
[    3.880699]        __thermal_zone_device_update.part.0+0x68/0x488
[    3.886765]        __thermal_cooling_device_register.part.0+0x374/0x3a0
[    3.893351]        __cpufreq_cooling_register.constprop.0+0x134/0x218
[    3.899763]        of_cpufreq_cooling_register+0x48/0x84
[    3.905049]        cpufreq_online+0x898/0xaec
[    3.909382]        cpufreq_add_dev+0xac/0xe8
[    3.913628]        subsys_interface_register+0x100/0x11c
[    3.918913]        cpufreq_register_driver+0x15c/0x380
[    3.924026]        dt_cpufreq_probe+0x2c0/0x408
[    3.928532]        platform_probe+0x5c/0xb8
[    3.932692]        really_probe+0xe0/0x400
[    3.936764]        __driver_probe_device+0x9c/0x1f0
[    3.941617]        driver_probe_device+0x30/0xc0
[    3.946210]        __driver_attach+0x124/0x1d4
[    3.950629]        bus_for_each_dev+0x70/0xc4
[    3.954962]        bus_add_driver+0xe0/0x200
[    3.959208]        driver_register+0x7c/0x114
[    3.963540]        do_one_initcall+0x70/0x310
[    3.967873]        kernel_init_freeable+0x2b0/0x314
[    3.972726]        kernel_init+0x18/0x12c
[    3.976712]        ret_from_fork+0x14/0x2c
[    3.980785]
[    3.980785] other info that might help us debug this:
[    3.980785]
[    3.988757]  Possible unsafe locking scenario:
[    3.988757]
[    3.994650]        CPU0                    CPU1
[    3.999155]        ----                    ----
[    4.003662]   lock(&tz->lock);
[    4.006695]                                lock(&data->lock#2);
[    4.012587]                                lock(&tz->lock);
[    4.018133]   lock(&data->lock#2);
[    4.021512]
[    4.021512]  *** DEADLOCK ***
[    4.021512]
[    4.027405] 5 locks held by swapper/0/1:
[    4.031304]  #0: c276848c (&dev->mutex){....}-{3:3}, at: __driver_attach+0x118/0x1d4
[    4.039017]  #1: c183caf4 (cpu_hotplug_lock){++++}-{0:0}, at: cpufreq_register_driver+0xc8/0x380
[    4.047768]  #2: c2c40298 (subsys mutex#8){+.+.}-{3:3}, at: subsys_interface_register+0x48/0x11c
[    4.056520]  #3: c1990a54 (thermal_list_lock){+.+.}-{3:3}, at: __thermal_cooling_device_register.part.0+0x294/0x3a0
[    4.066919]  #4: c422db98 (&tz->lock){+.+.}-{3:3}, at: __thermal_cooling_device_register.part.0+0x354/0x3a0
[    4.076624]
[    4.076624] stack backtrace:
[    4.080959] CPU: 7 PID: 1 Comm: swapper/0 Not tainted 6.3.1-armmp-lpae #exynos5
[    4.088236] Hardware name: Hardkernel ODROID-XU4
[    4.092833]  unwind_backtrace from show_stack+0x10/0x14
[    4.098029]  show_stack from dump_stack_lvl+0x58/0x70
[    4.103055]  dump_stack_lvl from check_noncircular+0xfc/0x168
[    4.108774]  check_noncircular from __lock_acquire+0x15e4/0x2a4c
[    4.114753]  __lock_acquire from lock_acquire+0x130/0x388
[    4.120125]  lock_acquire from __mutex_lock+0xa0/0x95c
[    4.125238]  __mutex_lock from mutex_lock_nested+0x1c/0x24
[    4.130697]  mutex_lock_nested from exynos_get_temp+0x3c/0xc8
[    4.136417]  exynos_get_temp from __thermal_zone_get_temp+0x48/0x11c
[    4.142743]  __thermal_zone_get_temp from __thermal_zone_device_update.part.0+0x68/0x488
[    4.150801]  __thermal_zone_device_update.part.0 from __thermal_cooling_device_register.part.0+0x374/0x3a0
[    4.160420]  __thermal_cooling_device_register.part.0 from __cpufreq_cooling_register.constprop.0+0x134/0x218
[    4.170298]  __cpufreq_cooling_register.constprop.0 from of_cpufreq_cooling_register+0x48/0x84
[    4.178877]  of_cpufreq_cooling_register from cpufreq_online+0x898/0xaec
[    4.185550]  cpufreq_online from cpufreq_add_dev+0xac/0xe8
[    4.191009]  cpufreq_add_dev from subsys_interface_register+0x100/0x11c
[    4.197595]  subsys_interface_register from cpufreq_register_driver+0x15c/0x380
[    4.204874]  cpufreq_register_driver from dt_cpufreq_probe+0x2c0/0x408
[    4.211372]  dt_cpufreq_probe from platform_probe+0x5c/0xb8
[    4.216919]  platform_probe from really_probe+0xe0/0x400
[    4.222204]  really_probe from __driver_probe_device+0x9c/0x1f0
[    4.228097]  __driver_probe_device from driver_probe_device+0x30/0xc0
[    4.234509]  driver_probe_device from __driver_attach+0x124/0x1d4
[    4.240575]  __driver_attach from bus_for_each_dev+0x70/0xc4
[    4.246207]  bus_for_each_dev from bus_add_driver+0xe0/0x200
[    4.251840]  bus_add_driver from driver_register+0x7c/0x114
[    4.257386]  driver_register from do_one_initcall+0x70/0x310
[    4.263018]  do_one_initcall from kernel_init_freeable+0x2b0/0x314
[    4.269170]  kernel_init_freeable from kernel_init+0x18/0x12c
[    4.274890]  kernel_init from ret_from_fork+0x14/0x2c
[    4.279915] Exception stack(0xf0835fb0 to 0xf0835ff8)
[    4.284944] 5fa0:                                     00000000 00000000 00000000 00000000
[    4.293094] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    4.301235] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000

Thanks for any support here and thanks for providing the debian bullseye

[meveric]

the u-boot is fine.. it's checking several possibilities on what to load and some of them fail.

The Kernel though I haven't noticed, but have the same issue, but it seems to work fine anyway.
I try to see if I can do anything about it, but it may take a while.

[AnotherLinuxUser]

Hello again, hope you don’t mind me asking about building upstream u-boot, but I’ve been trying to figure out how I can build current upstream u-boot on my own for the XU4. So far, can’t find anything much about building upstream u-boot for the XU4. Mainly, my biggest question is that will it need certain specific patches to get it to work? Would that mean rebasing upstream u-boot with HardKernel’s u-boot patches or does upstream u-boot already support the XU4 without any additional changes?

[meveric]

XU4 is supported upstream:
https://github.com/u-boot/u-boot/blob/m ... _defconfig

basically checkout u-boot repository:

Code: Select all

make odroid-xu3_defconfig
make all

and that's it.

You still need to sign the u-boot with the tools from hardkernel, but there's nothing else to do to actually build the u-boot.

[AnotherLinuxUser]

This might be useful for GPU accelerated Chromium (of course, adjusted for armv7h as the guide is for arm64): https://dev.to/yaegashi/howto-building- ... -rpi4-568e

I haven't tried it yet, I might give it a go in a moment. Apparently there's an option (that you might have to enable prior to building?) in Chromium: 'EGL_PLATFORM=surfaceless'

I wonder if that option might help us out here.

I found this some time ago: https://github.com/OSSystems/meta-browser/issues/234

I wonder how the Debian Team builds their Chromium package. Maybe we need: 'use_system_minigbm=false' if it isn't already like that. I'm not quite sure. I know that on MastaG's Fedora thread did Chromium GPU acceleration exist on Panfrost, albeit back when Chromium didn't exhibit GPU accelerated issues. Originally, the Mali Wayland blobs were targeted (and are shipped by default) but then Panfrost got some attention on there as well. I'm assuming maybe we need to compile Chromium in a certain way? The thing is that if Panfrost is being used, shouldn't the GBM be new enough anyways or is there something missing, broken, or both?

[TrevorH]

Network performance issue can be improved by setting the IRQ affinity to the faster cores. On my hc1 I have

# grep xhci /proc/interrupts
142: 5828 0 0 0 0 0 0 0 GICv2 104 Level xhci-hcd:usb3
143: 3449333 0 0 0 276607 0 0 0 GICv2 105 Level xhci-hcd:usb5

# cat /proc/irq/{142,143}/smp_affinity
ff
ff
# echo f0 > /proc/irq/142/smp_affinity
# echo f0 > /proc/irq/143/smp_affinity
# cat /proc/irq/{142,143}/smp_affinity
f0
f0

Now running iperf3 has improved from

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 3.43 GBytes 492 Mbits/sec 0 sender
[ 5] 0.00-60.01 sec 3.43 GBytes 491 Mbits/sec receiver

to

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 4.79 GBytes 686 Mbits/sec 0 sender
[ 5] 0.00-60.01 sec 4.79 GBytes 686 Mbits/sec receiver

Also, since the SATA port is connected to USB as well, this should also help with that too

I've now added this to /etc/rc.local until I can think of a better way to do it

Code: Select all

for irq in $(awk -F: '/xhci/ { print $1 }' /proc/interrupts);do echo f0 >/proc/irq/$irq/smp_affinity;done

[meveric]

installing irqbalance is also a good way to increase performance.
Rather than setting the IRQ fixed to a core each time, irqbalance will automatically switch over the IRQ to the big cores under big loads.
Does the same thing as you describe but automatically, but it may take a second to kick in.

[TrevorH]

I've tried irqbalance in the past and thought it was mostly junk. It assigns things to the wrong cores and moves them about all the time which may be good for "balance" but does nothing for cache coherency. I'd be quite surprised if it even knows about different speed cores and even more surprised if it did anything about preferring faster ones.

The command I posted there sets the affinity to any of the 4 faster cores, 0xf0, so it should take interrupts on any of them but it still seems to pick the lowest one it's allowed to (CPU4) so it still has a bottleneck but at least it's a faster bottleneck than it was before. In my test it ran cpu4 at about 80% so it looks like something else is now throttling it to just under 700Mbps. Unfortunately I can't remember what it used to get when I was running older kernels.

[AnotherLinuxUser]

Surrounding the Chromium situation, '--use-gl=egl' doesn't work as it should and is completely broken as I previously mentioned. It seems like this issue has almost always been present in some kind of way and isn't ARM specific. In fact, this is an issue that exists across all platforms including x86.

Though this thread isn't about Arch Linux, it seems like Arch Linux users haven looked at Chromium and native EGL (no ANGLE) acceleration and we're able to get a few clues/hints/information that should be helpful with getting Chromium to run with full GPU acceleration under Debian. The Arch Linux AUR package has a pinned comment for Wayland surrounding running Chromium with native (non-Angle) EGL (see https://aur.archlinux.org/packages/chro ... land-vaapi) that reads: "For EGL support, install the aur depend (https://aur.archlinux.org/packages/wayland-chromium) and try these flags (tested on Intel HD Graphics 4400)" "--use-gl=egl
--in-process-gpu"--I nearly got Chromium 114 (JeffyCN's meta-browser fork) to launch with '--in-process-gpu' and '--use-gl=egl' on Arch Linux ARM, but I encountered some strange error, though I got an entirely different error when using that Wayland AUR Package. Maybe I needed a different Wayland Chromium build that was older?? to get it to run?? as it seems like such kind of configuration most probably originated for ChromeOS (as it is a Linux based operating system using Wayland) which makes sense to why it needs patches to work on anything that is not ChromeOS (similar to that of V4L2 decoding). It seems like every update to Chrome under ChromeOS also means updating the entire operating system. Either way, it's very clear that Wayland and fully GPU accelerated Chromium isn't going to happen without a ton of work and investment, and theoretically, it *should* work on Panfrost.

As for X11, I'm going to assume that the regular 'meta-browser' Yocto layer can take care of that as it too has patches for native EGL (non-Angle) GPU acceleration under ARM. Looking at the HardKernel Chromium build that they ship with their Ubuntu Mate images for the XU3/4, it seems like they either used or modified 'meta-browser' Chromium for X11 (but for some reason kept Wayland compatibility). I haven't built 'meta-browser' for X11 yet, but it seems like there's no custom Xorg or anything extra needed really, '--in-process-gpu' and '--use-gl=egl' will be needed for 'meta-browser' Chromium for 100% GPU acceleration for the old Mali GPU. You could probably just rebuild the Chromium Debian package if you were to see what patches 'meta-browser' uses to get it easily running on X11.

[AnotherLinuxUser]

I have big news for you all today. I've successfully managed to achieve full GPU acceleration on the latest Chromium 119 on Panfrost with the Mali T628 MP6 GPU on a Wayland environment. To put it quite frankly, it was something that I was not expecting at all. This method should definitely work with X11 as well as and probably with native EGL rather than ANGLE. I still don't have an XU4 that works a the moment, but this should obviously work on Odroid devices as my device does have a Midgard T628 MP6 GPU.

I should note that this was achieved by using from the project 'meta-browser' which also provides a heavily patched Chromium that is more friendly towards embedded ARM platforms. You could probably achieve native EGL acceleration using Chromium instead of the also GPU accelerated ANGLE backend by using Google's Wayland fork. On X11 platforms, you will not need any Wayland forks and chances are native EGL would work as well. ANGLE is supposed to be pretty close to native when it comes to performance though. I might go ahead and try to learn how to proper construct a deb package and releases.

I should also note that you may or may not encounter slight, occasional, pheraps instances of rendering bugs, but to be honest, I've only encountered this with the Google snake game. In fact, this probably won't affect most people. Maybe native EGL via Google's Wayland fork fixes this, maybe this isn't an issue in X11; who knows, maybe it's a bug in 'meta-browser'--regardless of the cause, I've only seen only one instance of incorrect rendering. I've loaded several pages so far, and I've been unable to find any other rendering issues. Only the Google snake game is giving me some render bug, but it's nothing serious. I'm actually on the Odroid Forums right now on a Mali T628 MP6 device + Wayland + Chromium 119, tying this post at the moment.

The "Raw Draw" feature will have be enabled to get this to work without font rendering issues, probably because it's on ANGLE or something, not sure, but enabling the "Raw Draw" feature fixes it. More details will come in a moment, but I'm very much pleased to see that Chromium can be made to run on Wayland + Panfrost + this old Mali Midgard GPU. Hope you all have a fantastic evening and stay tuned for more.

[AnotherLinuxUser]

Hello everyone, would like to provide a quick update. When I had achieved GPU acceleration in that screenshot, it was on Ubuntu Noble 24.04 LTS beta, running a self-built Linux Zen 6.6.2 kernel with postmarketOS Exynos 5 kernel patches applied, and a custom config file based on the postmarketOS Exynos 5 defconfig. I've not tried this Chromium build on Debian Bullseye yet, but if it doesn't work, it should be fairly easy to bring it over to Bullseye from what I can tell. I'll be sure to test that at some point soon.

Surrounding native EGL and GLES, from what I understand, native EGL provides a significant performance boost as well as using OpenGL ES (the original OpenGL variant that the Mali GPUs target). I've been unable to get Google's Wayland fork working on Ubuntu with native EGL and GLES on Ubuntu so far but on Arch Linux have I managed to get either what appears to be a fork or a Wayland 21 build that has patches from Google's Wayland fork applied to it. I'm using it to type this right now and it most definitely does work and unlike my comment from back in October, no compromises on GPU accelerated features and it too 100% works. According to the meta-browser issue that first brought to light the fact that Chromium 110 needs a non-stock Wayland for functionality and from my experience with using ChromeOS, it most definitely appears that native GLES and EGL was designed for ChromeBooks running ChromeOS but it's a good thing that we as users have patched it out. Theoretically V4L2 decoding should also be possible. I think I've seen some work done on it somewhat recently? It's great to see Panfrost handling Chromium well though. I do wonder though if someone will eventually try to work on the errata surrounding the other 2 GPU cores that don't work properly with Panfrost, maybe someone will try MRT emulation on Midgard v4 so we get a newer GL and GLES? Have a great morning, everyone!

I've not tried it without '--in-process-gpu' (I'm currently using Chromium with '--in-process-gpu' along with '--use-gl=egl') but I'm pretty sure '--in-process-gpu' is required for native Wayland EGL + GLES acceleration. I'll take a closer look at the PKGBUILD and try to get this running on Ubuntu Noble and Debian Bullseye. EDIT: Also, another note, this time, with this new configuration, the original render error/bug/glitch/etc disappeared but in came another one. I'm sure that not many people here are going to care about the online game of Minecraft Classic. I only launched it to test GPU accelerated performance and it looks like for whatever reason, you get a ton of flickering and graphical bugs. It looks like this is specific to native GLES and/or EGL, but so far, I have found no other problems whatsoever. I'm not able to rule out whether Panfrost is a contributing/sole factor in such, but it could be. I'd have to bring back the Mali blobs to this unit to confirm/disprove that Panfrost is involved in some way with such.

Update EDIT: It looks like we're going to also get hardware accelerated V4L2 video decoding working on Chromium on ARM as well soon. Perfect! See: https://chromium-review.googlesource.co ... /+/5014723

Update EDIT #2: Just hit a new success here, I've just been able to get the Google Chromium fork to work properly with patched Chromium and native EGL and GLES. Patches from the Google Wayland fork found in the Arch AUR helped me getting it running on Ubuntu Noble 24.04 LTS. Able to confirm that '--in-process-gpu- is required for native EGL and GLES (which is enable by this flag: '--use-gl=egl')

[Adi563]

Hi!
I have installed bullseye (Linux odroid-bullseye 6.3.1-armmp-lpae #exynos5 SMP PREEMPT Mon May 8 15:27:00 UTC 2023 armv7l GNU/Linux) on my XU4.
I have also connected a 22 TB hard drive via USB3 and installed ntfs-3g. The drive is mounted (/dev/sda2 on /mnt/media type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)) and I am able to ready and write in general.
Although, whenever I try to write a large file to the drive the whole system starts crashing. From dmesg I get the following errors:

Code: Select all

[55744.152287] mount.ntfs-3g invoked oom-killer: gfp_mask=0x141cc0(GFP_USER|__GFP_COMP|__GFP_WRITE), order=0, oom_score_adj=0
[55744.161916] CPU: 6 PID: 12721 Comm: mount.ntfs-3g Not tainted 6.3.1-armmp-lpae #exynos5
[55744.169856] Hardware name: Samsung Exynos (Flattened Device Tree)
[55744.175929]  unwind_backtrace from show_stack+0x10/0x14
[55744.181123]  show_stack from dump_stack_lvl+0x58/0x70
[55744.186150]  dump_stack_lvl from dump_header+0x48/0x318
[55744.191348]  dump_header from oom_kill_process+0x328/0x33c
[55744.196808]  oom_kill_process from out_of_memory+0x104/0x454
[55744.202439]  out_of_memory from __alloc_pages+0x1174/0x13b4
[55744.207986]  __alloc_pages from __filemap_get_folio+0x194/0x4f8
[55744.213878]  __filemap_get_folio from pagecache_get_page+0xc/0x3c
[55744.219943]  pagecache_get_page from block_write_begin+0x1c/0xd0
[55744.225923]  block_write_begin from generic_perform_write+0xc0/0x1fc
[55744.232247]  generic_perform_write from __generic_file_write_iter+0x150/0x17c
[55744.239353]  __generic_file_write_iter from blkdev_write_iter+0x114/0x1cc
[55744.246114]  blkdev_write_iter from vfs_write+0x1bc/0x268
[55744.251486]  vfs_write from sys_pwrite64+0x98/0xc8
[55744.256250]  sys_pwrite64 from ret_fast_syscall+0x0/0x1c
[55744.261536] Exception stack(0xf1539fa8 to 0xf1539ff0)
[55744.266563] 9fa0:                   79784000 00000f83 00000003 0048a8c0 00002000 00000000
[55744.274709] 9fc0: 79784000 00000f83 b6f4df19 000000b5 b6fc34d0 00000000 b6f6817c 00000f83
[55744.282854] 9fe0: 000000b5 bef330f8 b6f77f49 b6f79676
[55744.287941] Mem-Info:
[55744.290168] active_anon:7930 inactive_anon:508 isolated_anon:0
                active_file:175265 inactive_file:172282 isolated_file:0
                unevictable:0 dirty:9694 writeback:0
                slab_reclaimable:14515 slab_unreclaimable:7265
                mapped:9493 shmem:1632 pagetables:335
                sec_pagetables:0 bounce:0
                kernel_misc_reclaimable:0
                free:118422 free_pcp:1156 free_cma:20597
[55744.329956] Node 0 active_anon:31720kB inactive_anon:2032kB active_file:701060kB inactive_file:689292kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:37980kB dirty:39016kB writeback:0kB shmem:6528kB writeback_tmp:0kB kernel_stack:1496kB pagetables:1340kB sec_pagetables:0kB all_unreclaimable? no
[55744.357137] Normal free:12424kB boost:0kB min:3408kB low:4260kB high:5112kB reserved_highatomic:12288KB active_anon:32kB inactive_anon:0kB active_file:552488kB inactive_file:73972kB unevictable:0kB writepending:38804kB present:786432kB managed:735028kB mlocked:0kB bounce:0kB free_pcp:4624kB local_pcp:732kB free_cma:0kB
[55744.385299] lowmem_reserve[]: 0 10064 10064
[55744.389355] Normal: 46*4kB (UM) 86*8kB (M) 56*16kB (M) 33*32kB (UME) 16*64kB (UM) 57*128kB (UME) 1*256kB (U) 0*512kB 1*1024kB (E) 0*2048kB 0*4096kB = 12424kB
[55744.403481] 349249 total pagecache pages
[55744.407293] 0 pages in swap cache
[55744.410665] Free swap  = 0kB
[55744.413445] Total swap = 0kB
[55744.416303] 518656 pages RAM
[55744.419163] 322048 pages HighMem/MovableOnly
[55744.423491] 12851 pages reserved
[55744.426616] 24576 pages cma reserved
[55744.430249] Tasks state (memory values in pages):
[55744.434848] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[55744.443549] [   1783]     0  1783    10791     3872    36864        0          -250 systemd-journal
[55744.452531] [   1798]     0  1798     4390      896    16384        0         -1000 systemd-udevd
[55744.461368] [   1864]   101  1864     4926     1120    20480        0             0 systemd-timesyn
[55744.470380] [   1873]     0  1873     7998      992    20480        0             0 dhclient
[55744.478698] [   1889]     0  1889     1874      480    14336        0             0 cron
[55744.486848] [   1890]   104  1890     1548      672    12288        0          -900 dbus-daemon
[55744.495513] [   1892]     0  1892     6311      768    20480        0             0 rsyslogd
[55744.503910] [   1893]     0  1893     2628     1152    16384        0             0 systemd-logind
[55744.512836] [   1913]     0  1913    47565     3328    45056        0             0 php-fpm7.4
[55744.521419] [   1931]     0  1931     1002      320    12288        0             0 agetty
[55744.529650] [   1934]     0  1934     1544      352    10240        0             0 agetty
[55744.537798] [   1944]     0  1944     2549     1280    14336        0         -1000 sshd
[55744.545950] [   1948]    33  1948    47565      984    36864        0             0 php-fpm7.4
[55744.554519] [   1949]    33  1949    47565      984    36864        0             0 php-fpm7.4
[55744.563104] [   1953]    33  1953      899      448     8192        0             0 lighttpd
[55744.571504] [   1966]    33  1966    45762     3104    38912        0             0 php-cgi
[55744.579850] [   1967]    33  1967    45762      752    30720        0             0 php-cgi
[55744.588058] [   1968]    33  1968    45762      752    30720        0             0 php-cgi
[55744.596450] [   1969]    33  1969    45762      752    30720        0             0 php-cgi
[55744.604773] [   1970]    33  1970    45762      752    30720        0             0 php-cgi
[55744.613093] [   2094]     0  2094     3099     1664    16384        0             0 systemd
[55744.621407] [   2095]     0  2095     8570      930    26624        0             0 (sd-pam)
[55744.629897] [  12366]     0 12366     3026     1410    18432        0             0 sshd
[55744.637797] [  12372]     0 12372      940      672    12288        0             0 sftp-server
[55744.646551] [  12499]     0 12499     3026     1413    18432        0             0 sshd
[55744.654655] [  12506]     0 12506     1979      704    14336        0             0 bash
[55744.662659] [  12721]     0 12721     1768      390     8192        0             0 mount.ntfs-3g
[55744.671500] [  12746]     0 12746     3026     1416    18432        0             0 sshd
[55744.679561] [  12753]     0 12753      940      704     8192        0             0 sftp-server
[55744.688143] [  12754]     0 12754     4088     2489    24576        0             0 sshd
[55744.696276] [  12760]     0 12760      934      704    12288        0             0 sftp-server

It repeats for every process that is running an kills it. Sometimes the device becomes irresponsive and I have to reboot it manually.
Do you have any idea how to solve this issue?

[meveric]

The XU4 is only a 32bit system and technically only supports sizes up to 4TB.
It may support larger devices, but it's no guarantee. I normally spit larger devices in partitions smaller than 4 TB to prevent issues.
Also NTFS is not very good supported under Linux while it generally works it does have issues here and there and a format like exFat or a native linux format like ext4 are way better supported.

The error you see shows in the first line that the "oom-killer" (Out Of Memory Killer) found it has no more memory (RAM) left and ended the ntfs-3g process which was using at that time the most memory.
You can try mitigating or delaying the issue by installing zram-odroid and extend the memory due to virtual swap space, but that's not a guarantee it will not happen again.

[meveric]

Small clarification.
The maximum storage device size is limited at 16TB on the 32bit system.
viewtopic.php?f=97&t=43289