XU4: Which Linux runs Netflix and VLC?

mctom · Post by **mctom** » Sun Mar 20, 2022 9:16 pm

Hi,

So the time has come to get rid of a swarm of single purpose SBCs and move everything onto XU4.

I want to run a few services on it:
- Pi Hole (DNS and DHCP)
- Deluge Daemon (Torrents)
- Borg backups
- Watch movies on VLC
- Watch stuff from Netflix

Hardware-wise, I want that to run with 200GB uSD card, and 1TB 3,5" HDD via USB-SATA dongle. The latter will be exclusively for Borg Backups.
The video output is 1024x768 projector, so I think software decode / rendering could be sufficient, if it helps.

I wanted to install official Ubuntu with MATE, but I see running VLC is a "known issue", so that pretty much rules it out for me.
Any suggestions will be appreciated.

mad_ady · Post by **mad_ady** » Sun Mar 20, 2022 11:32 pm

I doubt SD netflix works on anything but Coreelec or Android, but I wouldn't know, since I don't have it.

Regarding VLC - first make sure you have accelerated x11 support with glmark2-es2, then configure vlc to use GLES output and it should work. I'll try to test it next week, but it used to work a few years ago on a c2.

mctom · Post by **mctom** » Mon Mar 21, 2022 1:24 am

There is no Coreelec for XU4, as far as I can see.
My understanding is Netflix should "just work" on XU4, as Google's holy encryption library is available for 32-bit ARMs. It would be nice if it worked, especially because I have somewhat created an expectation that it will.

That leaves me with not many options really.. Either Debian/Ubuntu based, with official Ubuntu being an obvious choice, or Arch, with its "building from scratch" manual that I went through once and don't want to do it again.
Android is out of question. Kali? No...

I guess I'll try Ubuntu minimal with XFCE and see how it works.
Thanks for the tips regarding VLC, in the worst case I'll try some other player. All I want is GUI and subtitles support, so not much.

mad_ady · Post by **mad_ady** » Mon Mar 21, 2022 1:40 am

To save you some headache - if you install xfce, lightdm might produce a black screen on login. It's a lightdm issue and can be bypassed by installing gdm3 instead. I'm using xfce on Hardkernel's ubuntu 20 and it works ok as a light desktop. Try to install saraicot's chromium build. Not sure it has the widevine .so files or if you need to install them separately...

mctom · Post by **mctom** » Mon Mar 21, 2022 2:22 am

I have just found out there is "kodi-fbdev" package in the repository, sounds great. But apparently it works only with x264 movies, not so great anymore..
However it could make netflix work.

So I think I'll try XFCE and that specific Chromium build. And VLC or something else.
I'll probably wrestle with that next weekend, it's too late to start doing anything frustrating.

mctom · Post by **mctom** » Tue Mar 22, 2022 1:19 am

I have a hard time finding "saraicot's chromium". I assumed I'll just google it, but no luck

Anyway, I'm installing xfce as we speak, and picked gdm3 when it asked me what I wanted. We'll see if it works shortly.

mctom · Post by **mctom** » Tue Mar 22, 2022 3:40 am

Hmm, I did install a bunch of stuff, including mesa-x11 and did some Xorg configuration magic, and I think I'm almost there:

inxi -G

Code: Select all

Graphics:  Device-1: exynos5420-mali driver: mali v: N/A 
           Device-2: exynos5420-hdmi driver: exynos_hdmi v: N/A 
           Display: x11 server: X.Org 1.20.13 driver: exynos_hdmi resolution: 1920x1080~60Hz 
           OpenGL: renderer: llvmpipe (LLVM 12.0.0 128 bits) v: 4.5 Mesa 21.2.6

glmark2-es2 still won't work, and glmark2 works more or less the same. XU4 draws 10W in the process..

Video in mpv still works like crap, and VLC does not launch at all. I think I'm missing a decent OpenGL renderer for any reason. Could that be it?

I'm starting to think I should have picked that MATE image and install xfce on it...

Post by **odroid** » Tue Mar 22, 2022 9:23 am

Our Ubuntu 20.04 Mate image can run 'glmark2-es2' out of the box without any extra package configurations.
But, as far as I know, it is very hard to use the GPU and VPU hardware acceleration together on the Ubuntu x11 (Mate/XFCE) desktop.
Therefore, we supplied an old Kodi .deb package for our Ubuntu minimal image which use the direct rendering manager (DRM) like a bare framebuffer for using the VPU & GPU hardware acceleration.
If you install the Netflix add-on on the Kodi, you can enjoy up to HD resolution videos on the minimal image. But, there is no desktop GUI.

If you want to use the Netflix with some other features, consider trying the following all-in-one OS images.
viewtopic.php?f=147&t=39319
viewtopic.php?f=96&t=42354

mad_ady · Post by **mad_ady** » Tue Mar 22, 2022 3:53 pm

What do you know? It seems my install is running without GLES support, though I have mali-x11 installed:

Code: Select all

 adrianp  ~  $  glmark2-es2
Error: eglInitialize() failed with error: 0x3001
Error: eglInitialize() failed with error: 0x3001
Error: main: Could not initialize canvas
 adrianp  ~  $  sudo apt-get install mali-x11
Reading package lists... Done
Building dependency tree       
Reading state information... Done
mali-x11 is already the newest version (20200812-r17p0-cab1930-200).
The following packages were automatically installed and are no longer required:
  gdbserver libbabeltrace1 libdw1
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 61 not upgraded.

I remember starting from the Mate image, so that acceleration was builtin. Let me try something...

Edit1: For accelerated chrome I meant this repo:https://launchpad.net/~saiarcot895/+arc ... omium-beta, but it has builds only for amd64 and arm64...
Edit2: I've tried running Mate and Gnome on X11, but both show the same EGL error. So XFCE is not the cause of the problem. I've also reinstalled mali-x11 (sudo apt-get install --reinstall mali-x11) and restarted, but I get the same error. I'll need to read a bit more and see what's going on, because I remember testing it before and it worked on the official Mate image. Not sure what might have broken it.

mctom · Post by **mctom** » Tue Mar 22, 2022 5:52 pm

Thanks @odroid for new options. I really would prefer staying with Ubuntu and general purpose desktop, but if nothing works I'll give that Fedora thing a try.

Kodi-fbdev doesn't work here, I selected it in gdm3 and got black screen for some 10-20 seconds, and it came back to greeter. Hm, however if it worked I think that would be acceptable.
Now when I think about it I may have had network problems at that time, perhaps that was the reason to get all upset. Or perhaps kodi-fbdev cannot be run by just any user?

Is there a point in trying MATE image and making VLC work on it, or MATE image with kodi-fbdev?

Could Wayland improve the situation in any way? I wouldn't mind Wayland if I knew it has higher chances of success.

Netflix isn't my main concern here, I just want it to play videos smoothly for now. I believe I could install Kodi with Netflix plugins later on.

mad_ady wrote: ↑
Tue Mar 22, 2022 3:53 pm
Edit1: For accelerated chrome I meant this repo:https://launchpad.net/~saiarcot895/+arc ... omium-beta, but it has builds only for amd64 and arm64...

Oh well, at least I won't try something that surely won't work.

mad_ady wrote: ↑
Tue Mar 22, 2022 3:53 pm
I'll need to read a bit more and see what's going on, because I remember testing it before and it worked on the official Mate image. Not sure what might have broken it.

I'd appreciate if you came back with any additional clues.

BTW I believe my grandpa videophone project suffered from the very same problem - CPU was hogged because there is no hardware video acceleration. If we managed to solve it for X11 in general, there is a high chance that project would benefit too.

Post by **odroid** » Tue Mar 22, 2022 6:30 pm

We don't have any solution to use the GPU/VPU acceleration on X11.
Only OpenGL-ES GPU driver works on X11/Mate desktop.

You should flash this minimal Ubuntu 20.04 image first and install kodi-fbdev package on it. GPU/VPU accelerated Kodi should work out of the box.
https://wiki.odroid.com/odroid-xu4/os_i ... l/20210928

After checking the functionality of Kodi on the framebuffer, try installing a desktop(Mate or xfce or whatever) package to build your own system.
You can switch between the framebuffer console and x11 desktop by pressing Ctrl-Alt-F2 something like that.
But, it is far from your target usability for your grandpa proably.

mctom · Post by **mctom** » Tue Mar 22, 2022 8:14 pm

O-kay, I think we could have a working solution here.

This SBC will be used for low resolution projector based entertainment, so mostly Netflix and DVD rips. Kodi should support all that, right?
I think kodi-fbdev could be working, I just didn't run it properly (I tried running Kodi from greeter instead of tty1)

If x265 doesn't work in kodi-fbdev (I'm not sure if x264 implies that?), I'll either have to transcode all my ripped DVD movies, or carry on trying to make VLC work with GLES on XFCE. I'd try the latter, as my future downloads could be x265 by default.
... Or set up an automatic transcoder for all movies that I download, but I fear the XU4 fan would never stop

Here is a GLES2 plugin for VLC, but it looks very old and sparks barely any interest, which makes me assume this functionality is already supported by VLC itself.

rooted · Post by **rooted** » Tue Mar 22, 2022 8:24 pm

XU4 can't hardware decode h.265, it will software decode it but struggles with 1080p if it will play it at all. No hardware for vp9 either but luckily YouTube can play h.264 mpeg4 if that matters.

mad_ady · Post by **mad_ady** » Tue Mar 22, 2022 8:29 pm

VLC can display video out using GLES directly, no need for extra code. See Tools -> Preferences -> Video -> Output -> Open GL for embedded systems 2.
XU4's SOC can't decode x265 in hardware, but it can do so in software, provided the bitrate isn't too high.

I'll have to troubleshoot why GLES isn't working on my XU4... It may take a while. I'm using it for youtube in the browser, and it's a decent experience (albeit at 480/720p).

mctom · Post by **mctom** » Tue Mar 22, 2022 9:01 pm

Well if Kodi could handle youtube that would be sweet, no doubt about it, although I'm not a fan of youtube parties.

The output resolution is 1024x768, and when I download a movie I usually pick 720p, because that's the lowest resolution available.
My DVD rips are 1024x(??) as I'm making them with my projector in mind.
Although the decoding speed apparently depends on source file resolution and not the target.

mad_ady wrote: ↑
Tue Mar 22, 2022 8:29 pm
VLC can display video out using GLES directly, no need for extra code. See Tools -> Preferences -> Video -> Output -> Open GL for embedded systems 2.

So that's another thing I'll give a try when I'm back home. So far I only tried to play a movie using VLC, and the app window never appeared. Perhaps it will if I just open VLC with no specific file selected, or force some safer settings by modifying its config file.
Alas, there isn't much hope if glmark2-es2 doesn't work for now.

The Fedora XU edition thread contains some technical info, but I can't tell if this is useful in any way in this case..

mctom · Post by **mctom** » Wed Mar 23, 2022 5:48 am

Update: Kodi is a beast, it runs extra smooth and plays my x265 files too, possibly with software decoder. It takes some 5W to play a video, so well within reason.
It runs smooth on 1920x1080 test environment, shall I add.

Now in order to run Netflix on this thing, important question: Can I update Kodi, or is it outdated for a reason?

EDIT: Also, I'd like to run kodi on startup, so I understand I could add it to /etc/rc.local. But I would like to run it as a user "spectator" that has no way to mess anything up. Also I think rc.local should return zero before I exit kodi... How do I do that?

EDIT: I think this did the trick:
su spectator -c 'kodi'&

mad_ady · Post by **mad_ady** » Wed Mar 23, 2022 7:07 am

Also I think rc.local should return zero before I exit kodi...

No, it doesn't need to exit. It will just pause execution until you close kodi.

Though the cool kids would use a systemd service to start it...

rooted · Post by **rooted** » Wed Mar 23, 2022 8:45 am

It's great to hear it still has enough oomph to software decode 1080 HEVC, that is quite intensive.

Post by **odroid** » Wed Mar 23, 2022 9:32 am

mctom wrote: ↑
Wed Mar 23, 2022 5:48 am
Update: Kodi is a beast, it runs extra smooth and plays my x265 files too, possibly with software decoder. It takes some 5W to play a video, so well within reason.

Which OS image and Kodi version do you run now?

mctom · Post by **mctom** » Wed Mar 23, 2022 5:45 pm

mad_ady wrote: ↑
Wed Mar 23, 2022 7:07 am
No, it doesn't need to exit. It will just pause execution until you close kodi.

Though the cool kids would use a systemd service to start it...

Exactly, it is convenient to have it configured as a service, but maybe not now.

I always struggle to remember how to do it, and I've got only one monitor, so can't google anything quick for reference.

I have also used your odroid-cpu-control to tune down XU4 a bit. Apparently Kodi still uses some CPU even when I don't use Kodi, and I don't want my XU4 to buzz the fan 24/7. So I changed governor to schedutil, as it used slightly less power than ondemand. Reducing max CPU frequency didn't change a thing.
And that also landed in rc.local. Can't imagine making a service out of it.

But I'll probably have to do something to conveniently run and stop Kodi when not in use.. Some sort of Bash loop. When I exit Kodi, the only way from user's perspective would be to press any key to launch Kodi again.

rooted wrote: ↑
Wed Mar 23, 2022 8:45 am
It's great to hear it still has enough oomph to software decode 1080 HEVC, that is quite intensive.

I don't think I have tested 1080 material. I tried a few clips I have found lying around, but I think the maximum resolution I use is 720p. Nevertheless, every movie I tried worked very well, with vastly different codecs.

odroid wrote: ↑
Wed Mar 23, 2022 9:32 am
Which OS image and Kodi version do you run now?

That's Ubuntu Minimal downloaded a few days ago, and I believe kodi-fbdev contains version 17 as of now. It asked me once if I wanted to update, but I figured that might not be the good idea.
I am asking about the update because apparently the method of running Netflix on Kodi changes with Kodi releases, with Kodi 17 Netflix plugin being "unavailable" already.

I didn't reinstall the system after all, I just removed lightdm and gdm3, and left xfce just in case.

Now my girlfriend won't stop bugging me about that Netflix after I said it should work.

But at least we both agreed YouTube is "meh" on a projector so I won't struggle with API keys nonsense for that.

Post by **odroid** » Wed Mar 23, 2022 5:55 pm

We couldn't build a deb package of Kodi 19 nor 18 for XU4 due to tons of errors while compiling and running both.
It is the main reason why we still supply the very old version only.

mctom · Post by **mctom** » Wed Mar 23, 2022 6:18 pm

So I take it I should stick to Kodi 17 and not bother trying to update it.
Okay then, I'll see if I can run netflix on that somehow.

mctom · Post by **mctom** » Wed Mar 23, 2022 6:27 pm

Oooh, what about that?
viewtopic.php?f=98&t=34149

mad_ady · Post by **mad_ady** » Wed Mar 23, 2022 6:29 pm

Apparently Kodi still uses some CPU even when I don't use Kodi, and I don't want my XU4 to buzz the fan 24/7.

I used kodi 17 on C2 and yes, it had significant load when idle. It seems to be caused by the gpu redering the osd. I managed to lower it by disabling the rss feed in the main screen and avoiding scrolling texts in menus.

Note that AreaScout might have a NAS image for the cloudshell2 that may have a newer kodi.
When kodi moved from 17 to 18 (or was it 18 to 19?) plugins moved from python2 to python3. So it's close to impossible to find a working netflix addon for kodi 17.

Post by **odroid** » Wed Mar 23, 2022 7:07 pm

I already gave you a link of more recent version Kodi 18.9.
viewtopic.php?f=147&t=39319

mctom · Post by **mctom** » Wed Mar 23, 2022 7:25 pm

Hmm perhaps I would like a system where Kodi is automatically updated. Kodi seems to work like Android, a 2 years old release is obsolete beyond use

Like Electron - reaches EOL just 9 months after release.
What's up with software these days, imagine transistors going obsolete after 9 months, lol. There are electronic components still in active production after 50 years.

Anyway, in that case I could use one of somethingELEC builds and be done with this, but I want to run a few extra services on that machine, and I imagine these entertainment OSes don't have package manager such as apt.

odroid wrote: ↑
Wed Mar 23, 2022 7:07 pm
I already gave you a link of more recent version Kodi 18.9.
viewtopic.php?f=147&t=39319

Hmm, it is based on Debian Buster, so I think apt should still be there, and this could be what I'm looking for, after all. I'm sorry, I forgot about that one!

AreaScout's Kodi 18.7 build is also tempting and works on Ubuntu Minimal. But I'd have to switch to Bionic that ends support next year...

Okay then, tonight I'll try that AreaScout's OMV image.

mad_ady wrote: ↑
Wed Mar 23, 2022 6:29 pm
I used kodi 17 on C2 and yes, it had significant load when idle. It seems to be caused by the gpu redering the osd. I managed to lower it by disabling the rss feed in the main screen and avoiding scrolling texts in menus.

The power consumption was the same even when picture was completely static in Kodi menu. That implies it was still rendered even though nothing changed.
I think a method to shut down Kodi when not in use is a way to go, and not that hard to achieve really.
I'm running my tests on XU4 unit with custom, large heatsink and it stabilized at 52C. The target unit has a fan and much smaller heatsink, so I think it's not enough power saving to keep the fan off.

However, AreaScout wrote that his OMV + Kodi image shuts down Kodi when TV is off - I wonder if it's gonna work with my old projector via HDMI-VGA adapter.

rooted · Post by **rooted** » Wed Mar 23, 2022 7:52 pm

For plugins yeah Kodi gets outdated quickly, but if you don't need them I've found older releases to be better actually.

mctom · Post by **mctom** » Thu Mar 24, 2022 7:06 am

Okay, another few hours of basically trial and error.
I tried that OMV image and it didn't work as expected.. It caused my XU4 to reset all the time for any reason, especially when I wanted to perform actions on Kodi plugins.
So I figured I don't want Netflix THAT bad, if trouble is all I get in return.

I settled back with fresh Ubuntu Minimal again and kodi-fbdev which works really well. I felt reassured by @rooted that if I stay away from plugins, it may be a long term solution.
I also managed to run mpv in framebuffer in decent quality, using --vo=drm --profile=sw-fast - my usual settings on a netbook. That of course is a software render, and caused XU4 to draw some 5-6W in the process. Not critical to me, as the video projector is an old noisy junk on its own, but Kodi handles that in hardware, so that's the way to go. Not to mention mpv doesn't remember volume settings, and I have 1 cubic meter of speakers at home.
Although I admit I'd much prefer picking a movie in mc and displaying it in mpv, rather than navigate through that candy crush GUI.

Anyway, to give Netflix a chance, I figured we could stream desktop from my gf's laptop onto XU4 and play it in Kodi. Streaming online is a thing these days, so should be doable, right?
It seems that indeed it is. Here is some guy who described how he achieved that.

And surprisingly, it works. Gstreamer captures video on desktop machine, compresses it using x264 encoder, and flushes it down to udp sink, which is XU4. On Kodi side, there's a *.m3u file that lets Kodi accept the stream. I decided to send that stream directly to XU4, I don't do multicasts because I don't understand them

Doing so with 2K display wasn't the brightest idea and my desktop couldn't keep up with encoding all that, which resulted in random grey areas appearing on the bottom of the screen. However I modified the stream definition (with barely any clue what's going on in there), and by scaling the video down to projector's native resolution, the problem was kinda solved.
XU4 of course has no problem decoding that, 3.5W was a typical power consumption.

The impact on video quality is clearly visible, though, as all options are tuned towards low quality and fast encoding. However if I could make it run with hardware encoder, that I hope is present in my Celeron J4105, I'm curious to see if that could eventually become watchable.

Anyway, for the record, here's my experimental batch file to launch streaming. x264enc is a software encoder I believe, and replacing it by vaapi counterpart is expected to do the magic.

Code: Select all

[mctom@Tomusiomat ~]$ bat gst_save.sh 
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: gst_save.sh
       │ Size: 519 B
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ #!/bin/bash
   2   │ gst-launch-1.0 ximagesrc use-damage=0 ! video/x-raw,framerate=60/1 ! \
   3   │   videoconvert ! queue2 ! videoscale add-borders=true ! \
   4   │   x264enc bitrate=64000 speed-preset=superfast tune=zerolatency qp-min=30 \
   5   │   key-int-max=15 bframes=2 ! video/x-h264,profile=high,width=1024,height=768 ! queue2 ! \
   6   │   mpegtsmux alignment=7 name=mux ! rndbuffersize max=1316 min=1316 ! \
   7   │   udpsink host=192.168.0.171 port=1234 ttl-mc=1 auto-multicast=1 sync=0 \
   8   │   pulsesrc ! \
   9   │   audioconvert ! queue2 ! avenc_aac ! queue2 ! mux.
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────

BTW bat is a cool utility.

rooted · Post by **rooted** » Thu Mar 24, 2022 7:21 am

That's cool, I had no idea anyone had figured out how to stream Netflix in that way.

mad_ady · Post by **mad_ady** » Thu Mar 24, 2022 3:18 pm

If your gf's laptop has a dedicated gpu you coulc use something like sunlight/moonlight/parsec to stream the desktop at good quality and with low latency. I think there's an equivalent for intel gpu also. Not sure how linux-friendly the servers are though...
Also, a bitrate of 64kbps is way too low. For realtime encoder 1080p, try something like 10Mbps

mctom · Post by **mctom** » Thu Mar 24, 2022 5:53 pm

Nah, it's a laptop similar to my "desktop" in terms of performance, Intel HD something GPU in it. No wonder why we play supertuxkart and wesnoth mostly.

I don't think changing software at this point would do any good, but it's worth keeping in mind those exist.

Tonight I'll try to understand gstreamer's syntax and optimize this pipe for my use case. It's not as simple as it seems, in the command above there are actually multiple pipes defined, somehow..
And the tool seems to be super versatile, I won't regret learning its basics.
bitrate parameter is in kbps, so ~64Mbps should be enough. However I lurked into htop and clearly not the whole bitrate was in use on network level.
If for any reason I fail to use hardware encoder, or it won't be as great as I imagine, I could also fall back to some simpler encoding at the expense of bandwidth.
Latency is not of any concern, may be as well 5 seconds.

mad_ady · Post by **mad_ady** » Fri Mar 25, 2022 1:24 am

Try also the fedora image - it seems to have widevine support hacked-in:
https://r.tapatalk.com/shareLink/topic? ... source=app

Fedora 34 Workstation - XU Edition

mctom · Post by **mctom** » Fri Mar 25, 2022 2:18 am

Nah, I'd rather stick to good ol' Ubuntu for my services and stuff, and since video playback is sorted out I'm happy. Also, @odroid has already linked that I think.

Meanwhile I'm experimenting with desktop streaming, and I've made some progress. I am currently using vaapi, so encoding is handled by GPU. Yay!
It didn't work without a fight though, I managed to produce a stream that crashed Kodi on the other end.

It turns out careful queue placement does matter, and selecting queue types for a job.

Now I'm fiddling with parameters to squeeze some more performance from that. The delay dropped somewhere below 1s and picture quality is pretty much the same. It's hard to measure that, you've got to trust me.

It also turns out the frame scaling is also handled by GPU even though it's not said so anywhere. If I opt not to resize them, the CPU utilization is pretty much the same, and GPU "Render" time is well reduced (as reported by intel_gpu_top).
So in the end it might be more beneficial to not resize the frames, given that XU4 doesn't care that much, nor my network. I'll do some more scientific experiments after I'm done playing with parameters. So many of them!

Code: Select all

───────┬─────────────────────────────────────────────────────────────────────────────
       │ File: gst.sh
       │ Size: 569 B
───────┼─────────────────────────────────────────────────────────────────────────────
   1   │ #!/bin/bash
   2   │ #GST_DEBUG="GST_TRACER:7" \
   3   │ #GST_TRACERS="latency(flags=pipeline+element+reported)" \
   4   │ gst-launch-1.0 ximagesrc use-damage=0 ! \
   5   │   video/x-raw,framerate=60/1 ! queue ! \
   6   │   videoscale ! \
   7   │   videoconvert n-threads=1 ! queue ! \
   8   │   vaapih264enc cpb-length=500 tune=0 ! \
   9   │   video/x-h264,profile=high,width=1024,height=768 ! multiqueue name=q ! \
  10   │   mpegtsmux alignment=7 name=mux ! queue2 ! \
  11   │   rndbuffersize max=1316 min=1316 ! \
  12   │   udpsink host=127.0.0.1 port=1234 ttl-mc=1 auto-multicast=1 sync=0 \
  13   │   pulsesrc ! \
  14   │   audioconvert ! queue ! opusenc ! q. q. ! mux.

One thing puzzles me why buffer right before udpsink has size of exactly 1316 bytes. Does it have something to do with the nature of LAN?

mad_ady · Post by **mad_ady** » Fri Mar 25, 2022 2:43 am

What happens if you omit or change the value? Try going over 1480 (which would be max udp payload size on a lan with 1500 bytes packets.
What happens if you omit it?
My guess is the size is chosen to prevent packet fragmentation and maybe queing.

mctom · Post by **mctom** » Fri Mar 25, 2022 7:11 pm

Turns out nothing happens. Everything appears to work normally. I guess this might have been a leftover from original author's experiments. This element's intended use is for testing the pipeline with random buffer conditions.

So after another night of experiments, here's what I have found:
- The hardware h264 encoder for any reason produces a stream that doesn't work on Kodi (symptoms similar to this). The same stream worked with mpv, but flooded dmesg.
- So I switched to h265 vaapi encoder and decoding worked just fine, there was barely any difference in power consumption, so it might have been at least partially handled by hardware on XU4 side.
- Encoding video in GPU is cute and all, but I have completely forgotten that GPU is also used for decoding Netflix / YouTube stream in the first place

So when I played a YouTube video for tests, new problem emerged. Intel GPU seems to be tailored for Full HD encoding or decoding, but only one of those at a time. With additional encoding pipeline in place, Youtube started dropping about half of the frames on a host machine.
- So I wanted to try a simpler, software-based algorithm, such as MPEG2 or 4, as network bandwidth is of little concern, to let GPU handle its everyday tasks undisturbed and use CPU juice for streaming. I tried h264 and mpeg4, both load CPU at some 50-80% and the results are "meh". Watchable, but not "market ready". The passive cooled CPU is not happy about it either.

- Gstreamer debug shows that the majority of time is spent not on video encoding, but "videoconvert" component, that converts one raw video format into another raw video format prior to encoding. I'm not sure what is going on in there, and whether it could be avoided somehow.

Yep, for now that's another failed experiment, but I firmly believe even failed experiments contribute to the state of knowledge.

So, where do I go from here? There are only a few possibilities left:
- To find a video encoder that is not CPU intensive and does mild compression with few losses, at the expense of practically unlimited bandwidth for this purpose,
- Splitting the pipeline between host machine and XU4, but there's no good spot to split it really.
- Studying "videoconvert" component bottleneck and doing something about it.

I'm starting to think that HDMI switch could be a better solution than trying to set up that streaming function, lol

mad_ady · Post by **mad_ady** » Fri Mar 25, 2022 8:22 pm

To find a video encoder that is not CPU intensive and does mild compression with few losses, at the expense of practically unlimited bandwidth for this purpose

mjpeg. It's light on encoding, heavy on the network and light on decoding. Though not sure where you can multiplex it with sound - what container supports mjpeg + sound...

rooted · Post by **rooted** » Fri Mar 25, 2022 8:41 pm

You said you wanted to try mpeg2, I would definitely do that as it's fast and light. Bandwidth usage will be high of course but it should also look great.

mctom · Post by **mctom** » Fri Mar 25, 2022 10:53 pm

Thanks guys, I'll try both.
mpeg2 was intimidating with so many parameters I didn't even attempt to fiddle with them. I think mpeg2 has limited selection of resolutions to choose from, but should be able to do XGA, I hope?
With mpeg2 I also had grey or discolored areas randomly appearing on the bottom of the screen, especially when it wasn't moving - some problems with lack of key frames or something? Probably could be fixed with proper configuration.

I'm not familiar with mjpeg, but I may as well try it too.
mpeg_ts container that I use won't support mjpeg, though.

mpeg_ts is dedicated to streaming. I couldn't find any other container that supports streaming. Trying to stream Matroska didn't work

If you know any other options let me know.

And which audio codec is the lightest? I use opus right now, but aac and mp2 all worked too. mp2 was mono and I could tell it sounds weird.

EDIT: I have found a list of video containers.
Matroska should be streamable, and indeed, matroska mux in gstreamer does have "Streamable" property that I overlooked...
I think I'll give it a try, it's the only free video container I can think of. And should work with mpeg2, mjpeg and all other video formats.

rooted · Post by **rooted** » Sat Mar 26, 2022 3:59 am

You could try webm and vp8

mctom · Post by **mctom** » Sat Mar 26, 2022 5:35 am

Yeah, I could. It is a lot of trial and error, as many plugins with seemingly similar functionality have different behavior, it's messy to say the least. For example there are a few mpeg2 codecs, no idea why didn't they just make one that works.
The same could be said about 500 Linux distros, I know.
No wonder why plugins are grouped in categories: good, bad and ugly. The categories don't seem to match the actual performance of these plugins in any way.

I had moderate success with mpeg2 today, the picture looks decent in overall, except of the artifacts when nothing changes on the screen, or everything changes at once. I think this could be solved by tuning the parameters, or trying another mux.
The host CPU is less thrashed, at 40% or so. But apparently XU4 decodes that in software and draws 4W, which again is not a big issue as long as it copes.
And also sound stopped working for any reason, even though it always worked with any codec. I'm yet to discover why.

I had a few attempts at mjpeg, but that essentially requires mp4 or QuickTime container (almost the same thing). I wasn't able to make any mp4 mux to work to date. Either gstreamer errored out or stream wasn't readable by anything. So far only mpeg_ts works for me.

I also discovered a very interesting format, huffyuv. Dating back to early 2000s, it was apparently able to do lossless compression (or nearly-lossless), even on Pentium II. Sadly, this format faded to obscurity, and there is no streaming-friendly container that would support it. There's frankly no container that supports it at all. And indeed a basic Huffman compression would do.
I remember I saw *.m3u files in the past that glued together two streams, audio and video. If I could get rid of a container, that would mark a breakthrough in this effort. However I'd have to keep both latencies down to a bare minimum to keep both streams in sync.

So the chain looks like this: video enc - mux - transport.
I successfully implemented h264, h265 and mpeg2 encoders so far.
Only one mux works okay: mpeg_ts.
For transport, I used raw udp, or rtp, there was virtually no difference, as neither offers QoS anyway.

rooted · Post by **rooted** » Sat Mar 26, 2022 9:24 am

I used a USB based TV tuner for years and years which is high bitrate mpeg2 ts, it played back great with no artifacts on every device I've owned including the Odroid C1. I believe the XU4 has hardware decoding for it also but I can't remember for certain.

*Edit*
I have to add I listed the C1 because it is relatively underpowered compared to later devices, the XU4 also played back fine as well.

mad_ady · Post by **mad_ady** » Sat Mar 26, 2022 2:05 pm

To see if the macroblocks you're seeing are caused by the network or the xu4 decoding, try to view the stream locally on your laptop (on a second screen). They might be caused by dropped udp packets, so it might help go increase the kernel udp receive buffer on the xu4 side. I do it at work for udp-intensive applications (e.g. syslog servers).
https://askubuntu.com/questions/1115636 ... uffer-size

mctom · Post by **mctom** » Sat Mar 26, 2022 9:16 pm

Thanks again to both of you for all that precious insight. Tomorrow I'll be decommissioning the Pi4 that served us as a media player in favor of XU4, as it will take some time I cannot afford outside weekend.

@mad_ady, confirmed. I streamed into 127.0.0.1 and ran the stream in mpv, and no artifacts appear. Sadly I have only one monitor, so I made a sort of video looper, but nevertheless there were no visible errors.

mctom wrote: ↑
Fri Mar 25, 2022 2:18 am
One thing puzzles me why buffer right before udpsink has size of exactly 1316 bytes. Does it have something to do with the nature of LAN?

mad_ady wrote: ↑
Fri Mar 25, 2022 2:43 am
What happens if you omit it?

mctom wrote: ↑
Fri Mar 25, 2022 7:11 pm
Turns out nothing happens. Everything appears to work normally.

I thought this may have something to do with this problem, so I added a buffer limited to 1300 bytes back. It doesn't seem to do anything.

I tried what you suggested, but it didn't help, perhaps I'm doing something wrong.

Code: Select all

sudo sysctl -w net.core.rmem_max=83886080
sudo sysctl -w net.core.rmem_default=83886080

I also tried this, but didn't help either:

Code: Select all

sudo sysctl -w net.ipv4.udp_mem='8388608 8388608 8388608'

I still can see buffer errors counter accumulating more and more cases:

Code: Select all

mctom@odroid:~$ netstat -su
IcmpMsg:
    InType3: 40
    OutType3: 103
Udp:
    1754083 packets received
    33535 packets to unknown port received
    165634 packet receive errors
    253 packets sent
    165634 receive buffer errors
    0 send buffer errors
    IgnoredMulti: 5
UdpLite:
IpExt:
    InMcastPkts: 384
    OutMcastPkts: 191
    InBcastPkts: 5
    InOctets: 2630178034
    OutOctets: 387666
    InMcastOctets: 91513
    OutMcastOctets: 29322
    InBcastOctets: 876
    InNoECTPkts: 1956765

And I tried restarting NetworkManager after all these changes, but still no luck.
Please help! We're so close to cracking this case!

And about all the other stuff... Sound stopped working because sound in Kodi stopped working in overall. This must have something to do with user privileges and pulseaudio, as I started using a dedicated user for Kodi.

XU4 does not decode MPEG2 in hardware, it takes about 5W playing my stream (compared to 3W with h264 stream and 2W idle).
Smart Power 3 is such a cool device. I don't need htop anymore!

But htop reports some 140% CPU time on kodi.bin while playing the stream. Still well within XU4 capabilities of course, but I won't avoid fan action.

mad_ady · Post by **mad_ady** » Sun Mar 27, 2022 12:28 am

Ok, the next step is trickier - we need to work out if stream packets are dropped by the network or kernel/application.
You'll need to use tcpdump on both ends to do a packet capture of the stream )let's say it's on port 1234:

Code: Select all

laptop# tcpdump -i any -n -s 0 -w /tmp/server.pcap port 1234
xu4# tcpdump -i any -n -s 0 -w /tmp/client.pcap port 1234

Let the glitch happen and compare both captures. Find the same start packet on both ends (search by checksum) with wireshark, and find the same end packet on both ends. Count the number of packets in-between. If it's the same, all packet arrive and the xu4 must be dropping some of them.

For udp - try running kodi on the big cores, checking if the usb port interrupts run on the big core (see which odroid-tweaks).

Or... try streaming over tcp instead? That should resolve drops...

mctom · Post by **mctom** » Sun Mar 27, 2022 5:07 am

Okay, before I try that let me reiterate what I understood:
So either the network drops packets, or Odroid does. Until I was testing it on my desk, both machines were separated by just one 1Gbps switch.
Now, I have put it in its place with two network devices in between - the same switch, and ISP router box, that acts as another switch in this case.
The results are the same, when you look at the picture.

I could dive deeper into understanding the problem, but I'm worried it won't give us any solution. If network drops packets, I'm not replacing it. If Odroid is too slow, well.. Not much we can do about it.
Oh, and I switched governor back to performance, didn't change anything. Kodi uses some 90% CPU time (out of 800%) so I can't imagine that being a bottleneck.
Honestly, to me it looks like the network is losing packets. Any way to test against that exactly?

I'll try analyzing packets tomorrow, I'm too tired today. I'll post my today's build in Banter for everyone to point fingers at.

TCP would be an elegant solution, so I'd try and do that straight away. I already tried using gstreamer's tcp[server/client][src/sink] objects, but that doesn't work for me, and I can't find any examples working outside localhost. Whenever I try to launch a client from either side, it says it cannot bind to host, as if servers didn't have ports open at all.
Is there any generic Linux way to transfer raw data through TCP? nc or something?

rooted · Post by **rooted** » Sun Mar 27, 2022 5:45 am

You can put the USB on the big cores, I can't remember how but I bet Adrian can.

*edit*
He mentioned it above.

mad_ady · Post by **mad_ady** » Sun Mar 27, 2022 5:25 pm

Until I was testing it on my desk, both machines were separated by just one 1Gbps switch.

So, what is the stream's current bandwidth?
Also, you may need to measure pps count (try iptraf-ng). The switches on the way may have limited buffers, so they might cause packet drops.
Unfortunately, there is no way to guarantee no drops over udp... But you can try the following:
- increase packet size to 1480 to reduce pps
- if your switches support jumbo frames (you need to measure what jumbo frame size is supported), go over 1500 bytes

This reduces pps (and reduces interrupts on the nics) and may help. But at the same time streaming delay increases (though you might not mind), but in case of loss, you'll see larger macroblocks.

You can test iperf3 with udp and various bandwidths between xu4 and laptop to count packet loss. Something like iperf3 -c 192.168.x.y -u -b 50M -i 10 -t 120.
Also, to rule out intermediary network, you can test with a direct cable between laptop and xu4.

A while ago I used ffmpeg to stream with rtsp over tcp, but I would advise against changing everything again.
Yes, you can use netcat with tcp to stream data, but I don't know how you'd pipe it in and out with gstreamer/kodi...

mctom · Post by **mctom** » Sun Mar 27, 2022 7:30 pm

I do use ffmpeg codec already, but I guess you meant invoking ffmpeg directly. I do use a few exciting features of gstreamer, such as resizing with black borders preserving aspect ratio, and so on.

The current streaming bandwidth, as indicated in htop, is in orders of 1MBps - at least when the picture is mostly static, but artifacts appear regardless of what is displayed.
I ran iperf3, and there were no losses at 50Mbps, but some losses every few seconds at 300Mbps.

Hmm I suppose that 1MBps figure is a mean value over 1 or more seconds, but I fear data might be sent in short bursts of extreme throughput. That's what I'd expect from a video encoder.
So maybe the UDP buffers of ~8Mb that I previously configured on XU4 side were insufficient? It did say all errors were buffer errors..

But then again, I don't like that udp business and if there is a way to abandon it in favor of tcp, that would at least ensure proper operation in long term. I don't want to go back fixing it in 2026.

Last night I managed to make gstreamer write data to a file handle #1, stdout.

From there I can pipe it into nc and do whatever.
And I did. I piped it out to | nc -l -p 1233 and played back locally with mpv tcp://localhost:1233

The side effect is that once I quit mpv, nc see it as connection closed and the whole streaming pipe exits. Let's call it a safety feature

Now the question is whether Kodi likes that or not. I'll try playing it using a *.strm file after princess sleepy-head wakes up and won't be bothered by the projector.

mctom · Post by **mctom** » Sun Mar 27, 2022 8:18 pm

Hehey, it works! And it works brilliantly!

No stuttering or errors anywhere. The delay is about 1.5 or 2 seconds, but that doesn't matter in my case.

Here's the script:

Code: Select all

[mctom@Tomusiomat gst]$ bat tcp.sh
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: tcp.sh
       │ Size: 532 B
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ #!/bin/bash
   2   │ #GST_DEBUG="GST_TRACER:7" \
   3   │ #GST_TRACERS="latency(flags=pipeline+element+reported)" \
   4   │ 
   5   │ gst-launch-1.0 ximagesrc use-damage=0 ! \
   6   │   video/x-raw,framerate=30/1 ! queue ! \
   7   │   videoscale add-borders=true ! \
   8   │   videoconvert n-threads=1 ! queue ! \
   9   │   avenc_mpeg2video bitrate=20000000 quantizer=1 qmax=22 ! \
  10   │   video/mpeg,width=1024,height=768 ! multiqueue name=q ! \
  11   │   mpegtsmux alignment=7 name=mux ! queue2 ! \
  12   │   fdsink fd=1 sync=0 \
  13   │   pulsesrc ! \
  14   │   audioconvert ! queue ! \
  15   │   avenc_ac3 ! q. q. ! mux. \
  16   │   | nc -l -p 1233
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────

And the stream file for Kodi is completely unsurprising:

Code: Select all

spectator@odroid0:~$ cat tomusiomat.strm 
tcp://192.168.0.113:1233

The mpeg video encoder has rather poor documentation. Many of its parameters are labeled as "may be ignored", so I have no idea whether my parameters are respected in any way. Like I said, I measured some 1MBps despite setting 20Mbps bitrate.

XU4 has temperature between 50 and 60C, but I tuned the fan to work at 60 PWM in this temperature range, so it's not noticeable.

Thank you guys for making this happen! Yet again I owe you a round of Żołądkowa.

Now I've got a few minor issues to address:
- Audio has one channel only, pulsesrc component does a poor job on default settings apparently
- Audio is quiet, probably because it takes into account the volume control on the host machine,
- To add isolation transformers to physical audio path (gotta borrow them from kitchen audio setup, lol)
- To reduce framebuffer size on XU4 - instead of 1024x768 I'd prefer something like 1024x700. The movies will still fit, and my drum set blocks the view of Kodi controls that are on the bottom. fbset shoud do, I think

mad_ady · Post by **mad_ady** » Sun Mar 27, 2022 9:50 pm

Glad to hear tcp was a more reliable solution for you!

For audio - perhaps your recording script can record current volume level, increase it and restore it when the stream ends. But won't you hear audio from the laptop as well? Coupled with a 2s delay, it will get annoying really fast...