IPP/FIMC backend for mpv (Exynos4412)

Share here your ideas for new projects

Moderators: meveric, mdrjr, odroid

IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Tue Oct 06, 2015 5:47 am

Hey,

starting this thread to document a new small project of mine. As you might know the media player software mpv has a DRM vo backend, which can be used on the Odroid-X/X2/U2/U3 to display video without relying on X11 or fbdev.

The downside is that the backend does colorspace conversion and scaling in software. So while 1080p playback works, the CPU usage is still very high because of the overhead of CSC and scaler.

Why is this interesting? There is only a small set of player software that provides some kind of hw acceleration for presentation. XBMC/Kodi, VLC (with EGL backend) or mpv with x11egl backend. All these methods involve using EGL and therefore pull in dependency on the Mali blob.

Idea: Modify the current DRM backend in mpv so that it uses the IPP infrastructure. The FIMC accelerates CSC and scaling and works directly on GEM buffers. This leaves just decoding for the CPU.

Status: Currently adding IPP support to libdrm. I can already copy RGB to RGB buffers (first step, so I choose something trivial), but I'm not yet happy with the design. Once the design is fixed I'm going to focus on the (multi-planar) YUV to XRGB conversions, since this is going to be the most used path in the backend.

No public code yet.

- lq
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby memeka » Tue Oct 06, 2015 6:59 am

if/when mpv gets a gstreamer backend, it would add more choice for players... as it stands, for 1080p decoding, Exynos 4412 is not fast enough...
best of luck on your project
User avatar
memeka
 
Posts: 3631
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Thu Oct 08, 2015 2:56 am

I finished designing a simple public IPP API which uses a source-pipeline-destination approach. Most of the API is already implemented, but things may still need some changes here and there.

The test application currently loops over XRGB8888 buffers, letting the FIMC copy them from source to destination and then doing a compare. The compare is 'fuzzy' since the internal color representation of the FIMC seems to be some YUV format. So the engine converts RGB to YUV when reading the source, does its work (currently just copying), and converts back from YUV to RGB when writing to the destination. RGB/YUV conversion is lossy, so this introduces some small errors to the color channels.

Tested this with three 1920x1080 buffers and 1000 iterations. FIMC engine seems stable and fast. What's taking the most time is filling the buffers with random data and doing the compare.

Stuff to test:
- YUV to RGB and vice versa
- YUV to YUV (different formats)
- source/destination with different size parameters (the scaler is currently inactive)
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Fri Oct 09, 2015 6:17 am

Hacked together a very crude acceleration implementation on top of the DRM vo backend, so that I can get an impression on how the API design works for the intended purpose.

Test sample is Big Buck Bunny / 1080p / H264. Decoded color format is yuv420p. The FIMC nicely handles this format.

During playback load on each CPU core stays below 30% (performance governor is selected). So first evaluation looks promising.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Mon Oct 19, 2015 9:08 pm

I cleaned up the code a bit (still needs more work though) and pushed it to this branch.

Current limitations:
- only yuv420 supported at the moment
- aspect ratio
- OSD rendering not implemented

Also performance can probably be further improved. Still it shows that 1080p decoding in software is not an issue on the Exynos4412 devices.

EDIT: You need recent odroid-4.2.y for this backend.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Thu Oct 29, 2015 1:08 am

Rebased the mpv and libdrm code. I noticed some apparantly refcounting related problem in the IPP kernel code, which leads to exhaustion of the CMA when starting and quitting mpv repeatedly. This doesn't happen for the test application, so I'm still investigating.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby memeka » Thu Oct 29, 2015 5:59 am

does mpv use any clutter/cogl library?
User avatar
memeka
 
Posts: 3631
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Wed Nov 11, 2015 1:26 am

Marek has posted a large series of IPP related patches. This changes the kernel API and also currently doesn't expose CSC functionality, but it still looks promising.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Sat Sep 16, 2017 6:21 am

There is new activity on the IPP front:
https://www.spinics.net/lists/dri-devel/msg152040.html

So basically this rips out the old IPP (let's call it IPPv1) API and replaces it with the vastly simpler IPPv2 API. I have started to convert my libdrm code to use v2 and so far things look good. Next step is to also adapt the mpv vo backend to the new API.

I was planning on just doing CSC with the FIMC and leave scaling to the video plane, i.e. CSC to NV12 and the use the VP for the rest of the playback chain. The rationale is to keep bandwidth use low.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2

Re: IPP/FIMC backend for mpv (Exynos4412)

Unread postby LiquidAcid » Sat Oct 14, 2017 1:52 am

First draft on the mpv video backend can be found here:
https://github.com/tobiasjakobi/mpv

Needs IPPv2 patchset v2 applied to the kernel. The userspace part can be found here:
https://github.com/tobiasjakobi/libdrm/tree/ippv2

Fairly limited at the moment, but should also work on Exynos5 based boards. Bandwidth requirements are higher though.
LiquidAcid
 
Posts: 1076
Joined: Fri Oct 11, 2013 11:07 pm
languages_spoken: english
ODROIDs: X2


Return to The Ideas

Who is online

Users browsing this forum: No registered users and 1 guest