V4L2 M2M decoders on mainline: need testers !

Moderators: odroid, mdrjr

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby memeka » Mon Aug 13, 2018 3:42 pm

@Ely thanks.

btw, are there plans or is it hard to add S805 support to video dec? (Also, video enc)? I am just referring to VPU, not video plane support.
You mentioned there's limited support for it ... what are the limitations? :)

cheers.
User avatar
memeka
 
Posts: 4063
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Mon Aug 13, 2018 6:43 pm

memeka wrote:@Ely thanks.

btw, are there plans or is it hard to add S805 support to video dec? (Also, video enc)? I am just referring to VPU, not video plane support.
You mentioned there's limited support for it ... what are the limitations? :)

cheers.


So I actually started the vdec work a few months ago on an Odroid-C1, and most of the current code is compatible with that platform (S805). The main "problem" with S805 is that it only outputs the tiled pixel formats and doesn't allow for regular NV12 or YUV420P.
It's entirely possible that S805 gets added to the supported SoCs list in the future, but it'll require plumbing with everything else to support those formats.
Not to mention, there's no display support (DRM) for S805 as it's a completely different IP, so the vdec work on it was kinda "meh".

VENC, first I'll start working on it on S905+, and then we'll see ;) . It's a big, big chunk of code.
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby memeka » Mon Aug 13, 2018 7:43 pm

thanks for the reply.
dec+enc is still useful for S805 even w/o DRM cause it can make for an efficient iot device (cortex A5 is very low power), with a camera attached it can still stream high-res; or even an emby server it can transcode to clients.
but i am really impressed by the progress :) can't wait for encoders too :D
User avatar
memeka
 
Posts: 4063
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby miskol » Mon Aug 13, 2018 7:58 pm

I would really like to see multiple decoding.

@Ely can you explain what is difference between canvas that amlogic and you use in drivers and some DMA buffer ?
miskol
 
Posts: 145
Joined: Wed Jan 15, 2014 2:58 am
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Mon Aug 13, 2018 8:09 pm

miskol wrote:I would really like to see multiple decoding.

@Ely can you explain what is difference between canvas that amlogic and you use in drivers and some DMA buffer ?


A canvas is a pixel buffer descriptor (you program it with the dma buffer's phy addr, width, height, and other properties). Then, IPs like vdec and display are configured with canvas IDs rather than the phy addrs. This is only the case up to H.264, starting with VP9/HEVC you program the phy addr directly.

Multiple decoding is a very long way unfortunately, because each codec has a different firmware and kernel code for whether it's the single or multi-instance version. And the multi-instance versions are more complex.
And then you need to implement vdec scheduling and context switching. So a lot of work overall.
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby martos » Tue Aug 14, 2018 8:43 pm

Sadly i think encoder with aml drivers is a dream.
Amlogic take no time for dev and it's only because some guy ( i see Maxime Jourdan name https://patchwork.linuxtv.org/project/l ... itter=7555 , and a lot of other) make something to have this feature.
And without a team to support i am afraid this feature still not complete ...
I am a behinner so i hope i write an error and we can have more.
Just for information, coud some one can explain the diff betwwen a drivers than rockchip mpp and the way to use V4L2 and the specific v4l2_m2m ?
martos
 
Posts: 85
Joined: Fri Jun 29, 2018 12:42 am
languages_spoken: english
ODROIDs: MC1

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby OverSun » Tue Aug 14, 2018 9:02 pm

It's just because the whole world got crazy about android. All the consumers jerk off to android I have no idea why. Look for any guide how to make SBC a video player, 99% of them are going to start "download Android image ...".
So here we go, there is no reason for manufacturer to care about anything else than Android where most things works thanks to google that most of the time even do the job for manufacturers writing drivers and userspace libraries.
Would you invest months of developers time and salaries to make bunch of geeks happy?
User avatar
OverSun
 
Posts: 1397
Joined: Mon Apr 29, 2013 5:12 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby miskol » Mon Sep 03, 2018 3:34 pm

@Ely you wrote on phoronix that amlogic drivers support multiple decoding instances. Since when? I didn't check code for year. Is it really true?
miskol
 
Posts: 145
Joined: Wed Jan 15, 2014 2:58 am
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Mon Sep 03, 2018 6:13 pm

miskol wrote:@Ely you wrote on phoronix that amlogic drivers support multiple decoding instances. Since when? I didn't check code for year. Is it really true?


They've been supporting it for a while now, but I don't know since when. They had to rewrite a driver and a firmware for each codec (vmh264, vmmjpeg, vmmpeg4, vmmpeg12..) which are significantly more complex. the HEVC and VP9 drivers remain the same though as they already supported it. They also added a bunch of code for vdec scheduling and context switching, as well as manual esparser control.
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby memeka » Fri Oct 12, 2018 1:16 pm

@Ely

any news about encoding support? :D
User avatar
memeka
 
Posts: 4063
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby rubdos » Sat Oct 20, 2018 7:12 pm

Hi Ely, thanks for your cool work! I've watched your talk at embedded recipes this week, and I'm impressed :-)

I want you to destroy one of my dreams: can we ever have 10 bit output from 10 bit HEVC files on the C2? Let me explain.

It the meanwhile it's well known that, while the C2 reads and decodes 10 bit HEVC, it does "throw out" two bits and spits out NV12 framebuffers (at 12bbp average).

I have been reading your source code, and I noticed that you have a proprietary framebuffer format (called AM21C). I personally cannot find any documentation on the format itself, but your source code hints that it has 16 bbp of memory requirements. HEVC 4:2:0 10 bit video requires 15 bbp on average, so could it be that this pixel format supports 10 bit output?

EDIT: feel free to hit me up on IRC (#linux-meson or whatever, same nick)
rubdos
 
Posts: 4
Joined: Fri Jan 06, 2017 9:51 pm
Location: Brussels
languages_spoken: english,dutch,french
ODROIDs: C2

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby moon.linux » Sat Oct 20, 2018 8:27 pm

@Ely good talk nice inputs. very good in-dept knowledge. "We are not their yet but its looks right !!!".
moon.linux
 
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Sun Oct 21, 2018 9:45 pm

rubdos wrote:Hi Ely, thanks for your cool work! I've watched your talk at embedded recipes this week, and I'm impressed :-)

I want you to destroy one of my dreams: can we ever have 10 bit output from 10 bit HEVC files on the C2? Let me explain.

It the meanwhile it's well known that, while the C2 reads and decodes 10 bit HEVC, it does "throw out" two bits and spits out NV12 framebuffers (at 12bbp average).

I have been reading your source code, and I noticed that you have a proprietary framebuffer format (called AM21C). I personally cannot find any documentation on the format itself, but your source code hints that it has 16 bbp of memory requirements. HEVC 4:2:0 10 bit video requires 15 bbp on average, so could it be that this pixel format supports 10 bit output?

EDIT: feel free to hit me up on IRC (#linux-meson or whatever, same nick)


Hi rubdos,

You're definitely on the right track :P .

The hevc vdec can output two kind of formats:

- 8-bit NV12 (even with 10-bit hevc input, but in that case you lose 2 bits of precision per component)
- a proprietary losslessly compressed format that is similar to AFBC (ARM FrameBuffer Compression). This is indeed AM21C as you saw. There's no documentation, seems to be an in-house design.

AM21C is only handled by the decoder and display IPs, and it's the only format you can use if you want to actually pass through "true" 10-bit content. But even then, it seems like 10-bit HDMI output on the display side is only real for some SoCs, although I'm not sure which (@narmstrong knows more about this if you want to poke him). I'm pretty sure that the vendor driver already passes AM21C through, so maybe that display issue is the one you were talking about ?

When you decode 10-bit HEVC to a regular 10-bit pixel format (something AML SoCs can't do), then you need components to be 16-bit large, wasting 6 bits as padding. But if you use a compressed format like AM21C, then it can be smart about it and trim that padding (although that's not the only thing it does, but the details are proprietary and not known to me). Thus, you reduce RAM usage and bandwidth.

Currently, we support AM21C output from the decoder, but the drm/kms driver is yet to be able to support it as input. This'll come in @narmstrong's HDMI 2.0/10-bit display support work.

moon.linux wrote:@Ely good talk nice inputs. very good in-dept knowledge. "We are not their yet but its looks right !!!".


Thanks moon!

memeka wrote:@Ely

any news about encoding support? :D


Woops, missed your post. Nothing regarding encoding, sorry :( . It might come in the future but it's very low-priority at the moment (we still need VP9 decoding!)
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby moon.linux » Sun Oct 21, 2018 10:50 pm

Where can we get the correct format like RFC for encodes or decoders for video format/frames.
moon.linux
 
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Fri Oct 26, 2018 1:43 am

moon.linux wrote:Where can we get the correct format like RFC for encodes or decoders for video format/frames.


Can you expand on that ? Not sure I fully understand what you mean
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby martos » Fri Oct 26, 2018 6:00 pm

Little video to explain V4L2M_M2M

https://www.youtube.com/watch?v=pF6i19oscqg

In english i can understand
It's a french guy
Maxime Jourdan is my new hero ...
martos
 
Posts: 85
Joined: Fri Jun 29, 2018 12:42 am
languages_spoken: english
ODROIDs: MC1

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby moon.linux » Fri Oct 26, 2018 7:05 pm

Ely wrote:
moon.linux wrote:Where can we get the correct format like RFC for encodes or decoders for video format/frames.


Can you expand on that ? Not sure I fully understand what you mean

Sorry for asking some dum question and wasting your time.
moon.linux
 
Posts: 1161
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby ujsrwidc » Sat Nov 03, 2018 9:44 pm

I am also interested into having mainline support for decoding for the S805 (Odroid C0/C1/C1+). The OEM kernel is fully out of date (Kernel 3.10 - unsupported since a year now: https://www.linux.com/news/linux-kernel ... x-44-lts-1 ). Without mainline support i don't get any kernel security updates. Your work is really needed for security reasons. Without kernel security updates the odroid c0/c1/c1+ is a dead product and that would be really sad. Would be awesome to see your work in mainline kernel and all future odroid images build on mainline kernel instead on being build on a dead kernel version.
PS: I dont care about DRM-crap. DRM can be removed easy. Mainlining your work is by far more complex and important.
ujsrwidc
 
Posts: 37
Joined: Sat Nov 03, 2018 9:00 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby Ely » Mon Nov 05, 2018 6:13 pm

ujsrwidc wrote:I am also interested into having mainline support for decoding for the S805 (Odroid C0/C1/C1+). The OEM kernel is fully out of date (Kernel 3.10 - unsupported since a year now: https://www.linux.com/news/linux-kernel ... x-44-lts-1 ). Without mainline support i don't get any kernel security updates. Your work is really needed for security reasons. Without kernel security updates the odroid c0/c1/c1+ is a dead product and that would be really sad. Would be awesome to see your work in mainline kernel and all future odroid images build on mainline kernel instead on being build on a dead kernel version.
PS: I dont care about DRM-crap. DRM can be removed easy. Mainlining your work is by far more complex and important.


When referring to DRM, I was talking about the Direct Rendering Manager driver, briefly put the driver for HDMI/CVBS output. I understand there are usecases where you want to decode video without displaying it, but it's rare, so S805 support is not a priority.. sorry..
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby memeka » Wed Nov 21, 2018 7:54 am

@Ely

Is there any API support for scaling/rotation? Like v4l2m2m drivers?

Thanks.
User avatar
memeka
 
Posts: 4063
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART

Re: V4L2 M2M decoders on mainline: need testers !

Unread postby csucsu » Thu Dec 06, 2018 5:44 pm

Hi @Ely,

thank you all of your work!
I can't wait the multiple hw decode development!
The hardware can decode H.264 4K stream. Can you imagine the hw can decode 4x1080p HD stream or 16x480p in a matrix?

Do you have any plan about your development on this board in the (near) future?

best regards, Gabor
csucsu
 
Posts: 26
Joined: Tue Jul 12, 2016 12:14 am
languages_spoken: english
ODROIDs: C2

Previous

Return to Projects

Who is online

Users browsing this forum: No registered users and 0 guests