[UPDATE] GStreamer-1.4 for odroid

alexei
Posts: 41
Joined: Sun Oct 20, 2013 8:35 am
languages_spoken: english, russian
ODROIDs: XU+E, U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by alexei » Sun Jan 18, 2015 3:21 pm

Additional notes to previous post:
* For v4l2video8dec to appear at all in gstreamer's list (output of 'gst-inspect-1.0'), I had to clear the gstreamer's registry 'rm ~/.cache/gstreamer-1.0/registry.armv7l.bin'. Probably has to do with particular order of installation that happened.
* To use s5p_mfc and s5p_fimc as modules, a workaround is needed for a null dereference upon inserting the modules. The issue is a race condition in the driver: v4l_id udev utility runs immediately after /dev/video* node is created, which is in the middle of the probe routine, so v4l_id calls open before the probe finishes (the driver should take the lock before creating the device node). A workaround is to disable the udev rule 60-persistent-v4l.rules (by adding a blank file of same name to /etc/udev/rules.d).
at 3) you mean CPU reduction from software decoding to mfc decoding?; clutter or egl unfortunately will use some CPU too...
Yep. I was comparing cpu usage two pipeline sections 'v4l2video8dec ! v4l2video10convert ! fakesink' and 'avdec_h264 ! videoconvert ! fakesink'. The rough cpu load is for first 5 to 10 secs of big_buck_bunny_1080p_h264.mov. Yeah, a real sink will add to this.

Unfortunately, end-to-end HW decoding does not work for me. With glimagesink the framerate I get is about 5-10frames/sec. Some evidence suggests that the root cause is in the gstreamer/v4l2videodec, but other suggests that its in glimagesink.

TL;DR: Did anybody actually get a smooth playback of the above-mentioned Big Buck Bunny 1080p with glimagesink and the mfc pipeline?

Code: Select all

gst-launch-1.0 \
        filesrc location=/mnt/ext/samples/big_buck_bunny_1080p_h264.mov \
        typefind=true ! \
        qtdemux name=m m.video_0 ! \
        h264parse ! video/x-h264, stream-format=byte-stream, alignment=au ! \
        v4l2video8dec ! videoconvert ! \
        fpsdisplaysink video-sink=glimagesink
Inconclusive attempts at isolating the problem:

1. pure software decoding 'avdec_h264 ! videoconvert ! glimagesink ' shows a much higher frame rate than mfc 'avdec_h264 ! videoconvert ! glimagesink'. MFC is unwatchable ~5-10 fps, software is watchable (but fpsdisplaysink does report that all frames are still oficially late). This suggests that glimagesink is not the bottleneck.

2. If I comment out all the code in 'show_frame', effectively turning glimagesink into a fakesink (but with a log entry when show_frame is called), then compare the show_frame call timestamps in software vs MFC pipeline. The software pipeline calls it roughly every 20-40ms (on-time) consistenly. The MFC pipeline calls it roughly every 200ms and very erratically. This suggests that the issue is in gstreamer/v4l2videodec.

3. 'videotestsrc ! fpsdisplaysink video-sink=glimagesink' shows stable 30fps. This is a weak test though, since it's not a 1920x1080 stream.

4. The hardware converter is not the issue, since in all experiments there is no noticeable effect when 'videoconverter' is exchanged with 'v4l2video10converter'.

5. In v4l2videodec log, there are strange gaps in timestamps, when looking at 'Handling frame' events. Usually these exact events are spaced at around 30-40ms, but once in a while (every 10 or so frames), there is a gap on the order of 100ms:

Code: Select all

 0:00:01.376665265 27542 0xb5504380 DEBUG           videodecoder gstvideodecoder.c:2797:gst_video_decoder_clip_and_push_buf:<v4l2video8dec0> pushing buffer 0xb5567658 of size 3137536, PTS 0:00
:00.208333333, dur 0:00:00.041666666
0:00:01.377676847 27542   0x196660 LOG             videodecoder gstvideodecoder.c:2147:gst_video_decoder_chain:<v4l2video8dec0> chain PTS 0:00:00.500000000, DTS 0:00:00.458333333 duration 0:0
0:00.041666666 size 51180
0:00:01.474289401 27542 0xb5504380 DEBUG                   v4l2 gstv4l2bufferpool.c:1285:gst_v4l2_buffer_pool_release_buffer:<v4l2video8dec0:pool:src> release buffer 0xb5567658
These gaps do not happen with 'fakesink' instead of 'glimagesink'. This suggests that glimagesink is bottlenecking the pipeline.

5. The actual hw decoding operation seems to correspond to these events, which always show a fast ~15-20ms decode time. So, the low level decode operation seems to not be the root cause of the problem.

Code: Select all

0:00:01.361191915 27542   0x196660 LOG                     v4l2 gstv4l2bufferpool.c:988:gst_v4l2_buffer_pool_poll:<v4l2video8dec0:pool:sink> polling device
0:00:01.375507350 27542 0xb5504380 LOG                     v4l2 gstv4l2bufferpool.c:1091:gst_v4l2_buffer_pool_dqbuf:<v4l2video8dec0:pool:src> dequeueing a buffe

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Sun Jan 18, 2015 3:30 pm

* you can output to ximagesink directly, with options to disable scaling, and you can smoothly play any video in their native resoltion
* glimage sink is kind of slow ... i found cluttersink the best. BUT, cluttersink is OK only when running in full-screen mode; in windowed mode it's slower.
* i think your problem here is videoconvert; are you using fimc to do the conversion (i can't remember what;s the gstreamer name for FIMC, but you should use it explicitely... something like v4l2video12conv ....)

alexei
Posts: 41
Joined: Sun Oct 20, 2013 8:35 am
languages_spoken: english, russian
ODROIDs: XU+E, U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by alexei » Mon Jan 19, 2015 12:04 am

ximagesink: gets me about 1 frame a second with either software decode+convert or hw decode+convert pipeline, scaling is disabled by default, I full-screen the X window. I quickly gave up on it. Does it actually get you a reasonable frame rate with that 1080p file?

But, on the other hand, ffplay, which uses X image method (via SDL) just like ximagesink, does playback the 1080p smoothly (ironically) despite having no hw acceleration whatsoever. This suggests the problem is in gstreamer, not the X server + X video driver.

cluttersink: how did you build clutter/clutter-gst for GLES? I built clutter with '--enable-egl-backend', but it falls back to X backend, and if I explicitly say CLUTTER_BACKEND=eglnative, then the pipeline silently does not display anything (but no error). Gdb confirms that 'eglInitialize' from mali-egl/libEGL.so is never called. In any case, I expect cluttersink to be very similar to glimagesink.

conversion: the 'videoconvert' is the software converter element, the 'v4l2video10conv' is the FIMC converter. All previous observations are the same regardless of which I use, so it's unlikely to be the cause.

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Mon Jan 19, 2015 7:43 am

* try different options for ximagesink ... it did work for me when i disaled scaling, subtitle layers, etc. Of course, even when maximizing the window, the image would remain at original video resolution though
* eglnative is not the case here. clutter uses x11, since x11 actually does egl acceleration. also, don't expect 1080p even with clutter smoothly in window mode - 720p works ok in window mode. However, clutter in full-screen mode can play smoothly (probably because of the wm). egl acceleration i think it's using shaders to do some operations. also try disabling scaling etc to get better performance.
* make sure you are using v4l2video10conv. in my package, i actually set a flag when building to make that the default one. w/o v4l2video10conv it's not smooth.

https://www.youtube.com/watch?v=J5kJ7oJCZtQ in this video i am using snappy player, which uses cluttersink.

alexei
Posts: 41
Joined: Sun Oct 20, 2013 8:35 am
languages_spoken: english, russian
ODROIDs: XU+E, U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by alexei » Mon Jan 19, 2015 5:01 pm

Thanks so much for more info and demo.
* clutter: so x11 backend is the goal then -- ok, I'll play around with clutter+snappy+parole some more.

I finally see the reason behind my issues. As you pointed out earlier, it's all about colorspace conversion. A sink might accept different color formats, but each format comes at a (very) different performance cost. In addition, a color converter takes different times to convert between different pairs of formats. As a result, whether "hardware accelerated" decoding actually accelerates things depends on what colorspace format the decoder produces, what color format the sink accepts and at what cost, and what the cost of conversion is (might be zero if the two formats match). That's why I was seeing awful performance with HW "acceleration" and very good performance with purely software pipeline -- (in some pipelines) the software decoder produced a format that did not require conversion at all and could be consumed efficiently by the sink.

I will update this post with concrete details on tests I did with framebuffer (no X server) and with isolation of decoding, conversion, and display stages by creating raw video files.

Giuliano69
Posts: 7
Joined: Mon Jan 26, 2015 11:34 pm
languages_spoken: english
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by Giuliano69 » Tue Jan 27, 2015 12:08 am

Thanks memeka for your work.

Does the hw ACCELERATED GStreamer-1.4 for odroid,
works specificly ONLY with the "new" C1 (with Mali 450MP)
or ALSO with the "old2 U3 (with Mali 400MP) ?

The Mali data sheet says nothing about 400MP video decoding capability..... so I tought I is not able to HW decode any video stream...
For the Mali 450MP I've found something abouyt H264 only...

Giuliano

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Tue Jan 27, 2015 12:45 am

U3 has something called MFC for video acceleration.
C1 has some amlogic chip that does it.
mali is just for 3D, nothing to do with video.
so they're totally incompatible.

Giuliano69
Posts: 7
Joined: Mon Jan 26, 2015 11:34 pm
languages_spoken: english
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by Giuliano69 » Tue Jan 27, 2015 1:30 am

Thanks memeka
Cuold you please give some reference to dig into (url, pdf,...) ,
to discover WHICH video protocols are acceralated in the two boards (mpg, h264, vp8,...) ?

The Odroid C1 (with Amlogic S805 Cortex A5 ) seems to offer ONLY H264 hardware decoding (as Rpi does) e NOT mpg or vp8.
So other video format should NOT benefit of hardware IC.
http://www.amlogic.com/product05.htm

What about U3 ?

BR

Giuliano69
Posts: 7
Joined: Mon Jan 26, 2015 11:34 pm
languages_spoken: english
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by Giuliano69 » Tue Jan 27, 2015 2:57 am

For exynos 4 chipset (4412) informations are not clear.
There exist devices based on it, that support hw decoding for many format, at interesting speed
http://www.cnx-software.com/2013/04/04/ ... s-tegra-3/

But from the documentation of the chipset, NO hw decodint in the SoC seems to be present by default
http://www.samsung.com%2Fglobal%2Fbusiness%2Fsemiconductor%2Ffile%2Fproduct%2FExynos_4_Dual_32nm_User_Manaul_Public_REV100-0.pdf

May be Odroid tech service may help us :)))

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Tue Jan 27, 2015 5:55 am

The only references I have are google.
Maybe someone from hardkernel can answer your questions.
But you can google yourself, or look at the source code.
For example, source code for odroid u3: https://github.com/hardkernel/linux/blo ... c_cmd_v5.c

Code: Select all

	case S5P_MFC_CODEC_H264_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_H264_DEC;
		break;
	case S5P_MFC_CODEC_VC1_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_VC1_DEC;
		break;
	case S5P_MFC_CODEC_MPEG4_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_MPEG4_DEC;
		break;
	case S5P_MFC_CODEC_MPEG2_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_MPEG2_DEC;
		break;
	case S5P_MFC_CODEC_H263_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_H263_DEC;
		break;
	case S5P_MFC_CODEC_VC1RCV_DEC:
		h2r_args.arg[0] = S5P_FIMV_CODEC_VC1RCV_DEC;
		break;
show you have decoding for H264, MPEG2, VC1, H263 and MPEG4.

source code for C1 https://github.com/mdrjr/c1_aml_libs/bl ... dec_ctrl.c has:

Code: Select all

pcodec->video_type == VFORMAT_H264
pcodec->video_type == VFORMAT_VC1
pcodec->video_type == VFORMAT_AVS
pcodec->video_type == VFORMAT_HEVC

sciuipis
Posts: 9
Joined: Thu Jan 29, 2015 8:08 pm
languages_spoken: english, italian, french
ODROIDs: U3, U2, C1
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by sciuipis » Mon Feb 02, 2015 11:55 pm

Hi guys!
I'm waiting for the new release... but in meanwhile the older would be perfect!
Where can I download it?
thank you memeka!

stefano

sciuipis
Posts: 9
Joined: Thu Jan 29, 2015 8:08 pm
languages_spoken: english, italian, french
ODROIDs: U3, U2, C1
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by sciuipis » Tue Feb 10, 2015 7:20 am

Hi memeka.
Can I have your old release that was on http://oph.mdrjr.net/memeka/gstreamer-1.4.0.tar.gz? I need them to use with qtmultimedia 5.4.

Thank you very much!
stefano

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Tue Feb 10, 2015 8:09 am

Seems like I don't have the archive anymore.. maybe somebody else can upload it, or wait until I start working on U3 (dont know when :()

User avatar
SideEffECt
Posts: 7
Joined: Fri Jul 18, 2014 10:57 am
languages_spoken: english
ODROIDs: U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by SideEffECt » Sat Apr 18, 2015 1:03 am


aiwarrior
Posts: 11
Joined: Sat Apr 18, 2015 11:53 pm
languages_spoken: english
ODROIDs: U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by aiwarrior » Sun Apr 19, 2015 12:27 am

Does anybody confirm success with the Odroid U3?

I have compiled gstreamer 1.4 as per instructions and although I get the v4l2video8dec and v4l2video10convert, the following pipelines crash in __egl_platform_acquire_buffer in libMali.so

Code: Select all

gst-launch-1.0 -v -e v4l2src device=/dev/video0 ! video/x-h264,width=1920,height=1080,framerate=30/1 ! h264parse ! v4l2video8dec ! v4l2video10convert ! glimagesink sync=false

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Thu Apr 23, 2015 11:11 am

does it work with ximagesink?

PartyRaptor
Posts: 1
Joined: Tue Apr 28, 2015 12:08 am
languages_spoken: english
ODROIDs: U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by PartyRaptor » Tue Apr 28, 2015 12:12 am

Hello there, I'm not quite sure whether this is the right thread, yet I've encountered a problem while trying to play videos.

I simply solved this by installing Gnome MPlayer, maybe that helps.

hmmdhassan
Posts: 31
Joined: Fri Jan 23, 2015 1:44 am
languages_spoken: english
ODROIDs: Odroid-U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by hmmdhassan » Fri Apr 15, 2016 7:48 pm

Himemeka,

I have read the thread and your work done with MFC+Gstreamer. I have been working on U3 taking opencv processed frame passing it to the MFC and then transmitting the video using ffmpeg via pipe (stdout). But this process of sending video on stdout and taking as input with pipe is not effective. I need your code details for working with gstreamer, actualy to see how you are calling gstreamer, is it with shared memory or pipe. Please share your input on it.

Thanks,
Hammad.

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Sun Apr 17, 2016 9:12 am

does opencv supports gstreamer? can you build a gstreamer pipeline that includes your opencv processing?

hmmdhassan
Posts: 31
Joined: Fri Jan 23, 2015 1:44 am
languages_spoken: english
ODROIDs: Odroid-U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by hmmdhassan » Tue Apr 19, 2016 9:07 pm

no i don't think so. But what i need your Gstreamer input implementation taken from the output of mfc encoder.

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Wed Apr 20, 2016 12:50 pm

there's no "my" gstreamer input implementation - it's all gstreamer pipeline. also there's no mfc encoder in gst.

hmmdhassan
Posts: 31
Joined: Fri Jan 23, 2015 1:44 am
languages_spoken: english
ODROIDs: Odroid-U3
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by hmmdhassan » Wed Apr 20, 2016 2:30 pm

Okay thanks. But can you tell me how the pipeline input is taken into gstreamer. I wanted to do it with shared memory . Please help me on it .

User avatar
memeka
Posts: 4315
Joined: Mon May 20, 2013 10:22 am
languages_spoken: english
ODROIDs: XU rev2 + eMMC + UART
U3 + eMMC + IO Shield + UART
Has thanked: 1 time
Been thanked: 12 times
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by memeka » Wed Apr 20, 2016 3:24 pm

all elements are gstreamer elements

gstreamer source element -> gstreamer parser element -> gstreamer decoder element -> gstreamer converter element -> gstreamer output element
movie file -> h264parse -> gst mfc element -> optional converter (e.g. gst fimc in u3) -> gst opengl output

best_odroidfreak
Posts: 364
Joined: Wed May 04, 2016 1:07 am
languages_spoken: english
Has thanked: 0
Been thanked: 0
Contact:

Re: [UPDATE] GStreamer-1.4 for odroid

Unread post by best_odroidfreak » Tue May 31, 2016 5:54 pm

memeka wrote:I have compiled and packaged gstreamer 1.4.0 for Ubuntu. this is the latest stable gstreamer, with support for mfc and fimc.
currently gstreamer-1.4 did not hit the official ubuntu repos, just parts of it in utopic (not the parts with mfc and fimc though).
these packages should work on 14.04 images for all odroids (hopefully the upcoming XU3 too).

you can download the archive from: http://oph.mdrjr.net/memeka/gstreamer-1.4.0.tar.gz

untar/unzip then dpkg install the debs. to make sure they don't get overwritten by some other versions, you might want to use apt hold.

for those that want to replicate what i did (for example to update to latest git master):

* get the sources from git (gstreamer, gst-plugins-base, gst-plugins-good, gst-plugins-bad, optional gst-libav if you want ffmpeg)
* get ubuntu packaging rules from http://ppa.launchpad.net/ricotz/experim ... ol/main/g/ (the *ubuntu1~14.04~ricotz0.debian.tar.xz files), then unzip and copy the debian folder in each project folder
* update debian/changelog with the correct version
* make the following changes for odroid:
  • gstreamer: edit debian/rules and comment out the line with --enable-gtk-doc (somehow building the documentation fails on my system)
  • gst-plugins-base: edit gst/playback/Makefile.am and replace -DCOLORSPACE=\"videoconvert\" with -DCOLORSPACE=\"v4l2video10convert\" - this will force gstreamer to always use FIMC converter instead of the software converter
  • gst-plugins-good: edit configure.ac and comment out all the section about libv4l2. Make sure you only leave "have_libv4l2=no". if you don't do this, gstreamer will use the buggy libv4l2 and won't be able to use FIMC at all
  • gst-plugins-bad: (optional, only if you want egl support) edit debian/rules and add these flags to DEB_CONFIGURE_EXTRA_FLAGS: --enable-opengl=no --enable-glx=no --enable-egl=yes --enable-gles2=yes
* now you can build the debs with "dpkg-buildpackage -b -j5" in each project folder. you need to build gstreamer1.0, install the debs, then build gst-plugins-base, install the debs, then build the rest (gst-plugins-good, gst-plugins-bad, gst-libav) and finally install them.

Enjoy!
What would you like to say about this topic?
http://forum.odroid.com/viewtopic.php?f ... 94#p144294

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 2 guests