V4L2 M2M decoders on mainline: need testers !

[Ely]

Hello everyone,

I'm currently developing a V4L2 M2M mainline driver for the video decoders found within AMLogic SoCs.

I currently have MPEG 1,2,4, H.263, H.264 decoding working on S905X.
I also tested on S805 (odroid-c1), it works but there are limitations.

However I don't have a S905 board, such as an odroid-c2, so I'm coming here looking for guinea pigs to try it out .

• Repo: https://github.com/Elyotna/linux
• Pre-compiled Image + dtb + firmware for c2/S905: http://s000.tinyupload.com/index.php?fi ... 1643420975
  Kernel/dtb is compiled from my branch, version is 4.17-rc1 (soon to be out..), though are are slight modifications that I did locally (firmware paths changed..)
  Please extract the firmwares in /lib/firmware/meson/gxbb/

To test, you will need either a very recent FFmpeg (> 3.4), or GStreamer.

Examples pipelines:

• FFmpeg (dump raw NV12 frames to a file)

  Code: Select all

  ffmpeg -c:v h264_v4l2m2m -i input.mkv -f rawvideo output_nv12

• GStreamer 1.12 (display frames to KMS, please kill x11 beforehand)

  Code: Select all

  gst-launch-1.0 filesrc location=input.mkv ! matroskademux ! h264parse ! v4l2video0dec ! videoconvert n-threads=4 ! kmssink driver-name=meson force-modesetting=true connector-id=29 sync=false

• GStreamer 1.14 (display frames to KMS, please kill x11 beforehand)

  Code: Select all

  gst-launch-1.0 filesrc location=input.mkv ! matroskademux ! h264parse ! v4l2h264dec ! videoconvert n-threads=4 ! kmssink driver-name=meson force-modesetting=true connector-id=29 sync=false

For GStreamer pipelines, make sure you install all kind of plugins (gstreamer1.0-plugins-good, gstreamer1.0-plugins-bad, gstreamer1.0-plugins-ugly, gstreamer1.0-libav ..) beforehand. Also, don't use matroskademux for something else than mkv. I think it's qtdemux for mp4.

I obviously haven't tested those commands on a C2 so I'm hoping all the parameters are correct..

4K H.264 (3840x2160) should work, but please note that the speed will be slow. Since DRM doesn't support overlay planes / NV12 as of yet, the frames need to be color converted and scaled before being sent to KMS. And because the buffers are in cache-less coherent memory, it's very slow.

Don't try HEVC! It's advertised by the driver but does not work, and you'll end up freezing your board.

Please report here for any kind of problem.. crashes, freezes, bad picture, etc etc. Preferably with a dmesg or what was in your uart before the error .

Thank you !

Ely

[moon.linux]

@Ely Great will give this a try...

[odroid]

Great news!
I couldn't expect the VPU driver implementation appears with V4L2 interfaces on Kernel 4.17.

@moon.linux,
Since H.264 VPU in S905 can support up to 30fps at 4K resolution while H.265 decoder can handle up to 60fps.
So, consider the limitation when when you try the FFMPEG/FFPLAY test.

[Ely]

@moon.linux Thanks !
@odroid Well it's far from being mainlined (targeting 4.19), but it's coming along nicely. Also H.265 is not currently supported so this format won't work .

[rooted]

Need to get a C2 to the OP

[moon.linux]

@Ely

First I tried to copy the per-compiled image and it failed to boot up.

Code: Select all

[ 2334.620841] reboot: Restarting system
INFO:    PSCI Affinity Map:
INFO:      AffInst: Level 0, MPID 0x0, State ON
INFO:      AffInst: Level 0, MPID 0x1, State ON
INFO:      AffInst: Level 0, MPID 0x2, State ON
INFO:      AffInst: Level 0, MPID 0x3, State ON
bl31 reb¦GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;                                                                     READ:0;CHK:0;
TE: 322758
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000018c0
Wait bl30...Done
Sending bl301.......OK.
Run bl301...
from SD, src: 0x00020200, des: 0x10100000, size: 0x00011130


--- UART initialized after reboot ---
[Reset cause: unknown]
[Image: unknown, amlogic_v1.1.3046-00db630-dirty 2016-08-31 09:24:14 tao.zeng@dr                                                                     oid04]
bl30: check_permit, count is 1
bl30: check_permit: ok!
chipid: ef Load bl33 from SD, src: 0x00034200, des: 0x01000000, size: 0x000769f0
be ad de d f0 ad ba ef be ad de not ES chip
[0.437652 Inits done]
secure task start!
high task start!
low task start!
NOTICE:  BL3-1: v1.0(debug):4d2e34d
NOTICE:  BL3-1: Built : 17:08:35, Oct 29 2015
INFO:    BL3-1: Initializing runtime services
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address = 0x1000000
INFO:    BL3-1: Next image spsr = 0x3c9


U-Boot 2015.01-00014-g2e4380b-dirty (Jan 16 2018 - 02:09:02)

DRAM:  2 GiB
Relocation Offset is: 76f2f000
-------------------------------------------------
* Welcome to Hardkernel's ODROID-C2
-------------------------------------------------
CPU : AMLogic S905
S/N : HKC213254DFDA0AD
MAC : 00:1e:06:33:ed:2d
BID : HKC2211612
-------------------------------------------------
register usb cfg[1][0] = 0000000077f95010
register usb cfg[0][1] = 0000000077f95030
vpu detect type: 5
vpu clk_level = 7
set vpu clk: 666667000Hz, readback: 666660000Hz(0x300)
MMC:   aml_priv->desc_buf = 0x0000000073f27d30
aml_priv->desc_buf = 0x0000000073f29ec0
SDIO Port B: 0, SDIO Port C: 1
ret = 1 .[mmc_init] mmc init success
In:    serial
Out:   serial
Err:   serial
----------------------------------
MMC Size : 32 GB
----------------------------------
reading boot-logo.bmp.gz
** Unable to read file boot-logo.bmp.gz **
reading boot-logo.bmp
** Unable to read file boot-logo.bmp **
movi: the partiton 'logo' is reading...

MMC read: dev # 0, block # 61024, count 2048 ... 2048 blocks read: OK
hpd_state=1
[CANVAS]addr=0x3f800000 width=3840, height=1440

set hdmitx VIC = 16
hdmitx phy setting done
set hdmitx VIC = 16
hdmitx phy setting done
Error: Bad gzipped data
There is no valid bmp file at the given address
Net:   Meson_Ethernet
Hit [Enter] key twice to stop autoboot:  0
reading boot.ini
3347 bytes read in 6 ms (543.9 KiB/s)
cfgload: applying boot.ini...
cfgload: setenv bootrootfs "boot=/dev/mmcblk1p1 disk=/dev/mmcblk1p2 root=/dev/mm                                                                     cblk1p2"
cfgload: setenv console "console=tty0 console=ttyAML0,115200n8"
cfgload: setenv odroid   "no_console_suspend consoleblank=0 loglevel=8 debug ro                                                                      rootwait fsck.repair=yes net.ifnames=0"
cfgload: setenv maxcpus "4"
cfgload: setenv max_freq "1536"  # 1.536GHz
cfgload: setenv kernel  "fatload mmc 0:1 0x1080000 Image"
cfgload: setenv dtb     "fatload mmc 0:1 0x1000000 meson-gxbb-odroidc2.dtb"
cfgload: setenv bootseq "bootm 0x1080000 - 0x1000000"
cfgload: setenv bootcmd "${kernel}; ${dtb}; ${bootseq}"
cfgload: setenv bootargs "${console} ${bootrootfs} ${odroid} ${libreelec}"
cfgload: run bootcmd
reading Image
15581696 bytes read in 701 ms (21.2 MiB/s)
reading meson-gxbb-odroidc2.dtb
23416 bytes read in 7 ms (3.2 MiB/s)
ee_gate_off ...
Wrong Image Format for bootm command
ERROR: can't get kernel image!
ee_gate_on ...
reading boot-logo.bmp.gz
** Unable to read file boot-logo.bmp.gz **
reading boot-logo.bmp
** Unable to read file boot-logo.bmp **
movi: the partiton 'logo' is reading...

MMC read: dev # 0, block # 61024, count 2048 ... 2048 blocks read: OK
hpd_state=1
set hdmitx VIC = 16
hdmitx phy setting done
set hdmitx VIC = 16
hdmitx phy setting done
Error: Bad gzipped data
There is no valid bmp file at the given address
movi: the partiton 'dtb' is reading...

MMC read: dev # 0, block # 1504, count 128 ... 128 blocks read: OK
movi: the partiton 'boot' is reading...

MMC read: dev # 0, block # 1632, count 32768 ... 32768 blocks read: OK
Bad Linux ARM64 Image magic!
odroidc2#

Then I tried to compile the linux tree by enable CONFIG_VIDEO_AML_MESON_VDEC=m and succefully booted into new compiled linux.

Code: Select all

[    6.889112] libphy: stmmac: probed
[    7.129018] meson_pwrc_vdec_power_on
[    7.129024] meson_pwrc_vdec_setup_clk
[    7.129505] Requesting IRQ 32
[    7.129702] meson_pwrc_vdec_power_off
[    7.762147] vdec_open
[    7.762166] vdec_querycap
[    7.762305] vdec_close
[    8.080933] meson-drm d0100000.vpu: Queued 2 outputs on vpu
[    8.108954] meson-drm d0100000.vpu: Failed to create d0100000.vpu debugfs directory
[    8.108979] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    8.108981] [drm] No driver support for vblank timestamp query.
[    8.109042] meson-drm d0100000.vpu: CVBS Output connector not available
[    8.137391] meson-dw-hdmi c883a000.hdmi-tx: Detected HDMI TX controller v2.01a with HDCP (meson_dw_hdmi_phy)
[    8.137813] meson-dw-hdmi c883a000.hdmi-tx: registered DesignWare HDMI I2C bus driver
[    8.138958] meson-drm d0100000.vpu: bound c883a000.hdmi-tx (ops meson_dw_hdmi_ops [meson_dw_hdmi])
[    8.426270] Console: switching to colour frame buffer device 240x67
[    8.463220] meson-drm d0100000.vpu: fb0:  frame buffer device
[    8.493815] [drm] Initialized meson 1.0.0 20161109 for d0100000.vpu on minor 0
[    8.794884] FAT-fs (mmcblk1p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[    9.298810] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.344148] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.357110] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.373549] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.392786] new mount options do not match the existing superblock, will be ignored
[    9.404479] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.421010] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.432965] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.446087] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.663661] RTL8211F Gigabit Ethernet stmmac-0:00: attached PHY driver [RTL8211F Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:00, irq=37)
[    9.673345] meson8b-dwmac c9410000.ethernet eth0: PTP not supported by HW
[    9.673627] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    9.793194] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.
[    9.804934] systemd-journald[1447]: Successfully sent stream file descriptor to service manager.

But when I use the command it failed to evoke

Code: Select all

odroid@odroid64:~$ gst-launch-1.0 filesrc location=elephant.mkv ! matroskademux ! h264parse ! v4l2h264dec ! videoconvert n-threads=4 ! kmssink driver-name=meson force-modesetting=true connector-id=29 sync=false
WARNING: erroneous pipeline: no element "v4l2h264dec"
odroid@odroid64:~$ gst-launch-1.0 filesrc location=elephant.mkv ! matroskademux ! h264parse ! videoconvert n-threads=4 ! kmssink driver-name=meson force-modesetting=true connector-id=29 sync=false
WARNING: erroneous pipeline: no property "n-threads" in element "videoconvert0"
odroid@odroid64:~$ gst-launch-1.0 filesrc location=elephant.mkv ! matroskademux ! h264parse ! kmssink driver-name=meson force-modesetting=true connector-id=29 sync=false
WARNING: erroneous pipeline: no element "kmssink"

please let me know If I missed some thing...thanks

[Ely]

Thanks for trying moon.linux!

Did you copy the vdec module to the rootfs ? You can also compile with CONFIG_VIDEO_AML_MESON_VDEC=y so that it's builtin. You should get a /dev/video0 node if the decoder probe was successful. Also make sure that you updated the DTB!

The name of the v4l2 element is not the same depending on GStreamer version. Make sure to use the 1.12 or 1.14 commands that I provided depending on your version.

Also, can you provide a dmesg log ?

edit: oh, and if you compiled from tree, make sure you used the branch "4.17/amlogic-v4l2-m2m", and you'll have to put the firmware in /lib/firmware/meson/gxl. And rename vh264_mc to gxtvbb_vh264_mc.

[moon.linux]

@Ely
As suggested by you I copied the firmware file in /lib/firmware/meson/gxbb/

Code: Select all

root@odroid64:/usr/src/testing# ls -l /lib/firmware/meson/gxbb/
total 84
-rw-r--r-- 1 root root 16384 Apr 18 23:40 h263_mc
-rw-r--r-- 1 root root 36864 Apr 18 23:40 vh264_mc
-rw-r--r-- 1 root root 16384 Apr 18 23:40 vmpeg12_mc
-rw-r--r-- 1 root root 16384 Apr 18 23:40 vmpeg4_mc_5
root@odroid64:/usr/src/testing#

Yes I have compile using following branch

Code: Select all

root@odroid64:/usr/src/linux-elyotna# git b
* 4.17/amlogic-v4l2-m2m
root@odroid64:/usr/src/linux-elyotna#

CONFIG_VIDEO_AML_MESON_VDEC=y I will give this a try.

Here is the g stream package installed at my end.

Code: Select all

root@odroid64:~# dpkg -l | grep gstreamer
ii  gir1.2-gstreamer-1.0                        1.8.3-1~ubuntu0.1                            arm64        GObject introspection data for the GStreamer library
ii  gstreamer1.0-alsa:arm64                     1.8.3-1ubuntu0.2                             arm64        GStreamer plugin for ALSA
ii  gstreamer1.0-clutter-3.0                    3.0.18-1                                     arm64        Clutter PLugin for GStreamer 1.0
ii  gstreamer1.0-libav:arm64                    1.8.3-1ubuntu0.2                             arm64        libav plugin for GStreamer
ii  gstreamer1.0-nice:arm64                     0.1.13-0ubuntu2                              arm64        ICE library (GStreamer plugin)
ii  gstreamer1.0-plugins-bad:arm64              1.8.3-1ubuntu0.2                             arm64        GStreamer plugins from the "bad" set
ii  gstreamer1.0-plugins-bad-faad:arm64         1.8.3-1ubuntu0.2                             arm64        GStreamer faad plugin from the "bad" set
ii  gstreamer1.0-plugins-bad-videoparsers:arm64 1.8.3-1ubuntu0.2                             arm64        GStreamer videoparsers plugin from the "bad" set
ii  gstreamer1.0-plugins-base:arm64             1.8.3-1ubuntu0.2                             arm64        GStreamer plugins from the "base" set
ii  gstreamer1.0-plugins-good:arm64             1.8.3-1ubuntu0.4                             arm64        GStreamer plugins from the "good" set
ii  gstreamer1.0-plugins-ugly:arm64             1.8.3-1ubuntu0.1                             arm64        GStreamer plugins from the "ugly" set
ii  gstreamer1.0-plugins-ugly-amr:arm64         1.8.3-1ubuntu0.1                             arm64        GStreamer plugins from the "ugly" set
ii  gstreamer1.0-pulseaudio:arm64               1.8.3-1ubuntu0.4                             arm64        GStreamer plugin for PulseAudio
ii  gstreamer1.0-tools                          1.8.3-1~ubuntu0.1                            arm64        Tools for use with GStreamer
ii  gstreamer1.0-x:arm64                        1.8.3-1ubuntu0.2                             arm64        GStreamer plugins for X11 and Pango
ii  libgstreamer-plugins-bad1.0-0:arm64         1.8.3-1ubuntu0.2                             arm64        GStreamer development files for libraries from the "bad" set
ii  libgstreamer-plugins-base1.0-0:arm64        1.8.3-1ubuntu0.2                             arm64        GStreamer libraries from the "base" set
ii  libgstreamer-plugins-good1.0-0:arm64        1.8.3-1ubuntu0.4                             arm64        GStreamer development files for libraries from the "good" set
ii  libgstreamer1.0-0:arm64                     1.8.3-1~ubuntu0.1                            arm64        Core GStreamer libraries and elements
ii  libreoffice-avmedia-backend-gstreamer       1:5.1.6~rc2-0ubuntu1~xenial3                 arm64        GStreamer backend for LibreOffice

gst-launch-1.0 --version

Code: Select all

root@odroid64:~# gst-launch-1.0 --version
gst-launch-1.0 version 1.8.3
GStreamer 1.8.3
https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

How can upgrade gstreamer to 1.12 or 1.14 is their any PPA for latest version upstream.

Thanks.

[moon.linux]

I tried gstreamer PPA https://launchpad.net/~ddalex/+archive/ubuntu/gstreamer

Code: Select all

root@odroid64:/usr/src/linux-elyotna# sudo add-apt-repository ppa:ddalex/gstreamer
 This PPA brings in updated version of GStreamer packages.
 More info: https://launchpad.net/~ddalex/+archive/ubuntu/gstreamer
Press [ENTER] to continue or ctrl-c to cancel adding it

gpg: keyring `/tmp/tmpij5ks8pm/secring.gpg' created
gpg: keyring `/tmp/tmpij5ks8pm/pubring.gpg' created
gpg: requesting key 4BBC8F7F from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpij5ks8pm/trustdb.gpg: trustdb created
gpg: key 4BBC8F7F: public key "Launchpad PPA for ddalex" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
root@odroid64:/usr/src/linux-elyotna# sudo apt-get update
Ign:1 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial InRelease
Hit:2 http://ports.ubuntu.com/ubuntu-ports xenial InRelease
Get:3 http://ports.ubuntu.com/ubuntu-ports xenial-updates InRelease [102 kB]
Hit:4 http://deb.odroid.in/c2 xenial InRelease
Hit:5 http://ppa.launchpad.net/saiarcot895/myppa/ubuntu xenial InRelease
Hit:6 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease
Get:7 http://ports.ubuntu.com/ubuntu-ports xenial-backports InRelease [102 kB]
Ign:8 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial Release
Ign:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Get:13 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease [102 kB]
Ign:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Get:14 http://ports.ubuntu.com/ubuntu-ports xenial-proposed InRelease [253 kB]
Ign:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Ign:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Ign:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Err:9 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main arm64 Packages
  404  Not Found
Ign:10 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main armhf Packages
Ign:11 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main all Packages
Ign:12 http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial/main Translation-en
Fetched 560 kB in 7s (72.9 kB/s)
Reading package lists... Done
W: The repository 'http://ppa.launchpad.net/ddalex/gstreamer/ubuntu xenial Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch http://ppa.launchpad.net/ddalex/gstreamer/ubuntu/dists/xenial/main/binary-arm64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.

But no binary for arm64 ?

[Ely]

Ah, if you're running an old ubuntu it's not gonna be easy :/ . I'd go with an archlinux or something if I were you.

I received a C2 yersteday (Thanks HK!) and confirmed along with narmstrong that the video decoding works as well on S905, I'll try to whip up images that include everything necessary to test.

[moon.linux]

Thanks for your input, I have archlinux install at my end.
If their is some input to compile the kernel image for archlinux I would be happy to test this thing.

[moon.linux]

@Ely
I have installed ArchLinux on odroidc2 micro sdcard, after few minutes of working it freezes, I need to reboot the device to start working again.
Any input on this one.

This is on archlinux

Code: Select all

[alarm@archlinuxc2 ~]$ uname -a
Linux archlinuxc2 4.16.2-1-ARCH #1 SMP Thu Apr 12 22:25:04 MDT 2018 aarch64 GNU/Linux
[alarm@archlinuxc2 ~]$

I usually get following kernel crash and then it freezes.

Code: Select all

[  758.671306] ------------[ cut here ]------------
[  758.671336] driver forgot to call drm_crtc_vblank_off()
[  758.675628] WARNING: CPU: 3 PID: 639 at drivers/gpu/drm/drm_atomic_helper.c:986 drm_atomic_helper_commit_modeset_disables+0x398/0x3f8 [drm_kms_helper]
[  758.688831] Modules linked in: fuse arc4 rc_cec dw_hdmi_cec rt2800usb rt2x00usb rt2800lib rt2x00lib mac80211 cfg80211 rfkill joydev meson_dw_hdmi dw_hdmi dwmac_generic cec meson_drm realtek drm_kms_helper dwmac_meson8b stmmac_platform drm stmmac drm_panel_orientation_quirks ptp pps_core syscopyarea meson_ir sysfillrect rc_core sysimgblt fb_sys_fops meson_gxbb_wdt nvmem_meson_efuse crypto_user
[  758.723334] CPU: 3 PID: 639 Comm: Xorg Not tainted 4.16.2-1-ARCH #1
[  758.729540] Hardware name: Hardkernel ODROID-C2 (DT)
[  758.734458] pstate: 00000005 (nzcv daif -PAN -UAO)
[  758.739256] pc : drm_atomic_helper_commit_modeset_disables+0x398/0x3f8 [drm_kms_helper]
[  758.747179] lr : drm_atomic_helper_commit_modeset_disables+0x398/0x3f8 [drm_kms_helper]
[  758.755070] sp : ffff00000d12bab0
[  758.758347] x29: ffff00000d12bab0 x28: 0000000000000030
[  758.763608] x27: 0000000000000000 x26: ffff80006d96e028
[  758.768870] x25: ffff000000f90a10 x24: ffff80004bddb800
[  758.774131] x23: ffff000000f10448 x22: ffff000000f0fe10
[  758.779392] x21: ffff80006d969800 x20: 0000000000000000
[  758.784653] x19: ffff80003e051e00 x18: 0000000000000010
[  758.789915] x17: 0000ffffbd9a7c38 x16: ffff0000082affe8
[  758.795176] x15: ffffffffffffffff x14: ffff0000896548cf
[  758.800437] x13: ffff0000096548dd x12: ffff000009432000
[  758.805699] x11: ffff000009409b38 x10: ffff00000878e6a8
[  758.810960] x9 : 00000000ffffffd0 x8 : 0000000000000006
[  758.816221] x7 : 7264206c6c616320 x6 : 0000000000000134
[  758.821482] x5 : 0000000000000000 x4 : 0000000000000000
[  758.826743] x3 : 0000000000000000 x2 : ffff80007ffc1eb8
[  758.832005] x1 : 0000800076bcd000 x0 : 000000000000002b
[  758.837267] Call trace:
[  758.839725]  drm_atomic_helper_commit_modeset_disables+0x398/0x3f8 [drm_kms_helper]
[  758.847314]  drm_atomic_helper_commit_tail+0x28/0x78 [drm_kms_helper]
[  758.853696]  commit_tail+0x7c/0x80 [drm_kms_helper]
[  758.858527]  drm_atomic_helper_commit+0xd8/0x168 [drm_kms_helper]
[  758.864648]  drm_atomic_commit+0x54/0x70 [drm]
[  758.869016]  drm_atomic_connector_commit_dpms+0x110/0x118 [drm]
[  758.874876]  drm_mode_obj_set_property_ioctl+0x194/0x2e8 [drm]
[  758.880654]  drm_mode_connector_property_set_ioctl+0x4c/0x60 [drm]
[  758.886776]  drm_ioctl_kernel+0x70/0xd8 [drm]
[  758.891088]  drm_ioctl+0x198/0x410 [drm]
[  758.894887]  do_vfs_ioctl+0xb0/0x848
[  758.898419]  SyS_ioctl+0x8c/0xa8
[  758.901612]  el0_svc_naked+0x30/0x34
[  758.905146] ---[ end trace bb872b0cab4fd6fb ]---

I feel this happens when the screensaver disable the monitor it goes to sleep.

[Ely]

Yeah I get this backtrace as well, it's harmless on my side (does not cause freeze).

Are you running a graphical environment ? I'm not, all I have is the console login on screen until I run a gstreamer command.

I think that to update your kernel/dtb, all you have to do is replace /boot/Image and /boot/dtbs/meson64_odroidc2.dtb with your own.

Although, it's weird you have 4.16.2, those seem to be the old (3.14) namings.. In mainline the dtb for odroidc2 is called amlogic/meson-gxbb-odroidc2.dtb.

PS: for my current branch, you'll have to change firmware path. /lib/firmware/meson/gxbb/vh264_mc becomes /lib/firmware/meson/gxl/gxtvbb_vh264_mc. This will be cleaned up in the future..

[moon.linux]

After I cross compiled your your linux kernel on archlinux platfrom it failed to boot.

Code: Select all

ERprocess cCEC cfg:0x2f
WAKEUP GPIO cfg:0x00000000
set vddee to 0x0352mv
06f1930000000000GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;
TE: 381801
no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015.
gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8
set vcck to 1100 mv
set vddee to 1050 mv
CPU clk: 1536MHz
DDR channel setting: DDR0 Rank0+1 same
DDR0: 2048MB(auto) @ 912MHz(2T)-13
DataBus test pass!
AddrBus test pass!
Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0
Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0
Sending bl30........................................OK.
Run bl30...
Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000018c0
Wait bl30...Done
Sending bl301.......OK.
Run bl301...
om SD, src: 0x00020200, des: 0x10100000, size: 0x00011130


--- UART initialized after reboot ---
[Reset cause: unknown]
[Image: unknown, amlogic_v1.1.3046-00db630-dirty 2016-08-31 09:24:14 tao.zeng@droid04]
bl30: check_permit, count is 1
bl30: check_permit: ok!
chipid: ef be ad deLoad bl33 from SD, src: 0x00034200, des: 0x01000000, size: 0x00055da0
 d f0 ad ba ef be ad de not ES chip
[0.497153 Inits done]
secure task start!
high task start!
low task start!
NOTICE:  BL3-1: v1.0(debug):4d2e34d
NOTICE:  BL3-1: Built : 17:08:35, Oct 29 2015
INFO:    BL3-1: Initializing runtime services
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address = 0x1000000
INFO:    BL3-1: Next image spsr = 0x3c9


U-Boot 2017.07-2 (Sep 03 2017 - 07:34:08 -0600) Arch Linux ARM

DRAM:  2 GiB
MMC:   mmc@72000: 0, mmc@74000: 1
Using default environment

In:    serial@4c0
Out:   serial@4c0
Err:   serial@4c0
Net:   eth0: ethernet@c9410000
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
726 bytes read in 29 ms (24.4 KiB/s)
## Executing script at 1f000000
18063936 bytes read in 842 ms (20.5 MiB/s)
22745 bytes read in 50 ms (443.4 KiB/s)
5084347 bytes read in 265 ms (18.3 MiB/s)
Bad Linux ARM64 Image magic!
SCRIPT FAILED: continuing...
Card did not respond to voltage select!
mmc_init: -95, time 9
MMC Device 2 not found
no mmc device at slot 2
Speed: 1000, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
DHCP client bound to address 10.0.0.140 (904 ms)
*** Warning: no boot file name; using '0A00008C.img'
Using ethernet@c9410000 device
TFTP from server 0.0.0.0; our IP address is 10.0.0.140; sending through gateway 10.0.0.1
Filename '0A00008C.img'.
Load address: 0x1000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
missing environment variable: pxeuuid
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/01-00-1e-06-33-37-7a
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A00008C
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A00008
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A0000
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A000
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A00
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A0
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0A
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/0
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/default-arm-meson
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/default-arm
Speed: 1000, full duplex
*** ERROR: `serverip' not set
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/default
Speed: 1000, full duplex
*** ERROR: `serverip' not set
Config file not found
Speed: 1000, full duplex
BOOTP broadcast 1
DHCP client bound to address 10.0.0.140 (237 ms)
Using ethernet@c9410000 device
TFTP from server 0.0.0.0; our IP address is 10.0.0.140; sending through gateway 10.0.0.1
Filename 'boot.scr.uimg'.
Load address: 0x1f000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Speed: 1000, full duplex
BOOTP broadcast 1
DHCP client bound to address 10.0.0.140 (170 ms)
Using ethernet@c9410000 device
TFTP from server 0.0.0.0; our IP address is 10.0.0.140; sending through gateway 10.0.0.1
Filename 'boot.scr.uimg'.
Load address: 0x1080000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
=>

could it be possible to share your steps to either build pacman kernel package or the linux kernel for arch linux.

[Ely]

I don't really understand why the Image gives you "Bad Linux ARM64 Image magic!". I checked both the archlinux Image and the one you get by cross compiling, and they both have the correct header magic.

Did you cross compile with "export ARCH=arm64" ?

PS: I don't use archlinuxarm, so I wouldn't know about the package, sorry. But replacing the Image should work, it's really weird.

[moon.linux]

Following are the instruction I follow to cross compile the kernel.

Code: Select all

export PATH=/opt/toolchains/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu/bin/:$PATH
export GCC_COLORS=auto

#make clean
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- odroidc2_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LOCALVERSION=-ARCHM -j4 dtbs Image
mkimage -A arm64 -O linux -T kernel -C none -a 0x1080000 -e 0x1080000 -n linux-next -d arch/arm64/boot/Image ../Image

mkdir -p media
sudo umount media

sudo mount /dev/mmcblk0p1 ./media
sudo cp -v ../Image arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dtb ./media/boot && sync && sudo umount ./media

sudo mount /dev/mmcblk0p1 ./media
sudo make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- LOCALVERSION=-ARCHM modules modules_install INSTALL_MOD_PATH=./media && sync && sudo umount ./media

[Ely]

Ohh, I don't think you need the call to mkimage, it's only for older u-boots. On this archlinuxarm, all you should have to do is replace /boot/Image with arch/arm64/boot/Image.

Also make sure that you either rename the dtb to the archlinux dtb name (/boot/dtbs/meson64_odroidc2.dtb), or that you edit /boot/boot.ini to reflect the dtb name change.

[csucsu]

Hi,

this is very good news!
Looking for venc also!

best regards, Gabor

[miskol]

@Ely
Are you able to decode more then one video stream ? Is it somehow possible?
Are you able to provide decoded frames in dma-bufs(without copy) so I can use them as textures in opengl ?

[Ely]

@Ely
Are you able to decode more then one video stream ? Is it somehow possible?
Are you able to provide decoded frames in dma-bufs(without copy) so I can use them as textures in opengl ?

1. Unfortunately no. AML drivers do support multi-instance but it's tricky, complex & dirty. So for now the mainline drivers are limited to single instance.
2. Yes, the buffers are exported via dmabuf. The endgoal is to have a copyless display pipeline using dmabuf V4L2 / DRM Prime.

[miskol]

1. btw do you understand amlogic video api now? Do you have any documentation or you just investigate amlogic source codes?

[Ely]

1. btw do you understand amlogic video api now? Do you have any documentation or you just investigate amlogic source codes?

The biggest part of my work is done through reading AML's kernel source code. Luckily I don't need to bother with userspace reversing shenanigans since the kernel code is enough to run the vdecs.
I actually don't understand amlogic video API that much, but I am starting to have a good understanding about how the hardware works. I have many codecs working including H.264, and HEVC is starting to look good (although far from done).

I do get a bit of help here and there though from private channels as well as public code, such as the endless mobile V4L2 glue.

[rooted]

It's nice to hear everything is going well, I'm glad odroid sent you a C2 also. I'm sure it makes testing much faster.

[Ely]

It's nice to hear everything is going well, I'm glad odroid sent you a C2 also. I'm sure it makes testing much faster.

It allows me to test on S905, which is nice! My main dev board is a Libre Computer "Le Potato" (S905X). I also do some tests on a odroid-c1 I acquired a long time ago, but unfortunately S805 has some limitations when it comes to pixel formats (can't do regular NV12/NV21).

[miskol]

Is it posible to use frame in dmabuf with mali drivers ?
I found that mali has some special format of texture for this purpose. So I am not sure if NV12/NV21 is that format.
And I am not sure if I want to run some pixel shader to do transformation as mali has really small L1/L2 cache

[Ely]

Is it posible to use frame in dmabuf with mali drivers ?
I found that mali has some special format of texture for this purpose. So I am not sure if NV12/NV21 is that format.
And I am not sure if I want to run some pixel shader to do transformation as mali has really small L1/L2 cache

Yes, in fact I have already seen in action a bleeding edge version of kodi able to display frames with v4l2 + mali/dmabuf on S905. The work was mostly done by lrusak, a kodi/libreelec member.

[rooted]

Is it posible to use frame in dmabuf with mali drivers ?
I found that mali has some special format of texture for this purpose. So I am not sure if NV12/NV21 is that format.
And I am not sure if I want to run some pixel shader to do transformation as mali has really small L1/L2 cache

Yes, in fact I have already seen in action a bleeding edge version of kodi able to display frames with v4l2 + mali/dmabuf on S905. The work was mostly done by lrusak, a kodi/libreelec member.

This is reassuring to know.

[martos]

OK i am going to test :

My source is an enigma2 reveiver
i want to use ffmpeg to transcode source to h264 stream
viewtopic.php?f=95&t=31358

I try with an S905
I compil old aml ffmpeg on Armbian but it don't work :
viewtopic.php?f=138&t=27624

And roman rybalko say it's better to use V4L2 :
I would recommend to you give the ffmpeg up and try to enable Amlogic v4l2 driver

So i start to test your ffmpeg solution ...

[Ely]

OK i am going to test :

My source is an enigma2 reveiver
i want to use ffmpeg to transcode source to h264 stream
viewtopic.php?f=95&t=31358

I try with an S905
I compil old aml ffmpeg on Armbian but it don't work :
viewtopic.php?f=138&t=27624

And roman rybalko say it's better to use V4L2 :
I would recommend to you give the ffmpeg up and try to enable Amlogic v4l2 driver

So i start to test your ffmpeg solution ...

Transcoding is unfortunately not supported as I haven't implemented any encoder. It's only decoding for now.

[martos]

Ok for transcoding only decode.
How i can make one armbian kernel for use V4l2m2m ?
for now i use :
Armbian_5.44_S9xxx_Ubuntu_bionic_4.18.0-rc5_server_20180727.img.xz
https://github.com/150balbes/Build-Armbian
because for the moment is have with ffmpeg 4.01 with -vcodec h264_v4l2m2m :
[h264_v4l2m2m @ 0x2278490] Could not find a valid device
[h264_v4l2m2m @ 0x2278490] can't configure decoder

[csucsu]

Hi,
you need to use kernel from Ely's repo https://github.com/Elyotna/linux/tree/4 ... c-v4l2-m2m

best regards, Gabor

[martos]

could you help me ? ( it's my firts kernel compil )



i try :

sudo apt-get -y install bc curl gcc git libssl-dev libncurses5-dev lzop make u-boot-tools

make menuconfig

make -j3 ( in console)


sortextable.c:(.text+0x408): undefined reference to `get_unaligned_le32'
collect2: error: ld returned 1 exit status
scripts/Makefile.host:90: recipe for target 'scripts/sortextable' failed
make[1]: *** [scripts/sortextable] Error 1
Makefile:1045: recipe for target 'scripts' failed
make: *** [scripts] Error 2

I think i need the PC to use cross-compilation, it seem easier to run the build on the device BUT i don't find an "how to" for make a VM for cross compiling vm kernel.
Could some one can give me some link to make it ?

[martos]

So install in my PC
https://www.virtualbox.org/wiki/Downloads
create vm ( 10 CPU 16 G Ram ) with
http://archive.ubuntu.com/ubuntu/dists/ ... t/mini.iso
minimal +

sudo apt-get install sed wget cvs subversion git-core \
coreutils unzip texi2html texinfo docbook-utils \
gawk python-pysqlite2 diffstat help2man make gcc build-essential g++ \
desktop-file-utils chrpath default-jre gettext zip

sudo apt-get update -qq && sudo apt-get -y install \
autoconf \
automake \
build-essential \
cmake \
git-core \
libass-dev \
libfreetype6-dev \
libsdl2-dev \
libtool \
libva-dev \
libvdpau-dev \
libvorbis-dev \
libxcb1-dev \
libxcb-shm0-dev \
libxcb-xfixes0-dev \
pkg-config \
texinfo \
wget \
zlib1g-dev

sudo apt-get install yasm

sudo apt-get -y install bc curl gcc git libssl-dev libncurses5-dev lzop make u-boot-tools

sudo apt-get install locate mc

download kernel

git clone https://github.com/Elyotna/linux

sudo apt-get install flex bison

sudo apt-get install elfutils

cd linux
make menuconfig
save

make -j10



waiting ...

[martos]

But how i put the type of the cpu or other ? it's on the menuconfig ?
I make a mistake ?

[mad_ady]

Before menuconfig there should be a make odroidc2_defconfig or something similar

[Ely]

Before I write down a small guide, I want to make sure of a few things:

- The driver is not suitable for any production workflow at the moment. It's been tested with limited samples and will have bugs.
- There's no deinterlacing support, so if you capture from a DVB source it's likely going to look bad.
- What are you going to do with the decoded frames if you can't transcode them ? Encoding them with software might prove to be painfully slow as the frames' buffers are uncached for CPU.

With that said, here's a quick cross-compile-your-kernel guide:

1) Grab a toolchain such as gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu.
Uncompress it somewhere like /opt

2) cd into your kernel repository and create a file source-build. Edit it and put this as the content:

Code: Select all

export ARCH=arm64
export CROSS_COMPILE=/opt/gcc-linaro-7.2.1-2017.11-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-

3) source it

Code: Select all

$ source ./source-build

4) Only to do once - initialize your default .config from defconfig

Code: Select all

$ make defconfig

This defconfig enables most arm64 things for every SoC.

5) Menuconfig to disable what you don't need (all the archs except MESON for instance..), and enable things like the vdec (CONFIG_VIDEO_AML_MESON_VDEC)

Code: Select all

$ make menuconfig

6) Build the Image and the dtb

Code: Select all

$ make -j10 Image amlogic/meson-gxbb-odroidc2.dtb

And then you can push those 2 files to your board, I don't know specifically how it's done with your distro though. Hopefully you have a mainline u-boot and the kernel/dtb are somewhere in /boot.
The dtb will end up at ./arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dtb, while Image will be at the root directory.

If you want to use the vdec, you'll also need to download additional firmware from here. Place the meson directory in /lib/firmware on your board.

[martos]

Perfect !!!
I will try it this evening

[martos]

compile ok,
I try to put in my armbian img
or find archlinux img to put this kernel.

[martos]

i try last librelec dev ( LibreELEC-Khadas_VIM.arm-9.0-devel-20180731153357-778b48a.img.gz )
i see the meson firmware installed

When i launch
ffmpeg -vcodec h264_v4l2m2m -i http://192.168.1.50:8001/+1:0:19:401:4: ... 0000:0:0:0: -s 320x240 /tmp/test.mp4

the drivers (firmware hack) is used :
[h264_v4l2m2m @ 0x295e490] driver 'meson-vdec' on card 'AMLogic Video Decoder'
[h264_v4l2m2m @ 0x295e490] Using device /dev/video0
[h264_v4l2m2m @ 0x295e490] driver 'meson-vdec' on card 'AMLogic Video Decoder'
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> mpeg4 (native))
Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x29582d0] Using a PCE to encode channel layout

i begin to test .

[martos]

With
ffmpeg -vcodec h264_v4l2m2m -i http://192.168.1.50:8001/+1:0:19:401:4: ... 0000:0:0:0: -vcodec mpeg4 -s 320x240 /tmp/test.mp4
log:
Input #0, mpegts, from 'http://192.168.1.50:8001/+1:0:19:401:4: ... 0000:0:0:0:':
Duration: N/A, start: 13552.700422, bitrate: N/A
Program 1025
Stream #0:0[0x78]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x82](fra): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, 5.1(side), fltp, 192 kb/s
Stream #0:2[0x83](fra): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 96 kb/s (visual impaired)
Stream #0:3[0x84](qaa): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 128 kb/s
Stream #0:4[0x8c](fra): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired)
Stream #0:5[0x8d](fra): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:6[0xaa]: Unknown: none ([5][0][0][0] / 0x0005)
Stream #0:7[0xab]: Unknown: none ([11][0][0][0] / 0x000B)
Program 1026
Program 1031
Program 1045
Program 1046
File '/tmp/test.mp4' already exists. Overwrite ? [y/N] y
[h264_v4l2m2m @ 0x30374a0] driver 'meson-vdec' on card 'AMLogic Video Decoder'
[h264_v4l2m2m @ 0x30374a0] Using device /dev/video0
[h264_v4l2m2m @ 0x30374a0] driver 'meson-vdec' on card 'AMLogic Video Decoder'
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> mpeg4 (native))
Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x30324a0] Using a PCE to encode channel layout
Output #0, mp4, to '/tmp/test.mp4':
Metadata:
encoder : Lavf58.12.100
Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 320x240, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
Metadata:
encoder : Lavc58.18.100 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream #0:1(fra): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1(side), fltp, 394 kb/s
Metadata:
encoder : Lavc58.18.100 aac
frame= 12 fps=0.0 q=2.0 size= 0kB time=00:00:00.81 bitrate= 0.4kbits/s dup=11 drop=0 speed=0.848x
and freeze, ffmpeg is always up in memory but don't mouve ( wotks with vcodec h264 )

with files :
https://download.blender.org/durian/mov ... 0.720p.mkv
ffmpeg -vcodec h264_v4l2m2m -i Sintel.2010.720p.mkv -f rawvideo output_nv12
log:
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> rawvideo (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264_v4l2m2m @ 0x1e4a5e0] capture: NM12 20 buffers initialized: 1280x0544, sizeimage 00720896, bytesperline 00001280
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 25 times
detected 4 logical cores
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'video_size' to value '1280x544'
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'pix_fmt' to value '23'
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'time_base' to value '1/1000'
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x1e41100] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x1e41100] w:1280 h:544 pixfmt:nv12 tb:1/1000 fr:24/1 sar:1/1 sws_param:flags=2
[AVFilterGraph @ 0x1e9bff0] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed
Output #0, rawvideo, to 'output_nv12':
Metadata:
encoder : Lavf58.12.100
Chapter #0:0: start 0.000000, end 103.125000
Metadata:
title : Chapter 01
Chapter #0:1: start 103.125000, end 148.667000
Metadata:
title : Chapter 02
Chapter #0:2: start 148.667000, end 349.792000
Metadata:
title : Chapter 03
Chapter #0:3: start 349.792000, end 437.208000
Metadata:
title : Chapter 04
Chapter #0:4: start 437.208000, end 472.075000
Metadata:
title : Chapter 05
Chapter #0:5: start 472.075000, end 678.833000
Metadata:
title : Chapter 06
Chapter #0:6: start 678.833000, end 744.083000
Metadata:
title : Chapter 07
Chapter #0:7: start 744.083000, end 888.032000
Metadata:
title : Chapter 08
Stream #0:0(eng), 0, 1/24: Video: rawvideo, 1 reference frame (NV12 / 0x3231564E), nv12(left), 1280x544 [SAR 1:1 DAR 40:17], 0/1, q=2-31, 200540 kb/s, 24 fps, 24 tbn, 24 tbc
Metadata:
encoder : Lavc58.18.100 rawvideo
Clipping frame in rate conversion by 0.000008

The same ffmpeg freeze ( wotks with vcodec h264 )

[martos]

If i make a reboot and i test the same again , it begin with very low performance ( max 7 fps ) .
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> mpeg4 (native))
Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x362d130] Using a PCE to encode channel layout
Output #0, mp4, to '/tmp/test.mp4':
Metadata:
encoder : Lavf58.12.100
Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 320x240, q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
Metadata:
encoder : Lavc58.18.100 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream #0:1(fra): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1(side), fltp, 394 kb/s
Metadata:
encoder : Lavc58.18.100 aac
frame= 18 fps=0.0 q=2.0 size= 0kB time=00:00:00.68 bitrate= 0.5kbits/sframe= 19 fps= 12 q=2.0 size= 0kB time=00:00:00.81 bitrate= 0.4kbits/sframe= 20 fps=9.3 q=2.0 size= 0kB time=00:00:00.91 bitrate= 0.4kbits/sframe= 21 fps=7.8 q=2.0 size= 0kB time=00:00:00.98 bitrate= 0.4kbits/sframe= 22 fps=6.8 q=2.0 size= 0kB time=00:00:01.00 bitrate= 0.4kbits/s dup=17 drop=0 speed=0.312x

If i stop and relauch it freeze directly.
[h264_v4l2m2m @ 0x2d48620] driver 'meson-vdec' on card 'AMLogic Video Decoder'
[h264_v4l2m2m @ 0x2d48620] Using device /dev/video0
[h264_v4l2m2m @ 0x2d48620] driver 'meson-vdec' on card 'AMLogic Video Decoder'
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> mpeg4 (native))
Stream #0:1 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x2e479f0] Using a PCE to encode channel layout

reboot and it's works some times.

For the video files Sintel.2010.720p.mkv always freeze

i try ffmpeg -vcodec mpeg4_v4l2m2m -i Sintel.2010.720p.mkv output_nv12.mp4 with same result ( it start but freeze quickly
log:

Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (mpeg4_v4l2m2m) -> mpeg4 (native))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[mpeg4_v4l2m2m @ 0x33ed770] capture: NM12 08 buffers initialized: 1280x0544, sizeimage 00720896, bytesperline 00001280
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
detected 4 logical cores
[graph_1_in_0_1 @ 0x3444e80] Setting 'time_base' to value '1/48000'
[graph_1_in_0_1 @ 0x3444e80] Setting 'sample_rate' to value '48000'
[graph_1_in_0_1 @ 0x3444e80] Setting 'sample_fmt' to value 'fltp'
[graph_1_in_0_1 @ 0x3444e80] Setting 'channel_layout' to value '0x60f'
[graph_1_in_0_1 @ 0x3444e80] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x60f
[format_out_0_1 @ 0x3444390] Setting 'sample_fmts' to value 'fltp'
[format_out_0_1 @ 0x3444390] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph @ 0x3444650] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
[aac @ 0x33e68c0] Using a PCE to encode channel layout
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 18 times
frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbcur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times

[Ely]

Hi Martos, thanks for testing!

I just pushed a fix that could cause a freeze for many files, including the Sintel one. It's a force push so no new commits but the changes are there if you pull again. However you're using a libreelec release so I don't think you can update the Image/modules easily :/ .

The low performance is expected if you're going to do software processing/encoding unfortunately, with what I said earlier: "Encoding them with software might prove to be painfully slow as the frames' buffers are uncached for CPU."

[martos]

OK but now i decode and encode in software with some parameters the speed is 0.87 speed ( 250 to 380% of cpu )for 1920x1080 to 320x240.
So if we can gaind decode ( estimate 100% of cpu ) i can go to speed 1x. (continue to encode in software )

For your change, i read you use archlinux, how i can make one img ? ( i kwow how i can make your kernel new kernel compile ok)
if i use the same distrib it could be more easy to test your new change ...

[Ely]

I haven't used arch in a while and it was on a Le Potato board, you can look here but they seem to be using the old vendor kernel so I don't know how easily you'll be able to replace a new one.

Btw, I'm confident the performance is going to be poor when doing software scale/encode on v4l2 frames.

Code: Select all

ffmpeg -c:v h264_v4l2m2m -i Sintel.2010.720p.mkv -c:a copy -vcodec mpeg4 -s 320x240 /tmp/test.mp4
frame=  191 fps=8.5 q=4.7 Lsize=    1136kB time=00:00:10.52 bitrate= 884.2kbits/s speed=0.466x

Code: Select all

ffmpeg -i Sintel.2010.720p.mkv -c:a copy -vcodec mpeg4 -s 320x240 /tmp/test.mp4
frame= 1345 fps= 79 q=17.9 Lsize=    5990kB time=00:00:56.19 bitrate= 873.2kbits/s speed=3.28x

The reason behind this is that the frames that come out of the decoder via V4L2 have pages with CPU cache disabled, so any software access is very, very slow.

[memeka]

@Ely

can something like this: https://github.com/mihailescu2m/linux/c ... 9c5c91e9c8 be done for S905 SoC as well?

[Ely]

@Ely

can something like this: https://github.com/mihailescu2m/linux/c ... 9c5c91e9c8 be done for S905 SoC as well?

Most likely yeah, let me try. It's a bit different because it's arm64 instead of arm but shouldn't be hard.

[Ely]

So, I could *sorta* replicate the patches, I had to drop the cache maintenance because it was crashing but could still enable read cache (although of course it's not really usable since some of the data will be corrupted by cache).

Software decoding:

Code: Select all

ffmpeg -i Sintel.2010.720p.mkv -an -vcodec mpeg4 -s 320x240 -vframes 1500 -f null -
frame= 1500 fps= 79 q=10.2 Lsize=N/A time=00:01:02.50 bitrate=N/A speed=3.28x    

Code: Select all

ffmpeg -i bbb_sunflower_1080p_60fps_normal.mp4 -an -vcodec mpeg4 -s 320x240 -vframes 1500 -f null -
frame= 1500 fps= 40 q=31.0 Lsize=N/A time=00:00:25.03 bitrate=N/A speed=0.673x 

HW decoding + read cache:

Code: Select all

ffmpeg -c:v h264_v4l2m2m -i Sintel.2010.720p.mkv -an -vcodec mpeg4 -s 320x240 -vframes 1500 -f null -
frame= 1500 fps= 84 q=10.2 Lsize=N/A time=00:01:02.50 bitrate=N/A speed= 3.5x

Code: Select all

ffmpeg -c:v h264_v4l2m2m -i bbb_sunflower_1080p_60fps_normal.mp4 -an -vcodec mpeg4 -s 320x240 -vframes 1500 -f null -
frame= 1500 fps= 40 q=31.0 Lsize=N/A time=00:00:25.03 bitrate=N/A speed=0.661x

720p had a slight advantage (6%).
On the 1080p sample the bottleneck was elsewhere (probably mpeg4 encoder not being able to take full multi-thread advantage on such a small resolution), as CPU usage was 300% for software decoding and 100% for HW decoding.
The performance gains would probably be more significant if encoding at a higher resolution or with x264.

[memeka]

Even so, with low res, it looks like 30fps are achievable with 1x speed.
Do you mind sharing a patch with what it ended up working for you?
Thanks for taking the time to test.

[miskol]

Are you benchmarking software vs hw decoder? or what is the point ?

[Ely]

Even so, with low res, it looks like 30fps are achievable with 1x speed.
Do you mind sharing a patch with what it ended up working for you?
Thanks for taking the time to test.

Code: Select all

---
 arch/arm64/mm/dma-mapping.c                   |  2 +
 .../common/videobuf2/videobuf2-dma-contig.c   | 60 ++++++++++++-------
 drivers/media/platform/meson/vdec/vdec.c      |  1 +
 3 files changed, 41 insertions(+), 22 deletions(-)

diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index 61e93f0b5482..cac95736582b 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -37,6 +37,8 @@ static int swiotlb __ro_after_init;
 static pgprot_t __get_dma_pgprot(unsigned long attrs, pgprot_t prot,
 				 bool coherent)
 {
+	if (attrs & DMA_ATTR_NON_CONSISTENT)
+		return prot;
 	if (!coherent || (attrs & DMA_ATTR_WRITE_COMBINE))
 		return pgprot_writecombine(prot);
 	return prot;
diff --git a/drivers/media/platform/meson/vdec/vdec.c b/drivers/media/platform/meson/vdec/vdec.c
index b8b1edf38aae..32d9dfa49fa3 100644
--- a/drivers/media/platform/meson/vdec/vdec.c
+++ b/drivers/media/platform/meson/vdec/vdec.c
@@ -591,6 +591,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq,
 	if (ret)
 		return ret;
 
+	dst_vq->dma_attrs = DMA_ATTR_NON_CONSISTENT;
 	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
-- 
2.18.0

I unfortunately had to drop the cache sync thing with dma_sgt as it would badly crash, so this patch will trigger cache incoherencies when reading the buffers from CPU but it's a good base regardless.