FFMPEG HEVC x265 h265 hardware encoding

Post Reply
unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

Rockchip RK3568B2, Mali-G52 MP2 GPU supports x265 hardware encoding. See: https://www.96rocks.com/blog/2020/11/28 ... hip-rk3568
'ffmpeg -encoders | grep hevc' returns:
ffmpeg version 89daac5-1ubuntu2+202101121912~focal Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/usr --extra-version='1ubuntu2+202101121912~focal' --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=
/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --disable-avisynth -
-enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-l
ibcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --e
nable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-libru
bberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enabl
e-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enabl
e-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-l
ibdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
V..... libx265 libx265 H.265 / HEVC (codec hevc)
V..... nvenc_hevc NVIDIA NVENC hevc encoder (codec hevc)
V..... hevc_nvenc NVIDIA NVENC hevc encoder (codec hevc)
V..... hevc_v4l2m2m V4L2 mem2mem HEVC encoder wrapper (codec hevc)
V..... hevc_vaapi H.265/HEVC (VAAPI) (codec hevc)

the only one that worked was libx265 (software encoding), which expectedly is too slow... From what I understand hevc_v4l2m2m is the hardware method to encode in M1, but the test
ffmpeg -i vid0.avi -c:v hevc_v4l2m2m vid1.mp4
returned:
[hevc_v4l2m2m @ 0x5592eaac10] Could not find a valid device
[hevc_v4l2m2m @ 0x5592eaac10] can't configure encoder
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate,
width or height

h264_v4l2m2m also didn't work...

Can someone please help me with that?

thanks

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

I'm using the ubuntu server image from HK.
'sudo lshw -c video' returns nothing... why would that be?

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

Rockchip does not provide a V4L2 M2M interface for their codecs. They use RK MPP for both encoding and decoding.
https://github.com/hardkernel/rk3568_rk_mpp

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

yes I noticed that. I could build and install RockChip MPP (dev) from github rockchip-linux, ffmpeg with options --enable-rkmpp --enable-version3 --enable-libdrm, but ~/bin/ffmpeg -hwaccels:
Hardware acceleration methods:
drm
The encoders x264 and x265:
V....D libx264 libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
V....D libx264rgb libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
V..... h264_v4l2m2m V4L2 mem2mem H.264 encoder wrapper (codec h264)
V....D libx265 libx265 H.265 / HEVC (codec hevc)
V..... hevc_v4l2m2m V4L2 mem2mem HEVC encoder wrapper (codec hevc)
What you're saying is hevc_v4l2m2m (which didn't work), is not the right encoder for rk3568? Is it hevc_vaapi? I coudn't really find precise information on it. But I see you sugested the hardkernel repository. FFMPEG from apt wasn't compiled with --enable-rkmpp...

User avatar
mad_ady
Posts: 10590
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4 (HC1, HC2), C1+, C2, C4 (HC4), N1, N2, H2, Go, Go Advance, M1
Location: Bucharest, Romania
Has thanked: 644 times
Been thanked: 903 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by mad_ady »

See if you get a rkmpp decoder. I could only get decoder support when I tried the same thing for N1.

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

unt wrote:
Mon May 02, 2022 10:06 pm
What you're saying is hevc_v4l2m2m (which didn't work), is not the right encoder for rk3568? Is it hevc_vaapi?
Codecs ending with "_v4l2m2m" will not work. Codecs ending with "_vaapi" will not work either. Codecs ending with "_rkmpp" will work.
unt wrote:
Mon May 02, 2022 10:06 pm
But I see you sugested the hardkernel repository.
I linked the HardKernel repository because Rockchip has a history of removing repositories from their Github.
unt wrote:
Mon May 02, 2022 10:06 pm
FFMPEG from apt wasn't compiled with --enable-rkmpp...
The FFMPEG package is supplied by Ubuntu which focuses on PC hardware. A custom compiled version of FFMPEG is likely required.
https://github.com/JeffyCN/FFmpeg

I can see the Rockchip MPP decoder is supported, but Github truncates the file list so I can not tell whether the encoder is supported or not.

It is also worth mentioning that to use MPP, the correct permissions need to be set on it as detailed in this post.
viewtopic.php?p=347298#p347298

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

The FFMPEG package is supplied by Ubuntu which focuses on PC hardware. A custom compiled version of FFMPEG is likely required.
Yes I wandered about that too, it's logical since rkmpp is very specific to RockChip hardware.

I compiled again using HK's repo, the result was the same, no rkmpp encoders, only decoders:
V..... h264_rkmpp h264 (rkmpp) (codec h264)
V..... hevc_rkmpp hevc (rkmpp) (codec hevc)

Sad since RK3568 supports hevc encoder, which is important to reduce videos sizes while maintaining reasonable quality... So a bit of a misleading advertisement, since the harware supports it, but nobody can use it as there are no drivers available...

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

unt wrote:
Tue May 03, 2022 4:11 am
So a bit of a misleading advertisement, since the harware supports it, but nobody can use it as there are no drivers available...
There are drivers and hardware encode/decode is supported. It is FFMPEG that does not support it. Anybody can use it through the RK MPP library, GStreamer, or Android.

https://github.com/JeffyCN/mirrors/tree ... r-rockchip
https://github.com/JeffyCN/mirrors/blob ... stmppenc.c

User avatar
rooted
Posts: 9438
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by rooted »

Hardware encoding, quality, and small file size don't generally go together in my experience.

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

crashoverride:
There are drivers and hardware encode/decode is supported. It is FFMPEG that does not support it. Anybody can use it through the RK MPP library, GStreamer, or Android.
Ah ok... the limitation is FFMPEG's, thanks for the infos. I'll check GStreamer. The executables generated by MPP like mpi_enc_test seem to be quite limited for someone used fo ffmpeg, maybe it's possible to use it in a pipe mode with ffmpeg.
rooted:
Hardware encoding, quality, and small file size don't generally go together in my experience.
NVidia's nvenc is quick and has done a good job so far, but I imagine different hdw/drivers may be actually quite different in compression quality.

thanks

User avatar
rooted
Posts: 9438
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by rooted »

Yeah I was referring to hardware encoding on SBCs, it's certainly possible as it's done well on my various phones but that's vastly different since the amount of development dollars spent far exceeds what is put into these SBC devices.

User avatar
mad_ady
Posts: 10590
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4 (HC1, HC2), C1+, C2, C4 (HC4), N1, N2, H2, Go, Go Advance, M1
Location: Bucharest, Romania
Has thanked: 644 times
Been thanked: 903 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by mad_ady »

From my tests on xu4 and c2 the encoder is suitable for transcoding when streaming. Not for long term storage. The encoder is used when shooting video from Android and produces a higher bitrate stream for the same quality level compared to the software encoder (which can do two passes). So I gave up on hardware encoding (on sbcs) for my home video collection (I transcode all my videos to reduce size by up to 10 times for the same perceived quality level).

martinet
Posts: 9
Joined: Thu Apr 21, 2022 10:37 pm
languages_spoken: english
ODROIDs: N2+
Has thanked: 3 times
Been thanked: 3 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by martinet »

unt wrote:
Tue May 03, 2022 4:11 am
The FFMPEG package is supplied by Ubuntu which focuses on PC hardware. A custom compiled version of FFMPEG is likely required.
Yes I wandered about that too, it's logical since rkmpp is very specific to RockChip hardware.

I compiled again using HK's repo, the result was the same, no rkmpp encoders, only decoders:
V..... h264_rkmpp h264 (rkmpp) (codec h264)
V..... hevc_rkmpp hevc (rkmpp) (codec hevc)

Sad since RK3568 supports hevc encoder, which is important to reduce videos sizes while maintaining reasonable quality... So a bit of a misleading advertisement, since the harware supports it, but nobody can use it as there are no drivers available...
Hi. Any chance you could point me to instructions how to compile ffmpeg with this 2 decoders? For my use case I need decoders only.

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

martinet wrote:
Hi. Any chance you could point me to instructions how to compile ffmpeg with this 2 decoders? For my use case I need decoders only.
Hi,
I got the basic instructions from https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
The RKMPP library source code, to be used with ffmpeg .configure with enable-rkmpp, I got from https://github.com/hardkernel/rk3568_rk_mpp, but also tested https://github.com/rockchip-linux/mpp, see comments above. You'll have to adjust paths and variables to your liking:

Code: Select all

FFMPEG_BUILD_BASE=$HOME/inst/ffmpeg
export FFMPEG_BUILD_PATH=$FFMPEG_BUILD_BASE/build
export PKG_CONFIG_PATH=$FFMPEG_BUILD_PATH/lib/pkgconfig
export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH

mkdir -p $HOME/bin
mkdir -p $FFMPEG_BUILD_PATH

#Compile RKMPP
MPP_VER=develop
MPP_BASE_DIR=$FFMPEG_BUILD_BASE/rk3568_rk_mpp-rk3568-odroid
MPP_SRC_DIR=$MPP_BASE_DIR/build/linux/aarch64

#or use git
aria2c https://github.com/hardkernel/rk3568_rk_mpp/archive/refs/heads/rk3568/odroid.zip
unzip mpp-$MPP_VER.zip
cd $MPP_SRC_DIR

export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH
#Generate build system
./make-Makefiles.bash
#Build using 4 cores
cmake --build . -j4
#Install to $FFMPEG_BUILD_PATH
cmake --install $MPP_SRC_DIR --prefix $FFMPEG_BUILD_PATH
#CMake install didn't change prefix in pkg_config dir, so sedded it
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_mpp.pc
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_vpu.pc
#Test binaries generated at $FFMPEG_BUILD_PATH/bin
export LD_LIBRARY_PATH=~/inst/ffmpeg/build/lib:$LD_LIBRARY_PATH
#Check utilities, etc
#cd $FFMPEG_BUILD_PATH/bin
#./mpi_dec_test -i ./4k.h264 -o 1080.yuv -w 1920 -h 1080 -t 7

#FFMEG build
cd $FFMPEG_SRC_DIR

#Configure so 'make install' installs to $HOME/bin. If you also want ffplay remove corresponding line below.
./configure \
  --prefix=$FFMPEG_BUILD_PATH \
  --bindir=$HOME/bin \
  --extra-libs="-lpthread -lm" \
  --ld="g++" \
  --disable-ffplay \
  --enable-gpl \
  --enable-nonfree \
  --arch=aarch64 \
  --enable-static --disable-shared \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libvmaf \
  --enable-neon \
  --enable-rkmpp --enable-version3 --enable-libdrm 

#Compile ffmpeg
make -j4
#Install ffmpeg to ~/bin
make install
#You may need to adjust LD_LIBRARY_PATH to include libraries

hope that helps.
These users thanked the author unt for the post:
martinet (Fri May 06, 2022 7:43 am)

martinet
Posts: 9
Joined: Thu Apr 21, 2022 10:37 pm
languages_spoken: english
ODROIDs: N2+
Has thanked: 3 times
Been thanked: 3 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by martinet »

Thanks!

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

I started work on adding hardware encoder support for rkmpp to FFMPEG. My proof-of-concept that encodes frames with fixed parameters is working. A lot still remains to be done, but it seems viable.

User avatar
mad_ady
Posts: 10590
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4 (HC1, HC2), C1+, C2, C4 (HC4), N1, N2, H2, Go, Go Advance, M1
Location: Bucharest, Romania
Has thanked: 644 times
Been thanked: 903 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by mad_ady »

:shock: I thought you weren't into ffmpeg...

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

mad_ady wrote:
Fri May 06, 2022 4:55 pm
I thought you weren't into ffmpeg...
I did it for the greetz from my fellow developers. V4L2 is a crime against humanity! No developer should ever have to endure it.
:lol:

So, its not about FFMPEG, its about libavcodec/libavformat. Of all the linux media APIs i have evaluated, it is the winner. After completing the rkmpp implementation, I plan to write an amcodec implementation for N2/C4.
These users thanked the author crashoverride for the post:
mad_ady (Fri May 06, 2022 5:59 pm)

User avatar
mad_ady
Posts: 10590
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4 (HC1, HC2), C1+, C2, C4 (HC4), N1, N2, H2, Go, Go Advance, M1
Location: Bucharest, Romania
Has thanked: 644 times
Been thanked: 903 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by mad_ady »

Big hugs from me, then, though I'm not a fellow developer...

But wasn't there the issue that the N2/C4 encoder was stateful and v4l2 was stateless (most likely I'm confusing terms)?
Anyway, ffmpeg support would be awesome!
But - it would be limited to stock kernel, right? It wouldn't be useful on mainline...

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

mad_ady wrote:
Fri May 06, 2022 5:59 pm
But wasn't there the issue that the N2/C4 encoder was stateful and v4l2 was stateless (most likely I'm confusing terms)?
That is a V4L2-ism. If you are not using V4L2, its irrelevant.
mad_ady wrote:
Fri May 06, 2022 5:59 pm
But - it would be limited to stock kernel, right? It wouldn't be useful on mainline...
ffmpeg/libavcodec/libavformat are kernel version independent. If "mainstream" (actually BSP) supports rkmpp/amcodec, it will work. If it supports V4L2, it will work. See the beauty?

martinet
Posts: 9
Joined: Thu Apr 21, 2022 10:37 pm
languages_spoken: english
ODROIDs: N2+
Has thanked: 3 times
Been thanked: 3 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by martinet »

unt wrote:
Thu May 05, 2022 7:56 am
martinet wrote:
Hi. Any chance you could point me to instructions how to compile ffmpeg with this 2 decoders? For my use case I need decoders only.
Hi,
I got the basic instructions from https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
The RKMPP library source code, to be used with ffmpeg .configure with enable-rkmpp, I got from https://github.com/hardkernel/rk3568_rk_mpp, but also tested https://github.com/rockchip-linux/mpp, see comments above. You'll have to adjust paths and variables to your liking:

Code: Select all

FFMPEG_BUILD_BASE=$HOME/inst/ffmpeg
export FFMPEG_BUILD_PATH=$FFMPEG_BUILD_BASE/build
export PKG_CONFIG_PATH=$FFMPEG_BUILD_PATH/lib/pkgconfig
export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH

mkdir -p $HOME/bin
mkdir -p $FFMPEG_BUILD_PATH

#Compile RKMPP
MPP_VER=develop
MPP_BASE_DIR=$FFMPEG_BUILD_BASE/rk3568_rk_mpp-rk3568-odroid
MPP_SRC_DIR=$MPP_BASE_DIR/build/linux/aarch64

#or use git
aria2c https://github.com/hardkernel/rk3568_rk_mpp/archive/refs/heads/rk3568/odroid.zip
unzip mpp-$MPP_VER.zip
cd $MPP_SRC_DIR

export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH
#Generate build system
./make-Makefiles.bash
#Build using 4 cores
cmake --build . -j4
#Install to $FFMPEG_BUILD_PATH
cmake --install $MPP_SRC_DIR --prefix $FFMPEG_BUILD_PATH
#CMake install didn't change prefix in pkg_config dir, so sedded it
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_mpp.pc
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_vpu.pc
#Test binaries generated at $FFMPEG_BUILD_PATH/bin
export LD_LIBRARY_PATH=~/inst/ffmpeg/build/lib:$LD_LIBRARY_PATH
#Check utilities, etc
#cd $FFMPEG_BUILD_PATH/bin
#./mpi_dec_test -i ./4k.h264 -o 1080.yuv -w 1920 -h 1080 -t 7

#FFMEG build
cd $FFMPEG_SRC_DIR

#Configure so 'make install' installs to $HOME/bin. If you also want ffplay remove corresponding line below.
./configure \
  --prefix=$FFMPEG_BUILD_PATH \
  --bindir=$HOME/bin \
  --extra-libs="-lpthread -lm" \
  --ld="g++" \
  --disable-ffplay \
  --enable-gpl \
  --enable-nonfree \
  --arch=aarch64 \
  --enable-static --disable-shared \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libvmaf \
  --enable-neon \
  --enable-rkmpp --enable-version3 --enable-libdrm 

#Compile ffmpeg
make -j4
#Install ffmpeg to ~/bin
make install
#You may need to adjust LD_LIBRARY_PATH to include libraries

hope that helps.
One noob question - where did you get libdrm from? I'm trying to build ffmpeg for frigate inside docker container. Original docker file in master seems to be fuc*ed up: https://github.com/blakeblackshear/frig ... rch64#L400 as it points to not existing repository: https://github.com/rockchip-linux/libdrm-rockchip
I removed this entry and after running ffmpeg i get: ffmpeg: error while loading shared libraries: libdrm.so.2: cannot open shared object file: No such file or directory

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

martinet wrote:
Sun May 08, 2022 4:34 am
I removed this entry and after running ffmpeg i get: ffmpeg: error while loading shared libraries: libdrm.so.2: cannot open shared object file: No such file or directory
You can install the distribution provided package "libdrm2" on Ubuntu:

Code: Select all

$ apt-cache search libdrm2
libdrm2 - Userspace interface to kernel DRM services -- runtime

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

The rkmpp encoder is now in a usable state. I have tested realtime compression from a webcam with the following:

Code: Select all

./ffmpeg -hide_banner -f video4linux2 -input_format mjpeg -s 1280x720 -framerate 30 -i /dev/video9  -vcodec h264_rkmpp_enc -b:v 1M -maxrate 1M -g 24 output.mp4
I should point out that there is an issue with FFMPEG and the CSI camera making it unusable at the current time:

Code: Select all

$ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x55a83cf360] Not a video capture device.
/dev/video0: No such device
FFMPEG checks for the capability V4L2_CAP_VIDEO_CAPTURE but the rkisp driver only has V4L2_CAP_VIDEO_CAPTURE_MPLANE
https://github.com/JeffyCN/FFmpeg/blob/ ... 4l2.c#L174
https://github.com/hardkernel/linux/blo ... re.c#L1423

It then goes on to check the "channel" of the capture device, but I could not find either VIDIOC_G_INPUT or VIDIOC_S_INPUT implemented in the rkisp driver:
https://github.com/JeffyCN/FFmpeg/blob/ ... #L853-L868

After hacking FFMPEG to ignore both of the above, it still would not capture any video and simply faults.
These users thanked the author crashoverride for the post:
odroid (Sun May 08, 2022 5:12 pm)

User avatar
rooted
Posts: 9438
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 758 times
Been thanked: 479 times
Contact:

Re: Kodi Matrix 19.4 arm64, Kodi Matrix 19.4 32bit, Kodi Leia 18.9 32bit, Kodi 20.0 Alpha

Post by rooted »

I'm grateful to hear this, too bad it wouldn't just work with the camera as well.

Thank you for taking on the task.

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

martinet wrote:
Sun May 08, 2022 4:34 am
unt wrote:
Thu May 05, 2022 7:56 am
martinet wrote:
Hi. Any chance you could point me to instructions how to compile ffmpeg with this 2 decoders? For my use case I need decoders only.
Hi,
I got the basic instructions from https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
The RKMPP library source code, to be used with ffmpeg .configure with enable-rkmpp, I got from https://github.com/hardkernel/rk3568_rk_mpp, but also tested https://github.com/rockchip-linux/mpp, see comments above. You'll have to adjust paths and variables to your liking:

Code: Select all

FFMPEG_BUILD_BASE=$HOME/inst/ffmpeg
export FFMPEG_BUILD_PATH=$FFMPEG_BUILD_BASE/build
export PKG_CONFIG_PATH=$FFMPEG_BUILD_PATH/lib/pkgconfig
export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH

mkdir -p $HOME/bin
mkdir -p $FFMPEG_BUILD_PATH

#Compile RKMPP
MPP_VER=develop
MPP_BASE_DIR=$FFMPEG_BUILD_BASE/rk3568_rk_mpp-rk3568-odroid
MPP_SRC_DIR=$MPP_BASE_DIR/build/linux/aarch64

#or use git
aria2c https://github.com/hardkernel/rk3568_rk_mpp/archive/refs/heads/rk3568/odroid.zip
unzip mpp-$MPP_VER.zip
cd $MPP_SRC_DIR

export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH
#Generate build system
./make-Makefiles.bash
#Build using 4 cores
cmake --build . -j4
#Install to $FFMPEG_BUILD_PATH
cmake --install $MPP_SRC_DIR --prefix $FFMPEG_BUILD_PATH
#CMake install didn't change prefix in pkg_config dir, so sedded it
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_mpp.pc
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_vpu.pc
#Test binaries generated at $FFMPEG_BUILD_PATH/bin
export LD_LIBRARY_PATH=~/inst/ffmpeg/build/lib:$LD_LIBRARY_PATH
#Check utilities, etc
#cd $FFMPEG_BUILD_PATH/bin
#./mpi_dec_test -i ./4k.h264 -o 1080.yuv -w 1920 -h 1080 -t 7

#FFMEG build
cd $FFMPEG_SRC_DIR

#Configure so 'make install' installs to $HOME/bin. If you also want ffplay remove corresponding line below.
./configure \
  --prefix=$FFMPEG_BUILD_PATH \
  --bindir=$HOME/bin \
  --extra-libs="-lpthread -lm" \
  --ld="g++" \
  --disable-ffplay \
  --enable-gpl \
  --enable-nonfree \
  --arch=aarch64 \
  --enable-static --disable-shared \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libvmaf \
  --enable-neon \
  --enable-rkmpp --enable-version3 --enable-libdrm 

#Compile ffmpeg
make -j4
#Install ffmpeg to ~/bin
make install
#You may need to adjust LD_LIBRARY_PATH to include libraries

hope that helps.
One noob question - where did you get libdrm from? I'm trying to build ffmpeg for frigate inside docker container. Original docker file in master seems to be fuc*ed up: https://github.com/blakeblackshear/frig ... rch64#L400 as it points to not existing repository: https://github.com/rockchip-linux/libdrm-rockchip
I removed this entry and after running ffmpeg i get: ffmpeg: error while loading shared libraries: libdrm.so.2: cannot open shared object file: No such file or directory
Hi martinet,
As a general rule when building from source, the .configure script will need -dev libraries, there are libs without -dev for many reasons, some of them are used as tools, not necessarily to be linked to. So libdrm needed won't be satisfied by libdrm2 (I tested it), but libdrm-dev works fine.
Cheers

unt
Posts: 10
Joined: Sun May 01, 2022 2:05 am
languages_spoken: english
ODROIDs: XU4Q, N2+, M1
Has thanked: 0
Been thanked: 1 time
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by unt »

crashoverride wrote:
Sun May 08, 2022 6:51 am
The rkmpp encoder is now in a usable state. I have tested realtime compression from a webcam with the following:

Code: Select all

./ffmpeg -hide_banner -f video4linux2 -input_format mjpeg -s 1280x720 -framerate 30 -i /dev/video9  -vcodec h264_rkmpp_enc -b:v 1M -maxrate 1M -g 24 output.mp4
I should point out that there is an issue with FFMPEG and the CSI camera making it unusable at the current time:

Code: Select all

$ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x55a83cf360] Not a video capture device.
/dev/video0: No such device
FFMPEG checks for the capability V4L2_CAP_VIDEO_CAPTURE but the rkisp driver only has V4L2_CAP_VIDEO_CAPTURE_MPLANE
https://github.com/JeffyCN/FFmpeg/blob/ ... 4l2.c#L174
https://github.com/hardkernel/linux/blo ... re.c#L1423

It then goes on to check the "channel" of the capture device, but I could not find either VIDIOC_G_INPUT or VIDIOC_S_INPUT implemented in the rkisp driver:
https://github.com/JeffyCN/FFmpeg/blob/ ... #L853-L868

After hacking FFMPEG to ignore both of the above, it still would not capture any video and simply faults.
Hi crashoverride,
I couldn't get any ffmpeg x264, hevc rkmpp mpp encoders made available. I tested 2 sources for ffmpeg (githubs ffmpeg.org and JeffyCN_FFmpeg), and 2 rkmpp (githubs rockchip-linux/mpp, hardkernel/rk3568_rk_mpp). I looked into the rkmpp code and saw x264 abd x265 encoders are implemented, but still no luck, they don't show when .configure shows the options and after making also not. Would you please tell us which have you used, settings and so on?
thanks
Last edited by unt on Tue May 10, 2022 7:45 am, edited 1 time in total.

martinet
Posts: 9
Joined: Thu Apr 21, 2022 10:37 pm
languages_spoken: english
ODROIDs: N2+
Has thanked: 3 times
Been thanked: 3 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by martinet »

unt wrote:
Tue May 10, 2022 5:22 am
martinet wrote:
Sun May 08, 2022 4:34 am
unt wrote:
Thu May 05, 2022 7:56 am
martinet wrote:
Hi. Any chance you could point me to instructions how to compile ffmpeg with this 2 decoders? For my use case I need decoders only.
Hi,
I got the basic instructions from https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
The RKMPP library source code, to be used with ffmpeg .configure with enable-rkmpp, I got from https://github.com/hardkernel/rk3568_rk_mpp, but also tested https://github.com/rockchip-linux/mpp, see comments above. You'll have to adjust paths and variables to your liking:

Code: Select all

FFMPEG_BUILD_BASE=$HOME/inst/ffmpeg
export FFMPEG_BUILD_PATH=$FFMPEG_BUILD_BASE/build
export PKG_CONFIG_PATH=$FFMPEG_BUILD_PATH/lib/pkgconfig
export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH

mkdir -p $HOME/bin
mkdir -p $FFMPEG_BUILD_PATH

#Compile RKMPP
MPP_VER=develop
MPP_BASE_DIR=$FFMPEG_BUILD_BASE/rk3568_rk_mpp-rk3568-odroid
MPP_SRC_DIR=$MPP_BASE_DIR/build/linux/aarch64

#or use git
aria2c https://github.com/hardkernel/rk3568_rk_mpp/archive/refs/heads/rk3568/odroid.zip
unzip mpp-$MPP_VER.zip
cd $MPP_SRC_DIR

export CMAKE_INSTALL_PREFIX=$FFMPEG_BUILD_PATH
#Generate build system
./make-Makefiles.bash
#Build using 4 cores
cmake --build . -j4
#Install to $FFMPEG_BUILD_PATH
cmake --install $MPP_SRC_DIR --prefix $FFMPEG_BUILD_PATH
#CMake install didn't change prefix in pkg_config dir, so sedded it
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_mpp.pc
sed -i "s|prefix=/usr/local|prefix=${FFMPEG_BUILD_PATH}|g" $PKG_CONFIG_PATH/rockchip_vpu.pc
#Test binaries generated at $FFMPEG_BUILD_PATH/bin
export LD_LIBRARY_PATH=~/inst/ffmpeg/build/lib:$LD_LIBRARY_PATH
#Check utilities, etc
#cd $FFMPEG_BUILD_PATH/bin
#./mpi_dec_test -i ./4k.h264 -o 1080.yuv -w 1920 -h 1080 -t 7

#FFMEG build
cd $FFMPEG_SRC_DIR

#Configure so 'make install' installs to $HOME/bin. If you also want ffplay remove corresponding line below.
./configure \
  --prefix=$FFMPEG_BUILD_PATH \
  --bindir=$HOME/bin \
  --extra-libs="-lpthread -lm" \
  --ld="g++" \
  --disable-ffplay \
  --enable-gpl \
  --enable-nonfree \
  --arch=aarch64 \
  --enable-static --disable-shared \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libvmaf \
  --enable-neon \
  --enable-rkmpp --enable-version3 --enable-libdrm 

#Compile ffmpeg
make -j4
#Install ffmpeg to ~/bin
make install
#You may need to adjust LD_LIBRARY_PATH to include libraries

hope that helps.
One noob question - where did you get libdrm from? I'm trying to build ffmpeg for frigate inside docker container. Original docker file in master seems to be fuc*ed up: https://github.com/blakeblackshear/frig ... rch64#L400 as it points to not existing repository: https://github.com/rockchip-linux/libdrm-rockchip
I removed this entry and after running ffmpeg i get: ffmpeg: error while loading shared libraries: libdrm.so.2: cannot open shared object file: No such file or directory
Hi martinet,
As a general rule when building from source, the .configure script will need -dev libraries, there are libs without -dev for many reasons, some of them are used as tools, not necessarily to be linked to. So libdrm needed won't be satisfied by libdrm2 (I tested it), but libdrm-dev works fine.
Cheers
Hi unt.

Thank you for your reply. I managed to build it already inside frigates container, but have no luck running it with my camera stream.

Code: Select all

frigate-custom  | [2022-05-09 14:47:58] watchdog.office                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [flv @ 0x5599967ec0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [h264_rkmpp @ 0x55999cbfb0] Failed to send packet to decoder (code = -11)
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   :     Last message repeated 183 times
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : Finishing stream 1:0 without any data written to it.
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [swscaler @ 0x55999f2c90] [swscaler @ 0x5599a01740] deprecated pixel format used, make sure you did set range correctly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [swscaler @ 0x55999f2c90] [swscaler @ 0x5599990450] deprecated pixel format used, make sure you did set range correctly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [swscaler @ 0x55999f2c90] [swscaler @ 0x5599a1dde0] deprecated pixel format used, make sure you did set range correctly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [swscaler @ 0x55999f2c90] [swscaler @ 0x5599a2c630] deprecated pixel format used, make sure you did set range correctly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [swscaler @ 0x55999f2c90] [swscaler @ 0x5599969900] deprecated pixel format used, make sure you did set range correctly
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [flv @ 0x5599967ec0] Failed to update header with correct duration.
frigate-custom  | [2022-05-09 14:47:58] ffmpeg.office.detect           ERROR   : [flv @ 0x5599967ec0] Failed to update header with correct filesize.

I'm wondering, if any device from /dev/ should be made available for docker container for rkmpp decoder to run properly?

Also, when installing libdrm-dev the following additional packages were installed:

Code: Select all

Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libdrm-amdgpu1 libdrm-etnaviv1 libdrm-freedreno1 libdrm-nouveau2
  libdrm-radeon1 libdrm-tegra0 libpciaccess-dev libpciaccess0
Suggested packages:
  pciutils
The following NEW packages will be installed:
  libdrm-amdgpu1 libdrm-dev libdrm-etnaviv1 libdrm-freedreno1 libdrm-nouveau2
  libdrm-radeon1 libdrm-tegra0 libpciaccess-dev libpciaccess0
0 upgraded, 9 newly installed, 0 to remove and 1 not upgraded.
Need to get 261 kB of archives.
I'm wondering if libdrm-rockchip shouldn't be installed? like the one from: https://oph.mdrjr.net/madady/odroidn1/d ... -rockchip/

I checked out commit from this ticket: https://github.com/rockchip-linux/mpp/issues/272 and also no luck with the same error.

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

unt wrote:
Tue May 10, 2022 5:37 am
I tested 2 sources for ffmpeg (githubs ffmpeg.org and JeffyCN_FFmpeg), and 2 rkmpp (githubs rockchip-linux/mpp, hardkernel/rk3568_rk_mpp).
The source I have tested is JeffyCN_FFmpeg:
https://github.com/JeffyCN/FFmpeg

The rkmpp I am using is the one provided by HardKernel in the official image:

Code: Select all

$ apt-cache search librockchip-mpp-dev
librockchip-mpp-dev - Media Process Platform
unt wrote:
Tue May 10, 2022 5:37 am
I couldn't get any ffmpeg x264, hevc rkmpp mpp encoders made available.
Make sure you are using the version you compiled and not the distribution provided one:
System version

Code: Select all

$ ffmpeg
ffmpeg version 89daac5-1ubuntu2+202101121912~focal Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --prefix=/usr --extra-version='1ubuntu2+202101121912~focal' --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --disable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
Compiled version

Code: Select all

$ ./ffmpeg
ffmpeg version N-104475-g39576c97bc Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --bindir=./bin --extra-libs='-lpthread -lm' --ld=g++ --disable-ffplay --enable-gpl --enable-nonfree --arch=aarch64 --enable-static --disable-shared --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libx264 --enable-libx265 --enable-neon --enable-rkmpp --enable-version3 --enable-libdrm
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.101 / 59. 12.101
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
These users thanked the author crashoverride for the post:
odroid (Tue May 10, 2022 9:56 am)

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

martinet wrote:
Tue May 10, 2022 7:25 am
I'm wondering, if any device from /dev/ should be made available for docker container for rkmpp decoder to run properly?
The hardware codec device needs to be available for rkmpp to work:

Code: Select all

$ ls -l /dev/mpp_service 
crw-rw---- 1 root video 244, 0 May  6 04:17 /dev/mpp_service

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

Since many USB webcams use MJPEG, I added support for hardware decoding of it. In RKMPP, MJPEG uses an entirely different method than the other codecs, so I had to author a new FFMPEG decoder for it. Also of interest is that it would not output YUV420P like the other codecs. It only seems to support NV12 as a YUV format in addition to several RGB formats.

I tested using a webcam with hardware MJPEG decoding and hardware H264 encoding with the following command:

Code: Select all

$ ./ffmpeg -hide_banner -f video4linux2 -input_format mjpeg -s 1280x720 -framerate 60 -vcodec mjpeg_rkmpp -i /dev/video9  -vcodec h264_rkmpp_enc -b:v 1M -minrate 500K -maxrate 5M -g 24 output.mp4
It appears to work as expected, and a significant difference in CPU load can be observed versus using the software MJPEG decoder built into FFMPEG.
These users thanked the author crashoverride for the post:
odroid (Fri May 13, 2022 11:22 pm)

User avatar
mad_ady
Posts: 10590
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4 (HC1, HC2), C1+, C2, C4 (HC4), N1, N2, H2, Go, Go Advance, M1
Location: Bucharest, Romania
Has thanked: 644 times
Been thanked: 903 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by mad_ady »

Awesome! Great work!

crashoverride
Posts: 5660
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 535 times
Contact:

Re: FFMPEG HEVC x265 h265 hardware encoding

Post by crashoverride »

Some notes of interest:

I attempted to add MPEG2 hardware decoding. However, it fails in rkmpp. It uses the same decoder code already available, so a future version of the library may allow it work.

I attempted to add HEVC hardware encoding. However, it also also fails due to issues with rkmpp. While "AnnexB" can be disabled for the H264 encoder, it is hard-coded into the HEVC encoder. FFMPEG requires the use of "Extra Data" instead of AnnexB. The header issue will either need to be fixed in rkmpp or bitstream processing to extract it will need to be added to my code.

Buffers will always require an additional copy. I spent a lot of time trying to optimize buffer usage. However, there is a bug in the kernel MPP service driver that panics the kernel every time (I really love having a RESET button!). Since it should not be possible to panic the system from user space, this is considered a driver bug. There may be additional issues with RKMPP when attempting to setup a "Normal" buffer group.

The inability to use the CSI camera with FFMPEG is not a driver bug. From all the research I have done on the issue, the driver appears to implement a V4L2 MPLANE API only. This is even the case with the mainline kernel ISP driver. User space applications (like FFMPEG) need to be updated with MPLANE support. It appears that FFMPEG has never worked on any Rockchip SoC that has a CSI interface because of this. Additionally, it will not work with the mainline kernel. All the documentation for CSI use on Rockchip platforms have been with GStreamer instead of FFMPEG likely for this reason.
These users thanked the author crashoverride for the post:
odroid (Thu May 19, 2022 6:22 am)

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 1 guest