ffmpeg libav

Post Reply
smaram
Posts: 4
Joined: Tue Sep 28, 2021 6:00 pm
languages_spoken: english, german, spanish
ODROIDs: XU4, C4
Has thanked: 0
Been thanked: 1 time
Contact:

ffmpeg libav

Post by smaram »

hello,

i am using https://wiki.odroid.com/odroid-xu4/os_i ... 10-minimal

compiled ffmpeg 4.4 from source

Code: Select all

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
  configuration: --enable-gpl --enable-version3 --disable-static --enable-shared --enable-small --enable-chromaprint --enable-frei0r --enable-gmp --enable-gnutls --enable-ladspa --enable-libass --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-librtmp --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-openal --enable-opengl --enable-libdrm --enable-nonfree --enable-libbluray
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Splitting the commandline.
Finished splitting the commandline.
Parsing a group of options: global .
Successfully parsed a group of options.
Hyper fast Audio and Video encoder

Code: Select all

v4l2-ctl -D -d 11
Driver Info (not using libv4l2):
        Driver name   : s5p-mfc
        Card type     : s5p-mfc-enc
        Bus info      : platform:11000000.codec
        Driver version: 4.14.85
        Capabilities  : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
running this command everything works like expected and i have a playable output.

Code: Select all

 ffmpeg -i big_buck_bunny_720p_h264.mov -acodec copy -vcodec h264_v4l2m2m -pix_fmt nv21 video.mp4
i have added more log prints in libav ffmpeg-4.4/libavcodec/v4l2_buffers.c

Code: Select all

    av_log(logger(out), AV_LOG_DEBUG, "v4l2_buffer_swframe_to_buf::out->num_planes: %i\n", out->num_planes);
    for (i = 0; i < out->num_planes; i++) {
        av_log(logger(out), AV_LOG_DEBUG, "============================v4l2_buffer_swframe_to_buf::loop: %i\n", i);

        /* sskomudek */
        av_log(logger(out), AV_LOG_DEBUG, "frame ptr = %p\n", frame);
        av_log(logger(out), AV_LOG_DEBUG, "&frame->buf[%i] = %p\n", i, &frame->buf[i]);

        av_log(logger(out), AV_LOG_DEBUG, "frame out = %p\n", out);
        av_log(logger(out), AV_LOG_DEBUG, "sizeof frame->buf = %p\n", sizeof(frame->buf));
        //av_log(logger(out), AV_LOG_DEBUG, "frame->buf[i]->size = %d\n", frame->buf[i]->size);
        //av_log(logger(out), AV_LOG_DEBUG, "frame->buf[%i]->data[0] = 0x%x\n", i, frame->buf[i]->data[0]);

        ret = v4l2_bufref_to_buf(out, i, frame->buf[i]->data, frame->buf[i]->size, 0);

        // av_log(logger(out), AV_LOG_DEBUG, "::v4l2_bufref_to_buf ret: %i\n", ret);
        if (ret)
        {
            av_log(logger(out), AV_LOG_DEBUG, "v4l2_buffer_swframe_to_buf::7 (error)\n");
            return ret;
        }
    }

results with ffmpeg command line:

Code: Select all

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'big_buck_bunny_720p_h264.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 537199360
    compatible_brands: qt
    creation_time   : 2008-05-27T18:36:22.000000Z
    com.apple.quicktime.player.movie.audio.gain: 1.000000
    com.apple.quicktime.player.movie.audio.treble: 0.000000
    com.apple.quicktime.player.movie.audio.bass: 0.000000
    com.apple.quicktime.player.movie.audio.balance: 0.000000
    com.apple.quicktime.player.movie.audio.pitchshift: 0.000000
    com.apple.quicktime.player.movie.audio.mute:
    com.apple.quicktime.player.movie.visual.brightness: 0.000000
    com.apple.quicktime.player.movie.visual.color: 1.000000
    com.apple.quicktime.player.movie.visual.tint: 0.000000
    com.apple.quicktime.player.movie.visual.contrast: 1.000000
    com.apple.quicktime.player.version: 7.4.1 (14)
    com.apple.quicktime.version: 7.4.1 (14) 0x7418000 (Mac OS X, 10.5.2, 9C31)
    timecode        : 00:00:00:00
  Duration: 00:09:56.46, start: 0.000000, bitrate: 5589 kb/s
  Stream #0:0(eng), 1, 1/2400: Video: h264, 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709, topleft), 1280x720, 0/1, 5146 kb/s, 24 fps, 24 tbr, 2400 tbn, 4800 tbc (default)
    Metadata:
      creation_time   : 2008-05-27T18:36:22.000000Z
      handler_name    : Apple Video Media Handler
      vendor_id       : appl
      encoder         : H.264
  Stream #0:1(eng), 0, 1/2400: Data: none (tmcd / 0x64636D74), 0/1 (default)
    Metadata:
      creation_time   : 2008-05-27T18:36:22.000000Z
      handler_name    : Time Code Media Handler
      timecode        : 00:00:00:00
  Stream #0:2(eng), 45, 1/48000: Audio: aac (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 437 kb/s (default)
    Metadata:
      creation_time   : 2008-05-27T18:36:22.000000Z
      handler_name    : Apple Sound Media Handler
      vendor_id       : [0][0][0][0]

Code: Select all

[graph 0 input from stream 0:0 @ 0xb76990] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 0xb76990] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0xb76990] Setting 'time_base' to value '1/2400'
[graph 0 input from stream 0:0 @ 0xb76990] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0xb76990] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0xb76990] w:1280 h:720 pixfmt:yuv420p tb:1/2400 fr:24/1 sar:0/1
[format @ 0xb76d90] Setting 'pix_fmts' to value 'nv21'
[auto_scaler_0 @ 0xb772e0] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 0xb772e0] w:iw h:ih flags:'bicubic' interl:0
[format @ 0xb76d90] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0x749c70] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[auto_scaler_0 @ 0xb772e0] w:1280 h:720 fmt:yuv420p sar:0/1 -> w:1280 h:720 fmt:nv21 sar:0/1 flags:0x4
[h264_v4l2m2m @ 0x6b7ff0] probing device /dev/video21
[h264_v4l2m2m @ 0x6b7ff0] driver 'exynos-gsc' on card 'exynos-gsc gscaler' in mplane mode
[h264_v4l2m2m @ 0x6b7ff0] v4l2 capture format not supported
[h264_v4l2m2m @ 0x6b7ff0] probing device /dev/video20
[h264_v4l2m2m @ 0x6b7ff0] driver 'exynos-gsc' on card 'exynos-gsc gscaler' in mplane mode
[h264_v4l2m2m @ 0x6b7ff0] v4l2 capture format not supported
[h264_v4l2m2m @ 0x6b7ff0] probing device /dev/video11
[h264_v4l2m2m @ 0x6b7ff0] driver 's5p-mfc' on card 's5p-mfc-enc' in mplane mode
[h264_v4l2m2m @ 0x6b7ff0] Using device /dev/video11
[h264_v4l2m2m @ 0x6b7ff0] driver 's5p-mfc' on card 's5p-mfc-enc' in mplane mode
[h264_v4l2m2m @ 0x6b7ff0] requesting formats: output=NM21 capture=H264
[h264_v4l2m2m @ 0x6b7ff0] output: NM21 32 buffers initialized: 1280x0720, sizeimage 00921856, bytesperline 00001280
[h264_v4l2m2m @ 0x6b7ff0] capture: H264 08 buffers initialized: 0000x0000, sizeimage 00708608, bytesperline 00708608
[h264_v4l2m2m @ 0x6b7ff0] Encoder: number of B-frames = 0
[h264_v4l2m2m @ 0x6b7ff0] Encoder: header mode = 0
[h264_v4l2m2m @ 0x6b7ff0] Encoder: frame level rate control = 1
[h264_v4l2m2m @ 0x6b7ff0] h264_v4l2m2m encoder: enabling bit rate control: 200000
[h264_v4l2m2m @ 0x6b7ff0] Encoder: bit rate = 200000
[h264_v4l2m2m @ 0x6b7ff0] Encoder: gop size = 12
[h264_v4l2m2m @ 0x6b7ff0] Encoder Context: id (27), profile (-99), frame rate(24/1), number b-frames (0), gop size (12), bit rate (200000), qmin (-1), qmax (-1)
[h264_v4l2m2m @ 0x6b7ff0] Encoder: minimum video quantizer scale = 0
[h264_v4l2m2m @ 0x6b7ff0] Encoder: maximum video quantizer scale = 51
[mp4 @ 0x6d52b0] timecode: tbc=1/0 invalid, fallback on 24/1
all fine so far, now i want to libav in a project to convert RGB32 to NV21 and it just get stuck or i am just doing something simply wrong!

the initialization looks good to me.

Code: Select all

[h264_v4l2m2m @ 0xaff08640] probing device /dev/video21
[h264_v4l2m2m @ 0xaff08640] driver 'exynos-gsc' on card 'exynos-gsc gscaler' in mplane mode
[h264_v4l2m2m @ 0xaff08640] v4l2 capture format not supported
[h264_v4l2m2m @ 0xaff08640] probing device /dev/video20
[h264_v4l2m2m @ 0xaff08640] driver 'exynos-gsc' on card 'exynos-gsc gscaler' in mplane mode
[h264_v4l2m2m @ 0xaff08640] v4l2 capture format not supported
[h264_v4l2m2m @ 0xaff08640] probing device /dev/video11
[h264_v4l2m2m @ 0xaff08640] driver 's5p-mfc' on card 's5p-mfc-enc' in mplane mode
[h264_v4l2m2m @ 0xaff08640] Using device /dev/video11
[h264_v4l2m2m @ 0xaff08640] driver 's5p-mfc' on card 's5p-mfc-enc' in mplane mode
[h264_v4l2m2m @ 0xaff08640] requesting formats: output=NM21 capture=H264
[h264_v4l2m2m @ 0xaff08640] output: NM21 32 buffers initialized: 1344x0662, sizeimage 00903424, bytesperline 00001344
[h264_v4l2m2m @ 0xaff08640] capture: H264 08 buffers initialized: 0000x0000, sizeimage 00679936, bytesperline 00679936
[h264_v4l2m2m @ 0xaff08640] Encoder: number of B-frames = 0
[h264_v4l2m2m @ 0xaff08640] Encoder: header mode = 0
[h264_v4l2m2m @ 0xaff08640] Encoder: frame level rate control = 1
[h264_v4l2m2m @ 0xaff08640] h264_v4l2m2m encoder: enabling bit rate control: 8000000
[h264_v4l2m2m @ 0xaff08640] Encoder: bit rate = 8000000
[h264_v4l2m2m @ 0xaff08640] Encoder: gop size = 12
[h264_v4l2m2m @ 0xaff08640] Encoder Context: id (27), profile (77), frame rate(0/1), number b-frames (0), gop size (12), bit rate (8000000), qmin (-1), qmax (-1)
[h264_v4l2m2m @ 0xaff08640] Encoder: h264 profile = 2
[h264_v4l2m2m @ 0xaff08640] Encoder: minimum video quantizer scale = 0
[h264_v4l2m2m @ 0xaff08640] Encoder: maximum video quantizer scale = 51
then some log print from the application

Code: Select all

==============>>>>>> guac_video_prepare_frame dst = aff115f0
==============>>>>>> guac_video_prepare_frame dst->data[0] = ac414020, dst->data[1] = ac4ece60, dst->data[2] = 0
==============>>>>>> guac_video_prepare_frame dst->linezie[0] = 1344, dst->linesize[1] = 1344, dst->linesize[2] = 0
==============>>>>>> width = 1344, height = 661, stride = 5376, dst->width = 1344, dst->height = 661
==============>>>>>> guac_video_convert_frame frame = affbc670
==============>>>>>> guac_video_convert_frame dst_data = a0bc2020, dst_data1 = 0, dst_stride0 = 5376, dst_stride1 = 0, dst_stride2 = 0, height = 661
==============>>>>>> guac_video_prepare_frame src = affbc670
==============>>>>>> guac_video_prepare_frame src->data[0] = a0bc2020, src->data[1] = 0, src->data[2] = 0
==============>>>>>> guac_video_prepare_frame src->linezie[0] = dx, src->linesize[1] = 5376, src->linesize[2] = 0
==============>>>>>> guac_video_prepare_frame dst->data[0] = ac414020, dst->data[1] = ac4ece60, dst->data[2] = 0
==============>>>>>> guac_video_prepare_frame dst->linezie[0] = 1344, dst->linesize[1] = 1344, dst->linesize[2] = 0
then

Code: Select all

[h264_v4l2m2m @ 0xaff08640] v4l2_buffer_swframe_to_buf::out->num_planes: 2
[h264_v4l2m2m @ 0xaff08640] ============================v4l2_buffer_swframe_to_buf::loop: 0
[h264_v4l2m2m @ 0xaff08640] frame ptr = 0xaff0aba0
[h264_v4l2m2m @ 0xaff08640] &frame->buf[0] = 0xaff0ac98
[h264_v4l2m2m @ 0xaff08640] frame out = 0xaff0ad40
[h264_v4l2m2m @ 0xaff08640] sizeof frame->buf = 0x20
[h264_v4l2m2m @ 0xaff08640] ::IN v4l2_bufref_to_buf
[h264_v4l2m2m @ 0xaff08640] We are in MultiPlanar
[h264_v4l2m2m @ 0xaff08640] v4l2_bufref_to_buf::3
[h264_v4l2m2m @ 0xaff08640] ============================v4l2_buffer_swframe_to_buf::loop: 1
[h264_v4l2m2m @ 0xaff08640] frame ptr = 0xaff0aba0
[h264_v4l2m2m @ 0xaff08640] &frame->buf[1] = 0xaff0ac9c
[h264_v4l2m2m @ 0xaff08640] frame out = 0xaff0ad40
[h264_v4l2m2m @ 0xaff08640] sizeof frame->buf = 0x20
now it got stuck in the second loop in /root/ffmpegtemp44/ffmpeg-4.4/libavcodec/v4l2_buffers.c at

Code: Select all

ret = v4l2_bufref_to_buf(out, i, frame->buf[i]->data, frame->buf[i]->size, 0);
looks like frame->buf->data has no usefull data but we have sizeof frame->buf = 0x20

i am not sure whats going on and any help is appreciated :idea: and here what i am trying todo and sorry for the mess in the code, to much experiments:

Code: Select all


#include "config.h"
#include "ffmpeg-compat.h"
#include "guacamole/client.h"
#include "guacamole/protocol.h"
#include "guacamole/timestamp.h"
#include "guacamole/video.h"
#include "video-private.h"

#include <cairo/cairo.h>
#include <libavcodec/avcodec.h>
#include <libavutil/common.h>
#include <libavutil/imgutils.h>
#include <libavutil/opt.h>
#include <libswscale/swscale.h>
#include <libavutil/log.h>

/*#include <libavutil/mathematics.h>
#include <libavformat/avformat.h>
#include <x264.h>*/

#include <sys/types.h>
#include <sys/stat.h>
#include <assert.h>
#include <fcntl.h>
#include <inttypes.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

/**
 * The framerate at which video should be encoded, in frames per second.
 */
#define GUACENC_VIDEO_FRAMERATE 25

guac_video* guac_video_alloc(guac_socket* socket, guac_stream* stream,
        const char* codec_name, int width, int height, int bitrate) {     

    /* Pull codec based on name */
    AVCodec* codec = avcodec_find_encoder_by_name(codec_name);
    if (codec == NULL) {
        fprintf(stderr, "Failed to locate codec \"%s\".",
                codec_name);
        goto fail_codec;
    }

    /* Retrieve encoding context */
    AVCodecContext* context = avcodec_alloc_context3(codec);
    if (context == NULL) {
         fprintf(stderr, "Failed to allocate context for "
                "codec \"%s\".", codec_name);
        goto fail_context;
    }


    /* Init context with encoding parameters */
    context->bit_rate = bitrate;
    context->width =  width;
    context->height = height;
    context->time_base = (AVRational) { 1, GUACENC_VIDEO_FRAMERATE };
    context->max_b_frames = 0;
    context->pix_fmt = AV_PIX_FMT_NV21;
    context->profile = 77;
    context->time_base.num = 1;    
    context->time_base.den = 25;  

    



    /*if (av_opt_set(context->priv_data, "num_output_buffers", 32 , 0))
        fprintf(stderr, "=========>>>>>Failed to set preset\n");
    */
    //if (av_opt_set(context->priv_data, "num_capture_buffers", 32 , 0))
    //  fprintf(stderr, "=========>>>>>Failed to set preset\n");
    
    //if (av_opt_set(context->priv_data, "tune", "zerolatency", 0))
    //    fprintf(stderr, "Failed to set tune\n");

  
    //if (av_opt_set(context->priv_data, "pass", "1", AV_OPT_SEARCH_CHILDREN))
    //    fprintf(stderr, "Failed to set pass\n");

    //if (av_opt_set(context->priv_data, "coder", "0", 0))
    //    fprintf(stderr, "Failed to set coder\n");

    //if (av_opt_set(context->priv_data, "flags", "-loop", 0))
    //    fprintf(stderr, "Failed to set flags\n");
    
    /*
    if (av_opt_set(context->priv_data, "bsf", "h264_mp4toannexb", 0))
        fprintf(stderr, "Failed to set bsf\n");
    */

    //if (av_opt_set(context->priv_data, "wpredp", "0", 0))
    //    fprintf(stderr, "Failed to set wpredp\n");
    
    //av_opt_set(context->priv_data, "quality", "realtime", AV_OPT_SEARCH_CHILDREN);
    //av_opt_set(context->priv_data, "x264opts", "opencl", AV_OPT_SEARCH_CHILDREN);
    //av_opt_set(context->priv_data, "crf", "25", AV_OPT_SEARCH_CHILDREN);
    //av_opt_set(context->priv_data, "b:v", "2M", AV_OPT_SEARCH_CHILDREN);

    /* Open codec for use */
    if (avcodec_open2(context, codec, NULL) < 0) {
        fprintf(stderr,  "============>>>>>Failed to open codec \"%s\".", codec_name);
        /* guacenc_log(GUAC_LOG_ERROR, "Failed to open codec \"%s\".", codec_name); */
        goto fail_codec_open;
    }

    /* Allocate corresponding frame */
    AVFrame* frame = av_frame_alloc();
    if (frame == NULL) {
        goto fail_frame;
    }

    /* Copy necessary data for frame from context */
    frame->format = context->pix_fmt;
    frame->width = context->width;
    frame->height = context->height;

    /* Allocate actual backing data for frame */
    if (av_image_alloc(frame->data, frame->linesize, frame->width,
                frame->height, frame->format, 32) < 0) {
        goto fail_frame_data;
    }
fprintf(stderr, "==============>>>>>> guac_video_alloc dst_data0 = %x, dst_data1 = %x, dst_stride0 = %d, dst_stride1 = %d, dst_stride2 = %d\n", frame->data[0], frame->data[1], frame->linesize[0], frame->linesize[1], frame->linesize[2]);
    /* Allocate video structure */
    guac_video* video = malloc(sizeof(guac_video));
    if (video == NULL) {
        goto fail_video;
    }

    /* Init properties of video */
    video->socket = socket;
    video->stream = stream;
    video->context = context;
    video->next_frame = frame;
    video->width = width;
    video->height = height;
    video->bitrate = bitrate;

    /* No frames have been written or prepared yet */
    video->last_timestamp = 0;
    video->next_pts = 0;

fprintf(stderr, "==============>>>>>> guac_video_alloc frame = %x\n", frame);
    return video;

    /* Free all allocated data in case of failure */
    fail_video:
        av_freep(&frame->data[0]);

    fail_frame_data:
        av_frame_free(&frame);

    fail_frame:
    fail_codec_open:
        avcodec_free_context(&context);

    fail_context:
    fail_codec:
        return NULL;

}

/**
 * Flushes the specied frame as a new frame of video, updating the internal
 * video timestamp by one frame's worth of time. The pts member of the given
 * frame structure will be updated with the current presentation timestamp of
 * the video. If pending frames of the video are being flushed, the given frame
 * may be NULL (as required by avcodec_encode_video2()).
 *
 * @param video
 *     The video to write the given frame to.
 *
 * @param frame
 *     The frame to write to the video, or NULL if previously-written frames
 *     are being flushed.
 *
 * @return
 *     A positive value if the frame was successfully written, zero if the
 *     frame has been saved for later writing / reordering, negative if an
 *     error occurs.
 */
static int guac_video_write_frame(guac_video* video, AVFrame* frame) {
    
    /* Set timestamp of frame, if frame given */
    if (frame != NULL)
        frame->pts = video->next_pts;

    /* Write frame to video */
    int got_data = guac_avcodec_encode_video(video, frame);
    if (got_data < 0)
        return -1;

    /* Update presentation timestamp for next frame */
    video->next_pts++;

    /* Write was successful */
    return got_data;

}

/**
 * Flushes the frame previously specified by guac_video_prepare_frame() as a
 * new frame of video, updating the internal video timestamp by one frame's
 * worth of time.
 *
 * @param video
 *     The video to flush.
 *
 * @return
 *     Zero if flushing was successful, non-zero if an error occurs.
 */
static int guac_video_flush_frame(guac_video* video) {
    
    /* Write frame to video */
    return guac_video_write_frame(video, video->next_frame) < 0;

}

int guac_video_advance_timeline(guac_video* video,
        guac_timestamp timestamp) {

    guac_timestamp next_timestamp = timestamp;

    /* Flush frames as necessary if previously updated */
    if (video->last_timestamp != 0) {

        /* Calculate the number of frames that should have been written */
        int elapsed = (timestamp - video->last_timestamp)
                    * GUACENC_VIDEO_FRAMERATE / 1000;

        /* Keep previous timestamp if insufficient time has elapsed */
        if (elapsed == 0)
            return 0;

        /* Use frame time as last_timestamp */
        next_timestamp = video->last_timestamp
                        + elapsed * 1000 / GUACENC_VIDEO_FRAMERATE;

        /* Flush frames to bring timeline in sync, duplicating if necessary */
        do {
            if (guac_video_flush_frame(video)) {
                fprintf(stderr, "Unable to flush frame to video stream.\n");
                return 1;
            }
        } while (--elapsed != 0);

    }

    /* Update timestamp */
    video->last_timestamp = next_timestamp;
    return 0;

}




/**
 * Converts the given raw image data into an AVFrame object usable by FFMpeg.
 *
 * @param src_data
 *     Raw image data in RGB32 format, with the lowest-order byte being the
 *     blue component and the highest-order byte being ignored (identical to
 *     Cairo's CAIRO_FORMAT_RGB24 format).
 *
 * @param width
 *     The width of the image data, in pixels.
 *
 * @param height
 *     The height of the image data, in pixels.
 *
 * @param src_stride
 *     The number of bytes between each row of image data within src_data.
 *
 * @return
 *     A newly-allocated AVFrame containing an identical copy of the provided
 *     image data.
 */
static AVFrame* guac_video_convert_frame(unsigned char* src_data, int width,
        int height, int src_stride) {

    /* Prepare source frame for buffer */
    AVFrame* frame = av_frame_alloc();
    if (frame == NULL)
        return NULL;

    /* Copy buffer properties to frame */
    frame->format = AV_PIX_FMT_RGB32;
    frame->width = width;
    frame->height = height;

    /* Allocate actual backing data for frame */
    if (av_image_alloc(frame->data, frame->linesize, frame->width,
                frame->height, frame->format, 32) < 0) {
        av_frame_free(&frame);
        return NULL;
    }

    /* Get pointer to destination image data */
    unsigned char* dst_data = frame->data[0];
    int dst_stride = frame->linesize[0]; 
fprintf(stderr, "==============>>>>>> guac_video_convert_frame frame = %x\n", frame);
fprintf(stderr, "==============>>>>>> guac_video_convert_frame dst_data = %x, dst_data1 = %x, dst_stride0 = %d, dst_stride1 = %d, dst_stride2 = %d, height = %d\n", dst_data, frame->data[1], frame->linesize[0], frame->linesize[1], frame->linesize[2], height);
    /* Copy all data from source buffer to destination frame */
    while (height > 0) {

    
        
        memcpy(dst_data, src_data, width * 4); 

        //memcpy(dst_data, src_data, width*height*3/2); 

        dst_data += dst_stride;
        src_data += src_stride;

        height--;

    }

    /* Frame converted */
    return frame;

}
#include <stdio.h>
FILE *pfile;
void guac_video_prepare_frame(guac_video* video, cairo_surface_t* surface) {
    av_log_set_level(AV_LOG_TRACE);

    int width = cairo_image_surface_get_width(surface);
    int height = cairo_image_surface_get_height(surface);
    int stride = cairo_image_surface_get_stride(surface);
    unsigned char* data = cairo_image_surface_get_data(surface);
pfile = fopen("/root/guacamole-server/temp.rgb", "wb+");
fwrite(data, stride * height, 1, pfile);
fclose(pfile);

    AVFrame* dst = video->next_frame;
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame dst = %x\n", dst);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame dst->data[0] = %x, dst->data[1] = %x, dst->data[2] = %x\n", dst->data[0], dst->data[1], dst->data[2]);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame dst->linezie[0] = %d, dst->linesize[1] = %d, dst->linesize[2] = %d\n", dst->linesize[0], dst->linesize[1], dst->linesize[2]);

    /* Prepare scaling context */
    struct SwsContext* sws = sws_getContext(width, height,
            AV_PIX_FMT_RGB32, dst->width, dst->height, AV_PIX_FMT_NV21,
            SWS_BICUBIC, NULL, NULL, NULL);
fprintf(stderr, "==============>>>>>> width = %d, height = %d, stride = %d, dst->width = %d, dst->height = %d\n", width, height, stride, dst->width, dst->height);
            //CAIRO_FORMAT_ARGB32

    /* Abort if scaling context could not be created */
    if (sws == NULL) {
fprintf(stderr, "==============>>>>>>Failed to allocate software scaling ");
        /* guacenc_log(GUAC_LOG_WARNING, "Failed to allocate software scaling "
                "context. Frame dropped."); */
        return;
    }

    /* Flush pending operations to surface */
    cairo_surface_flush(surface);

    /* Prepare source frame for surface */
    AVFrame* src = guac_video_convert_frame(data, width, height, stride);
    if (src == NULL) {
        /* guacenc_log(GUAC_LOG_WARNING, "Failed to allocate source frame. "
                "Frame dropped."); */
        return;
    }
    /* Apply scaling, copying the source frame to the destination */
//    sws_scale(sws, (const uint8_t* const*) src->data, src->linesize,
//            0, height, dst->data, dst->linesize);


fprintf(stderr, "==============>>>>>> guac_video_prepare_frame src = %x\n", src);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame src->data[0] = %x, src->data[1] = %x, src->data[2] = %x\n", src->data[0], src->data[1], src->data[2]);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame src->linezie[0] = dx, src->linesize[1] = %d, src->linesize[2] = %d\n", src->linesize[0], src->linesize[1], src->linesize[2]);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame dst->data[0] = %x, dst->data[1] = %x, dst->data[2] = %x\n", dst->data[0], dst->data[1], dst->data[2]);
fprintf(stderr, "==============>>>>>> guac_video_prepare_frame dst->linezie[0] = %d, dst->linesize[1] = %d, dst->linesize[2] = %d\n", dst->linesize[0], dst->linesize[1], dst->linesize[2]);
    //fprintf(stderr, "==============>>>>>> sws_scale job done\n");

    /* Free scaling context */
    sws_freeContext(sws);

    /* Free source frame */
    av_freep(&src->data[0]);
    av_frame_free(&src);

    guac_video_flush_frame(video);

    guac_protocol_send_sync(video->socket, video->last_timestamp, 1);
    guac_socket_flush(video->socket);

}

int guac_video_free(guac_video* video) {
        /* Ignore NULL video */
    if (video == NULL)
        return 0;

    /* Write final frame */
    guac_video_flush_frame(video);

    /* Init video packet for final flush of encoded data */
    AVPacket packet;
    av_init_packet(&packet);

    /* Flush any unwritten frames */
    int retval;
    do {
        retval = guac_video_write_frame(video, NULL);
    } while (retval > 0);

    /* Video is now completely written */
    guac_protocol_send_end(video->socket, video->stream);

    /* Free frame encoding data */
    av_freep(&video->next_frame->data[0]);
    av_frame_free(&video->next_frame);

    /* Clean up encoding context */
    avcodec_close(video->context);
    avcodec_free_context(&(video->context));

    free(video);
    return 0;

}


crashoverride
Posts: 5429
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 472 times
Contact:

Re: ffmpeg libav

Post by crashoverride »

smaram wrote:
Tue Sep 28, 2021 6:30 pm
looks like frame->buf->data has no usefull data but we have sizeof frame->buf = 0x20
The sizeof function will return the type size. Since "frame->buf" appears to be an array, it is likely not returning the data (type size) that you are interested in (buffer size). Its really hard to tell the intent from the limited code provide, but it appears to be a coding error.

Other parts of the code posted use:

Code: Select all

frame->buf[i]->size

smaram
Posts: 4
Joined: Tue Sep 28, 2021 6:00 pm
languages_spoken: english, german, spanish
ODROIDs: XU4, C4
Has thanked: 0
Been thanked: 1 time
Contact:

Re: ffmpeg libav

Post by smaram »

i dont know exactly why but commenting out in video.c:

Code: Select all

    /* Allocate actual backing data for frame */
    /*if (av_image_alloc(frame->data, frame->linesize, frame->width,
                frame->height, frame->format, 32) < 0) {
        fprintf(stderr, "=======================================>>>>>> av_image_alloc fail_frame_data");
        goto fail_frame_data;
    }*/
and doing it by my self.

Code: Select all

    int ret, i , padded_height;
    int align = 32;
    int plane_padding = FFMAX(16 + 16/*STRIDE_ALIGN*/, align);

    if (!frame->linesize[0]) {
        if (align <= 0)
            align = 32; /* STRIDE_ALIGN. Should be av_cpu_max_align() */

            //fwrite(dst->data[0], dst->linesize[0] * height, 1, pfile);
            //fwrite(dst->data[1], dst->linesize[1] * height / 2, 1, pfile);

        for(i=1; i<=align; i+=i) {
            ret = av_image_fill_linesizes(frame->linesize, frame->format, FFALIGN(frame->width, i) );
            if (ret < 0)
                return ret;
            if (!(frame->linesize[0] & (align-1)))
                break;
        }

        for (i = 0; i < 4 && frame->linesize[i]; i++)
            frame->linesize[i] = FFALIGN(frame->linesize[i], align);
    }

    padded_height = FFALIGN(frame->height, 32);
    if ((ret = av_image_fill_pointers(frame->data, frame->format, padded_height, NULL, frame->linesize)) < 0)
        return ret;

    frame->buf[0] = av_buffer_alloc(ret  + 4*plane_padding); 
    if (!frame->buf[0]) {
        ret = AVERROR(ENOMEM);
        goto fail_frame_data;
    }

    frame->buf[1] = av_buffer_alloc(ret  + 4*plane_padding );
    if (!frame->buf[1]) {
        ret = AVERROR(ENOMEM);
        goto fail_frame_data;
    }

    frame->data[0] = frame->buf[0]->data;
    frame->data[1] = frame->buf[1]->data;
    frame->data[2] = frame->data[1] + ((frame->width * padded_height) / 4);
solved my problem so far, just performance is not that good with -> 8-10fps, but one step further.

going to optimize now. :shock:
These users thanked the author smaram for the post:
odroid (Thu Sep 30, 2021 9:28 am)

smaram
Posts: 4
Joined: Tue Sep 28, 2021 6:00 pm
languages_spoken: english, german, spanish
ODROIDs: XU4, C4
Has thanked: 0
Been thanked: 1 time
Contact:

Re: ffmpeg libav

Post by smaram »

45fps and still tuning.

Image

User avatar
odroid
Site Admin
Posts: 38170
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2051 times
Been thanked: 1222 times
Contact:

Re: ffmpeg libav

Post by odroid »

Interesting progress.

BTW, is there any specific reason to use Ubuntu 18.04+Kernel 4.14 instead of 20.04+Kernel 5.4 image?
18.04 has better performance?

smaram
Posts: 4
Joined: Tue Sep 28, 2021 6:00 pm
languages_spoken: english, german, spanish
ODROIDs: XU4, C4
Has thanked: 0
Been thanked: 1 time
Contact:

Re: ffmpeg libav

Post by smaram »

indeed ubuntu 20.04 kernel 5.4 is slower compared to 18.04 4.14 i didnt look exactly why, but the same project gets half of the fps.

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 1 guest