I am trying to integrate this hardware encoder into mediastreamer (which is the media framework of linphone). MediaStreamer already has 2 filters for software H264 encoding : one based on libX264 and one based on OpenH264. I just took one of them, deleted all software encoding stuff and called the Amlogic hardware encoder by using the library from :
The ultimate goal is to have linphone integrated into the box (libreelec), so that I can start video conferences between the box and any other SIP phone. Everything seems to be ok. The encoding starts, it generates a H264 bitstream. I am continuously encoding the same image, just for testing :
I am absolutely sure that the H264 bitstream from the hardware encoder is good. I converted it to MOV with ffmpeg and the result is perfect.
The problem is with the RTP transmission. The bitstream has to be split into NAL units. MediaStreamer already has a function for that.
Code: Select all
int outCnt = vl_video_encoder_encode(handle, type, in, in_size, &out); // perform hardware encoding (i am sure that "out" contains valid H264 data)
ms_h264_bitstream_to_nalus((unsigned char*)out, outCnt, &nalus); // split into NAL units and store in a queue
rfc3984_pack(packetizer, &nalus, f->outputs, timestamp); // pack
But the produced NALs are too big for the max UDP MTU size. The RFC3984 packetizer included in MediaStreamer supports mode 0 (no split) and mode 1 (split).
When I use mode 0, it gives warnings about the size of NALs. This is normal, since my MTU is about 1300 and the NAL sizes are about 18 KB to 22 KB for I-Frames, and something like 3 or 4 KB for the others frames. Playing at the other end with a softphone gives a strange result. The top half of the image is ok. The bottom one is corrupt :
(please note that the image is flipped down and the color are interpreted as RGB instead of BGR but this is not a big deal, I just have to adjust my RGB->NV12 conversion code).
When I use mode 1, I have a black screen.
I am testing on a LAN, there is only one router/hub between the box and the softphone. I tried several softphones at the receiver end :
With eyebeam, I have the half-frame corruption problem
With jitsi, no video at all (just black screen)
With linphone, no video at all (just black screen)
Do you think my problem is due to fragmentation ? Maybe the softphones I am using do not support packet-mode 1.
Is it possible to tell the hardware encoder what is the max NAL unit size ? That way I could give him the size of my MTU (which is I thing 1300) and force him to give me small NAL units.
I really don't understand why the bottom half of the movie is moving from left to right. It repeats over and over after 10 frames. 10 is the GOP size that I used for encoding. The input image is always the same (I am continuously encoding the same picture), so the output should be a fixed image...