JPEG Hardware decoding support

Post Reply
LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

JPEG Hardware decoding support

Unread post by LongChair » Thu May 05, 2016 2:17 pm

Hello,

I wanted to fiddle with Jpeg Hardware decoding as C2 S905 is advertised supporting it.

I have found tracks of this in older kernels, but it seems that in 3.14 kernel it has disapeared by unsetting option CONFIG_AM_JPEGDEC

Trying to set this option leads to :

Code: Select all

drivers/amlogic/amports/jpegdec.c:31:21: fatal error: plat/io.h: No such file or directory
compilation terminated.
Looks like part of the include files have been removed between 3.10 and 3.14.

Are there any plans to restore this fucntionality in 3.14 kernel ? that would be sweet !

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Sat May 07, 2016 3:59 pm

The JPEG option should be available in the Android Kernel.
Please compare the Android Kernel tree.
https://github.com/hardkernel/linux/tre ... .y-android

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Sun May 08, 2016 2:20 am

Well, i am not very familiar with the android tree, but i doubt it would even build it if it's using the same headers :

https://github.com/hardkernel/linux/blo ... /jpegdec.h

as you can see here, it wouldn't build because this enum :

https://github.com/hardkernel/linux/blo ... gdec.h#L45

is used with the wrong name like 10 lines below :

https://github.com/hardkernel/linux/blo ... gdec.h#L80

If it's using different source files & headers somewhere else in that tree, could you let me know where ? :)

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Tue May 10, 2016 8:11 pm

I checked again the Android tree and it looks exactly the same as the linux one regarding that feature.
I don't have an android image to test with, but i'm pretty sure that android probably doesn't have the ability to decode jpegs either.

@odroid : i there any chance you could ask amlogic if they plan to reintroduce support for jpeg harware decoding ?

Thanks !

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Wed May 11, 2016 9:03 am

I think someone should look into the Android media framework to check how it uses the JPEG hw accelerator since they already claimed the Android platform uses the JPEG decoder.

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Wed May 11, 2016 6:46 pm

Where is that framework source available ?

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Wed May 11, 2016 7:12 pm

http://odroid.com/dokuwiki/doku.php?id= ... ng_android

We will check the directory name of the media framework tomorrow.

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Thu May 12, 2016 12:53 am

@odroid : i grabbed the whole android repo as per you link .... and it's huge (like 23 gigs).

I digged around in the vendor/amlogic/framework/av and other directories ... and there is no track of any use of jpeg harware decoder.
So unless i really missed it i kinda doubt android uses it either.

If you find any track of it, pointers woudl be appreciated :)

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Thu May 12, 2016 12:06 pm

C2 Android uses the hardware accelerator in the Camera framework for the webcam MJPEG video input processing.
Check below two directories.
hardware/amlogic/camera/jpegenc_hw
hardware/amlogic/camera/mjpeg

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Sun May 15, 2016 1:48 am

That is true, i had seen that, but what i need is jpeg decoding .. not encoding and that seems to be a quite different process if i look the encoder / decoder process.

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Sun May 15, 2016 10:10 pm

After hours of fiddling around, i got the jpegdec to build again in kernel.

I had to make some assumptions on registers values and actually dig them from older versions
Typically

Code: Select all

define PSCALE_RBUF_START_BLKX	0x0925 
#define PSCALE_RBUF_START_BLKY	0x0926 
#define PSCALE_CANVAS_RD_ADDR		0x092c 
#define PSCALE_CANVAS_WR_ADDR		0x092d 
These are not defined anymore in the 3.14 kernel. Some other regs could be found in the dvb registers.

I was able to start the decoding process, as kicking a jpeg data buffer seems to start partly the decoding process (microprogram will read correctly the jpeg header at least).

I'm getting that kind of log in kernel :

Code: Select all

[   59.312647] codec:vdec_clock_set-----2
[   59.312736] codec:vdec1_set_clk 0, 1
[   59.312799] codec:vdec_clock_set 2 to 500
[   59.313034] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[   59.313155] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[   59.313304] codec:vdec base memory alloced 000000005f200000
[   59.314200] codec:video first pts = 0
[   59.315633] codec:vdec_clock_set-----2
[   59.315728] codec:vdec1_set_clk 0, 1
[   59.315795] codec:vdec_clock_set 2 to 500
[   59.315906] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[   59.316035] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[   59.316234] codec:vdec base memory alloced 000000005f200000
[   59.317181] codec:video first pts = 0
[   59.385656] codec:loading firmware jpegdec_mc to vdec ram  ok!
[   59.386673] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFOCONFIG
[   59.386820] codec:7amjpegdec: ucode report picture size 1920x1080, comp_num 3
[   59.387103] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFO, 1920x1080
[   59.387381] codec:7amjpegdec: amjpegdec_ioctl:config,target (0-0-1920-1080)
[   59.387504] codec:7amjpegdec: planes (0x0-0x0-0x0), pbufAddr=0x5f200000
[   59.387692] codec:7amjpegdec: Enable decoding
[   60.974422] switch_vpu_mem_pd: viu_vd1 OFF
[   60.974499] switch_vpu_mem_pd: afbc_dec OFF
[   60.974566] switch_vpu_mem_pd: di_post OFF

But it will never complete. When i look into the jpegdec code, it seems that after reading the size header, i should get another interrupt request to notify that the decoding is complete, but i never get it.

This is getting me exhausted as we don't have any info, and or doc / samples / datasheet to work with ...

@odroid : is there any chance you could ping amlogic and see if they plan to fix that advertised feature ? That would be much appreciated.

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Mon May 16, 2016 12:14 pm

I think the MJPEG should be identical to the JPEG.
So the "hardware/amlogic/camera/mjpeg" is still utilizing the hardware JPEG decoder driver probably.
Did you check the Android Kernel .config file?

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Mon May 16, 2016 2:23 pm

yes the CONFIG_AM_JPEGDEC doesn't seem to be enabled on android either.

regarding the mjpegdec, do you mean it's able to decode jpeg as well or just that it should more or less use the jpeg block in the same way ?

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Mon May 16, 2016 3:56 pm

Multiple JPEG stream is MJPEG as far as I understood.
If we didn't enable the MJPEG option in Android build configuration, the preview function of camera app was extremely slow because our 720p webcam output is MJPEG.

LongChair
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2
Contact:

Re: JPEG Hardware decoding support

Unread post by LongChair » Tue May 17, 2016 2:56 pm

@odroid : i ended up fixing the jpegdec module and it seems to work according to the dmesg log.

Code: Select all

[ 1008.846516] codec:vdec_clock_set-----2
[ 1008.850350] codec:vdec1_set_clk 0, 1
[ 1008.854175] codec:vdec_clock_set 2 to 500
[ 1008.858072] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[ 1008.861928] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[ 1008.865705] codec:vdec base memory alloced 000000005f200000
[ 1008.869805] codec:BufSize =33554432
[ 1008.873708] codec:physicall address = 405f200000
[ 1008.877908] codec:video first pts = 0
[ 1008.882074] codec:enabling amvdec
[ 1008.882187] codec:loading firmware jpegdec_mc to vdec ram  ok!
[ 1008.889551] codec:jpegdec ucode loaded
[ 1008.915496] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFOCONFIG
[ 1008.919371] codec:7amjpegdec: ucode report picture size 1280x1024, comp_num 3
[ 1008.923770] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFO, 1280x1024
[ 1008.927774] codec:7amjpegdec: amjpegdec_ioctl:config,target (0-0-1280-1024)
[ 1008.931569] codec:7amjpegdec: planes (0x0-0x0-0x0), pbufAddr=0x5f200000
[ 1008.935372] codec:7amjpegdec: Enable decoding
[ 1008.939972] codec:7amjpegdec: ucode report decoding finished
I have tried to time the decoding and should it work, it looks really fast.
Decoding /storage/car_720p.jpg...
OPEN es DEVICE
codec_init amstream version : 2.0
Decoding took 1.711 ms for a 1280x720 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/tiger.jpg...
OPEN es DEVICE
Decoding took 1.381 ms for a 1600x900 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/tiger_fhd.jpg...
OPEN es DEVICE
Decoding took 1.488 ms for a 1920x1080 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/ferarri_4k.jpg...
OPEN es DEVICE
Decoding took 1.359 ms for a 3840x2160 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/F1.jpg...
OPEN es DEVICE
Decoding took 1.320 ms for a 1280x1024 image
[codec_close_cntl]video codec close return=0!
total operation times including loading : 21.177 ms
Now that it seems to work, i want to grab the output data. I might need some clarification on how this works.
When looking at the code here https://github.com/hardkernel/linux/blo ... dec.c#L302

it seems that the input data is grabbed at pBufAddr, am i correct ?
this address is logged in the kernel log as 0x5f200000.

I wanted to doublecheck that i was right, using mmap and mapping that address on user space pointer, but when I dump that area, i don't fidn the jpeg header there.

@odroid : could you tell me if i'm right assuming that pBufAddr is the input buffer address ?

tolszak
Posts: 35
Joined: Fri Dec 30, 2016 6:24 pm
languages_spoken: english
ODROIDs: C2
Contact:

Re: JPEG Hardware decoding support

Unread post by tolszak » Tue May 09, 2017 6:45 pm

Has anyone found how to decode jpegs using HW decoder?

User avatar
odroid
Site Admin
Posts: 29750
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID
Contact:

Re: JPEG Hardware decoding support

Unread post by odroid » Tue May 09, 2017 7:41 pm

HW Encoder seems to be working.
viewtopic.php?f=135&t=26642

But we have no idea about the HW decoder. :(

Ely
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english
Contact:

Re: JPEG Hardware decoding support

Unread post by Ely » Mon Aug 13, 2018 11:37 am

Digging this thread out to add some info: I got the mjpeg/jpeg decoder working in my V4L2 M2M driver (https://github.com/Elyotna/linux). There's a catch though (that was here on AML 3.14 as well) : it only supports decoding YUV 4:2:0 jpeg. So if your input is 4:2:2 or 4:4:4 MJPEG/JPEG, you're screwed.

Post Reply

Return to “Hardware and peripherals”

Who is online

Users browsing this forum: No registered users and 0 guests