JPEG Hardware decoding support

Moderators: odroid, mdrjr

JPEG Hardware decoding support

Unread postby LongChair » Thu May 05, 2016 2:17 pm

Hello,

I wanted to fiddle with Jpeg Hardware decoding as C2 S905 is advertised supporting it.

I have found tracks of this in older kernels, but it seems that in 3.14 kernel it has disapeared by unsetting option CONFIG_AM_JPEGDEC

Trying to set this option leads to :

Code: Select all
drivers/amlogic/amports/jpegdec.c:31:21: fatal error: plat/io.h: No such file or directory
compilation terminated.


Looks like part of the include files have been removed between 3.10 and 3.14.

Are there any plans to restore this fucntionality in 3.14 kernel ? that would be sweet !
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Sat May 07, 2016 3:59 pm

The JPEG option should be available in the Android Kernel.
Please compare the Android Kernel tree.
https://github.com/hardkernel/linux/tre ... .y-android
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Sun May 08, 2016 2:20 am

Well, i am not very familiar with the android tree, but i doubt it would even build it if it's using the same headers :

https://github.com/hardkernel/linux/blo ... /jpegdec.h

as you can see here, it wouldn't build because this enum :

https://github.com/hardkernel/linux/blo ... gdec.h#L45

is used with the wrong name like 10 lines below :

https://github.com/hardkernel/linux/blo ... gdec.h#L80

If it's using different source files & headers somewhere else in that tree, could you let me know where ? :)
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby LongChair » Tue May 10, 2016 8:11 pm

I checked again the Android tree and it looks exactly the same as the linux one regarding that feature.
I don't have an android image to test with, but i'm pretty sure that android probably doesn't have the ability to decode jpegs either.

@odroid : i there any chance you could ask amlogic if they plan to reintroduce support for jpeg harware decoding ?

Thanks !
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Wed May 11, 2016 9:03 am

I think someone should look into the Android media framework to check how it uses the JPEG hw accelerator since they already claimed the Android platform uses the JPEG decoder.
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Wed May 11, 2016 6:46 pm

Where is that framework source available ?
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Wed May 11, 2016 7:12 pm

http://odroid.com/dokuwiki/doku.php?id= ... ng_android

We will check the directory name of the media framework tomorrow.
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Thu May 12, 2016 12:53 am

@odroid : i grabbed the whole android repo as per you link .... and it's huge (like 23 gigs).

I digged around in the vendor/amlogic/framework/av and other directories ... and there is no track of any use of jpeg harware decoder.
So unless i really missed it i kinda doubt android uses it either.

If you find any track of it, pointers woudl be appreciated :)
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Thu May 12, 2016 12:06 pm

C2 Android uses the hardware accelerator in the Camera framework for the webcam MJPEG video input processing.
Check below two directories.
hardware/amlogic/camera/jpegenc_hw
hardware/amlogic/camera/mjpeg
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Sun May 15, 2016 1:48 am

That is true, i had seen that, but what i need is jpeg decoding .. not encoding and that seems to be a quite different process if i look the encoder / decoder process.
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby LongChair » Sun May 15, 2016 10:10 pm

After hours of fiddling around, i got the jpegdec to build again in kernel.

I had to make some assumptions on registers values and actually dig them from older versions
Typically

Code: Select all
define PSCALE_RBUF_START_BLKX   0x0925
#define PSCALE_RBUF_START_BLKY   0x0926
#define PSCALE_CANVAS_RD_ADDR      0x092c
#define PSCALE_CANVAS_WR_ADDR      0x092d


These are not defined anymore in the 3.14 kernel. Some other regs could be found in the dvb registers.

I was able to start the decoding process, as kicking a jpeg data buffer seems to start partly the decoding process (microprogram will read correctly the jpeg header at least).

I'm getting that kind of log in kernel :

Code: Select all
[   59.312647] codec:vdec_clock_set-----2
[   59.312736] codec:vdec1_set_clk 0, 1
[   59.312799] codec:vdec_clock_set 2 to 500
[   59.313034] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[   59.313155] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[   59.313304] codec:vdec base memory alloced 000000005f200000
[   59.314200] codec:video first pts = 0
[   59.315633] codec:vdec_clock_set-----2
[   59.315728] codec:vdec1_set_clk 0, 1
[   59.315795] codec:vdec_clock_set 2 to 500
[   59.315906] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[   59.316035] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[   59.316234] codec:vdec base memory alloced 000000005f200000
[   59.317181] codec:video first pts = 0
[   59.385656] codec:loading firmware jpegdec_mc to vdec ram  ok!
[   59.386673] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFOCONFIG
[   59.386820] codec:7amjpegdec: ucode report picture size 1920x1080, comp_num 3
[   59.387103] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFO, 1920x1080
[   59.387381] codec:7amjpegdec: amjpegdec_ioctl:config,target (0-0-1920-1080)
[   59.387504] codec:7amjpegdec: planes (0x0-0x0-0x0), pbufAddr=0x5f200000
[   59.387692] codec:7amjpegdec: Enable decoding
[   60.974422] switch_vpu_mem_pd: viu_vd1 OFF
[   60.974499] switch_vpu_mem_pd: afbc_dec OFF
[   60.974566] switch_vpu_mem_pd: di_post OFF



But it will never complete. When i look into the jpegdec code, it seems that after reading the size header, i should get another interrupt request to notify that the decoding is complete, but i never get it.

This is getting me exhausted as we don't have any info, and or doc / samples / datasheet to work with ...

@odroid : is there any chance you could ping amlogic and see if they plan to fix that advertised feature ? That would be much appreciated.
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Mon May 16, 2016 12:14 pm

I think the MJPEG should be identical to the JPEG.
So the "hardware/amlogic/camera/mjpeg" is still utilizing the hardware JPEG decoder driver probably.
Did you check the Android Kernel .config file?
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Mon May 16, 2016 2:23 pm

yes the CONFIG_AM_JPEGDEC doesn't seem to be enabled on android either.

regarding the mjpegdec, do you mean it's able to decode jpeg as well or just that it should more or less use the jpeg block in the same way ?
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby odroid » Mon May 16, 2016 3:56 pm

Multiple JPEG stream is MJPEG as far as I understood.
If we didn't enable the MJPEG option in Android build configuration, the preview function of camera app was extremely slow because our 720p webcam output is MJPEG.
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby LongChair » Tue May 17, 2016 2:56 pm

@odroid : i ended up fixing the jpegdec module and it seems to work according to the dmesg log.

Code: Select all
[ 1008.846516] codec:vdec_clock_set-----2
[ 1008.850350] codec:vdec1_set_clk 0, 1
[ 1008.854175] codec:vdec_clock_set 2 to 500
[ 1008.858072] codec:Video stbuf alloced at 000000005e300000, size = 15728640
[ 1008.861928] codec:vdec_dev_reg.mem[0x0 -- 0x0]
[ 1008.865705] codec:vdec base memory alloced 000000005f200000
[ 1008.869805] codec:BufSize =33554432
[ 1008.873708] codec:physicall address = 405f200000
[ 1008.877908] codec:video first pts = 0
[ 1008.882074] codec:enabling amvdec
[ 1008.882187] codec:loading firmware jpegdec_mc to vdec ram  ok!
[ 1008.889551] codec:jpegdec ucode loaded
[ 1008.915496] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFOCONFIG
[ 1008.919371] codec:7amjpegdec: ucode report picture size 1280x1024, comp_num 3
[ 1008.923770] codec:7amjpegdec: amjpegdec_ioctl:  JPEGDEC_IOC_INFO, 1280x1024
[ 1008.927774] codec:7amjpegdec: amjpegdec_ioctl:config,target (0-0-1280-1024)
[ 1008.931569] codec:7amjpegdec: planes (0x0-0x0-0x0), pbufAddr=0x5f200000
[ 1008.935372] codec:7amjpegdec: Enable decoding
[ 1008.939972] codec:7amjpegdec: ucode report decoding finished


I have tried to time the decoding and should it work, it looks really fast.

Decoding /storage/car_720p.jpg...
OPEN es DEVICE
codec_init amstream version : 2.0
Decoding took 1.711 ms for a 1280x720 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/tiger.jpg...
OPEN es DEVICE
Decoding took 1.381 ms for a 1600x900 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/tiger_fhd.jpg...
OPEN es DEVICE
Decoding took 1.488 ms for a 1920x1080 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/ferarri_4k.jpg...
OPEN es DEVICE
Decoding took 1.359 ms for a 3840x2160 image
[codec_close_cntl]video codec close return=0!
Decoding /storage/F1.jpg...
OPEN es DEVICE
Decoding took 1.320 ms for a 1280x1024 image
[codec_close_cntl]video codec close return=0!
total operation times including loading : 21.177 ms


Now that it seems to work, i want to grab the output data. I might need some clarification on how this works.
When looking at the code here https://github.com/hardkernel/linux/blo ... dec.c#L302

it seems that the input data is grabbed at pBufAddr, am i correct ?
this address is logged in the kernel log as 0x5f200000.

I wanted to doublecheck that i was right, using mmap and mapping that address on user space pointer, but when I dump that area, i don't fidn the jpeg header there.

@odroid : could you tell me if i'm right assuming that pBufAddr is the input buffer address ?
LongChair
 
Posts: 153
Joined: Mon Mar 28, 2016 2:32 pm
languages_spoken: english
ODROIDs: ODROID C2

Re: JPEG Hardware decoding support

Unread postby tolszak » Tue May 09, 2017 6:45 pm

Has anyone found how to decode jpegs using HW decoder?
tolszak
 
Posts: 35
Joined: Fri Dec 30, 2016 6:24 pm
languages_spoken: english
ODROIDs: C2

Re: JPEG Hardware decoding support

Unread postby odroid » Tue May 09, 2017 7:41 pm

HW Encoder seems to be working.
viewtopic.php?f=135&t=26642

But we have no idea about the HW decoder. :(
User avatar
odroid
Site Admin
 
Posts: 29098
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: JPEG Hardware decoding support

Unread postby Ely » Mon Aug 13, 2018 11:37 am

Digging this thread out to add some info: I got the mjpeg/jpeg decoder working in my V4L2 M2M driver (https://github.com/Elyotna/linux). There's a catch though (that was here on AML 3.14 as well) : it only supports decoding YUV 4:2:0 jpeg. So if your input is 4:2:2 or 4:4:4 MJPEG/JPEG, you're screwed.
Ely
 
Posts: 39
Joined: Sat Jan 31, 2015 9:27 pm
languages_spoken: english


Return to Hardware and peripherals

Who is online

Users browsing this forum: No registered users and 7 guests