Page 1 of 1

New mkfw implementation

Posted: Wed Feb 26, 2020 7:38 pm
by KorKozan
Hey guys,
I've reimplemented the mkfw tool over the weekend, adding quite a few new features, fixes and improvements:
  • help/documentation
  • parameter validation
  • error checking and error reporting via messages and return codes
  • symbolic names for some of the parameters (auto, app, data, ota_, ...)
  • numeric values can be specified in either decimal notation or hex notation
  • size values can be specified using k/K or m/M suffixes to denote KB or MB
  • automatically calculated partition sizes based on the actual sizes
  • quiet mode
  • versioning
  • no memory allocations and proper cleanup
  • fixed potential endianness problems on big endian hosts
  • proper Makefile, with release and debug targets
  • dual license: Public Domain and Zero Clause BSD (PD-like)
https://github.com/KorKozan/odroid-go-f ... tools/mkfw

A pull request has also been sent to upstream.
Enjoy!

Re: New mkfw implementation

Posted: Tue Mar 03, 2020 7:02 am
by ducalex
That is quite an improvement, thanks for the work!

I think it would be great to have PNG (or BMP or GIF or whatever is easier) support for the tile, RAW565 is just annoying.

Re: New mkfw implementation

Posted: Tue Mar 03, 2020 7:30 pm
by KorKozan
ducalex wrote:
Tue Mar 03, 2020 7:02 am
That is quite an improvement, thanks for the work!

I think it would be great to have PNG (or BMP or GIF or whatever is easier) support for the tile, RAW565 is just annoying.
Thanks! My ultimate goal is producing a launcher that works flawlessly on the device, but in the meantime the original mkfw was getting in my way so I channelled my inner NIH spirit and rewrote it 8-) . While I don't plan on making mkfw my life's masterpiece, I'm ok with adding the occasional feature and improvement when it makes sense.
Which brings us to your second point: support for PNG totally makes sense to me, so I'll look into it. For now I'll just perform the PNG -> RAW656 conversion on the fly, but I'm thinking it would also make sense to be able to decode PNG on the device (ie. in the bootloader/flasher as well as the launcher).

Re: New mkfw implementation

Posted: Tue Mar 03, 2020 7:46 pm
by KorKozan
One thing I forgot in my original post: here's an example on how things can improve with the new implementation, taken from RetroESP32:
  • before:

    Code: Select all

    $MKFW_PATH/mkfw "${NAME}" $FIRMWARE_PATH/Assets/${TILE}.raw \
    0 16 524288 ${BIN} $FIRMWARE_PATH/Bins/${BIN}.bin \
    0 17 786432 nesemu-go $FIRMWARE_PATH/Bins/nesemu-go.bin \
    0 18 720896 gnuboy-go $FIRMWARE_PATH/Bins/gnuboy-go.bin \
    0 19 1441792 smsplusgx-go $FIRMWARE_PATH/Bins/smsplusgx-go.bin \
    0 20 524288 spectrum $FIRMWARE_PATH/Bins/spectrum.bin \
    0 21 1703936 stella-go $FIRMWARE_PATH/Bins/stella-go.bin \
    0 22 786432 prosystem-go $FIRMWARE_PATH/Bins/prosystem-go.bin \
    0 23 1507328 handy-go $FIRMWARE_PATH/Bins/handy-go.bin \
    0 24 720896 pcengine-go $FIRMWARE_PATH/Bins/pcengine-go.bin
    
    mv firmware.fw "$FIRMWARE_PATH/Releases/${NAME}.fw"
    
  • after:

    Code: Select all

    "${MKFW_PATH}/mkfw" -o "${FIRMWARE_PATH}/Releases/${NAME}.fw" "${NAME}" "${FIRMWARE_PATH}/Assets/${TILE}.raw" \
    app ota_0 auto "${BIN}"     "${FIRMWARE_PATH}/Bins/${BIN}.bin" \
    app ota_1 auto nesemu-go    "${FIRMWARE_PATH}/Bins/nesemu-go.bin" \
    app ota_2 auto gnuboy-go    "${FIRMWARE_PATH}/Bins/gnuboy-go.bin" \
    app ota_3 auto smsplusgx-go "${FIRMWARE_PATH}/Bins/smsplusgx-go.bin" \
    app ota_4 auto spectrum     "${FIRMWARE_PATH}/Bins/spectrum.bin" \
    app ota_5 auto stella-go    "${FIRMWARE_PATH}/Bins/stella-go.bin" \
    app ota_6 auto prosystem-go "${FIRMWARE_PATH}/Bins/prosystem-go.bin" \
    app ota_7 auto handy-go     "${FIRMWARE_PATH}/Bins/handy-go.bin" \
    app ota_8 auto pcengine-go  "${FIRMWARE_PATH}/Bins/pcengine-go.bin"
    
Features used in the after example:
  • -o to specify the output path/filename, so no mv needed afterwards
  • app instead of 0 for the partition type
  • ota_1 to ota_8 instead of 16 to 24 for the app partition subtypes
  • auto to automagically calculate the partition sizes based on the sizes of the respective partition .bin
  • everything is validated under the hood and both error messages and error codes are emitted if anything goes wrong. This can be handy if mkfw is used as part of a build process (ie. make)
  • the original mkfw syntax is fully supported (with the added benefit of validation and error reporting) and you can actually mix and match the old and the new style, so transitioning can be done incrementally

Re: New mkfw implementation

Posted: Wed Mar 04, 2020 6:38 am
by ducalex
KorKozan wrote:
Tue Mar 03, 2020 7:30 pm
For now I'll just perform the PNG -> RAW656 conversion on the fly, but I'm thinking it would also make sense to be able to decode PNG on the device (ie. in the bootloader/flasher as well as the launcher).
I think it's a bit too late in the Odroid GO's life to revamp the .fw format unfortunately. I'd be happy to add support for your .fw improvements to my multi-firmware bootloader for example, but I'd still distribute my projects using the old .fw format to reach a greater audience. So my vote goes to simple conversion.

However I think the Retro-ESP32 project is young enough to adopt and benefit from an improved build and packaging system!

Re: New mkfw implementation

Posted: Wed Mar 04, 2020 5:40 pm
by KorKozan
ducalex wrote:
Wed Mar 04, 2020 6:38 am

I think it's a bit too late in the Odroid GO's life to revamp the .fw format unfortunately. I'd be happy to add support for your .fw improvements to my multi-firmware bootloader for example, but I'd still distribute my projects using the old .fw format to reach a greater audience. So my vote goes to simple conversion.

However I think the Retro-ESP32 project is young enough to adopt and benefit from an improved build and packaging system!
I just pushed png support to my repo. Not all png formats are supported (ie. the interlaced and palette-based formats are currently unsupported), but I'll add support for the palette-based ones later.
In doing this I also became more conservative wrt adding png support to the device (ie. bootloader/apps), mostly because the png decoder is a pig, and all the format conversions are a pain in the ass to handle. In the end, the benefits of using png on the device are offset by the size of the decoder as well as the memory and cpu requirements. My experience so far is that RAW565 is both more compact and easier to handle than png.
Enjoy!