Odroid N1 SPI Boot flash

Moderators: odroid, mdrjr

Odroid N1 SPI Boot flash

Unread postby mad_ady » Fri Feb 09, 2018 4:33 pm

I'd like to start this thread to discuss more about the SPI Boot flash that @odroid said the N1 would get.
@odroid: please move this to the N1 forum thread once it's created.

To recap what was discussed on the N1 announcement thread: viewtopic.php?f=29&t=29932&p=213491
The N1 can boot from SPI, eMMC, sd and USB - according to @odroid.
It also supports Orange/Black/Green/Red/Blue eMMCs.

Regarding SPI flash - it will likely be a 16MB flash and it will probably be onboard.
There was some discussion about using a multiplexer and selecting SPI flash at boot and disabling it afterwards so that other SPI devices would work. This needs to be reversible - e.g. to have the ability to disable SPI at boot (either via a jumper or a button press) to avoid getting it bricked in case of a bad flash. Also, having a boot switch like the xu4 might be useful (e.g. so that one can reflash a broken emmc in the N1 without the need of a emmc adapter). There also needs to be a way to access/change files from the OS for backup/restore.

Regarding what the SPI flash should contain - it has to have uboot and maybe there's enough room for a kernel/initrd? It should have IMHO a minimal kernel for recovery that is triggered when a button is kept pressed during boot - so that you can boot a known kernel in case you flash a broken kernel. @crashoverride suggested having a minimal kernel which could access the SATA ports and enable booting from SATA via kexec.

Any other ideas I've missed?
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby odroid » Fri Feb 09, 2018 4:53 pm

Please hold on.
We will make a sub-forum early next week to start the N1 Debug Party.
User avatar
odroid
Site Admin
 
Posts: 27688
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Fri Feb 09, 2018 5:10 pm

The SPI flash also needs a way "disable it" so that eMMC, SD, or USB boot can be used for recovery. Yes, I said USB boot!. The RK3399 supports USB boot and enough public information is provided to support it. One of the USB2 ports on N1 likely exposes this, and it may be possible to use with a USB-A to USB-A cable (like Rock64 does).

Ideally, the contents of the SPI flash should only contain the Rockchip pre-boot and u-boot. Kernels and associated ram disks should NOT be placed there. Instead, they should reside on what ever boot medium is being used. A minimal kernel may be placed in SPI flash provided there is enough space (16MB). However, its only job should be similar to what a BIOS/UEFI provides. It will NOT be the kernel that runs the system, rather it will chain load the kernel that will run the system. This would be similar to how NOOBS on RPi works.

TL;DR = The SPI flash is like a BIOS. Its contents should rarely change and its only job is to boot the board, not hold an OS/Kernel.

[edit]
In the context used above, USB boot means a special protocol between RK3399 and a computer ("recovery" mode). It does not mean booting from a USB device such as flash or HDD. The latter functionality would be provded by u-boot or the minimal kernel.

[edit2]
DISCLAIMER: The above is all speculation on what may be possible. It is not a statement of what will be possible. That will not be known until hardware is available and tests have been conducted.
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby tkaiser » Fri Feb 09, 2018 11:44 pm

mad_ady wrote:I'd like to start this thread to discuss more about the SPI Boot flash that @odroid said the N1 would get.


A bit background info (read at least through the slides): https://fosdem.org/2018/schedule/event/ ... _in_linux/

Andre is also member of linux-sunxi community (Allwinner hardware) where we started almost 2 years ago to explore SPI boot capabilities: http://linux-sunxi.org/Bootable_SPI_flash

Some Allwiner board makers followed the request (funnily the first device was the el cheapo $7 Orange Pi Zero!), later TL Lim from Pine Inc. introduced the idea to Rockchip when starting on ROCK64 and bridged Rockchip engineers with community. Andre and the OpenSource team at Rockchip went then into details and RK fully supported the idea. Not it's up to the board makers to make it happen.

As @crashoverride said the idea behind is that the stuff in SPI NOR flash acts just as a regular BIOS so that once there's a bootloader able to initialize the hardware and providing a proper hardware description (device-tree) you should be able to boot simply every OS as long as it comes with a kernel suitable for ARM (aarch64). This does not only apply to Linux but *BSD folks are pretty active here too (or as usual a bit advanced: Jared McNeill did some great job last summer almost nobody noticed: https://www.cnx-software.com/2017/07/11 ... ent-544109 )
tkaiser
 
Posts: 354
Joined: Mon Nov 09, 2015 12:30 am
languages_spoken: english
ODROIDs: C1+, C2, XU4, HC1

Re: Odroid N1 SPI Boot flash

Unread postby elatllat » Mon Feb 12, 2018 11:29 pm

Would be nice to make signing and verification for of the SPI flash easy.
https://youtu.be/Dfl2JI2eLc8
elatllat
 
Posts: 959
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: Odroid N1 SPI Boot flash

Unread postby joy » Tue Feb 13, 2018 5:07 pm

Hi all.
With ODROID-N1, we are checking the possibility of SPI booting and have done a very basic implementation with SPI NOR flash and SATA disk.
So far, only booting scenario is done and there are more jobs remaining to do like update, recovery and so on.
We hope you guys look over it and give us any feedback. :)

Basic concept of SPI Booting

The basic concept of SPI Booting is that boot loaders and kernel are built in SPI flash and rootfs partition in SATA disk is loaded.
Before this idea, we've checked the approach that SPI flash has only boot loaders and SATA disk manages rootfs and kernel-related including boot.ini.
But there is no rockchip PCIe and PHY driver in u-boot yet, so we tried the current logic.

Booting Flow
Here is the booting flow based on the concept.
    1. RK3399 Maskrom (1st loader) tries to find boot storage with the order, SPI → SD → eMMC.
    2. If there is a specific header in 32kB offset of SPI Flash, Maskrom keeps booting from SPI flash and loading and running 2nd boot loader, idbspl.
    3. idbspl loads u-boot.itb and then atf, m0 firmware and u-boot will be executed.
    4. On u-boot stage, kernel image, ramdisk and dtb are loaded from each mtd partition in SPI flash.
    5. In case of kernel, gunziped kernel image is used because of the size issue.
    6. After booting with booti, the rootfs partition specified in bootargs with root and rootfstype is mounted and the node must be one of SATA disk's partitions.
Code: Select all
### kernel load
# sf read $kernel_addr1  kernel
# unzip $kernel_addr1 $kernel_addr2
### ramdisk load
# sf read $ramdisk_addr initrd
### dtb load
# sf read $dtb_addr dtb
### booting
# booti $kernel_addr2 - $dtb_addr


Image Layout of SPI Flash and SATA Disk
Here is a SPI flash layout and an example of SATA disk layout.
1. SPI Flash Layout
As aforementioned, All of boot loaders and kernel, dtb, uInitrd including boot.ini should be built in spi flash.
For current test, boot.ini is excluded but there is a free space to include it.
Image
2. Example of SATA Disk Layout
For this test, we're using /dev/sda2 node from SATA disk as rootfs partition those format is ext4,
and another partition of SATA disk can be assigned by modifying bootargs.
Code: Select all
bootargs=earlyprintk swiotlb=1 console=ttyFIQ0,115200n8 rw root=/dev/sda2 rootfstype=ext4 rootwait


SPI NOR Flash Sample
We use a winbond SPI NOR flash W25Q128JVSIQ for this test.
    128M-bit / 16M-byte
    3V IO
    4KB sectors
    8-pin SOIC type package

Hw Configuration
The group to support RK3399 SPI booting is SPI1
and those ports are exposed as pins of 40pin expansion connector.

The following pins of expansion connector are used as SPI I/O.
Image
Image

Image
Image
Image
Image
Last edited by joy on Wed Feb 14, 2018 10:41 am, edited 2 times in total.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby umiddelb » Tue Feb 13, 2018 6:08 pm

If possible double the u-boot env from 8kB to 16kB please. This would make my life a little bit more easy.
umiddelb
 
Posts: 438
Joined: Thu Jan 29, 2015 6:42 am
languages_spoken: English, German
ODROIDs: ODROID-C1, ODROID-XU4, ODROID-C2

Re: Odroid N1 SPI Boot flash

Unread postby mad_ady » Tue Feb 13, 2018 6:11 pm

It looks great, but for the final implementation I would suggest the following:
1. have an option to disable CS on boot (jumper, shortable pad), so that if there is a broken uboot/kernel/dtb in the boot flash you can bypass it by shorting out a pad on the board on boot so that it skips the SPI.
2. It's good that SD is before emmc in the boot process. This allows you to recover and reflash a broken emmc by booting from SD.
3. Flashing the SPI on each kernel change could cause it to wear sooner than normal. How many write cycles does it have? I think it would be worthwhile to explore crashoverride's suggestion and booting a minimal kernel with kexec support that can load a full kernel from a storage device. @crashoverride: Can you use the same kernel to do kexec of both Android and Linux? This way the kernel in SPI doesn't get updated very often and reduces the write cycles.
4. Regarding boot.ini - it is inconvenient to edit if it's in the SPI flash. Ideally uboot should look for it in SPI, emmc and sdcard in a deterministic order. By default it should be stored with the OS, and for power users who want to use network boot, there should be instructions on how to flash a boot.ini into the SPI. The boot.ini update scripts should not write it to SPI, but maybe have a hook to run a user-script to flash it to SPI after an update.
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby joy » Tue Feb 13, 2018 6:41 pm

umiddelb wrote:If possible double the u-boot env from 8kB to 16kB please. This would make my life a little bit more easy.

Thank you for the feedback.
OK. It's available!
Last edited by joy on Tue Feb 13, 2018 7:06 pm, edited 1 time in total.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby joy » Tue Feb 13, 2018 6:58 pm

mad_ady wrote:It looks great, but for the final implementation I would suggest the following:

Thank you for your feedback, mad_ady. :)
mad_ady wrote:1. have an option to disable CS on boot (jumper, shortable pad), so that if there is a broken uboot/kernel/dtb in the boot flash you can bypass it by shorting out a pad on the board on boot so that it skips the SPI.

You're right. We need the precaution for unexpected cases like broken uboot/kernel/dtb.
So we're considering to add a DIP switch to pull-up CS.
And also it will be available to control boot order in 2nd boot loader if SPI flash is built in N1 board by default.

mad_ady wrote:2. It's good that SD is before emmc in the boot process. This allows you to recover and reflash a broken emmc by booting from SD.

The boot order I mentioned is a fixed one with RK3399 bootrom.
Maskrom of RK3399 scans boot storage in order of SPI -> SD -> eMMC.
But we can adjust the order in 2nd boot loader so it will be possible as you suggest
and I think it's a good suggestion. :)
Also we will try it to control boot order DIP switch control by manual that 2nd boot loader detects the status and run booting flexibly.
mad_ady wrote:4. Regarding boot.ini - it is inconvenient to edit if it's in the SPI flash. Ideally uboot should look for it in SPI, emmc and sdcard in a deterministic order. By default it should be stored with the OS, and for power users who want to use network boot, there should be instructions on how to flash a boot.ini into the SPI. The boot.ini update scripts should not write it to SPI, but maybe have a hook to run a user-script to flash it to SPI after an update.

Thank you for the good idea. It must be inconvenient to modify boot.ini in SPI flash so it's one of main issue we're agonizing over.
We will keep your idea to implement the logic.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby mad_ady » Tue Feb 13, 2018 7:50 pm

mad_ady wrote:
2. It's good that SD is before emmc in the boot process. This allows you to recover and reflash a broken emmc by booting from SD.


The boot order I mentioned is a fixed one with RK3399 bootrom.
Maskrom of RK3399 scans boot storage in order of SPI -> SD -> eMMC.
But we can adjust the order in 2nd boot loader so it will be possible as you suggest
and I think it's a good suggestion. :)
Also we will try it to control boot order DIP switch control by manual that 2nd boot loader detects the status and run booting flexibly.

You know, people will complain about security of a collocated N1 box. An attacker with physical access could use a rogue SD card to hijack the boot process and access data on the emmc. But I think realistically the benefit will be greater than the risk. As long as an attacker has physical access, nothing is safe.
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby elatllat » Tue Feb 13, 2018 8:11 pm

I see the advantage of using SPI is the ability to boot without eemc/sd to ssd so long as config and kernel are [re]loaded from ssd. One could use SPI to decrypt the boot volume to make tampering harder...

Maybe leave SPI zeroed by default to make it simple for everyone else.
elatllat
 
Posts: 959
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Tue Feb 13, 2018 8:55 pm

joy wrote:We use a winbond SPI NOR flash W25Q128JVSIQ for this test.

I just happen to have lots of these laying around along with 8SOIC breakout boards for them. I look forward to testing the SPI boot feature.

mad_ady wrote:An attacker with physical access could use a rogue SD card to hijack the boot process and access data on the emmc.

There is no way to secure any ARM development board from someone with physical access.

[edit]
Also, if I recall correctly, the chip has a 100,000 erase/write cycle life. That means you could update it once every day for almost 274 years (100,000 / 365 = 273.972).
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby willmore » Tue Feb 13, 2018 9:22 pm

crashoverride wrote:
joy wrote:We use a winbond SPI NOR flash W25Q128JVSIQ for this test.
[edit]
Also, if I recall correctly, the chip has a 100,000 erase/write cycle life. That means you could update it once every day for almost 274 years (100,000 / 365 = 273.972).


You're being pessimistic. :) The spec normally goes "Device can tolerate X erase/write cycles and then Y years at Z degrees."

Typical values for X are 10K, 100K, and 1M. Y and Z are normally (25,25) and (10,80) or similar. That can vary depending if the part is rated for automotive, industrial, or consumer temperature ranges.

So, for a part rated for 100K, not only will it survive those 100K E/W cycles, but it'll retain that data for at least a decade at the temps it will experience on an SBC in typical usage.

I wouldn't worry about writing env, kernel, etc. to the SPI NOR flash. I wouldn't put a filesystem on it, though. That's a good way to kill it.

[edit]
Come to think of it, the best arguement to avoid writing to it is that every time you enable writes, you add a small chance that junk will get written to the chip by accident--noise from the environment, etc.--and you'll brick your device. Focus on recovering from those errors and don't worry about the E/W cycle endurance. Just please, don't put a filesystem on it....
willmore
 
Posts: 10
Joined: Mon Aug 29, 2016 2:37 am
languages_spoken: english
ODROIDs: C1 x1
C2 x2

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Tue Feb 13, 2018 9:34 pm

willmore wrote:You're being pessimistic.

I was being hyperbolic. I will not likely be around in 274 years for anyone to prove me wrong! :lol:
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby willmore » Tue Feb 13, 2018 9:54 pm

mad_ady wrote:I'd like to start this thread to discuss more about the SPI Boot flash that @odroid said the N1 would get.
There was some discussion about using a multiplexer and selecting SPI flash at boot and disabling it afterwards so that other SPI devices would work. This needs to be reversible - e.g. to have the ability to disable SPI at boot (either via a jumper or a button press) to avoid getting it bricked in case of a bad flash. Also, having a boot switch like the xu4 might be useful (e.g. so that one can reflash a broken emmc in the N1 without the need of a emmc adapter). There also needs to be a way to access/change files from the OS for backup/restore.


I believe I was the one who brought up the idea of making the one SPI port useable for flash as well as for the 40 pin GPIO header by multiplexing the CS signal. The basic idea is that a GPIO from the SoC will control the select input of a demultiplexer (which will take the SoCs CS signal and route it to either the GPIO header or the SPI NOR flash). This signal will need to be pulled high/low depending on how the board designer decides to wire the demultiplexer.

To address the recovery issue, you could put a jumper on it to pull the select line to the state which selects the GPIO header. This will prevent the SPI NOR from appearing to the SoC and hence, no SPI boot. Once the system comes up to the recovery environment, the jumper could be removed and the SPI NOR selected by the SoC by setting the GPIO line appropriately.

The cost is one 1-2 demultiplexer, one jumper header, and one resistor. I haven't designed a board for a while, so I'm not sure what options exist for this these days. I have to assume there's some little 6 pin SOT23 or smaller device that does this as it only needs Vcc, GND, select, input, and two outputs. Back in the day I would have used a 74HC138--which is good for 8 CS signals. Ask your EE or board design person, they'll know better than I what parts are available these days.

I'd be glad to talk to your designer if there are any questions.
willmore
 
Posts: 10
Joined: Mon Aug 29, 2016 2:37 am
languages_spoken: english
ODROIDs: C1 x1
C2 x2

Re: Odroid N1 SPI Boot flash

Unread postby willmore » Tue Feb 13, 2018 9:59 pm

crashoverride wrote:
willmore wrote:You're being pessimistic.

I was being hyperbolic. I will not likely be around in 274 years for anyone to prove me wrong! :lol:

Back in the day, I wrote a tester for EEPROM endurance. I was testing some 100K endurance parts. I killed them with a test run over a weekend. IIRC, I started seeing bit errors around 140K cycles. And that's with practically zero retention time--write the data, read it back. Lacking a time machine, I didn't have a good way to test the 25 year retention. :)

This is why I caution not to put a filesystem on them. They have no write accumulation, no wear leveing, etc. You can wear one out pretty fast if you try--or don't work to prevent it.
willmore
 
Posts: 10
Joined: Mon Aug 29, 2016 2:37 am
languages_spoken: english
ODROIDs: C1 x1
C2 x2

Re: Odroid N1 SPI Boot flash

Unread postby AreaScout » Wed Feb 14, 2018 2:08 am

crashoverride wrote:
willmore wrote:You're being pessimistic.

I was being hyperbolic. I will not likely be around in 274 years for anyone to prove me wrong! :lol:


@crashoverride lol :lol:

@all

sadly that there is no ISP connector for programming ...
User avatar
AreaScout
 
Posts: 482
Joined: Sun Jul 07, 2013 3:05 am
languages_spoken: english, german
ODROIDs: X2, U3, XU3, C2, XU4Q

Re: Odroid N1 SPI Boot flash

Unread postby moon.linux » Wed Feb 14, 2018 2:56 am

@joy
Thanks for this valuable input. Promotion of Winter Olympics 2018 + Odroid N1. ;)
moon.linux
 
Posts: 1056
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: Odroid N1 SPI Boot flash

Unread postby rooted » Wed Feb 14, 2018 5:27 am

moon.linux wrote:@joy
Thanks for this valuable input. Promotion of Winter Olympics 2018 + Odroid N1. ;)
I caught that as well :)
User avatar
rooted
 
Posts: 4693
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1, C1+, C2
XU3 Lite, XU4
N1
VU7+
HiFi Shield 2
Smart Power (original)

Re: Odroid N1 SPI Boot flash

Unread postby umiddelb » Wed Feb 14, 2018 6:45 am

Usually I wouldn't store anything else than the firmware in flash memory. If you have a special use case like running OpenWRT or a small lightweight container OS (Atomic, CoreOS, ...) the flash memory might be an appropriate place to store kernel and userland (initrd).

If you consider a multiboot environment where you want to switch between different OS, you don't want to spread userland und kernel over different storage entities. You want to keep them in exactly one partition which makes it easier to relocate one OS from eMMC to SATA,USB,... and vice versa.
umiddelb
 
Posts: 438
Joined: Thu Jan 29, 2015 6:42 am
languages_spoken: English, German
ODROIDs: ODROID-C1, ODROID-XU4, ODROID-C2

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Wed Feb 14, 2018 1:26 pm

mad_ady wrote:@crashoverride: Can you use the same kernel to do kexec of both Android and Linux?

No idea since I don't use android. I do not know of anything preventing it if properly configured.

umiddelb wrote:the flash memory might be an appropriate place to store kernel and userland (initrd).

There is a very limited amount of space (16MB, not GB) to store everything.
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby umiddelb » Thu Feb 15, 2018 5:32 am

crashoverride wrote:
umiddelb wrote:the flash memory might be an appropriate place to store kernel and userland (initrd).

There is a very limited amount of space (16MB, not GB) to store everything.

Take a look at this table, OpenWrt supports boards with even 4MB Flash only and the majority is equipped with 16 MB. But I will agree with you that this is not the usual use case for the N1.
umiddelb
 
Posts: 438
Joined: Thu Jan 29, 2015 6:42 am
languages_spoken: English, German
ODROIDs: ODROID-C1, ODROID-XU4, ODROID-C2

Re: Odroid N1 SPI Boot flash

Unread postby umiddelb » Thu Feb 15, 2018 5:59 am

joy wrote:...
mad_ady wrote:4. Regarding boot.ini - it is inconvenient to edit if it's in the SPI flash. Ideally uboot should look for it in SPI, emmc and sdcard in a deterministic order. By default it should be stored with the OS, and for power users who want to use network boot, there should be instructions on how to flash a boot.ini into the SPI. The boot.ini update scripts should not write it to SPI, but maybe have a hook to run a user-script to flash it to SPI after an update.

Thank you for the good idea. It must be inconvenient to modify boot.ini in SPI flash so it's one of main issue we're agonizing over.
We will keep your idea to implement the logic.

In this case the file `boot.ini` has to be placed on SD/eMMC storage in order to be accessible during u-boot.

Did you consider to facilitate something similar to petitboot (kexec based boot loader) for the N1? This might help to overcome the current u-boot limitations.
umiddelb
 
Posts: 438
Joined: Thu Jan 29, 2015 6:42 am
languages_spoken: English, German
ODROIDs: ODROID-C1, ODROID-XU4, ODROID-C2

Re: Odroid N1 SPI Boot flash

Unread postby mad_ady » Fri Feb 16, 2018 5:23 am

I've ordered two spi flash chips from here: https://www.aliexpress.com/snapshot/0.h ... 2834346596, but they are surface mounted. I could solder pins to it and use a breadboard, but I'd like to avoid soldering.

I could use a connector with a spring and pins that is used to attach to the pins of live circuits for flashing bioses and such, but I don't know what it's called. Anyone know what I'm talking about?
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby elatllat » Fri Feb 16, 2018 5:30 am

elatllat
 
Posts: 959
Joined: Tue Sep 01, 2015 8:54 am
languages_spoken: english
ODROIDs: XU4, N1

Re: Odroid N1 SPI Boot flash

Unread postby mad_ady » Fri Feb 16, 2018 3:10 pm

No, that looks expensive. I found what I was looking for: https://m.aliexpress.com/item/32234174753.html
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby joy » Sun Feb 18, 2018 11:46 am

AreaScout wrote:@all
sadly that there is no ISP connector for programming ...

It's available to program on SPI flash through eMMC or SD interface with U-Boot command like fatload and sf commands without additional connector to program serial flash.
    - Booting from eMMC or SD
    - Probe and Erase serial flash
    - Load boot loader and kernel or something for SPI booting
    - Write images to SPI flash
Later, we need to check and establish set-up and update scenario cases with all of bootable storages,
but this can be a brief and base instruction for initial programming on SPI flash.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Sun Feb 18, 2018 12:04 pm

joy wrote:Later, we need to check and establish set-up and update scenario cases with all of bootable storages,

The RK3399 supports USB based boot using a proprietary protocol. Unlike other SoCs, the support for this is open source and publicly available.

The hardware functionality is exposed via a USB-C connector on other boards. This implies one of the USB3 ports on N1 should have the USB2 block lines for booting. Without USB-C or USB-OTG connectors, the hardware will need to support USB-A to USB-A cabling. I have not tried this since I do not want to risk setting my board (or myself) on fire. ;)
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby joy » Sun Feb 18, 2018 12:07 pm

umiddelb wrote:
joy wrote:...
mad_ady wrote:4. Regarding boot.ini - it is inconvenient to edit if it's in the SPI flash. Ideally uboot should look for it in SPI, emmc and sdcard in a deterministic order. By default it should be stored with the OS, and for power users who want to use network boot, there should be instructions on how to flash a boot.ini into the SPI. The boot.ini update scripts should not write it to SPI, but maybe have a hook to run a user-script to flash it to SPI after an update.

Thank you for the good idea. It must be inconvenient to modify boot.ini in SPI flash so it's one of main issue we're agonizing over.
We will keep your idea to implement the logic.

In this case the file `boot.ini` has to be placed on SD/eMMC storage in order to be accessible during u-boot.

Did you consider to facilitate something similar to petitboot (kexec based boot loader) for the N1? This might help to overcome the current u-boot limitations.

Right!
Based on the opinions of you guys, we're considering to use pre-OS that is built-in serial flash and its role will be like a BIOS or pre-process to select bootable storage and load boot.ini and kernel from the selected boot storage.
Rather than the first idea that I mentioned before, this approach must be more expandable for various of user scenario.
No need for frequent update of images on serial flash and easy access and update on each bootable storage.....
Also I will check kexec examples including petitboot to design better booting and updating logic.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Sun Feb 18, 2018 12:37 pm

I keep forgetting to "connect the dots" regarding USB boot. It is possible to "flash" the device from USB. This takes care of the "how to initialize SPI" issue (theoretically; It may take some extra work). You can also use it to flash a blank eMMC (and likely SD card).
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby joy » Sun Feb 18, 2018 2:01 pm

crashoverride wrote:I keep forgetting to "connect the dots" regarding USB boot. It is possible to "flash" the device from USB. This takes care of the "how to initialize SPI" issue (theoretically; It may take some extra work). You can also use it to flash a blank eMMC (and likely SD card).

Yes. It's a good method for the initial program on SPI flash and also eMMC/SD. :)
But as you know, there is no USB-C or OTG connector on N1 (RK3399 supports only USB device booting, not host.)
and a host application that control program protocol via USB must be needed.
Compared with initial program via eMMC/SD with auto-script, it looks harder to implement.

crashoverride wrote:I have not tried this since I do not want to risk setting my board (or myself) on fire. ;)

I also hope you not to try it. :D
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby MimCom » Mon Feb 19, 2018 2:37 am

What about SPI -> PXE/netboot? This would be very helpful for distributed cloud applications (like ceph, mining, etc)?
MimCom
 
Posts: 29
Joined: Sun Mar 12, 2017 3:24 am
languages_spoken: english

Re: Odroid N1 SPI Boot flash

Unread postby mad_ady » Mon Feb 19, 2018 3:24 am

It should be possible, because spi holds uboot and uboot can boot from network (tftp + nfs for rootfs)
User avatar
mad_ady
 
Posts: 3636
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Odroid N1 SPI Boot flash

Unread postby umiddelb » Mon Feb 19, 2018 3:31 am

joy wrote:Right!
Based on the opinions of you guys, we're considering to use pre-OS that is built-in serial flash and its role will be like a BIOS or pre-process to select bootable storage and load boot.ini and kernel from the selected boot storage.
Rather than the first idea that I mentioned before, this approach must be more expandable for various of user scenario.
No need for frequent update of images on serial flash and easy access and update on each bootable storage.....
Also I will check kexec examples including petitboot to design better booting and updating logic.

It's a trade off, either you put some effort into things like PCI/SATA & USB3 support in u-boot (once again for every new board) or you invest some time in a high level bootmanager.
umiddelb
 
Posts: 438
Joined: Thu Jan 29, 2015 6:42 am
languages_spoken: English, German
ODROIDs: ODROID-C1, ODROID-XU4, ODROID-C2

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Fri Feb 23, 2018 7:40 am

I assembled some flash+breakout boards today. So, I am ready to help in testing when something is publicly available.
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Tue Feb 27, 2018 5:42 pm

I am attempting to add the required information for the flash to the device tree. However, with recent kernel patches, there appears to be a "fake" SPI device using a GPIO CS.

Can you clarify which SPI device is the actual hardware SPI device on the GPIO? There is "spi0" and "spi1".
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Tue Feb 27, 2018 7:43 pm

Got it working (spi1):
Code: Select all
[    2.934932] m25p80 spi32766.0: w25q128 (16384 Kbytes)

Code: Select all
$ mtdinfo /dev/mtd0
mtd0
Name:                           spi32766.0
Type:                           nor
Eraseblock size:                4096 bytes, 4.0 KiB
Amount of eraseblocks:          4096 (16777216 bytes, 16.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size:                  1 byte
Character device major/minor:   90:0
Bad blocks are allowed:         false
Device is writable:             true


Code: Select all
$ sudo flash_eraseall /dev/mtd0
flash_eraseall has been replaced by `flash_erase <mtddev> 0 0`; please use it
Erasing 4 Kibyte @ fff000 -- 100 % complete
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby joy » Wed Feb 28, 2018 8:34 am

crashoverride wrote:I am attempting to add the required information for the flash to the device tree. However, with recent kernel patches, there appears to be a "fake" SPI device using a GPIO CS.

Can you clarify which SPI device is the actual hardware SPI device on the GPIO? There is "spi0" and "spi1".

Hello crashoverride,
Very sorry for late response. :(
"SPI1" is the one as you found.

I need to upload related-source codes of boot loader and share a detailed instructions to boot SPI flash,
then we can sync our environment.
I will do it till next week.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby joy » Sun Mar 11, 2018 1:46 pm

Hi.
I've updated SPI boot-related boot loaders here in my github.
We're trying to fix some other issues of u-boot and a new u-boot version will be released soon.
Before that, please refer to the temporary repository and the following instructions.
Once it's ready, SPI-related source codes will be merged into Hardkernel official u-boot repository.

[ NOTE ]
The post is to share the existing status with forum users who wants test SPI flash booting.
viewtopic.php?f=155&t=29976#p214764
Still we're under implementation and designing the whole booting flow considering SPI flash and flexible boot options,
so booting sequence will be changed with more practical way.

1. Download and Build (1) - SPI boot Target
You can get and build the boot loaders for SPI flash booting from this repository.
Code: Select all
$ git clone https://github.com/JeonghwaCho/u-boot.git -b odroidn1-v2017.07
$ cd u-boot
$ ./build_spi.sh

https://github.com/JeonghwaCho/u-boot/c ... 1f461467b3
https://github.com/JeonghwaCho/u-boot/c ... a86ccccadf

Then, under the folder sd_fuse, two files will be generated.
Code: Select all
idbspl.bin : secondary program loader that doesn't use miniloader
u-boot.itb : third loader including u-boot, atf, m0 firmware and fdt


2. Download and Build (2) - eMMC/SD Target
For initial update to SPI Flash, eMMC/SD interfaces are considered.
From the same repository, you can build a new boot loader for eMMC/SD including SPI commands.
Code: Select all
$ cd u-boot
$ ./build.sh

Then, update your eMMC/SD card.
Code: Select all
$ cd sd_fuse
$ ./sd_fusing.sh <device/path/of/your/card>


3. Fusing
    1. Copy idbspl.bin and u-boot.itb into VFAT into your eMMC/SD card.
    2. Copy Image.gz, rk3399-odroidn1-linux.dtb and uInitrd into your eMMC/SD card, too.
    3. Boot from eMMC/SD card
    4. Hold booting at u-boot stage by typing ENTER before kernel booting
    5. Type the following instructions to update SPI flash from eMMC/SD card
Code: Select all
# setenv loadaddr 0x400000
 
# sf probe
# sf erase 0x0 0x1000000
 
# fatload mmc 0 $loadaddr idbspl.bin
# sf write $loadaddr 0x8000 $filesize
 
# fatload mmc 0 $loadaddr u-boot.itb
# sf write $loadaddr 0x40000 $filesize
 
# fatload mmc 0 $loadaddr Image.gz
# sf write $loadaddr 0x142000 $filesize
 
# fatload mmc 0 $loadaddr rk3399-odroidn1-linux.dtb
# sf write $loadaddr 0x842000 $filesize
 
# fatload mmc 0 $loadaddr uInitrd
# sf write $loadaddr 0x853800 $filesize


4. Booting from SPI flash with SATA disk
After updating SPI flash, try reset.
Then, you can get the u-boot message from SPI flash booting now.

Now try kernel booting with SATA disk partitions.
Based on the exiting scenario, we're considering SATA disk is configured same as the current file systems of SD/eMMC.
Code: Select all
Partition 1 : VFAT that has same UUID as released image
Partition 2 : EXT4 that has same UUID as released image, e139ce78-9841-40fe-8823-96a304a09859

So bootargs must be
Code: Select all
bootargs=earlyprintk swiotlb=1 console=ttyFIQ0,115200n8 rw root=/dev/sda2 rootfstype=ext4 rootwait

To boot, try run bootcmd on u-boot prompt.
Code: Select all
# run bootcmd

The commands is same as this detailed instructions.
Code: Select all
# mtdparts
# sf probe
# sf read 0x01000000 kernel
# unzip 0x01000000 0x02000000
# sf read 0x04000000 initrd
# sf read 0x01f00000 dtb
# booti 0x02000000 - 0x01f00000
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby joy » Sun Mar 11, 2018 1:58 pm

And you can encounter no booting even into u-boot when some boot loader part in SPI Flash is crashed.
Because SPI is the first order if RK3399, so it's not available to enter SD or eMMC when the idbspl section exists in SPI flash.
RK3399 maskrom will continuously try to get boot loaders from SPI flash in this case.

In this case, connect PIN#24 (SPI1_CSN) and PIN#1 (3.0V) of 40pin connector to hold SPI Chip select as HIGH.
Then, RK3399 will be ignore SPI node then try eMMC and SD.

Or erase the first 256KB part of SPI flash using this command.
note : the basic unit for erase is 4KB.
Code: Select all
sf erase 0x0 0x1000
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby moon.linux » Sun Mar 11, 2018 2:12 pm

crashoverride wrote:Got it working (spi1):
Code: Select all
[    2.934932] m25p80 spi32766.0: w25q128 (16384 Kbytes)

Code: Select all
$ mtdinfo /dev/mtd0
mtd0
Name:                           spi32766.0
Type:                           nor
Eraseblock size:                4096 bytes, 4.0 KiB
Amount of eraseblocks:          4096 (16777216 bytes, 16.0 MiB)
Minimum input/output unit size: 1 byte
Sub-page size:                  1 byte
Character device major/minor:   90:0
Bad blocks are allowed:         false
Device is writable:             true


Code: Select all
$ sudo flash_eraseall /dev/mtd0
flash_eraseall has been replaced by `flash_erase <mtddev> 0 0`; please use it
Erasing 4 Kibyte @ fff000 -- 100 % complete


16 MB is little bit less consider the size of Image get increase as we add debug option to the kernel.
HK should increase the size to at laleast 64 MB to support big kernel image.
moon.linux
 
Posts: 1056
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: Odroid N1 SPI Boot flash

Unread postby odroid » Sun Mar 11, 2018 2:17 pm

@moon.linux
512Mbit must be better than 128Mbit for development. But it is very far from economical selection.
128Mbit(16MB) serial flash is already expensive.
Do you really want to pay additional several dollars for 512Mbit flash only for debugging purpose?
User avatar
odroid
Site Admin
 
Posts: 27688
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Odroid N1 SPI Boot flash

Unread postby moon.linux » Sun Mar 11, 2018 4:02 pm

@odroid
If SPI boot is optional then it's not a issue. Do not know the cost of the SPI NOR flash.
But as the kernel grows in features it become difficult task to fit in the kernel images on to correct partition sector of spi flash as per my understanding.
moon.linux
 
Posts: 1056
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Re: Odroid N1 SPI Boot flash

Unread postby tobetter » Mon Mar 12, 2018 1:00 am

moon.linux wrote:@odroid
If SPI boot is optional then it's not a issue. Do not know the cost of the SPI NOR flash.
But as the kernel grows in features it become difficult task to fit in the kernel images on to correct partition sector of spi flash as per my understanding.

I don't think it's a good idea to put the kernel into SPI memory in terms of a cost and a kernel maintenance, an only bare minimal feature supporting by SPI like a BIOS would be fair enough.
tobetter
 
Posts: 2159
Joined: Mon Feb 25, 2013 10:55 am
Location: Kitchener, ON, Canada
languages_spoken: Korean, English
ODROIDs: X, X2, U2, U3, XU3, C1

Re: Odroid N1 SPI Boot flash

Unread postby crashoverride » Mon Mar 12, 2018 1:10 am

Why not just get a SPI enabled miniloader from Rockchip? I believe this is what the Rock64 ended up doing.
crashoverride
 
Posts: 3465
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1

Re: Odroid N1 SPI Boot flash

Unread postby odroid » Mon Mar 12, 2018 9:23 am

Kernel loading from eMMC or USB storage should be fine.
But we can not load the kernel from SATA storages.
The main reason is the missing Rockchip specific PCIe driver in the u-booot.
User avatar
odroid
Site Admin
 
Posts: 27688
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Odroid N1 SPI Boot flash

Unread postby joy » Mon Mar 12, 2018 9:58 am

moon.linux wrote:16 MB is little bit less consider the size of Image get increase as we add debug option to the kernel.
HK should increase the size to at laleast 64 MB to support big kernel image.

Yes, it's reasonable opinion. But we will try another approach so I think 16MB is enough.

N1 kernel image is about 16MB so I consider gziped kernel image those size under 7MB is used.
On u-boot stage, it's available to uncompress gzipped image and I've confirmed it's working with Image.gz under kernel/arch/arm64/boot/.
Please refer to the image in this link.
viewtopic.php?f=155&t=29976#p214764

For kernel size, we expect actual kernel image size will be reduced
because currently some driver modules that can be separated as module driver are included in kernel image.

And actual reaseon is...
the kernel image in SPI flash will be used only to judge boot storage and control the next booting process from the target boot storage (like BIOS).
It will be independent and optimized one for the purpose.

And It's better to exclude the scenario that actual kernel is built in SPI flash and updated every release.
Last edited by joy on Mon Mar 12, 2018 6:14 pm, edited 3 times in total.
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby joy » Mon Mar 12, 2018 10:11 am

crashoverride wrote:Why not just get a SPI enabled miniloader from Rockchip? I believe this is what the Rock64 ended up doing.

Yes you're right. It's easier approach.
But there is a history for that.

We've already asked rockchip korea agency the miniloader that supports SPI
, and they sent us a new miniloader that is not officially released one.
But it's not working. :(
It was very difficult to debug where the cause is because we got just a binary file
And they also recommended u-boot SPL approach from mainline.
So I made the SPL and u-boot based on other RK3399 products references.

I have no idea about Rock64 history.
I will also check it, too. :)
joy
 
Posts: 455
Joined: Fri Oct 02, 2015 1:44 pm
languages_spoken: english
ODROIDs: ODROID-C1+, XU4, X

Re: Odroid N1 SPI Boot flash

Unread postby moon.linux » Mon Mar 12, 2018 10:41 am

@joy @tobetter @odroid.
Thanks for your input on my query.
moon.linux
 
Posts: 1056
Joined: Thu Oct 02, 2014 11:42 pm
languages_spoken: english

Next

Return to Projects

Who is online

Users browsing this forum: No registered users and 1 guest