[Just FYI] Project TinyMiniMicro

Post Reply
domih
Posts: 274
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 90 times
Been thanked: 104 times
Contact:

[Just FYI] Project TinyMiniMicro

Post by domih »

A contact point between Enterprise computing and the SBC world finally got a name: Project TinyMiniMicro.

See https://www.servethehome.com/introducin ... evolution/

Roughly I guess it aspires at establishing a set of best practices for building, using and maintaining clusters of mini PCs or SBCs at home or in a SOHO environment.

Enterprise computing people love to create names, labels and acronyms.

Like I do for White wood mini-tower of 3 x H2 (see https://forum.odroid.com/viewtopic.php?f=172&t=38711) it includes buying used hardware (e.g. from eBay) to reuse Enterprise hardware in a home/SOHO environment at cheap prices. In my case I buying the Mellanox network cards while using Odroid H2 and H2+.

At this point I see one of the major differences/hurdles between the two worlds is the use of BMC/IPMI, Intel vPro and AMD Pro on the Enterprise side and missing in the SBC world (putting aside the > $500 industrial motherboards). Almost all Enterprise motherboard makers use AST2500 or earlier to provide BMC/IPMI (see https://www.aspeedtech.com/products.php ... 20&rId=440). Unfortunately Aspeed is not known to be flexible with small OEMs, making the adoption of its solution virtually impossible in the non-Enterprise (read low cost) SBC world.

The Enterprise people would also love to apply the lessons of Hyper-converged infrastructure https://en.wikipedia.org/wiki/Hyper-con ... astructure to TinyMiniMicro. Hyper-converged infrastructure is a mouthful marketing buzz-word. But it does correspond to something real from a technological viewpoint: virtualize everything and manage it via software.
These users thanked the author domih for the post:
ecogit (Fri Jul 24, 2020 4:49 pm)

fvolk
Posts: 451
Joined: Sun Jun 05, 2016 11:04 pm
languages_spoken: english
ODROIDs: C2, C4, H2
Has thanked: 0
Been thanked: 41 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by fvolk »

By three they come... "Why did you come here?" "I came here for knowledge"

Image

I think it's the cheapest, most low-power, most silent way to test distributed code.
In fact mixing low resource C2 cores, with a bit more potent C4 cores, over H2 cores, up to the DeskMini high-powered (but still low-power and silent) 8700T cores is THE way to test software. All cores are 64bit and mixing aarch64 and x86-64 nodes in one networked application breaks non-robust code fast.

All C2 and C4 nodes boot diskless from the same rootfs, as all H2 do from another rootfs -> easy upgrade and identical configuration.
One H2 (not in photo) plays the (NFS) server for all, the DeskMinis are standalone installs.
3x H2 powered by one 19V 75W laptop brick,
3x C4 powered by one H2 15V PSU,
3x C2 powered by one 5V4A XU4 PSU
....praise Odroid for variable V power input on newer models! (and no USB as power input)
These users thanked the author fvolk for the post (total 2):
PigLover (Wed Jul 15, 2020 1:18 am) • domih (Tue Jul 21, 2020 10:11 am)

mad_ady
Posts: 8316
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 573 times
Been thanked: 434 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by mad_ady »

Regarding the lack of ipmi/bmc - what is the #1 usage for it? Remote power on/off. For SBCs you could easily replace it with relays on power input - either home made or commercial (e.g. wifi), so you can power-cycle any board remotely.
For kvm, you could wire the serial ports in a serial multiplexer and use that instead...
These users thanked the author mad_ady for the post:
PigLover (Tue Jul 14, 2020 11:57 pm)

fvolk
Posts: 451
Joined: Sun Jun 05, 2016 11:04 pm
languages_spoken: english
ODROIDs: C2, C4, H2
Has thanked: 0
Been thanked: 41 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by fvolk »

FYI, some power numbers - note that they vary with ambient, device and PSU temperatur:

Code: Select all

        idle   rsa512*  whirlpool*  shutdown
3xH2    6,9W     39 W       43 W     0,8W
3xC4    3,8W     8,8W      11,1W     1,1W
3xC2    2,9W     8,0W       9,7W     0,9W
*openssl speed [whirlpool|rsa512] on all 4 cores

1xDeskMini: 10-12W idle headless, ~55-60W all cores max.
These users thanked the author fvolk for the post (total 2):
odroid (Wed Jul 15, 2020 8:18 am) • domih (Tue Jul 21, 2020 10:11 am)

domih
Posts: 274
Joined: Mon Feb 11, 2019 4:48 pm
languages_spoken: English, French
ODROIDs: UX4, HC2, N2, H2, C4, H2+
Location: San Francisco Bay Area
Has thanked: 90 times
Been thanked: 104 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by domih »

mad_ady wrote:
Tue Jul 14, 2020 11:15 pm
Regarding the lack of ipmi/bmc - what is the #1 usage for it? Remote power on/off. For SBCs you could easily replace it with relays on power input - either home made or commercial (e.g. wifi), so you can power-cycle any board remotely.
For kvm, you could wire the serial ports in a serial multiplexer and use that instead...
Yes and no. Yes you can use relays to switch on/off restart a bunch of SBCs.

On the other hand, IPMI/BMC goes much further like for instance deploying new BIOS on hundreds of nodes, change settings and so on. IMHO, you do not want to "emulate" IPMI features, you have it or you don't have it, no need to reinvent the wheel. If you have it, you want it to follow the standard so that it can be integrated with the usual enterprise software from HP, Dell, and so on.

In other words, if you design a board and you want to sell it in the Enterprise world meaning you sell a lot of them at high unit price with discount over qty, IPMI/BMC is an unavoidable check mark. This is a conundrum for small OEMs because it requires a huge investment.

For individual or SOHO usage who cares, because you won't to have to automate the management of hundreds of nodes anyway.

Unrelated:

These days the frontier between the solutions become blurry and the blur comes from both sides (low X86, high ARM).

QNAP for example is selling semi-pro NAS with no IPMI/BMC. See for example their new series https://www.qnap.com/en/product/ts-453du, https://www.qnap.com/en/product/ts-853du-rp, https://www.qnap.com/en/product/ts-1253du-rp. They "look" like enterprise solutions, but they are not. They are based on the Celeron J4125, the specs are like candies and unicorns, meaning you can't have it all at the same time. The CPU still only has 6 HSIO lanes. Looking into the details of the user guide one can read: PCIe Gen 2 x2, PCIe bandwidth limitations will restrict the NAS 10GbE performance Duh... The SATA III ports are probably multiplexed. And no IPMI/BMC. Add on top of it the very heavy QNAP software with all the add-ons. I pretty sure the overall IO performance is not optimal. You won't see a data center with hundred of these rack servers. At the same time if you are an individual with some minimal knowledge you'll probably get better performance from an H2 or H2+ (e.g. PCIe Gen 2 x4 is exposed). These QNAP individual/SOHO NAS are for people who want turn key solutions, if they just knew a little bit more, they would be better served with an H2/H2+ NAS running only what they need.

fvolk
Posts: 451
Joined: Sun Jun 05, 2016 11:04 pm
languages_spoken: english
ODROIDs: C2, C4, H2
Has thanked: 0
Been thanked: 41 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by fvolk »

Hmm... Question at @odroid:
If stacking multiple C2, C4, H2, N2, etc. - and they are all powered from the same power supply with a Y-cable,
a) Should one always use plastic spacers to electrically isolate them? Like in my picture above?
b) Or can one use metal spacers - which are more robust - but don't isolate the individual boards from each other?

User avatar
odroid
Site Admin
Posts: 34851
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean, Japanese
ODROIDs: ODROID
Has thanked: 932 times
Been thanked: 759 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by odroid »

I think any (conductive) metal spares should be fine. There must be no negative side effect.

fvolk
Posts: 451
Joined: Sun Jun 05, 2016 11:04 pm
languages_spoken: english
ODROIDs: C2, C4, H2
Has thanked: 0
Been thanked: 41 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by fvolk »

FYI, a happy cluster of 3xC2+3xC4+3xH2 -> 36 cores working together.
Central scheduler runs on another H2 server that is also the rootfs server for the diskless clients,
JupyterLab runs on a normal PC and connects to the scheduler.
On x86 standard Anaconda Python,
on aarch64 Conda Miniforge (https://github.com/conda-forge/miniforge/releases)
Unfortunately the library versions on x86 and aarch64 are never exactly the same, so mismatches happens, but if they are close enough in minor revisions cross-platform works.

Note that this is just a demo computation for checking of setup, each task is just a random delay/sleep, so CPUs are bored.
ocluster.png
ocluster.png (526.44 KiB) Viewed 123 times

fvolk
Posts: 451
Joined: Sun Jun 05, 2016 11:04 pm
languages_spoken: english
ODROIDs: C2, C4, H2
Has thanked: 0
Been thanked: 41 times
Contact:

Re: [Just FYI] Project TinyMiniMicro

Post by fvolk »

too many cores and not enough parallel tasks :-)
ocluster2.png
ocluster2.png (99.66 KiB) Viewed 122 times
These users thanked the author fvolk for the post:
odroid (Fri Jul 24, 2020 8:23 am)

Post Reply

Return to “Projects”

Who is online

Users browsing this forum: No registered users and 0 guests