Stack-able micro computers

moon.linux · Unread post by **moon.linux** » Sun Aug 27, 2017 5:46 pm

MC1 will only play with sdcard + ethernet and cpu.
It need to be seen if we could build MC1 using 4 x HC1.

memeka · Unread post by **memeka** » Tue Aug 29, 2017 8:08 am

Did anyone (or is anyone willing to) build a RAID array from multiple HC1s by exporting each HDD as a iSCSI target, then build on one odroid RAID via iSCSI?
I think this is a good way to pool all storage together, while still having some form of reliability ....

it would be interesting to see some performance numbers, and also how reliable the setup is ...

EDIT: this is how to do it: https://www.mylinuxplace.com/distribute ... ver-iscsi/

ard · Unread post by **ard** » Tue Aug 29, 2017 11:00 pm

I wonder why you want iSCSI over vn2vn FCoE?
FCoE has lower overhead, works out of the box, and I was able to access my *DVD* drive over FCoE 8-D. And it only needs 2 commands to use it as initiator.
As a target you can just install targetcli.

If you want real pooled reliability, I would opt for ceph/rbd, as that's in the protocol, and the client has to arrange for failover of the store nodes. The downside of ceph is you need at least 3 "supervisors", which can of course be handled by the same systems.

Another solution is to split disks in 2 and use drbd ("raid" 1), that's for mutual disk recovery.

Of course, md over FCoE is possible too. In the end, if you want to combine mc1 and hc1 I would go for ceph using IPv6 link-local only. In worst case use a seperate ceph vlan.
Ah, yes, I assume you have a switch that's capable of handling vlans.

Raymond Day · Unread post by **Raymond Day** » Fri Sep 08, 2017 7:56 am

It works super good!

Got it with a 5V 6 amp power supply and 16GB loaded SD card. All so the USB-UART or RS-232.

Installed htop and it shows 8 cores!

Got WordPress installed on it and it's coping a import of WordPress to it right now. The RS-232 from putty looks like this with htop:

: This is a nice system and this htop show how good it works. 8 cores and 2GB RAM nice server.; Oddoid-HC1_htop.jpg (89.28 KiB) Viewed 321703 times

I got my Odroid-HC1 a few days ago. Put a 320GB HDD on it.

Still testing this but so fare so good.

Thank you for making this Odroid-HC1 nice job!

-Raymond Day

msperl · Unread post by **msperl** » Sat Sep 09, 2017 7:51 pm

We like the idea of the hc1 (and maybe also the mc1).
Our main focus is to get a nice, cheap and reliable setup for a ceph storage cluster.
So obviously we want the sata port for that, but then we would also like 3.5" support for bigger cheap disks.

Still there are a few shortcomings for our preferred usecase (some of them I guess are already mentioned in this thread):
* no direct 3.5" support right now
* no reset pin - means we need some power control via:
* a Relais
* POE device in front (connectable)
* a fast secondary storage for journal (typical ssd)
* this obviously can not get done without a second device, so maybe an accessible usb3 port may be nice (or a m2-sata port), both would require more of a redesign/additional HW.
* second gibt port for backend Traffic.
* rear facing led for identification purposes (or a gpio that can get used for such purposes)

For clarification: there seems to be a 12v connector to add 12v for 3.5" disks, but it is not coming with a connected connector. can you confirm this?

Finally: is there an official last order date for this product?

mad_ady · Unread post by **mad_ady** » Sat Sep 09, 2017 9:40 pm

There is a 12V pad on the board but you'd need to solder pins/wires to it and provide 12V for a bigger drive. Also a 3.5" drive would not fit in the enclosure. A HC2 is in the works to support 3.5" drives and should be available by november

mad_ady · Unread post by **mad_ady** » Sat Sep 09, 2017 9:42 pm

@odroid: you haven't added the hc1 to the front page yet!

moon.linux · Unread post by **moon.linux** » Sat Sep 09, 2017 9:54 pm

mad_ady wrote:A HC2 is in the works to support 3.5" drives and should be available by november

Sounds great...I would like to see similar type of product from HK with Amlogic SOC 64 bit.

Sentilix · Unread post by **Sentilix** » Mon Sep 11, 2017 5:55 pm

I still miss holes in the heat sink so the HC1 can be mounted on the wall. As of now I'm trying to make some kind of buckle to hold it, but it does not look stable - especially when 4 holes (one each corner) could have solved everything.

Telac · Unread post by **Telac** » Tue Sep 12, 2017 4:15 pm

odroid wrote:I think we are able to sell the MC1 from the middle of September at $200 including fully assembled four units with a USB cooling fan.

Any update on this? I'm about to build a cluster for an online game, and have been considering the XU4's for it for a while, but need to test out their performance, as I really hope this MC-1 will be exactly what I hoped for.
Really good project, can't wait to try it out

Unread post by **odroid** » Tue Sep 12, 2017 5:13 pm

@Telac
Some key components are not ready and we need a few more weeks to start to sell.
Sorry for the delay.

BTW, can you tell me which online game server will run on the MC1?

Telac · Unread post by **Telac** » Tue Sep 12, 2017 5:44 pm

odroid wrote:BTW, can you tell me which online game server will run on the MC1?

We are building a new version of the multiplayer game http://bloodfeud-game.com/ that now offers realtime movement, and combat etc (like a real game), so we hope to split the game into instances of maps, and run map servers as processes distributed on odroid mc-1 nodes

Raymond Day · Unread post by **Raymond Day** » Tue Sep 12, 2017 6:44 pm

I got another Odroid-HC1.

Thought they would slide in to each other but they just stack on top of each other. They don't lock in just set on the top in a grove. Photos don't show that good.

Because have 2 I guess can install a OS that can make them seem like one with 16 cores?

-Raymond Day

elatllat · Unread post by **elatllat** » Wed Sep 13, 2017 3:23 am

Raymond Day wrote:...I guess can install a OS that can make them seem like one with 16 cores?

No;
The OS is the wrong layer for this, apps or frameworks are the correct layer, because local cores have significantly less lag than remote cores so your application needs to know how to use them correctly.

https://en.wikipedia.org/wiki/Distributed_computing
https://en.wikipedia.org/wiki/Computer_cluster
https://en.wikipedia.org/wiki/Grid_computing

Molorius · Unread post by **Molorius** » Wed Sep 13, 2017 10:02 am

The MC1 could be very useful for anybody wanting to do intensive nanoscale structure modeling: http://www.quantum-espresso.org/. Universities in particular could benefit from a cheap cluster computer that science and engineering students could use and abuse.

If/when I get it, I will be posting benchmarks (and perhaps guides) of the MC1 doing these calculations vs the C1, C2, Pi 3, Pi 0, and a few x86_64 processors. It should be interesting from a price/performance perspective. I've been wanting to do it with the computers I have now anyway.

Veker · Unread post by **Veker** » Fri Sep 15, 2017 2:06 pm

Hi, this is my first post. I bought a C2 about half a year ago which which led me to buying an HC1. I'm using it as a NAS and a programming/development computer for Python and Ruby.

It's taken me a few days to get it set up the way I want it but I now have AIDE, Smb, Nginx, Puma, Rails, and some other apps on it. I'm thinking about getting a second one so I can learn about clusters.

merbst · Unread post by **merbst** » Tue Sep 19, 2017 10:04 am

Hello Odroid! Thank you for developing these innovative products!
I have been building a home Linux compute cluster for 2 years, out of a mix of Dell & HP servers manufactured within the past 10 years, connected via 2xInfiniband DDR 20gb cables to a common switch and configured in Linux to each share 8gb of their DDR3 via RDMA to be pooled together as a grid-accessible zero-copy NUMA shared memory space for working on a common dataset.
The individual servers are mostly multi-node machines with a common 24 SAS 2.5" bay backplane, thanks to the magic of M.2 SSDs available larger than a TB, and sophisticated software data redundancy tools, (see servethehome.com ) most of the uses providing the rationale for having 24 HDD with moving platters attached to a hardware SAS RAID card are now obsolete, leaving a large proportion of the 2U rack mount units presently installed having a large number of vacant hot-swap 3.5" or 2.5" bays designed to provide power at 12V, 5v, and 3.3V, as well as a pair of redundant dedicated data channels, each operating in full-duplex using the RDMA capable Serial Attached SCSI protocol at 3Gb/Second for SAS 1.0, 6Gb/second for SAS 2.0, 12GB/second for SAS 3.0, all of which use a physical connector nearly identical to the SATA PHY you have added to your MHC board, backwards compatible, and not even requiring more expensive electrical considerations.

The point of all this being, there is a great opportunity for an innovative ARM64 devboard manufacturer to market "Heterogeneous Computational Accellerator Boards, in a variety of configurations to extend the utility of the millions of servers installed in Network Operations Centers recently!

I put a month of research into this idea last year, planning to propose this to Odroid when I was ready to launch a joint Kickstarter. I even wrote a project outline for myself to write a custom usermode Infiniband over SAS bridge protocol Linux kernel module so the worker processes of the docker.io images on each ARM64 board could access the NUMA shared RDMA memory space to collaborate on solving large scale math and physics problems with extremely low latency and transactional data integrity. I have also been utilizing my 2 Laptops, and Odroid U3+ & XU4 as distcc nodes networked directly to the GB Ethernet TOE ports of the master server along with a few of my servers connected via 2*10GBE Direct Copper for OpenMPI message passing Algorithms experiments using Julia & Jupyter ... Also as an icecc & dist-cc build farm.

I had nearly forgotten to tell you guys about this dream, I have been so busy learning Linux clustering internals and building from spare parts sourced on a hobby budget my various servers motherboards, and sourcing matching RAM & Xeon X5-2600 X7-8800, and Amd Opteron 16 core CPUs, a few older Nvidia Tesla accellerator cards, assembled into a rack in my garage I assembled by hand.

I have a Google drive full of SFF technical specifications for the electrical engineering work (not my specialty, but I think I have a thorough understanding of how to accomplish the electrical interface at minimum cost, and emulate all the familiar interfaces using software virtualization. (I am a software engineer at heart, not afraid of getting deep into low-level stuff. I have dreams of becoming a mathmatician or physics theory guy)

Anybody who would like to work with me to make this dream an OPEN SOURCE HARDWARE reality, please email the address I used to sign up for this PHPBB account, all I ask is co-credit and enough free samples to keep me interested in remaining involved!

Actually, I started a Nonprofit organization this year to fund face to face meetings to advance my Open Source Ambitions & swap knowledge with people from around the world, so here is my email address & real name:
Matt (.) Erbst (@) gmail dot com

mitchw · Unread post by **mitchw** » Wed Sep 27, 2017 10:05 am

I realise this is a derivative of the XU4, but if/when you guys decide to do this with a newer ARMv8 SoC I'd be really interested. Right now 2GB of RAM, with 32bit registers doesn't really interest me at mass scale. older 32bit ARMv7s are fine for low power / small footprint devices, but at scale I'm interested in utilising as many cycles as I can on each node, doubling of registers and a bit more RAM are essential for this task.

I do have one request for future iterations though: Drop the sata controller bridge, and re-expose the USB 3 - it's much more versatile in my opinion, and nodes in a cluster should be using a remote block storage solution 'or' a much smaller local scratch space anyway (small enough that eMMC or microSD would be fine, though modern eMMC would be ideal for the speed/latency). This would also simplify your PCB, reduce your BoM, and probably even make the PCB a bit smaller overall.

A dream request would be moving to an SoC that has USB 3.1 Gen 2, I believe this isn't entirely impractacle - newer external SSDs, eg: from samsung have 3.1 Gen 2 controllers built in - so the economies of scale are surely starting to kick in on these controllers. This would give us the option to 1) attach incredibly fast DAS/NAS to nodes if required, or much more importantly 2) attach 10GbE interfaces as required (at our own cost) for iWarp/RoCE style RDMA or 3) more of a niche but still possible with modern USB 3.1 - attach external GPUs to nodes for GPGPU processing.

Moving to a modern USB 3 spec truly enables so many options, and exposing the USB port directly is so much more versatile for us (and simpler for you guys!), USB 3.1+ even lets you move to USB-C reducing the size of the board. If you could expose 2x USB-C ports, and move your USB->Ethernet bridge off the SBC and instead develop your own cheap USB-C->GbE adapter, that makes things even smaller still (and more modular/flexible).

Imagine the density of nodes you could achieve if you didn't have those bulky RJ45/USB-A/HDMI ports taking up all that space, just two USB-C!

You could go even further dropping the microSD and forcing eMMC on everyone, which isn't a horrible idea either - but I can understand why people may prefer microSD (simpler/more-common tools for mass-formatting cards during setup).

Edit: One more way more advanced idea would be integrating an on-board FPGA (nothing big/expensive, just big enough to implement USB 3.1, InfinBand, or PCIe - the number of gates required for these IPs isn't huge, the most important part is ensuring the FPGA has the SerDes/transceivers required for the bandwidth of these protocols), exposed with a standard mezzanine connector that could have USB/Ethernet/HDMI/Fiber/whatever endpoints connected depending on how you used the FPGA. That would make for a truly flexible cluster computing node that could meet everyones needs (and better yet, the FPGA being able to fill the technical gaps of the cheaper/slower ARM SoCs means you have a lot more freedom on the ARM side as you can implement missing features on the FPGA). This definitely brings up the BoM though by a few dollars at least, and complicates the boot process, programming the FPGA, tooling requirements, etc... probably not what most HardKernel customers want.

wtarreau · Unread post by **wtarreau** » Thu Sep 28, 2017 12:53 pm

Hi,

odroid wrote:@Telac
Some key components are not ready and we need a few more weeks to start to sell.
Sorry for the delay.

Are the components still missing ? I'm asking because I'm really insterested in testing how MC1 compares to my MiQi-based build farms (using a *very* powerful RK3288) : https://forum.mqmaker.com/t/miqi-based- ... ing/605/23.

Thanks!
Willy

Unread post by **odroid** » Thu Sep 28, 2017 3:42 pm

MC1 is under production state.
We will start to sell it from 16 October probably.

tkaiser · Unread post by **tkaiser** » Sat Sep 30, 2017 9:54 pm

@odroid: If you want tons of useful feedback for cluster use cases (especially wrt overall memory bandwidth and ideas how to deal with HMP 'correctly' or better say in the most efficient way) I strongly suggest getting in touch with Willy and maybe even sending him a pre-production sample if available.

piper · Unread post by **piper** » Tue Oct 10, 2017 4:43 pm

@odroid - any news on the plastic cover for HC1?

JagDoc · Unread post by **JagDoc** » Thu Oct 19, 2017 2:04 pm

odroid wrote:MC1 is under production state.
We will start to sell it from 16 October probably.

Any news when we can get it?

Unread post by **odroid** » Thu Oct 19, 2017 2:10 pm

piper wrote:@odroid - any news on the plastic cover for HC1?

It will be available in the middle of next week.

JagDoc wrote:
odroid wrote:MC1 is under production state.
We will start to sell it from 16 October probably.

Any news when we can get it?[/qu
MC1 will be available two weeks later. We've just started the production.

golgoth · Unread post by **golgoth** » Thu Oct 19, 2017 10:17 pm

odroid wrote:
piper wrote:@odroid - any news on the plastic cover for HC1?
It will be available in the middle of next week.

JagDoc wrote:
odroid wrote:MC1 is under production state.
We will start to sell it from 16 October probably.
Any news when we can get it?[/qu
MC1 will be available two weeks later. We've just started the production.

Hey guys,
just wondering if the HC2 is still planned for november or have been rescheduled?

thanks

Unread post by **odroid** » Fri Oct 20, 2017 8:10 am

HC2 will be released in the end of November or early December.

Unread post by **odroid** » Tue Oct 24, 2017 3:27 pm

HC1 cover case is available now.
http://www.hardkernel.com/main/products ... 0878897712

mad_ady · Unread post by **mad_ady** » Tue Oct 24, 2017 3:49 pm

it looks great!

trilby · Unread post by **trilby** » Tue Oct 24, 2017 8:37 pm

Looks great, hopefully Odroid.co.uk will stock it soon as it's hard to justify $16 UK shipping for a $4 case

Tinkerisk · Unread post by **Tinkerisk** » Fri Nov 03, 2017 12:35 am

odroid wrote:HC1 cover case is available now.
http://www.hardkernel.com/main/products ... 0878897712

Will it be available in translucent black color as well?

Unread post by **odroid** » Fri Nov 03, 2017 8:56 am

We have no plan to make a black cover.

fvolk · Unread post by **fvolk** » Sat Nov 04, 2017 11:00 pm

odroid wrote:HC1 cover case is available now.

Just to be sure, the cover case also fits for 15mm disks?
From the photo/product page it is not clear - I would suggest you explicitly add this information to the product page.

Unread post by **odroid** » Sun Nov 05, 2017 2:19 pm

We've tested Seagate 5TB HDD which is 15mm thickness.
This rendering images are based on 15mm HDD.

giminni · Unread post by **giminni** » Sun Nov 05, 2017 9:05 pm

Hi Team,
I am currently working with Ubuntu Core 16 on Raspberry Pi and was attracted by your new ODROID-MC1 cluster and your offer to be involved in building a docker cluster with it.
I have deep knowledge in docker and docker swarm, and I am willing to partecipate in building a cluster solution with ODROID-MC1, Ubuntu Core 16 and docker-snap 17.06-2-ce.

My Idea is to build physical and/or logical IoT clusters (MANagement, REPository, STOrage, CONTainer) and pin or dynamically green balance docker container services from 4 high to 4 low CPU cores based on demand or based on cron loads and auto-refresh core, kernel, gadgets, docker and containers automatically to guarantee 24/7 operation.

Even I want to try to build a Kubernetes cluster with minio/minfs as object storage server

Can I get at least four of them and share some knowledge with you and others?

Unread post by **odroid** » Thu Nov 09, 2017 10:42 am

ODROID-MC1 has been released. We are ready to ship.

http://www.hardkernel.com/main/products ... 0152508314

moon.linux · Unread post by **moon.linux** » Thu Nov 09, 2017 1:33 pm

Excellent

venkatbo · Unread post by **venkatbo** » Thu Nov 09, 2017 2:48 pm

Congrats, HK team, gr8 variant. Maybe the MC1 & HC1 should nudge the others in the homepage and take the "top" positions

PigLover · Unread post by **PigLover** » Wed Nov 15, 2017 4:08 am

I think the HC1 and MC1 are great. Thank you very much!

Future idea:

Distributing power becomes a real limitation as you build up a cluster of HC1/MC1. Running multiple plug adapters is not practical and doing a 'DIY' cable job with a 5v PSU presents a number of challenges.

For a next-spin of these things, why not consider this: build in a "stacking plug" - power in from the unit below, power out to the unit above. Would probably make sense to distribute 12v and put a small regulator on each board to get 5v. This requires smaller connectors and also allows same design to support 12v devices like 3.5" HDDs.

Then build a "bottom plate" that can take power from a standard 12v power brick and distribute it to the "stack". 12v bricks are cheap an plentiful thanks to all the laptops on the world that use them. Better still, make it flexible enough to accept 12-19v so that you could use almost any power brick made...

I would think this could really expand your market opportunity as clustering them becomes a simple matter of stacking the units and connecting the LAN and a common power adapter - something that almost anyone can do.

Even better if built so that HC1 and MC1 can be stacked together interchangeably.

andrewsmithty · Unread post by **andrewsmithty** » Thu Nov 16, 2017 10:56 pm

@odroid, is there any news or specs on ODROID-HC2?
google results all over the web are stating "Update: ODROID-HC2 is in the works, to be released in November 2017"
however i can see no news about it here...
could you enlighten me?
Thanks
Endre

Unread post by **odroid** » Fri Nov 17, 2017 9:32 am

HC2(or HC1+) for 3.5inch HDD will be available in the middle of December even we start the production in late November.

trohn_javolta · Unread post by **trohn_javolta** » Fri Nov 17, 2017 11:46 am

odroid wrote:HC2(or HC1+) for 3.5inch HDD will be available in the middle of December even we start the production in late November.

I see it's getting rescheduled... so no HC2 christmas present for me

Will it be for 1 3.5" HDD or 2 HDDs? I mean both via sata obviously, I know I can hook up an external HDD via usb 2.0.

elatllat · Unread post by **elatllat** » Sat Nov 18, 2017 12:17 am

memeka wrote:... exporting each HDD as a iSCSI target ....

https://www.backblaze.com/blog/the-clou ... backblaze/

Waffle · Unread post by **Waffle** » Thu Nov 23, 2017 4:48 am

I used a 200W power suppy (40A @ 5V) to power the initial 2 MC1's I ordered. They came up without a problem (though I did have to power-cycle each unit after the initial boot). Making custom cables to connect to the ethernet switch made for a neater installation, and the large fan is remarkably quiet. To connect to the power supply, I used an Anderson Powerpole connector for each 4 of the power cables. This allows me to power down each MC1 with a simple disconnect, and opens the possibility of a power distribution block to be used as well. The small Powerpole connectors are rated at 15A, so it's pushing them a little if you believe each board can draw 4A at 5V, but the boards do seem stable.

The performance is _very_ good, better than the XU4, probably due to the lack of throttling as a result of the increased heat sink mass.

Using MDXfind, I saw the XU4 running 13.58M MD5 hashes per second, at 80 C (brief peak to 85 C)
On the MC1, I got 14.78M MD5 hashes per second, at 75C (brief peak to 76C).

I'm looking forward to running many more of these! All in all, a very good value.

Thank you for building them, guys. Great job.

Unread post by **odroid** » Thu Nov 23, 2017 12:30 pm

Waffle wrote:I used a 200W power suppy (40A @ 5V) to power the initial 2 MC1's I ordered. They came up without a problem (though I did have to power-cycle each unit after the initial boot). Making custom cables to connect to the ethernet switch made for a neater installation, and the large fan is remarkably quiet. To connect to the power supply, I used an Anderson Powerpole connector for each 4 of the power cables. This allows me to power down each MC1 with a simple disconnect, and opens the possibility of a power distribution block to be used as well. The small Powerpole connectors are rated at 15A, so it's pushing them a little if you believe each board can draw 4A at 5V, but the boards do seem stable.

The performance is _very_ good, better than the XU4, probably due to the lack of throttling as a result of the increased heat sink mass.

Using MDXfind, I saw the XU4 running 13.58M MD5 hashes per second, at 80 C (brief peak to 85 C)
On the MC1, I got 14.78M MD5 hashes per second, at 75C (brief peak to 76C).

I'm looking forward to running many more of these! All in all, a very good value.

Thank you for building them, guys. Great job.

Thank you for sharing your MC1 experience. Happy to know MC1 has better thermal characteristics than XU4.

meveric · Unread post by **meveric** » Thu Nov 23, 2017 6:28 pm

With the big fan on the back that's no surprise I guess.
I was also thinking about using the Cloudshell 2 for compiling and heavy duty.
The HDD fan cools the XU4 in there quite nicely as well and I think even under full load there isn't any throttling.

golgoth · Unread post by **golgoth** » Mon Dec 04, 2017 6:48 pm

Hello Odroid,
have the HC2 production been launched as planned?

thanks again for the good work.

Unread post by **odroid** » Mon Dec 04, 2017 6:53 pm

golgoth wrote:Hello Odroid,
have the HC2 production been launched as planned?

Nope. the schedule is pushed to January due to some raw material lead time issues.
Sorry about that.

andrewsmithty · Unread post by **andrewsmithty** » Mon Dec 04, 2017 9:33 pm

hi Odroid!
Can you share the specs of the HC2?
it would be nice to know / share ideas / usecases about it!
Thanks!
Endre

exenGT · Unread post by **exenGT** » Sat Dec 09, 2017 7:36 am

Hi Odroid,

I plan to purchase several sets of MC1 as a personal cluster for my simulation work. Just wondering if it is possible to attach an external hard drive to MC1 for storage purposes, like HC1?

Unread post by **odroid** » Sat Dec 09, 2017 12:53 pm

MC1 has a USB 2.0 port which can connect an external HDD.