Can you make boards with more RAM and/or modules...?

Share here your ideas for new projects

Moderators: meveric, mdrjr, odroid

Can you make boards with more RAM and/or modules...?

Unread postby dave_t » Fri Jul 14, 2017 9:54 am

Hi folks. I've been using ODROIDs for several years now for parallel computing education (dual node clusters for my students) and for a couple of years for building clusters for scientific computing (see attached image of my XU3-Lite cluster). For certain applications, my cluster of ODROIDs is far superior to a server (a journal paper is due out in a few days about this), but for other applications, I simply need more RAM per core.

Would you consider putting out a board like the XU4, but with 4 or 8 GB of RAM, like UDOO or other SBC manufacturers have done?

Also, I would LOVE to have a setup where I could snap a whole bunch of nodes like XU4's into an enclosure that provides power, networking connectivity, and cooling (think the most killer ARM server) - basically a cluster in a box. I know some companies make computer-on-a-modules and have carrier boards, but the boards only support 1 module. I'd love to have a carrier board where I can snap lots of ODROIDs into (an alternative for the cluster in a box I just mentioned).

Is there any chance of any of this happening? Thanks!
Attachments
cluster.png
cluster.png (1.88 MiB) Viewed 946 times
dave_t
 
Posts: 19
Joined: Wed Dec 24, 2014 9:34 am
languages_spoken: english
ODROIDs: C1
U3
C1+
XU3-Lite
C2
XU4

Re: Can you make boards with more RAM and/or modules...?

Unread postby odroid » Fri Jul 14, 2017 10:55 am

We are also very interested in the cluster computing and server.
Recently we made a cluster with ~200pcs of XU4 and its uptime is around 10 days.
Image
But it was quite painful to build it. We spent a couple of weeks.
And it is too expensive even it has 1600 CPU cores and 400GB RAM. :o

So we decided to make a stripped down version of XU4 with minimal IO ports to reduce the cost and time.
Stackable metal heatsink cases must be very helpful for the scalability too.
There will be a news in the middle of August.

BTW, we have no plan to add more RAM in near future because of low demand.
Can you tell me why do you need more RAM for cluster computing?
User avatar
odroid
Site Admin
 
Posts: 24662
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Can you make boards with more RAM and/or modules...?

Unread postby mad_ady » Sat Jul 15, 2017 3:56 am

Somebody's following the bitcoin trail :)
User avatar
mad_ady
 
Posts: 2603
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Can you make boards with more RAM and/or modules...?

Unread postby dave_t » Sat Jul 15, 2017 11:48 pm

Very cool cluster you have there! The reason I need more RAM is that the computational chemistry package we're trying to use (GAMESS - http://www.msg.ameslab.gov/GAMESS/) is a bit of a memory hog for all but the simple calculations. While I can run the computational chemistry package AutoDock Vina (http://vina.scripps.edu/) with very little memory per core, GAMESS just needs more RAM/core to make the cluster usable. We'd like to try other scientific software on the cluster next, but some of the packages are likely to need more RAM. As soon as our paper is published (any day now), I'll send you the link so you can get a better sense of what we've been doing and why we want more RAM. I can't wait to hear you announcement inmid-August. My research partner and I have about $5000 that we can spend to build a new cluster, so...
dave_t
 
Posts: 19
Joined: Wed Dec 24, 2014 9:34 am
languages_spoken: english
ODROIDs: C1
U3
C1+
XU3-Lite
C2
XU4

Re: Can you make boards with more RAM and/or modules...?

Unread postby venkatbo » Sun Jul 16, 2017 1:51 am

XU4-0 fanless, with 4gb RAM, rj45, 1 usb3, will make for a nice headless server, capable of running 3 containers with 1gb RAM each, and 2 cores each. Containers are the future. One hindrance for adoption is RAM/container. We routinely hit bottlenecks at 512mb RAM/container. Clusters can only scale this further. And most likely you'd be out with yet another ground breaking SBC. If it can priced below $99, all the better. Closest x86 SBCs are ~$150.

I think 4 per cluster is a sweetspot - many wifi rts can handle 4 SBCs, 1 12v 8a psu can drive rtr+4SBCs. Makes for a nice self-contained cluster. 32cores, 16gb for ~$400 (less than a laptop) should be within the reach of many!

If volume manufacture is the issue, you can explore the kickstarter route. Can amortize the R&D cost in there. Will also increase HK visibilty for other pdts as well.
User avatar
venkatbo
 
Posts: 444
Joined: Mon Feb 25, 2013 3:10 pm
Location: Bay Area, California
languages_spoken: english
ODROIDs: C0/C1+/C2 U3+ XU3-Lite/XU4 VU/VU7+ Touchscreen Show TFT Cloudshell SmartPower mAHRS DAC GPS Multiscope

Re: Can you make boards with more RAM and/or modules...?

Unread postby odroid » Sun Jul 16, 2017 11:40 am

@dave_t & @venkatbo,

Thank you for the valuable inputs.
But there is no way to add 4GB RAM PoP memory on Exynos5422 SoC because Samsung has no plan to make it at this moment.
Millions of orders are required to move Samsung :(

For the better flexibility, we have to consider a smaller SoC vendor in China instead of the giant in Korea.
User avatar
odroid
Site Admin
 
Posts: 24662
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Can you make boards with more RAM and/or modules...?

Unread postby dave_t » Sun Jul 16, 2017 8:29 pm

@odroid,

Thanks for your reply. I understand that you won't be able to add RAM to the XU4. I'm just hoping you'll produce a new SBC with more RAM in the future. Would you mind telling me which power supply you're using in your cluster's picture and where you got them? I'd love to start wiring up new clusters with those instead of using the individual power supplies. I've been told the way yours is wired is more electricity efficient. Thanks!
dave_t
 
Posts: 19
Joined: Wed Dec 24, 2014 9:34 am
languages_spoken: english
ODROIDs: C1
U3
C1+
XU3-Lite
C2
XU4

Re: Can you make boards with more RAM and/or modules...?

Unread postby odroid » Mon Jul 17, 2017 9:31 am

We used this 5V/80A (400Watt) PSU which could handle 20~25pcs of XU4 stably.
https://www.aliexpress.com/store/produc ... 49295.html
User avatar
odroid
Site Admin
 
Posts: 24662
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Can you make boards with more RAM and/or modules...?

Unread postby robroy » Tue Jul 18, 2017 8:44 am

@odroid and @davet, please document both of these awesome projects and create an article for ODROID Magazine!
Rob Roy, Editor-in-Chief of ODROID Magazine (submit an article | make a donation)
User avatar
robroy
 
Posts: 3340
Joined: Mon Feb 25, 2013 2:29 am
Location: San Francisco, California
languages_spoken: English
ODROIDs: Q2, X, X2, U2, U3, XU-E, XU3, XU4, C1, C1+, C2

Re: Can you make boards with more RAM and/or modules...?

Unread postby phaseshifter » Sat Jul 22, 2017 1:36 am

the more ram available the more threads it can run ..i think that is where they are going with wanting more ram ..
i have had my xu4 running 14 threads..but it slows down the the chain as it may be said
4.9.xx.xx odroids
phaseshifter
 
Posts: 2455
Joined: Fri May 08, 2015 9:12 am
languages_spoken: english
ODROIDs: opp sys.. ubuntu .kernel 4.9.xx.xx.c1+ ..c-2..xu3 lite,xu4...vu7,vu8,c-shell -II..c-shel-II,uart,hi-fi ,,hi-fi2,,show,w-board,6x16GB emmc`s 3.5 inch touch...other odroid acc`s as well

Re: Can you make boards with more RAM and/or modules...?

Unread postby dave_t » Sat Jul 22, 2017 2:33 am

@phaseshifter:

Actually, I just have calculations that require more RAM. I don't run more than one process per core, and they're all single-threaded processes. I'm not sure what others are doing, though.
dave_t
 
Posts: 19
Joined: Wed Dec 24, 2014 9:34 am
languages_spoken: english
ODROIDs: C1
U3
C1+
XU3-Lite
C2
XU4

Re: Can you make boards with more RAM and/or modules...?

Unread postby tchiwam » Sun Jul 23, 2017 5:41 pm

I will pre-order 32 units if they have >4GB of ram.

This will allows me to keep a full LUT in the ram, ideally I need 6 or 8 GB total, ~4GB for the E=0...16 LUT and 2 or 4 GB for the data.
tchiwam
 
Posts: 81
Joined: Wed Dec 30, 2015 4:53 am
Location: Greenland
languages_spoken: english, French, Finnish, Greenlandic(little)
ODROIDs: 5x XU4, 3x C2, 2 cloudshell, 1 VU7, 1 VU7+, UPS1, UPS2, UPS3, USB/SATA, and many other fine bits from Odroid

Re: Can you make boards with more RAM and/or modules...?

Unread postby dave_t » Fri Aug 04, 2017 12:58 am

robroy wrote:@odroid and @davet, please document both of these awesome projects and create an article for ODROID Magazine!



@robroy:
The paper on this was just published in the Journal of Computational Science Education (http://jocse.org/). Here's a link to the paper: http://www.jocse.org/articles/8/2/3/.

I'm not really sure what I would write for the ODROID Magazine about this. It's kind of a technical paper for the magazine. You could contact JOCSE and see if they would let you reprint some or all of the paper, but I signed the copyright over to them, so you'll have to contact them for permission.

Dave
dave_t
 
Posts: 19
Joined: Wed Dec 24, 2014 9:34 am
languages_spoken: english
ODROIDs: C1
U3
C1+
XU3-Lite
C2
XU4

Re: Can you make boards with more RAM and/or modules...?

Unread postby mad_ady » Fri Aug 04, 2017 3:34 am

I've read the article and it was pretty nice. I don't find it too technical (apart from all the references) - a lot of Odroid Magazine articles feature commands or code.
There are some interesting setails which are missing - perhaps you could tell us/expand in a future article:

1. More details about AutoDock Vina. Can regular people (outside of your domain) run it through boinc and participate/advance science?
2. How did you compile/run it? Did you use any optimization flags on any platform? What would the results be if compiled with -O3 and other flags (viewtopic.php?f=9&t=1645)?
3. What governor did you use (might affect power consumption)? Also, if not using gpu you might want to turn it off (or minimize frequency). Are you using gpu as well? If yes, can you compare performance of cpu only vs cpu+gpu? In my password cracking tests using the gpu nearly doubled performance.
4. It might be interesting to compare a xu4 @2GHz against a xu3 @1.8, though the speed boost may be smaller than the power used.
5. HK said in a different thread that they would release a uboot for xu4 with network boot capability, for clusters. This means reduced flash times for your SD. Though you might want gigabit or a ramdrive
6. I disagree that a cluster needs a manager node that can't do any work. Why can't it be both msnager and worker? It will be less efficient than the other workers, but still do work.
7. For memory-intensive applications I'd like to see tests with xu3/4 + swap on an emmc (or usb3 ssd + uas) against a pc. I wonder by how much it would slow down.
8. Regarding gigabit, on a different thread it was discovered that a c2 consumes about 100mW more with gigabit compared to 100Mbps. Would switching to Gb impact your power savings? (There's also the switch to consider - both hardware and power).
User avatar
mad_ady
 
Posts: 2603
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Can you make boards with more RAM and/or modules...?

Unread postby odroid » Fri Aug 04, 2017 2:37 pm

@dave_t,
Thank you for sharing the nice paper.
Our new coming Cluster oriented XU4 variant is less than $50 and total hardware cost will be much lower.
I hope you can write a user-friendly article with 4 free engineering samples for Cluster-computing beginners.
If you are interested in such a mini personal cluster, please give me a PM.
User avatar
odroid
Site Admin
 
Posts: 24662
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: Can you make boards with more RAM and/or modules...?

Unread postby nobe » Fri Aug 04, 2017 5:31 pm

@odroid
here is my advice : you should also contact Willy Tarreau and check if he wants to spend some time making a building farm using your upcoming XU4-Cluster version.
http://wiki.ant-computing.com/Choosing_ ... build_farm
among other things, he's a linux kernel maintainer https://www.kernel.org/category/releases.html
nobe
 
Posts: 96
Joined: Sun Feb 07, 2016 9:52 pm
languages_spoken: english, french
ODROIDs: Odroid-C2

Re: Can you make boards with more RAM and/or modules...?

Unread postby odroid » Fri Aug 04, 2017 6:14 pm

Thank you for the link. We will contact him once we have some samples in 1~2 weeks.

We will also contact below two people for the Docker Swarm implementation.
MrDreamBot: viewtopic.php?f=140&t=22671
gügü: viewtopic.php?f=114&t=18001
User avatar
odroid
Site Admin
 
Posts: 24662
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID


Return to The Ideas

Who is online

Users browsing this forum: No registered users and 2 guests