Porting project to ODROID devices.

Moderators: mdrjr, odroid

Porting project to ODROID devices.

Unread postby BaldNerd » Wed Sep 13, 2017 10:56 pm

Hello,
I'm brand new here!

I've only ever worked with Raspberry Pi as far as SBCs go, so that's where my experience lies. I'm very experienced with Linux.

My project, NEMS / Nagios Enterprise Monitoring Server has become quite popular on the Pi 3, and since Raspberry Pi doesn't intend to release the next gen board for nearly 2 more years and the specs of the Pi 3 are far behind other SBC manufacturers (like ODROID), I am very curious about porting NEMS over to other platforms - but have never, ever done such a thing. I'm familiar with multiarch for building say, a desktop version, but no clue how to go about making my image run on other boards.

ODROID is my first pick for a new port of NEMS. I'm very curious about the C2, and on the fence about the XU4 (since I think the C2 specs show it to be more than powerful enough for a very good NEMS server).

NEMS, at its heart, started with Raspbian Wheezy. It has followed the Debian upgrade path throughout its semi-annual release cycle and is currently Debian Stretch with Raspbian repos for drivers and so-on.

So is there anyone who could help me understand what is needed to build a port of the OS? I see DietPi and think, how the heck did they make so many ports?! Yes, that's envy. :)

I look forward to hearing from the community here and hope I can bring about support for ODROID devices!

Thanks!
Robbie // BaldNerd
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 12:46 am

Well, nagios doesn't need anything rpi-specific, so if you start with @meveric's debian image for c2 I expect the installation process will go the same way as for rpi.
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 12:52 am

Thanks mad_ady - yes of course I could take that route, but I'm asking specifically how to port it over, not rebuild the system on another distro: I want to take the already-built image and make it so it works on ODROID devices, if possible.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 1:39 am

Oh, in that case - it depends. Are your images server only? If yes, you don' need to worry about graphics/acceleration.

As a minimum you need just the kernel, boot.ini, uboot (and ideally the vfat partition) and /lib/modules on the rootfs.

You can extract those from the official ubuntu image.

Though:
* without proper integration you'd be missing kernel updates
* on C2 you'd be running with 64bit kernel and 32bit userspace. XU4 would be fine
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 1:58 am

Thanks. Yes it is strictly server... no GUI components at all - built to be headless.

So is what you're describing basically make a copy of the official image, then chroot to my image and replace the same?

Can it be done in a "proper" way (to use your term) where everything can remain updated via apt? I envision for example, removing the Raspbian sources.list.d entry, and replace it with an ODROID repo, then run a dist-upgrade. Does such a process exist?

Not sure what you mean about 64-bit. Do you mean the C2 won't run with an SMP Arm kernel?

Here's our current kernel.
Linux NEMS 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux

NEMS is Debian Stretch, not Ubuntu (in case that matters).
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 3:33 am

There are two ways of doing it:
1. Start from the official ubuntu/debian image and delete everything from the ext partition except /lib/modules, /etc/apt.sources.d and /etc/fstab. Rsync data from your rpi system to this partiton (except fstsb). Flash this image to the c2 and it should work (the original image resizes on first boot, you could keep this as well)
2. Use buildroot to package your image (harder to setup, easier to maintain)

Regarding kernel - c2 comes with 3.14, xu4 has 4.9. Both of them have custom patches and you'll need to get from HK. There is a mainline kernel for c2 that is suitable for headless opperation, but it's not yet supported by HK
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 3:40 am

Wow - great info, thank you mad_ady.

So do you think the XU4 is in fact better suited to this project? I suppose I could go with an older kernel, but if I can keep both images on similar kernel versions, it might help with easing my workload should something be in the newer kernel that is missing from the old.

I don't mind something being a challenge to set up... ultimately I have to look at the big picture... this is a distro I update with a new major version every 6 months. I don't want to have to duplicate work if I don't have to... if that makes sense. Though reading the first option, it sounds super easy.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 4:10 am

I don't know exactly what your image contains, but if it runs ok on a rpi, it will run just fine on a c1/c2/xu4. I think that unless you have a lot of nagios probes, the xu4 will be overkill.

In terms of porting complexity, both c2 and xu4 will be the same
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby meveric » Thu Sep 14, 2017 7:38 am

nagios can be very demanding in matter of hardware, especially on storage if you have many nodes and service checks you do.
At work our nagios server runs on SSDs in order to keep up with all the data it has to collect. In that matter the XU4 would be better, since it's USB 3.0 ports allow for faster storage devices as the ODROID C2 which only has access to USB 2.0.

I would be very careful with using eMMC, although rather fast, nagios is constantly writing data and I would be afraid, that the eMMC might wear down after a year or so.
Also in general write speed, a harddrive is faster than eMMC. While reading from eMMC can be faster than from an HDD (random read) the write will be very limited.

Also as some tests indicate, you need big eMMC modules (64GB) to be able to use full speed of the eMMC on writing (max. around 70MB/sec), smaller eMMC modules can limit your write speed even more (to below 30MB/sec) while regular USB 3.0 HDDs can probably write with an avg. of 100MB/sec (SSDs with UAS even up to 400MB/sec).

BTW I wouldn't try to port the entire OS you're using.
Nagios is part of many distros by default. You can simply do an apt-get install nagios3 on any Debian Jessie image and have nagios installed. After that it's just a matter of copying the configs from your RPi.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 7667
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: ODROID-X2,ODROID-U2,ODROID-U3,ODROID-XU-Lite, ODROID-XU3, ODROID-XU3-Lite, ODROID-C1, ODROID-XU4, ODROID-C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 10:20 pm

Thanks for the great info you two!

In terms of porting complexity, both c2 and xu4 will be the same

Okay, awesome. Are the images compatible between them? Ie., if I build on a C2 will it also work on an XU4? Or do I have to build separate images for each?

I would be very careful with using eMMC, although rather fast, nagios is constantly writing data and I would be afraid, that the eMMC might wear down after a year or so.

NEMS caches to RAM and then bursts to the drive as needed. Drive access is greatly reduced vs. a traditional Nagios installation. After all, the Pi is using SD :) Now, I have not used eMMC before, but is it (technology and reliability-wise) closer to SD or SSD? In my naivety I'd assumed it to be similar to say, an m.2 in regards to reliability, but you make it sound more like SD. Please clarify.

Nagios is part of many distros by default.

NEMS is not Nagios. Nagios is not NEMS. NEMS is not just "Debian with Nagios installed" - if it was, there'd be no point to it at all.. I could just post a blog telling people to
Code: Select all
apt install nagios3
:) I say this because it needs to be clear, NEMS is a distro. It is not just a bit of software stuck on Debian. It is highly optimized, heavily customized, and does a lot of things a "normal" Nagios install can't do (without an insane amount of work).

In that matter the XU4 would be better, since it's USB 3.0 ports allow for faster storage devices as the ODROID C2 which only has access to USB 2.0.

Excellent point, thank you. Using an external storage device would potentially be desirable. I could see an external SSD being useful, and certainly makes sense with the separate USB chip, unlike the Pi.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 11:04 pm

C2 and XU4 are not directly compatible in terms of firmware, because C2 runs arm64 while XU4 runs armhf (32bit). You could have a 32bit rootfs ported directly from your RPI image which would run on C1/C2/XU4, but you'd need different kernels, bootloaders and odroid package repos for each. The C2 can run armhf code, though it's not as efficient as arm64 (you're wasting register space). You can't run arm64 code on C1/XU4.
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 11:24 pm

Wow, that's actually surprising to hear. I'd have thought the XU4 would be 64-bit.

So, let's say I buy a C2... what would be the bullet list "how to" to make my image [hopefully] run?

This is what I gather from our discussion so far:
- Download and deploy the official Debian image for the C2
- Mount its filesystem and delete everything from the C2 filesystem except:
/lib/modules
/etc/apt.sources.d
/etc/fstab
- Copy all data, except /etc/fstab, from the NEMS image to the C2 filesystem.
- Boot the C2 on this new image.

Problems I foresee:
- NEMS runs a much newer kernel than the stock C2. Won't this be an issue if I replace /boot?
- Network interfaces will have different /dev assignments than the Pi and need to be updated. This is just a note to self.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Thu Sep 14, 2017 11:41 pm

You'd also need to review apt sources and remove anything rpi specific. Since rpi is debian based you should base it on top of meveric's images.
Boot files are not in /boot, but in a vfat partition mounted on /media/boot.
Before deleting anything check out /etc/rc.local. It references a resizing script which runs on first boot.

Kernel 3.14 should be fine, unless you have custom sysfs entries which might differ for older kernels.

There's also a 4.12 kernel in a separate thread you can use for a server image.
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Thu Sep 14, 2017 11:54 pm

meveric's images

Link please?

You'd also need to review apt sources and remove anything rpi specific.

Yes, so obvious I didn't think to add it to my list :p Will definitely do that, thanks for being thorough :)

Boot files are not in /boot

Oh okay - different filesystem layout than NEMS... so is there a specific partition I am to do this on? When you said "remove all except..." I expected /boot to be part of that, which I now understand is incorrect. So remove all ... from which partition / mountpoint?
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Fri Sep 15, 2017 12:14 am

You can replace /boot with your own copy. It's not used in the boot process. Read the manuals of C2 and XU4 to get a feeling about the system layout.
You can find the manuals and links to images on the wiki: https://wiki.odroid.com/odroid-c2/os_images/third_party
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Fri Sep 15, 2017 12:17 am

Okay great, thank you.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Fri Sep 15, 2017 12:22 am

Is there a reason the "current" image is old-stable and not stable?
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby meveric » Fri Sep 15, 2017 6:33 am

BaldNerd wrote:
Boot files are not in /boot

Oh okay - different filesystem layout than NEMS... so is there a specific partition I am to do this on? When you said "remove all except..." I expected /boot to be part of that, which I now understand is incorrect. So remove all ... from which partition / mountpoint?

mad_ady wrote:You can replace /boot with your own copy. It's not used in the boot process.
That's actually not true, this ONLY applies for the stock HardKernel Ubuntu image.
I believe most (if not all) other images including my Debian images mount the bootfs to /boot so replacing these files will break the system.

BaldNerd wrote:Is there a reason the "current" image is old-stable and not stable?

Both images are available for the C2:
Debian Jessie
Debian Stretch

Still I'm just a one man army and I support many boards from hardkernel with thousands of custom build packages for ODROIDs, and switching everything to Debian Stretch takes a while ;)
That's why generally Debian Jessie is better supported, although on the C2 it probably doesn't really matter, as it's not very good supported as it is anyway, arm64 is still very problematic under Linux.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 7667
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: ODROID-X2,ODROID-U2,ODROID-U3,ODROID-XU-Lite, ODROID-XU3, ODROID-XU3-Lite, ODROID-C1, ODROID-XU4, ODROID-C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Sat Sep 16, 2017 2:52 am

Excellent thank you!

Yeah - I hear ya. I can't believe you've done this much! Wow! I'm also just one man... and mine only works on a Raspberry Pi 3, and NOTHING else :p

Here I go, wanting to add it to a second device.

So meveric, do you suggest I stick with the C2, or jump right to the XU4 for this?
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby meveric » Sat Sep 16, 2017 3:56 am

I guess XU4 should be better as it's also an armhf device same as the RPi, and with that the image should be easier to port.
The C2 can run armhf as well, but still it's foreign architecture and may cause some trouble.

Aside from that the XU4 is the more powerful device, with faster CPU, RAM, USB 3.0 so depending on the size of your setup the XU4 has the better chances to keep up with a larger project.

I'm not saying the C2 wouldn't work, I just say the XU4 is probably easier to handle.
If you want to use the C2 anyway, I guess we should be able to figure out how to port this anyway, so shouldn't be too much to worry.
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 7667
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: ODROID-X2,ODROID-U2,ODROID-U3,ODROID-XU-Lite, ODROID-XU3, ODROID-XU3-Lite, ODROID-C1, ODROID-XU4, ODROID-C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Sat Sep 16, 2017 4:16 am

Thank you. Hmm... I'm really starting to think maybe I should do the XU4 just because it's so far different (performance) than the Pi 3. The C2 is still an enormous improvement over the Pi 3, but if I'm going to do the work to offer "a more powerful option" maybe the XU4 is the better one to focus on. The USB 3.0 thing is what's really pushing me that direction (as NEMS 2.0 will be introducing external storage options).

Do you concur with my assumptions?

I've been concerned about the heat of the XU4.

Amazon.ca has both devices, but they seem higher priced than they should be... is there a better place to buy a unit for development? I'm in Canada.

Thanks again.
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby mad_ady » Sat Sep 16, 2017 4:46 am

You can go with xu4q and lower cpu frequency to 1200MHz for a passive experience
User avatar
mad_ady
 
Posts: 2399
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU4, C1+, C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Sat Sep 16, 2017 5:06 am

Since the Pi 3 can scale up to 12000MHz, is there an advantage to the XU4 in this case (obviously the USB 3, UHS-I and 2GB RAM remain - but would the performance overall still be better?)
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english

Re: Porting project to ODROID devices.

Unread postby meveric » Sat Sep 16, 2017 7:18 am

BaldNerd wrote:Since the Pi 3 can scale up to 12000MHz, is there an advantage to the XU4 in this case (obviously the USB 3, UHS-I and 2GB RAM remain - but would the performance overall still be better?)

MHz values do not directly compare in the ARM world.
The ODROID C1 has about 1500 MHz per core same as the C2 still the C2 is a lot faster, cause MHz values can not directly be translated.

If you check https://en.wikipedia.org/wiki/List_of_A ... hitectures you can see why.

For example, the RPi3 has Cortex-A53 CPU cores if you check the table that means: 2.3 DMIPS/MHz per core
DMIPS = Dhrystone MIPS (Million Instructions per Second)
While the Cortex-A15 cores of the ODROID XU4 can do up to 4.01 DMIPS/MHz that's nearly twice as fast PER Mhz ;)
1200 MHz Cortex-A53 = 2760 DMIPS
1200 MHz Cortex-A15 = 4812 DMIPS

Still that's not all the advantages of the XU4, first you have 8 instead of 4 cores 4xCortex-A7 + 4xCortex-A15, second, RAM twice as much and it is a lot faster, means the data in the memory can be processed much faster on the XU4 than it can on the RPi3
RPi3 = 1GB LPDDR2 (900 MHz)
ODROID XU4 = 2GB LPDDR3 (933 MHz)
(DDR2 vs DDR3 Memory)

Also although 1200 MHz is a nice suggestion, you can easily run it on 1500, or 1800 MHz without heating issues.
You can also use a different governor, that clocks down or turns off cores that are not needed reducing the overall power consumption and heat.

Aside from external HDD, you also have the ability to use eMMC modules on the ODROID compared to the SDs on the RPis.
eMMC have reported to have as high as 190 MB/sec read speed some more values and comparison with RPi3 can be found on the HardKernel website: http://www.hardkernel.com/main/products ... 3452239825

Also don't forget about the Gigabit LAN, compared to the 100MBit on the RPi.

I won't talk about the blazing fast GPU performance, as you probably won't need it ;)
Donate to support my work on the ODROID GameStation Turbo Image for U2/U3 XU3/XU4 X2 X C1 as well as many other releases.
Check out the Games and Emulators section to find some of my work or check the files in my repository to find the software i build for ODROIDs.
If you want to add my repository to your image read my HOWTO integrate my repo into your image.
User avatar
meveric
 
Posts: 7667
Joined: Mon Feb 25, 2013 2:41 pm
languages_spoken: german, english
ODROIDs: ODROID-X2,ODROID-U2,ODROID-U3,ODROID-XU-Lite, ODROID-XU3, ODROID-XU3-Lite, ODROID-C1, ODROID-XU4, ODROID-C2

Re: Porting project to ODROID devices.

Unread postby BaldNerd » Sat Sep 16, 2017 9:58 pm

meveric, I so greatly appreciate you taking the time to lay this all out. Thank you.

Based on what I've learned in this thread and the direction I'd like to take NEMS, I have opted for the XU4Q.

Since my intent will be to run the XU4 in such a way (ie underclock) as to never have the fan spin up anyways, I opted for the larger heatsync. Like NEMS on the Raspberry Pi 3, I will determine the best governor... likely it can take care of throttling itself and I won't even have to do much there (after all, even at its low frequency of 1200 MHz, it'll still be a lot faster than the Pi 3 at its highest frequency) but when it's able, it can go up to 1.5+ if it's not too hot.

I've also bought the 16GB EMMC so I can build on that as well... would be a good option for our users.

And since the ultimate plan is to also allow NEMS to use external hard drives for its logs and regular data movement, I've also opted for the Cloudshell 2. I adore the SATA backplane, and the little screen makes me get ideas about the upcoming NEMS appliance. I can't wait to tinker with it.

Though I know a lot of our users will opt for the XU4 in a standard SBC case, this will give me the best possible development system to be able to build for all scenarios.

Perhaps a moderator should move this thread to the /XU4/Projects instead of C2... I'd come here thinking I was going to get a C2 for this... but the XU4 ultimately looks like the best board for my project (at least, for now - I can always branch out to more SBCs later, if needed).

And so it will be, NEMS on the Raspberry Pi 3 (as is now the case) will be the cheap and cheerful low-powered option. Stepping up from there will be the XU4 build for a more powerful, slightly more expensive version. And then the XU4Q with the CloudShell2 and 2x SSD drives for a very robust, high-powered solution.

Can't wait to start tinkering! Once the hardware arrives, this thread can go back to its original topic as I begin porting NEMS to the XU4Q.

Thanks again to you both,
Robbie Ferguson
http://www.category5.tv
BaldNerd
 
Posts: 13
Joined: Wed Sep 13, 2017 10:45 pm
languages_spoken: english


Return to Projects

Who is online

Users browsing this forum: No registered users and 1 guest