[Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Moderators: odroid, meveric, mdrjr

[Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Thu Jan 12, 2017 7:09 pm

Setting up your XU4 as a general-purpose NAS

I've got my XU4 with the intent of converting it into a NAS. However, I didn't want to settle on a specialized NAS distro like OpenMediaVault because I want my XU4 to do much more than being a plain old NAS. For instance, I plan on transcoding TV shows recorded on my TV to H264 using the XU4's hardware encoder (as described here: http://magazine.odroid.com/wp-content/u ... df#page=18) and also make use of the GPIO pins later on. One more issue I had with OpenMediaVault is that it runs on top of Debian and I wanted to keep using Ubuntu, to benefit from newer packages (that's a personal choice - I don't hate Debian with a passion).
So, I'll be losing much of the convenience of using a specialized distro and I'll have to discover alternate ways of doing things in a simple and user-friendly way. But hey, how else can you learn new things?

So - these are the steps we'll need to do:
  • Install the mainline kernel (4.9) (optional)
  • Install Webmin for easier management
  • Mount your disks
  • Set up network shares (Samba/NFS)
  • Install Owncloud
  • Secure and optimize the OS
Note that the instructions supplied do not go into the finest detail, but medium and advanced users should know their way around.

Install the mainline kernel (4.9)

Odroid XU3/4 has the benefit of having pretty good mainline kernel support - which I need specifically for video transcoding. Mainline kernel has benefits of newer drivers, better support but also comes with problems - buggy HMP support, USB3 instabilities and (at the time of this writing) no sound over HDMI. This means that you have to weigh in the pros and cons and make a decision for your needs.

In order to install mainline kernel, you will need to follow either the official instructions (Hardkernel is working on an official 4.9 kernel for XU3/4 and most likely will release it as a deb package) or follow the general instructions here: http://magazine.odroid.com/wp-content/u ... df#page=18

Remember to fully unplug your XU4 from all power sources (power, HDMI, USB) otherwise you will miss your network when you first boot.

Webmin
Every NAS needs a nice and shiny web gui. Unfortunately OpenMediaVault's GUI is not an option and after searching for a long while for alternatives I settled on using Webmin. Webmin has been around for decades (since 1997!) and has solid support for general server maintenance tasks. It has the advantage that even inexperienced users can find their way around and with the integrated help can set up and manage all kinds of servers like web, mysql, mail, dns, and others. It also has solid support for RAID and LVM management and also Samba and NFS file sharing. Unfortunately it lacks support for newer services like Transmission or Owncloud, but I can always configure them manually.
To install it, follow the steps below:
Code: Select all
$ echo "deb http://download.webmin.com/download/repository sarge contrib" | sudo tee /etc/apt/sources.list.d/webmin.list
$ wget http://www.webmin.com/jcameron-key.asc
$ sudo apt-key add jcameron-key.asc
$ rm jcameron-key.asc
$ sudo apt-get update
$ sudo apt-get install libapt-pkg-perl libnet-ssleay-perl libauthen-pam-perl libio-pty-perl apt-show-versions apt-transport-https
$ sudo apt-get install webmin
$ sudo systemctl enable webmin
$ sudo systemctl start webmin



You can login to your odroid's IP address on port 10000 to use the web interface:https://odroid-ip:10000. However after you log-in (with any system user with sudo access) you will likely be unimpressed by the default interface. It looks like out of the 1990s.
Image
Figure 1. Stock Webmin interface

The first thing we must do is beautify it via a theme. The best looking theme is called "Authentic Theme" and brings in a lot of features, including being mobile-friendly. You can get the latest version from https://github.com/qooob/authentic-theme/ and install it with:

Code: Select all
$ wget https://github.com/qooob/authentic-theme/releases/download/18.31/authentic-theme-18.31.wbt.gz

Navigate inside Webmin to Webmin Configuration->Webmin Themes->Install themes->From uploaded file and select the newly downloaded theme. After a short wait the new interface will refresh.

Image
Figure 2. Webmin with Authentic theme

Take your time to explore webmin's features by using the search tool in the interface. Also note that you can install third-party modules available at http://www.webmin.com/cgi-bin/search_th ... ?modules=1.

There is one thing you should change as soon as possible. By default webmin has a background process that monitors disk temperature, and for me it caused my disks to wake up every 5 minutes. After a laborious search I stumbled on this bug: https://sourceforge.net/p/webadmin/bugs/3999/ that explains how to turn off this "feature". Simply add the line collect_notemp=1 to /etc/webmin/system-status/config and restart the webmin process and you should be fine.

Mounting disks
First of all you will need to know/decide if you're going to use RAID or LVM on your disks and which filesystems you should use. I won't go into details about setting RAID/LVM because the subject has been discussed in previous Odroid Magazine articles, but even without having a lot of expertise, you can use Webmin to do the heavy lifting for you and you can use the built-in help to learn more. Webmin will also prompt you to install any missing dependencies. Once you have your partitions ready, you can consider mounting them.
The traditional way is to use /etc/fstab (and Webmin has a comprehensive module to handle that as well), but you may run into problems if you start your system with the disk not attached (systemd likes to wait around for the disk).
I prefer to use autofs, which mounts disks (local or network-based) on demand and unmounts it when not in use. Unfortunately it's not managed by webmin, so you'll need to use the shell. To install it run:
Code: Select all
$ sudo apt-get install autofs

You'll need to edit /etc/auto.master and add a mount entry for your disk specifying the base directory and its configuration file:
Code: Select all
$ sudo vi /etc/auto.master
# add at the end your mountpoint
/media/yourdisk /etc/auto.yourdisk --timeout 20

Replace yourdisk with the path you want to use. Next, edit this configuration file and add your partitions and their mount parameters:

Code: Select all
$ sudo vi /etc/auto.yourdisk
xfs-partition   -fstype=xfs,dev,exec,suid,noatime       :UUID=9d2d675d-cb08-45b2-b222-c981a8d00c06


Restart autofs and when you access /media/yourdisk/xfs-partition your partition will be mounted automatically:
Code: Select all
$ sudo service autofs restart


You will need to take care with the mount parameters - each filesystem has their own parameters and they might impact performance. For instance, without activating big_writes on NTFS you will get very poor performance. If in doubt, you can cheat and use Webmin to create entries in/etc/fstab, test them to ensure the parameters are ok and migrate them to autofs's layout later (that's what I used).
To force automounted disks to be unmounted, you can simply restart the autofs service.

Set up file shares
To set up Samba shares (and also install Transmission for torrent downloading) you can follow the guide "Designing your own seedbox" featured in Odroid Magazine (http://magazine.odroid.com/wp-content/u ... df#page=34). But you also can play around with Webmin's interface and easily create shares and users with a few clicks. For example, Figure 3 shows the "Create NFS share" dialog. Clicking on the form items shows a contextual help menu that explains in plain english what that item does. This can help you with things you might not be familiar with.

Image
Figure 3. Create NFS share

There is one thing I'd like to point out. When creating Samba/NFS shares take security into consideration from the start. Samba authenticates by user and password, but NFS authenticates users only by IP. If you know which hosts in your network may have access to specific shares, specify it in the configuration. For example, a NFS share might be exported to "Everyone", but access can still be limited with iptables or /etc/hosts.allowand/etc/hosts.deny (which are used by TCP Wrappers Webmin module).

Image
Figure 4. /etc/hosts.allow configuration for NFS to limit access from a few hosts

Used with its default configuration Samba will give decent performance, but with the tweaks below (extracted from the forum) you should get fewer "pauses" in large file transfers. Add the lines below to the [global] section of your /etc/samba/smb.conf:
Code: Select all
write cache size = 524288
getwd cache = yes
use sendfile = yes
min receivefile size = 16384


Install Owncloud

Owncloud is a personal "cloud" service that lets you share files with people over the internet. I'm not going to go into installation details too much (because have been discussed in a previous Magazine article http://magazine.odroid.com/wp-content/u ... pdf#page=8), but there are some things I'd like to point out.
First of all, the installation is pretty simple on Ubuntu 16.04. I used this guide: https://www.digitalocean.com/community/ ... untu-16-04 and was up and running in 10 minutes. If you have a DNS name (e.g. dynamic DNS for your home) you should take the time to get a valid SSL certificate from Let's Encrypt (https://letsencrypt.org/): https://www.digitalocean.com/community/ ... untu-16-04

You basically need to install the following prerequisites before installing OwnCloud:
Code: Select all
$ sudo apt-get install php libapache2-mod-php php-mcrypt php-mysql php-bz2 php-curl php-gd php-imagick php-intl php-mbstring php-xml php-zip mysql-server apache2

Next you install the OwnCloud repository for Ubuntu and refresh the available packages:
Code: Select all
$ sudo curl https://download.owncloud.org/download/repositories/stable/Ubuntu_16.04/Release.key | sudo apt-key add -
$ echo 'deb https://download.owncloud.org/download/repositories/stable/Ubuntu_16.04/ /' | sudo tee /etc/apt/sources.list.d/owncloud.list
$ sudo apt-get update

Finally, you can install owncloud:
Code: Select all
$ sudo apt-get install owncloud
$ sudo systemctl reload apache2


You will also need to create a database user for owncloud:
Code: Select all
$ sudo mysql -u root
> CREATE DATABASE owncloud;
> GRANT ALL ON owncloud.* to 'owncloud'@'localhost' IDENTIFIED BY 'databasePassword';
> FLUSH PRIVILEGES;
> exit


After all this work you can log in in the web interface at https://odroid-ip/owncloudand finish the installation.

Since the point of owncloud is to be accessible to the internet you should take some time to harden your installation. This is a good place to start: https://doc.owncloud.org/server/9.0/adm ... erver.html. In my case I want to run the owncloud service on a different port (so that external users don't have access to my internal sites), to set iptables rules to allow access only from my country (based on geo-ip data) and set up fail2ban to protect me against automated password guesses.

In order to run the owncloud virtual host on a different port you need to make a few adjustments to your apache config:
Code: Select all
$ sudo cp /etc/apache2/sites-available/default-ssl.conf /etc/apache2/sites-available/owncloud.conf
$ cd /etc/apache2/sites-available
$ sudo ln -s ../sites-available/owncloud.conf 020-owncloud.conf


Next, edit /etc/apache2/sites-available/owncloud.conf and make the following changes:
  • Add "Listen 8443" as the first row
  • Change the VirtualHost definition to use port 8443 instead of 443 (<VirtualHost _default_:8443>)
  • Change DocumentRoot to point to your owncloud installation "DocumentRoot /var/www/owncloud"
When done you can restart the apache daemon and you should be able to access only your owncloud instance onhttps://odroid-ip:8443/

To get started with GeoIP firewall rules, you'll need to have the kernel sources (or kernel headers) available. Next, you can install the extra iptables modules with:
Code: Select all
$ sudo apt-get install xtables-addons-dkms xtables-addons-common xtables-addons-source

The dkms package will fail to install cleanly because some of the modules fail to compile against kernel 4.9. You can disable the failed modules and recompile the rest by setting the following settings to "n" instead of "m" in /var/lib/dkms/xtables-addons/2.10/build/mconfig:
Code: Select all
$ sudo vi /var/lib/dkms/xtables-addons/2.10/build/mconfig
build_ACCOUNT=n
build_LOGMARK=n
build_SYSRQ=n
build_pknock=n
build_psd=n

Next you will need to manually compile the rest:
Code: Select all
$ cd /var/lib/dkms/xtables-addons/2.10/build/
$ sudo autoconf
$ sudo ./configure
$ sudo make
$ sudo make install

Before using the geoip module you will need to initialize the geoip database (the prefix to country mapping). You may need to repeat this step from time to time to benefit from the latest data:
Code: Select all
$ sudo apt-get install libtext-csv-xs-perl
$ sudo mkdir /usr/share/xt_geoip
$ sudo /usr/lib/xtables-addons/xt_geoip_dl
$ sudo /usr/lib/xtables-addons/xt_geoip_build -D /usr/share/xt_geoip /root/GeoIPCountryWhois.csv

All that's left to do now is to create (and test!) the iptables rules to allow only traffic that you want to reach your owncloud setup. An example rule looks like this:

Code: Select all
$ sudo iptables -N geo-owncloud
$ sudo iptables -A INPUT -p tcp -m tcp --dport 8443 -j geo-owncloud
$ sudo iptables -A geo-owncloud -s 192.168.1.0/24 -j ACCEPT
$ sudo iptables -A geo-owncloud -m geoip ! --src-cc RO -j DROP


Don't forget to save your rules and apply them at startup (either withiptables-save or with webmin). More details about geoip can be found here: http://blog.jeshurun.ca/technology/bloc ... bles-geoip

Configuring fail2ban is not very complicated once you follow the tutorial here: https://forum.owncloud.org/viewtopic.php?t=28678. Remember to install fail2ban first (and test it with some false credentials):
Code: Select all
$ sudo apt-get install fail2ban


Image
Figure 5. Fail2Ban doing its job on failed logins.

Since we've added a special port for owncloud, we'll need to tweak fail2ban's configuration to account for that. Edit /etc/fail2ban/jail.local and append port 8443 to the port line and restart fail2ban:
Code: Select all
$ sudo vi /etc/fail2ban/jail.local
port    =  http,https,8443
$ sudo service fail2ban restart


To manually lift the ban for a blacklisted IP address you can run:
Code: Select all
$ sudo fail2ban-client set owncloud unbanip 172.22.22.2


Tips and tricks

Emulating HMP (Heterogeneous Multi-Processing Device) (optional)
As you know, the XU4 comes with two types of CPU cores - 4 little cores that are low power and are best suited for background tasks and 4 big cores which are high power. The official 3.10 kernel comes with a "magic" scheduler from Samsung which knows the processor's true power and can switch tasks from the little cores to the big cores when load is high. This allows you to have a snappy experience. However, Samsung's patches have been rejected for the mainline kernel (https://lkml.org/lkml/2015/11/7/55) and this means the kernel will randomly assign tasks to different CPU and you will get inconsistent performance. There are plans to add this functionallity for big.Little systems (https://lwn.net/Articles/501501/), but there's nothing mature right now. We will need to emulate the functionality of HMP somehow… I know, let's use cgroups (https://www.youtube.com/watch?v=81j1WF5xEZc)!

CGroups is a feature of modern kernels to allow allocation of resources for various processes. In our case we will need the "cpuset" cgroup to create a "littlecores" and a "bigcores" group. Each group will force processes to run on specific cores by setting the affinity. So, littlecores will have cpus 0-3 and bigcores 4-7. Fortunately, creating the cgroups is easy:

Code: Select all
# mkdir -p /sys/fs/cgroup/cpuset/littlecores /sys/fs/cgroup/cpuset/bigcores
# echo "0-3" > /sys/fs/cgroup/cpuset/littlecores/cpuset.cpus
# echo "0"> /sys/fs/cgroup/cpuset/littlecores/cpuset.mems
# chmod -R 777 /sys/fs/cgroup/cpuset/littlecores
# echo "4-7"> /sys/fs/cgroup/cpuset/bigcores/cpuset.cpus
# echo "0"> /sys/fs/cgroup/cpuset/bigcores/cpuset.mems
# chmod -R 777 /sys/fs/cgroup/cpuset/bigcores


Unfortunately the commands will only last until the next reboot. So, let's create a service to set them as early as possible on boot:

Code: Select all
$ sudo wget -O /etc/systemd/system/cpuset.service https://raw.githubusercontent.com/mad-ady/odroid-xu4-optimizations/master/cpuset.service
$ sudo systemctl enable cpuset
$ sudo systemctl start cpuset


At this point the cgroups are created, but they are not actively used by anyone. To manually start a process in a specific cgroup you can use cgexec:

Code: Select all
$ sudo apt-get install cgroup-tools
$ cgexec -g cpuset:bigcores sysbench --test=cpu --cpu-max-prime=100000 --num-threads=8 run


Image
Figure 6. 8 sysbench threads are forced to run on 4 specific cores

But we're only halfway there. We will need to tell specific process to run on the little cores and others to run on the big cores. This is where you need to make a list and decide what you want. Start with a list of active services from webmin (System -> Bootup and Shutdown) and disable anything you're not using. In my case I've disabled the following services: ModemManager, NetworkManager-wait-online, NetworkManager, accounts-daemon, alsa-restore, alsa-state, apport, apport-forward.socket, bluetooth, cups-browsed, cups.path, cups.service, cups.socket, lightdm, lxc-net, lxc, lxcfs, plymouth*, rsync, saned, speech-dispatcher, whoopsie.

You will need to edit the startup scripts for the services you want and have them add their PID to the correct cgroup. Once the main process (and its children) are part of the correct cgroup any new children will inherit the cgroup. My plan was to add things like mysql, apache, samba, nfs and even webmin to the big group and things like ssh (and all my shell activity), cron, munin, transmission to the little group. This allows processes that are involved in the NAS functionality to be snappy, while other tasks can happily run on the little cores. If you're also using the X11 GUI, you might want to add lightdm to the bigcores as well.

There are two types of startup scripts - systemd native scripts and legacy sys-v (/etc/init.d/).
When editing a systemd script (for example nfs-mountd.service) you will need to add something like this to the [Service] section:

Code: Select all
ExecStartPost=-/bin/sh -c 'echo $MAINPID | tee -a /sys/fs/cgroup/cpuset/bigcores/tasks'


When editing an older sys-v script, it's trickier. You will need to find the start function, extract the PID (or PIDs) of the newly started process and add it to the tasks list. Below is an example for changing the apache startup script:

Code: Select all
pidof apache2 | tr " " "\0"| xargs -0 -n1 | sudo tee -a /sys/fs/cgroup/cpuset/bigcores/tasks


Image
Figure 7. Changing apache's startup configuration

Take care to restart each service after changing it and make sure to check that the process PID is in the correct cpuset tasks file. Do a full system reboot and check again after restart.

If this sounds too complicated and you're using kernel 4.9, there is a way to cheat and run all the tasks on the big cores. You can simply set systemd's affinity and all its children processes will inherit it. The affinity can be controlled by the CPUAffinity parameter in /etc/systemd/system.conf, but keep in mind you'll be wasting CPU cores.

Disk longevity
In order to prolong the life of your disk(s) you may want to spin them down after a period of inactivity. If you're using SSDs then you can skip this section because it only applies to old mechanical disks. Disks may receive a "stop" command to spin down either from an internal controller, from the USB-SATA bridge or directly from the operating system. But sometimes the controllers are not tuned correctly and a stop command never arrives. This causes the disk to keep spinning which generates a lot of heat and causes the drive to fail sooner than normal.
The normal way to handle this is to tell the disk to spin down after a period of inactivity, which can be done with hdparm.
Code: Select all
$ sudo apt-get install sdparm hdparm


To manually set the disk to sleep after 10 minutes of inactivity, you can run
Code: Select all
$ sudo hdparm -S 120 /dev/sda

If you get errors (like "bad/missing sense data") hdparm might not help you for that disk.

But, to handle disk mobility, it would be better to let udev run the command after a disk has been plugged in. Since different disks might have different roles and you may want different sleep timers (e.g. one disk is for backups and should sleep sooner, other is active and should sleep later), I decided on setting the UDEV rule based on the disk's serial number. You can get this serial number by looking in dmesg when plugging in a disk:
Code: Select all
[1885221.800435] usb 4-1.3: Product: My Passport 0730
[1885221.800436] usb 4-1.3: Manufacturer: Western Digital
[1885221.800437] usb 4-1.3: SerialNumber: 575844314141305636323937



To set-up the rule you can create a file like this and reload udev:
Code: Select all
$ sudo vi /etc/udev/rules.d/90-disk.rules
ACTION=="add", ENV{DEVNAME}=="/dev/sd?", SUBSYSTEM=="block", ENV{ID_SERIAL_SHORT}=="575844314141305636323937", RUN+="/sbin/hdparm -S 120 $env{DEVNAME}"
$ sudo udevadm control -R


If hdparm can't put your disk to sleep try other alternatives - like sdparm which can send a SCSI command to your disk, like ordering it to shut down in that instant:
Code: Select all
$ sudo sdparm -C stop /dev/sda


There are tools (like hd-idle - viewtopic.php?f=52&t=24239) or periodic scripts you can run to put your disk to sleep. In my case they didn't work, but make sure to try them manually before settling on a solution. Here's a manual script which checks a disk (identified by a partition's UUID) for activity in a 10s window and if there was no disk activity (data transferred), it uses sdparm to stop the disk. You can run it via cron:
Code: Select all
$ sudo wget -O /usr/local/bin/hdd-idle.sh https://raw.githubusercontent.com/mad-ady/odroid-xu4-optimizations/master/hdd-idle.sh
$ sudo chmod a+x /usr/local/bin/hdd-idle.sh
$ sudo /usr/local/bin/hdd-idle.sh "4283-E975"


You must be aware that there are tools and services which will wake up your disk periodically, even if no data is transferred. Such tools include smartctl (from smartmontools) and smartd. The smartd service periodically checks disk health and if not correctly configured it may keep your disk up needlessly. You can consult this thread in case you don't know what's keeping your disk awake: viewtopic.php?f=95&t=22049
You should be able to infer the disk's state by running this command:
Code: Select all
$ sudo smartctl -d sat -n standby -a /dev/sda

If it exits with an error, your disk is still in standby and should have been spun-down.

Flash disk performance
One more thing to keep in mind when using flash storage (eMMC or SSD) is that they need periodic trimming to maintain their speed. Basically, in order to write to a storage block you need to erase it first and this takes longer than writing to it. Normal filesystems do not do this erase when deleting data so after a while disk performance drops significantly. To "revive" the disk, the trim operation informs the disk controller to erase all empty blocks, thus restoring write speeds. The trim operation must be supported by the filesystem and the disk controller. Again, using cron once a week to run fstrim can save you from slowdowns in the long-run:
Code: Select all
$ sudo crontab -e
#trim the eMMC once a week
15 0 0 * *      /sbin/fstrim / >/dev/null 2>&1


Governor
Performance and heat are also directly dependent on what governor you're using for the CPU. Keeping "performance" on gets you top performance, but also generates a lot of useless heat. In my tests the best combination was a tweaked out "ondemand" governor based on the recommandations here: https://obihoernchen.net/1235/odroid-xu ... ediavault/
To enable it, make sure you select governor ondemand in /media/boot/boot.ini, and set the rest of the parameters inside /etc/rc.local (test out the commands before). The commands below work for a 4.9 kernel and may differ for the 3.10 kernel:

Code: Select all
$ sudo vi /etc/rc.local
echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy
echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
echo 80 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold



With the setting above the CPU will ramp up frequency sooner and will consider IO usage as CPU, making IO intensive tasks influence the CPU frequency. This allows you to have great performance when needed and low heat when idle. In my usage the little cores idle around 300MHz while the big cores idle at 200MHz.

Network performance - MTU
If you have a Gigabit network with proper cables you can increase the MTU (Maximum Transmission Unit) on the Odroid's onboard network. This will allow it to send larger packets which have less overhead and generate fewer interrupts on the receiving end. The thing is - to benefit from it you will need to have network devices (switches/routers) and end devices which support Jumbo frames. Ideally, Jumbo frames would need to be enabled on all network devices in your LAN, otherwise you might see dropped traffic or even devices unable to send large traffic to each other (e.g. SSH works because it uses small packets, but getting a web page or transferring a file stalls the connection). If you do decide to enable jumbo frames, the XU4's magic MTU value is 6975 (viewtopic.php?f=95&t=20642&p=171441). You can enable it on the XU4 inside /etc/rc.local:
Code: Select all
$ sudo vi /etc/rc.local
#MTU
/sbin/ifconfig eth0 mtu 6975 up


Network performance - interrupts
Another big cause of performance drops at a high traffic rate is the fact that the interrupt handler on the odroid runs on CPU0 by default, which is a little core. You can get another 150+Mbps in network speed if you move the network card's interrupt handler to a big core. I've setup a service that moves all USB interrupt handlers to the big cores. You can download it and install it with:

Code: Select all
$ sudo wget -O /etc/systemd/system/affinity.service https://raw.githubusercontent.com/mad-ady/odroid-xu4-optimizations/master/affinity.service
$ sudo systemctl enable affinity
$ sudo systemctl start affinity


You can check that it worked by looking inside /proc/interrupts and seeing values in the 6th column (corresponding to the 4th CPU) for the USB bus. Also, iperf will report faster download speeds. There are other possible improvements you can do, but in my use case I saw no difference in performance. You can review and discuss them here: viewtopic.php?f=98&t=14907&p=172122#p172122

Fastest transfers over sshfs/scp/sftp
Since SSH is a very flexible protocol and supports tunnelling and file transfer it would be wise to use it at full speed. If you attempt a scp transfer on an XU4 with the sshd process tied to the little cores you will get about 15MB/s top speed. If you tie the sshd process to the big cores you get 40MB/s. If you're feeling adventurous and don't mind sacrificing some security, you can squeeze 50MB/s by lowering the encryption algorithm used. I did that by starting a different sshd instance (on port 2222) with different settings:

Code: Select all
$ sudo wget -O /etc/systemd/system/ssh-big.service https://raw.githubusercontent.com/mad-ady/odroid-xu4-optimizations/master/ssh-big.service
$ sudo wget -O /etc/ssh/sshd_config_big https://raw.githubusercontent.com/mad-ady/odroid-xu4-optimizations/master/sshd_config_big
$ sudo systemctl enable ssh-big
$ sudo systemctl start ssh-big


To mount or transfer a file using this new ssh service you'll need to specifically specify the cipher (since it's disabled by default because it's considered weak). You can do so in an entry in ~/.ssh/config on the client:

Code: Select all
Host odroid-big
Hostname odroid-ip
Port 2222
Ciphers arcfour
Compression no



To transfer files you can simply do:
Code: Select all
$ scp bigfile odroid-big:/remote/path


Tune systemd timeouts
It can be irritating to wait around for systemd to finish waiting for something that will never finish. So, you can tweak systemd's timeouts by modifying the global timeout settings in /etc/systemd/system.conf:

Code: Select all
DefaultTimeoutStartSec=20s
DefaultTimeoutStopSec=10s



Note that some services (like networking) set explicit timeouts and you'll need to change those as well:
Code: Select all
$ sudo vi /etc/systemd/system/network-online.target.wants/networking.service
TimeoutStartSec=30sec


Performance to be expected

Here are some performance metrics you can expect with the tweaks above and a gigabit network. The client is a C2 running Ubuntu 16.04, while the server is the XU4. The download and upload directions are relative to the XU4. The disk attached to the XU4 has a write speed of 110MB/s. File transfers transferred an 8GB file filled with zeros (dd if=/dev/zero of=zero bs=1M count=8000 conv=fsync). Please note that part of the performance depends on your client as well. I was able to get better performance with a Linux PC than with the C2 as a client.

Image

Fell free to discuss possible optimizations on this support thread
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Mon Mar 20, 2017 5:15 am

Hello Adrian. Can you please recheck the guide? I git stuck at running cgexec command. I upgraded to Debian 8 and I want to apply your cpuset and affinity tweaks.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Mon Mar 20, 2017 1:25 pm

What happens when you run cgexec? Also, did you setup the two big/little cgroups? Any errors that you can post?
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Mon Mar 20, 2017 5:22 pm

I have enabled cpuset service, I have ran it, then I tried running the cgexec command. I even checked that there is not issue with the command because of Windows, I am connecting with KiTTY, a PuTTY clone. The error that appears after running cgexec is "no such file or directory".
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Tue Mar 21, 2017 4:28 am

Have you installed the cgroup-tools package? Anyway, cgexec is not the most important (it's just for convenience). You can just sudo tee $pid /sys/fs/cgroup/cpuset/bigcores/tasks
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Tue Mar 21, 2017 5:43 am

Yes, I have installed the cgroup-tools package. I will just drop the pids of the processes I want to run on big cores into that variable. I though it needed that command as well in order to function. I am in the process of creating a service or maybe a script that takes the list of programs from a file, looks up their pids and puts them in /sys/fs/cgroup/cpuset/bigcores/tasks. I cannot use pidof because it does not get all the pids, I can use pgrep, but for some (still) unknown reason for and tee do not like when several pids are being supplied by pgrep.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Tue Mar 21, 2017 1:40 pm

This worked for me to get multiple pids at once: pidof smbd | tr " " "\0"| xargs -0 -n1 | sudo tee -a /sys/fs/cgroup/cpuset/bigcores/tasks
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Wed Mar 22, 2017 3:36 am

Ok, I may have done something wrong. How can I take out PIDs from /sys/fs/cgroup/cpuset/bigcores/tasks? Rebooting is not an option, obviously :)
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby odroid » Wed Mar 22, 2017 6:58 am

@mad_ady
I've sent you a PM. Please check it when you have spare time. ;)
User avatar
odroid
Site Admin
 
Posts: 22340
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English
ODROIDs: ODROID

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Wed Mar 22, 2017 2:02 pm

@tmihai20: I haven't tried it but I think you can overwrite the tasks file with only the pids you want to keep. E.g. echo "" > /sys/.../tasks
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Thu Mar 23, 2017 4:45 am

mad_ady wrote:@tmihai20: I haven't tried it but I think you can overwrite the tasks file with only the pids you want to keep. E.g. echo "" > /sys/.../tasks


This does not work and I have not worked with sysfs that much. A normal echo does not do anything. I tried looking on Google, but I did not find anything yet.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby rooted » Thu Mar 23, 2017 5:59 am

@mad_ady

Could you run a transfer with netcat on both ends and see the speed results of a no overhead file transfer?

No hurry at all, just if you find time.
User avatar
rooted
 
Posts: 3561
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby mad_ady » Thu Mar 23, 2017 6:26 pm

@tmihai20: I played a bit and here is how you can move a running process between cgroups:
1. Start the process
Code: Select all
$htop
$pidof htop
7147

2. Assign to the big cores cgroup
Code: Select all
#grep 7147 /sys/fs/cgroup/cpuset/bigcores/tasks
#echo 7147| tee -a /sys/fs/cgroup/cpuset/bigcores/tasks
7147
#grep 7147 /sys/fs/cgroup/cpuset/bigcores/tasks
7147

3. Assign back the process to the cpuset (parent group that holds all processes by default)
Code: Select all
#grep 7147 /sys/fs/cgroup/cpuset/tasks
#echo 7147 | tee -a /sys/fs/cgroup/cpuset/tasks
7147
#grep 7147 /sys/fs/cgroup/cpuset/bigcores/tasks


Hope this helps you.

@rooted: Unfortunately my remote link home failed while doing the tests, so I only have a TCP test done. I will paste back exact commands when I can connect back, but here is a screenshot of netcat doing transfer (client sending is a C2, XU4 is receiving, TCP transfer):
Code: Select all
c2$ nc -w 3 192.168.1.5 1234 </dev/zero
xu4$ cgexec -g cpuset:bigcores nc -l -p 1234 > /dev/null


Transfer speed is in bytes/s.
Image

@rooted: It seems I can't do the UDP test remotely. After the traffic starts flowing I completely lose connectivity with my router. Most likely it's overwhelmed by the amount of UDP traffic and is too busy. Although the C2 and XU4 are connected in switch ports which should be processed completely in hardware. Anyway, all this ends when some processes on the XU4 run out of memory and are killed by oom-killer and thus breaks the netcat stream and I am able to ssh back into my network... I'll try to do it when I'm connected locally - but I generally don't have time when at home.
Last edited by mad_ady on Thu Mar 23, 2017 7:14 pm, edited 1 time in total.
User avatar
mad_ady
 
Posts: 1565
Joined: Wed Jul 15, 2015 5:00 pm
Location: Bucharest, Romania
languages_spoken: english
ODROIDs: XU3, C1+, C2

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby tmihai20 » Thu Mar 23, 2017 7:13 pm

@mad_ady: thank you for the ideea. I did not think to move the processes back to cpuset. I will try this tonight when I get back home.
Riddle me this, riddle me that
Who is afraid of the big, black bat?
I write (in Romanian mostly) on a blog (see my profile)
tmihai20
 
Posts: 40
Joined: Mon Nov 07, 2016 10:56 pm
Location: Romania
languages_spoken: english, french, italian, romanian
ODROIDs: XU4

Re: [Howto] XU4 4.9 kernel, NAS, Webmin, Owncloud and tweaks

Unread postby rooted » Fri Mar 24, 2017 6:12 am

Thanks mad_ady, TCP test is fine.

It's interesting the transfer was slower than samba, it shouldn't be it should be faster than any conventional method that doesn't use caching to increase thruput.
User avatar
rooted
 
Posts: 3561
Joined: Fri Dec 19, 2014 9:12 am
Location: Gulf of Mexico, US
languages_spoken: english
ODROIDs: C1
C1+
C2
XU3 Lite
XU4
VU7+
HiFi Shield 2
Smart Power (original)


Return to Ubuntu (All Linux'es)

Who is online

Users browsing this forum: No registered users and 3 guests