HOWTO: Create a Backup Server Using Clonezilla and PXE

Post Reply
lazlo
Posts: 143
Joined: Sun Oct 01, 2017 11:32 am
languages_spoken: english
ODROIDs: XU4
Has thanked: 0
Been thanked: 0
Contact:

HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by lazlo » Mon Jan 21, 2019 10:29 am

HOWTO: Create a Backup Server Using Clonezilla and PXE

Introduction


Purpose: The goal of this HOWTO is to walk the reader through the creation of a simple and minimal network boot environment that allows any computer that supports PXE to boot Clonezilla over the local subnet for the purpose creating and restoring disk images. This will require some knowledge of TCP/IP networking and the installation/configuration of a software packages on your *nix distribution of choice. This guide will try to be distro agnostic (hopefully) allowing the reader to use which ever flavor of *nix they like best. This also means that the location and layout of the configuration files on your system may differ from what is listed in this guide. If you get stuck please consult your distribution’s documentation for the given package and once you find a solution please leave a reply here to help other readers that come after you.

Why Clonezilla? Clonezilla is nothing short of an Enterprise Class disk imaging suite. It runs on any x86 (32bit or 64bit) computer that can run Debian, it can both create and restore images of whole disks or individual partitions, it can compress and encrypt disk images, it supports a wide variety of network protocols, and it is truly free software. Read more at https://www.clonezilla.org and while you are there download the .zip file version of the installer and place it on the system you will using as the PXE server. We will need that later.

Why PXE? For years my monthly backups went something like this: 1.) Look for my Clonezilla USB thumb drive, 2.) Realize I must have accidentally over written it in the past month, 3.) Create a new Clonezilla USB thumb drive, and 4.) A day later find my old Clonezilla USB thumb drive. Monthly backups have been so much simpler and less frustrating since I learned how to set up PXE.

How PXE booting works (in a nutshell): When the Client system is booted via PXE the first thing that happens is the client broadcasts a request for a DHCP Server to tell it not just what the Client’s place is on the network but also where to find the files it needs to boot. The Client contacts the TFTP Server it was referred it to by the DHCP Server and begins downloading the files it needs and stores them in RAM. Once this is complete the client then boots up from the files in RAM and is ready to be used.

What does this mean? This means we will need a DHCP Server, a TFTP Server, a Client to back up, and optionally a File Server to store the backups. The DHCP server can be a home router, or stand alone server (like the ISC DHCP server). If the DHCP server is a home router (or if you have some other DHCP server that you can’t reconfigure on your subnet) then we will need to use the dnsmasq package to act as a proxy between the client and the DHCP server. While dnsmasq is able to act as a TFTP (Trivial File Transfer Protocol) server, tftp-hpa is much more flexible if you ever want to expand your PXE server beyond the scope of this guide. It’s what I use on my home server and it is what I will explain in this guide. The Client can be any x86 (32bit or 64bit) computer that supports booting via PXE and can run Debian. UEFI only mode will not work with this guide. I will start working on that soon though. The File Server can be almost anything, it just needs a stable (and hopefully fast) network connection and enough disk space to hold the backups. On my home server I use NFS, but you can also use Samba/CIFS, SSH, Amazon AWS S3, and a few others. Finally, the DHCP, TFTP, and File Servers can all be installed in the same OS or they can all three be on different physical servers, VMs, and/or Containers.

What should you do before changing anything on your computers and/or network? You should read this guide from start to finish before doing anything or changing anything. Read the examples and follow the links. If you are not sure about something please remember that asking a question before you take action is better than asking how to fix a broken network. You should make a back up copy of all configuration files before editing them as you follow this guide. If the system(s) you are working with contain data you care about at all you should back that data up before you start. Remember that a backup is only a backup if you can actually restore it.
Last edited by lazlo on Tue Jan 22, 2019 4:22 am, edited 7 times in total.

lazlo
Posts: 143
Joined: Sun Oct 01, 2017 11:32 am
languages_spoken: english
ODROIDs: XU4
Has thanked: 0
Been thanked: 0
Contact:

Re: HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by lazlo » Mon Jan 21, 2019 10:30 am

Part One: DHCP


dnsmaq as a DHCP Proxy

A word of caution before we begin: The dnsmasq package is used in wide variety of software stacks and you may already have it installed and running on your system. If that is the case then it is wise to ensure any changes you make to it’s configuration will not interfere with how it is being used elsewhere on your computer. There are a lot of ways to configure dnsmasq and a lot of documentation for how to use it. If you have never read the example config file for the package you can usually find it located in /usr/share/doc/dnsmasq-something-something-depending-on-your-distro. The file is well documented and educational thanks to it’s large number of examples and comments. The man page for dnsmasq can be found at http://www.thekelleys.org.uk/dnsmasq/do ... q-man.html

If you do not already have dnsmasq then please install it at this point. The main config file is normally located at /etc/dnsmasq.conf or /etc/dnsmasq/dnsmasq.conf or /etc/dnsmasq.d/some_program_that_uses_dnsmasq.conf depending on your distro.

Here is the config for a bare bones dhcp proxy:

Code: Select all

#This option enables some extra logging which might help with troubleshooting:

log-dhcp

#This option disables DNS services.  You should be getting DNS server info from 
#elsewhere if you installed dnsmasq just for the DHCP proxy.  If dnsmasq was
#already installed as part of a virtualization stack this might break your VM’s so
#test it early.  You may have to ommit it:

port=0  

#A bit counter intuitive, this entry actually disables DHCP in dnsmasq and 
#tells it act as a proxy for all DHCP requests on the 192.168.0.0/24 subnet.  
#The safest option is to use the ip address of this server’s NIC, but any valid IP 
#address in your subnet will work:

dhcp-range=192.168.0.151,proxy,255.255.255.0

#This option is mainly for compatibility with older or broken DHCP clients.
#It doesn’t hurt to be careful:

dhcp-no-override

#This option tells the PXE what file it needs to start booting and where to get it.
#The path to the file pxelinux.0 is relative to the root directory for tftpd.
#Since this guide is about simple and minimal we will put it in the root dir.
#The IP address is for the TFTP server, which may or may not be the one running 
#our DHCP proxy:

dhcp-boot=pxelinux.0,192.168.0.151

#This option disables multicast and starts the download right away:

dhcp-option=vendor:PXEClient,6,2b
Don’t forget to restart the dnsmasq service after editing it’s config file.

ISC DHCP Server

The ISC DHCP Server is the standard DHCP server for most versions of Linux and Unix. You will want to use this if you want a powerful yet flexible DHCP server on your subnet. It is very well documented and it’s config is file well commented and that makes it educational as well. It’s home page is at https://www.isc.org/downloads/dhcp/ and a copy of the man pages can be found at https://www.isc.org/dhcp-manual-pages/ so take a bit of time and read over them as well as any distro specific documentation before you install it. The package name can vary from distro to distro but usually it is either “isc-dhcpd-server” or “dhcp” but you might have to search a bit. The main config file for the package is typically found in /etc/dhcp/dhcpd.conf and some distros might have a second file located at /etc/default/dhcp. Below is an excerpt from the dhcpd.conf on my home server with my own comments added:

Code: Select all

 
#Start the subnet definition:
subnet 192.168.123.0 netmask 255.255.255.0 {
#Authoritative means that this is the main DHCP server on this subnet:
        authoritative;  
#This is the range of IP addresses given out by dhcpd:
        range 192.168.123.50 192.168.123.150;
#These next lines define the network topology for the subnet and allow 
#dhcpd to pass the needed parameters to the dhcp clients:
        option broadcast-address 192.168.123.255;
        option domain-name-servers 192.168.123.1, 192.168.0.1;
#I have my server set up to act as a router so this is the address of the LAN
#side NIC.  Otherwise enter the default gateway for your subnet:
        option routers 192.168.123.1;
#This is name of the file needed to start the PXE boot process. It’s path
#is relative to the TFTP root directory:
        filename "pxelinux.0";
#This is the IP address of the TFTP server that will be sending files to
#our PXE clients:
        next-server 192.168.123.1;
}


After you edit the config to meet your needs restart the dhcpd service.
Last edited by lazlo on Mon Jan 21, 2019 8:40 pm, edited 1 time in total.

lazlo
Posts: 143
Joined: Sun Oct 01, 2017 11:32 am
languages_spoken: english
ODROIDs: XU4
Has thanked: 0
Been thanked: 0
Contact:

Re: HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by lazlo » Mon Jan 21, 2019 10:31 am

Part Two: PXElinux and tftp-hpa


PXElinux

In both of the previous examples we have configured DHCP to tell a PXE client trying to boot to download the file named “pxelinux.0” from a specific server. This file is part of the pxelinux package of boot loaders. So go ahead and install pxelinux on the same system you will be installing tftp-hpa on. You can find the Syslinux homepage at https://www.syslinux.org/wiki/index.php ... ux_Project and the pxelinux docs at https://www.syslinux.org/wiki/index.php?title=PXELINUX If you ever have plans for expanding your PXE server beyond this little Clonezilla exercise consider studying it.


Setting up tftp-hpa and making Clonezilla bootable over the network

The tftp-hpa package is a very small, light weight FTP server used almost exclusively for booting over a network. It is light on security so don’t deploy it in a hostile environment unless you have another layer or two of security covering it. Because it is so light weight configuration is pretty simple but not very standardized. The most important option is setting the root folder for tftpd. A lot of people use some place like /var/lib/tftpboot or /srv/tftpboot or even /opt/tftpboot. I have seen example configs that just have the user create the directory /tftp but I think it’s better not to clutter / with more top level directories. In the end I chose /srv/tftp.root just because I wanted it to stand out in my mind. After you have it installed pick a location, create the directory, and then edit the config file. The config file location varies pretty widely depending on the disto you are using. It could be /etc/default/tftpd-hpa or /etc/xinetd.d/tftp, or even /etc/conf.d/in.tftpd so you will need to consult your distros documentation to make sure you have if configured right. For the purposes of this HOWTO I will use /var/lib/tftpboot as the root directory.

First, lets get pxeliunx.0 in place. It’s installed location will again depend on your distro but it will usually be somewhere is /usr/share or /usr/lib. After we copy it over we will create one directory to hold the files we extract from the Clonezilla .zip file and another directory to hold our boot parameters.

Code: Select all

sudo cp /usr/lib/PXELINUX/pxelinux.0 /var/lib/tftpboot
sudo mkdir /var/lib/tftpboot/cz
sudo mkdir /var/lib/tftpboot/pxelinux.cfg
Next copy over the Clonezilla zip file, extract it, and copy a few files into place:

Code: Select all

sudo cp /path/to/clonezilla-live-2.6.0-37-amd64.zip /var/lib/tftpboot/cz/
cd /var/lib/tftpboot/cz/
sudo unzip -X *.zip
sudo cp /var/lib/tftpboot/cz/syslinux/*.c32 /var/lib/tftpboot
Then we create the PXE bootloader config file. For this HOWTO it will be nothing fancy, or even very pretty. It will just load what we tell it to after a 10 second countdown:

Code: Select all

cd /var/lib/tftpboot/pxelinux.cfg
sudo nano default
Place the following in the “default” config file and don't forget to change the IP address in the last line. As a side note, I found this template somewhere online a while back and I don’t remember where or I would give credit:

Code: Select all

# The default menu style - using vesa menu in this example
DEFAULT vesamenu.c32
# If you have a png image in the tftpr oot directory you can specify it here like so:
# Menu Background image.png
# Prompt user for selection
prompt 0

#Global label identifier
label Clonezilla
        # Set this entry as the default selection
        menu default
        # Actual viewable label text
        MENU LABEL Clonezilla
        # The timeout for the entry is a bit unclear, but 100 should be equivalent to 10 Seconds. 
        TIMEOUT 100
        TOTALTIMEOUT 100
        # The kernel image to load.  This entry would actually reside at /var/lib/tftpboot/cz/live/vmlinuz   
	  #The path is relative to /var/lib/tftpboot or your tftp root directory
        kernel cz/live/vmlinuz
        # The initrd relative to the /var/lib/tftpboot directory and specifying the netboot server, protocol, and file
        # In this example the tftp protocol is used on server 192.168.123.1. The file is filesystem.squashfs
        append initrd=cz/live/initrd.img boot=live username=user union=overlay config components quiet noswap edd=on nomodeset nodmraid locales= keyboard-layouts= ocs_live_run="ocs-live-general" ocs_live_extra_param="" ocs_live_batch=no net.ifnames=0 nosplash noprompt fetch=tftp://192.168.123.1/cz/live/filesystem.squashfs
Now restart the tftpd service and boot a client computer with PXE. It should work.


Extra Reading

https://help.ubuntu.com/community/UbuntuLTSP/ProxyDHCP <== An Oldie but a Goodie
https://clonezilla.org/livepxe.php <== The Document that Inspired this HOWTO
https://clonezilla.org/clonezilla-SE/ <== Setting up a Dedicated Clonezilla Server for Large Scale Deployment
https://wiki.gentoo.org/wiki/Home_router <== Using dnsmasq and iptables to turn a server with two NICs into a router
Last edited by lazlo on Mon Jan 21, 2019 9:30 pm, edited 4 times in total.

lazlo
Posts: 143
Joined: Sun Oct 01, 2017 11:32 am
languages_spoken: english
ODROIDs: XU4
Has thanked: 0
Been thanked: 0
Contact:

Re: HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by lazlo » Mon Jan 21, 2019 10:33 am

Part Three: Creating a File Server With the Network File System

The most common file sharing protocol in the entire *nix ecosystem has to be the Network File System. NFS is what you could call ancient technology. The first versions were developed in-house by Sun Microsystems in 1984. It will soon reach it’s 35th birthday and it’s last major revision was in 2016. So why is a file sharing protocol that old still in use? Well, for the same reason the wheel is still in use. It just works. You can use NFS to share your users’ /home directories from a single server so than no matter what *nix system they log into on your network they will have the exact same contents in /home/user-name. You can use NFS to store the root file systems of diskless workstations or virtual machine images. Basically, any part of a *nix computer’s file system can reside remotely on an NFS server. The network share can be mounted at boot time from an entry in /etc/fstab or by hand with the “mount” command.

NFS is not perfect. Back in 1980’s most software development efforts were focused of “works by default” and not on “secure by default.” Although there have been attempts add security to NFS over the last twenty years none of the solutions proposed has been both easy to set up and easy to maintain. So there really is no standard at this point for securing NFS by default. There is a cool proposal (https://tools.ietf.org/html/draft-cel-nfsv4-rpc-tls-01) that could make a huge impact if it gets adopted. For now I think you are safest installing your NFS file server behind a good firewall and restricting access to it with a sane configuration file. If you have the time to learn you can look into LDAP and Kerberos or even RPCSEC GSS.

Almost every modern Linux distro has the NFS server build into it’s kernel as a module. What is missing is the user-space interface to configure and control it. In Debian the package is called nfs-kernel-server and in most other distros it is named nfs-utils. Go ahead and install it now.

The next step is to decide where your File Server will store the backups and create the directory. For this HOWTO I will use /srv/backups as the shared directory. First we will create the directory and change it’s ownership to least privileged user and group:

Code: Select all

sudo mkdir /srv/backups
sudo chown nobody:nogroup /srv/backups
The configuration file /etc/exports controls what directories are shared and in what ways users and/or computers can access them. A copy of the man page for /etc/exports is available by visiting https://linux.die.net/man/5/exports or by executing “man exports” in a bash shell.

The exports file has three section per line. It starts with the directory to be shared by the server, followed by who may access it and kind of share options and access that person will have. As an example:

Code: Select all

/foo/bar		lazlo(rw,sync,root_squash)
Means that I have read/write access to /foo/bar, the server will synchronize any pending disk writes before accepting a new one, and if anyone takes action on this share as root their uid and gid will be changed from root:root (0:0) to nobody:nogroup (65535:65535) for the given action.

Another example:

Code: Select all

/home/lazlo		*(r,no_root_squash)
This would give any user or computer that can get to it read-only access to my home folder on the NFS server.

Feel free to play with the export options and the mount options for your new File Server and Clients. Both types of options can have a real impact on the usability, performance, and security of the NFS share. Just don’t use important directories while playing.

For our NFS File Server we will give anyone on the same subnet as the server read/write access to /srv/backups as nobody:nogroup. We will also use asynchronous disk writes on our share to speed things up. By doing this we are sacrificing a little security of a little speed. If the server crashes or powers down during the transfer it will cause data corruption in the files being written at that time. If you want your data to be as safe as possible the change “async” to “sync” make sure “-o sync” is included in the mount command of the Client. Make sure to adjust the network IP to meet your needs:

Code: Select all

/srv/backups		192.168.123.0/24(rw,async,all_squash,no_subtree_check)
At this point you can restart the NFS service and mount your share on a remote system. Test it by creating some files and directories and then running “ls -al” to make sure that everything is owned by nobody:nogroup. To manually mount the share on Linux Client you will need to install the NFS client package "nfs-common" on Debian based systems. The "nfs-utils" package for most the distros includes both the client and server programs. Once you have installed in you can make a new subdirectory in your home folder and mount the share (don't forget change the IP address in the example):

Code: Select all

mkdir nfstest
sudo mount -t nfs 192.168.123.1:/srv/backups ./nfstest
mkdir nfstest/testdir
dd if=/dev/zero of=nfstest/testdir/testfile bs=1M count=10000 status=progress
Once that is done check the ownership:

Code: Select all

ls -al nfstest/testdir
The file owner should be nobody and the group should be nogroup. If that is the case, congratulations: you are ready to use your File Server with Clonezilla.



Extra Reading

If you have the desire or the need you can tune NFS in a wide variety of ways to best suit your environment. A simple search on Google for “nfs tuning guide” will give you a great foundation. Just remember that there are such a things as “over tuning” and “a point of diminishing returns.”
Last edited by lazlo on Wed Jan 23, 2019 6:48 am, edited 7 times in total.

User avatar
mad_ady
Posts: 6152
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Has thanked: 121 times
Been thanked: 84 times
Contact:

Re: HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by mad_ady » Mon Jan 21, 2019 3:49 pm

Great guide - I hope it gets published in the magazine. There is one omission though - this works if the computers can boot in legacy mode, but H2 can only boot in uefi mode.

lazlo
Posts: 143
Joined: Sun Oct 01, 2017 11:32 am
languages_spoken: english
ODROIDs: XU4
Has thanked: 0
Been thanked: 0
Contact:

Re: HOWTO: Create a Backup Server Using Clonezilla and PXE

Unread post by lazlo » Mon Jan 21, 2019 8:09 pm

mad_ady wrote:
Mon Jan 21, 2019 3:49 pm
Great guide - I hope it gets published in the magazine. There is one omission though - this works if the computers can boot in legacy mode, but H2 can only boot in uefi mode.

Thanks for letting me know, ady. I've skimmed 2 guides for UEFI PXE and they seem to rely on the ipxe package and/or grub. After I finish the NFS section of the HOWTO I'll spend some time researching it more to see if it's a simple modification or if it's worthy of second HOWTO.

Edit: The NFS section is complete. I'll play with UEFI over the next week and see what I figure out. The pxelinux package might work but it will most likely need some scripting added to the config files of in the DHCP sections.

Post Reply

Return to “Ubuntu (All Linux'es)”

Who is online

Users browsing this forum: No registered users and 2 guests