|
Size: 3169
Comment:
|
Size: 8863
Comment: Add graphical user interface option
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 2: | Line 2: |
| #acl TeamGroup,paul:read,write,admin,delete,revert All:read | #acl Group/Team:read,write,admin,delete,revert All:read |
| Line 4: | Line 4: |
| = Features = This backup script is a graphical wrapper for duplicity. For the older encfs based backup see [[/encfs]]. |
<<TableOfContents(3)>> |
| Line 7: | Line 6: |
| * The backup tool is run manually on a desktop machine via a graphical user interface * The stored backup data is encrypted, encryption is done on the local workstation * Backup runs are possible for single folders (e.g. the "FSFE"-folder of employees) * Backup runs are incremental, so the amount of data transferred after initial backups is small. |
= Backup Office Computers = |
| Line 12: | Line 8: |
| = Limitations = ... as of 2016-08-25 (svn rev. 283) * Recovery of single files is not supported, the script always restores complete backups * single file recovery is possible from the command line * Changing the backup password is currently not possible from the GUI (and can be tedious on the CLI) * The tool does not allow removing old backups or forcing full backups out of schedule (but duplicity does on CLI) |
This guide explains the recommended way to do backups for people working in the FSFE's office, so staff and interns. It can be useful for other groups as well but may require some modifications. |
| Line 19: | Line 10: |
| = Setup and Use = | Currently, the programme we use to create backups is [[https://www.borgbackup.org/|BorgBackup]], often simply called `borg`. Its main features: great backup speed (even when using weak internet connections), space-efficiency, encryption by default, and clever ways for restoring data. It however lacks a stable graphical user interface so users are required to work on the command line for now. But don't be afraid: it is not hard! |
| Line 21: | Line 12: |
| == Software Requirements == | The default setting for the backups is saving your home directory, so for example `/home/LOCALUSER/` on your GNU/Linux computer, which is equivalent with `~`. This is the place where all personal data and application configuration should be stored. Please refrain from storing important information outside of this directory – if you decide otherwise, make sure to include it in your backup schema. |
| Line 23: | Line 14: |
| * zenity * duplicity * rsync * ssh |
= Preparation = |
| Line 28: | Line 16: |
| Use your package manager to make sure the above mentioned software is installed. Then download and run [[attachment:backup.sh]]. Usually you will have to set the executable flag before running a downloaded file. | First, you will have to install the necessary software packages and think of a strong password. Then, you need to configure access to the remote destination your backups will be stored in. Afterwards, you will setup the script which makes the backup really simple. |
| Line 30: | Line 18: |
| = Technical Idea = {{{#!wiki white '''From the duplicity manual:''' Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server. |
In this guide we assume that your FSFE user name is ''FSFEUSER''. Make sure to replace this throughout the guide accordingly. == Software Requirements (Graphical User Interface method - easy) == Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions. * `ssh` * `borgbackup` or `borg` (depending on your operating system it may have different names, see [[https://borgbackup.readthedocs.io/en/latest/installation.html#distribution-package|this overview]]) * `vorta` (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use) == Software Requirements (Terminal method - advanced) == Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions. * `ssh` * `borgbackup` or `borg` (depending on your operating system it may have different names, see [[https://borgbackup.readthedocs.io/en/latest/installation.html#distribution-package|this overview]]) * `borgmatic` (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use) == Set a backup password == Borg encrypts all your backups securely to protect your sensitive data. Therefore, their security heavily depends on the strength of your password and how well you protect it. If you lose the password, there is no way to recover your backups! This section will help you find a good password for borg, and how you can save it on your computer for (semi)-automated backups. Finding a good password is not trivial, and there are many strategies. [[https://xkcd.com/936/|This famous XKCD comic]] suggests using combined words, but you can also use a password manager like ''pass'', ''{{{KeePass}}}'', or ''KeePassXC'' to auto-generate lots of loooong passwords protected with only one strong master password you have to remember. Whatever you choose for your backups, make sure you only use it for this purpose only and have a safe place to store it, even if your computer is stolen, burns to death, explodes or is kidnapped by evil aliens. Ask your colleagues and/or the internet for tipps. Again, it is crucial that your password is securely stored somewhere else then on your work computer. You can only restore the backup with this password! So if you use a password manager, please make sure that you will be able to access its content if your computer, apartment, local backup disk explodes, is stolen, or kidnapped by aliens. == Create SSH key == You will backup your files to a remote place. In order to gain access to this space, you will have to set up a SSH key. Like a GnuPG key, this is a secure way to authenticate yourself. 1. Check whether you already have an SSH key: Type `ls -al ~/.ssh` in your GNU/Linux terminal. If `id_rsa` and `id_rsa.pub` appears, you already have a SSH key. If not, continue with step 2. 1. Generate a new SSH key by typing `ssh-keygen -t rsa -b 4096 -C "FSFEUSER@fsfe.org"`. Omit typing in a password if you are sure that you can protect your SSH private key (it's like a house key!). If you set a password, consider using `ssh-agent` to avoid having to type in the SSH key's password each time you use it. = Request backup storage creation = Now it is time to create the remote backup storage. This is handled by our System Hackers. Send an email to contact@fsfe.org asking for the backup store. Provide the following information: * Which folders would you like to have stored in your backup? By default, we save the whole home directory, but you can provide more if necessary * Attach your public SSH key `id_rsa.pub`. You will find it in `~/.ssh/`. Make sure to send the file suffixed with `.pub`, the other one is your private key! '''You can only continue with the next step as soon as you have received a reply.''' = Configure the backup (Graphical User Interface method)= It's probably easiest, if you simply watch the video that details all the steps using the graphical client called Vorta. You can find it on Nextcloud. Simple navigate to Staff > Tech Sessions > Tech Session 3 - Backups. There, you will find a short version (~ 20 minutes) in which everything you need to do in order to start making backups is shown. = Configure the backup (Terminal method)= In the reply, you will receive a configuration file for borgmatic. Borgmatic is a so-called wrapper for borg which makes initialisation and creation of backups easier for you. Let us configure borgmatic once: 1. Save the file `config.yaml` in the directory `~/.config/borgmatic/`. If this directory does not exist, please create it. 1. Open the file and replace `<REPLACE_WITH_PASSWORD>` with the strong password you have chosen before Now, you can initiate the borg repository. This only has to be done once. 1. Open a terminal and run the following command: `borgmatic -I -e repokey-blake2` 1. You will see something like ''The authenticity of host 'a3p79bh3.repo.borgbase.com (148.251.50.177)' can't be established.''. Type ''yes'' to confirm. 1. Then, you will see some information about your borg version and security considerations. If you see something completely different, reply back to the System Hacker you had contact with. = Run a backup (Terminal method)= Now you are set to run your first backup. Since you will send it to a remote server, it can take quite some time. In the Berlin office, an initial backup of 10 GB might well take ~1 hour. But all following backups will be much faster and a matter of a few minutes. The first command creates a backup. You will see some progress and information about the running and finished backup. If it's finished, the last lines of the output will indicate clearly whether it has been successful or not. Afterwards, the second command cleans older backups according to the retention policy in order to save worthy backup space. This will keep daily backups for the last 7 days, a weekly backup for 4 weeks, a monthly backup for 12 months, and a yearly backup for 2 years. This is a well-known policy in system administration. Please run this command: {{{ borgmatic --create --progress --stats borgmatic --prune --stats |
| Line 36: | Line 91: |
| Duplicity is a command line based tool. This script provides a graphical wrapper, handling authentication and setup. Bckup runs are kept incremental for 6 month, after 6 month a new full backup will be performed. | = Getting help and Error messages (Terminal method)= |
| Line 38: | Line 93: |
| == Authentication == | If after the first try to run a backup there was an error message and the backup process did not start you can try the following: |
| Line 40: | Line 95: |
| All data transfer to and from the backup server is done via Rsync/SSH. Each backup user has a separate system account on the backup server. Authentication to the server is performed via ssh public key. The backup program generates its own ssh keypair. The same passphrase is used for the ssh key and the backup encryption. | 1. Open the config.yaml in the editor of your choice (e.g. vim, gedit, nano, ...) 2. Replace the ~ in the source_directories with the folder you want to backup like /home/name_of_your_folder |
| Line 42: | Line 98: |
| === Changing a password === Should the passphrase be changed, it must be changed for the ssh key, and for existing backup archives. The latter is tricky. It is also possible to start a new backup chain, by performing a full backup manually. However, when this is done the old passphrase is still required for restoring existing archives. |
If you would like to learn more about borg, the different commands and useful parameters, please have a look at the [[https://borgbackup.readthedocs.io/en/stable/|borg documentation]]. It is well written and continuously maintained. Other resources might be the wikis of your GNU/Linux distributions, so for instance UbuntuUsers or the Arch Linux Wiki. |
| Line 45: | Line 100: |
| = Setup on Backup Server = The backup server must provide ssh public key logins for each user as well as rsync. The backup server in the Berlin office is a !RaspberryPi model 1B+ running raspbian wheezy. The system was installed via the Raspbian unattended installer. As storage device we use an external hard disk attached to the computer and running 24/7. Backup users currently have full shell access. |
The perspective of the System Hackers is explained [[TechDocs/TechnicalProcesses/Backup/BorgBase|here]]. |
Contents
Backup Office Computers
This guide explains the recommended way to do backups for people working in the FSFE's office, so staff and interns. It can be useful for other groups as well but may require some modifications.
Currently, the programme we use to create backups is BorgBackup, often simply called borg. Its main features: great backup speed (even when using weak internet connections), space-efficiency, encryption by default, and clever ways for restoring data. It however lacks a stable graphical user interface so users are required to work on the command line for now. But don't be afraid: it is not hard!
The default setting for the backups is saving your home directory, so for example /home/LOCALUSER/ on your GNU/Linux computer, which is equivalent with ~. This is the place where all personal data and application configuration should be stored. Please refrain from storing important information outside of this directory – if you decide otherwise, make sure to include it in your backup schema.
Preparation
First, you will have to install the necessary software packages and think of a strong password. Then, you need to configure access to the remote destination your backups will be stored in. Afterwards, you will setup the script which makes the backup really simple.
In this guide we assume that your FSFE user name is FSFEUSER. Make sure to replace this throughout the guide accordingly.
Software Requirements (Graphical User Interface method - easy)
Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions.
ssh
borgbackup or borg (depending on your operating system it may have different names, see this overview)
vorta (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use)
Software Requirements (Terminal method - advanced)
Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions.
ssh
borgbackup or borg (depending on your operating system it may have different names, see this overview)
borgmatic (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use)
Set a backup password
Borg encrypts all your backups securely to protect your sensitive data. Therefore, their security heavily depends on the strength of your password and how well you protect it. If you lose the password, there is no way to recover your backups! This section will help you find a good password for borg, and how you can save it on your computer for (semi)-automated backups.
Finding a good password is not trivial, and there are many strategies. This famous XKCD comic suggests using combined words, but you can also use a password manager like pass, KeePass, or KeePassXC to auto-generate lots of loooong passwords protected with only one strong master password you have to remember. Whatever you choose for your backups, make sure you only use it for this purpose only and have a safe place to store it, even if your computer is stolen, burns to death, explodes or is kidnapped by evil aliens. Ask your colleagues and/or the internet for tipps.
Again, it is crucial that your password is securely stored somewhere else then on your work computer. You can only restore the backup with this password! So if you use a password manager, please make sure that you will be able to access its content if your computer, apartment, local backup disk explodes, is stolen, or kidnapped by aliens.
Create SSH key
You will backup your files to a remote place. In order to gain access to this space, you will have to set up a SSH key. Like a GnuPG key, this is a secure way to authenticate yourself.
Check whether you already have an SSH key: Type ls -al ~/.ssh in your GNU/Linux terminal. If id_rsa and id_rsa.pub appears, you already have a SSH key. If not, continue with step 2.
Generate a new SSH key by typing ssh-keygen -t rsa -b 4096 -C "FSFEUSER@fsfe.org". Omit typing in a password if you are sure that you can protect your SSH private key (it's like a house key!). If you set a password, consider using ssh-agent to avoid having to type in the SSH key's password each time you use it.
Request backup storage creation
Now it is time to create the remote backup storage. This is handled by our System Hackers.
Send an email to contact@fsfe.org asking for the backup store. Provide the following information:
- Which folders would you like to have stored in your backup? By default, we save the whole home directory, but you can provide more if necessary
Attach your public SSH key id_rsa.pub. You will find it in ~/.ssh/. Make sure to send the file suffixed with .pub, the other one is your private key!
You can only continue with the next step as soon as you have received a reply.
= Configure the backup (Graphical User Interface method)=
It's probably easiest, if you simply watch the video that details all the steps using the graphical client called Vorta. You can find it on Nextcloud. Simple navigate to Staff > Tech Sessions > Tech Session 3 - Backups. There, you will find a short version (~ 20 minutes) in which everything you need to do in order to start making backups is shown.
= Configure the backup (Terminal method)=
In the reply, you will receive a configuration file for borgmatic. Borgmatic is a so-called wrapper for borg which makes initialisation and creation of backups easier for you. Let us configure borgmatic once:
Save the file config.yaml in the directory ~/.config/borgmatic/. If this directory does not exist, please create it.
Open the file and replace <REPLACE_WITH_PASSWORD> with the strong password you have chosen before
Now, you can initiate the borg repository. This only has to be done once.
Open a terminal and run the following command: borgmatic -I -e repokey-blake2
You will see something like The authenticity of host 'a3p79bh3.repo.borgbase.com (148.251.50.177)' can't be established.. Type yes to confirm.
- Then, you will see some information about your borg version and security considerations. If you see something completely different, reply back to the System Hacker you had contact with.
= Run a backup (Terminal method)=
Now you are set to run your first backup. Since you will send it to a remote server, it can take quite some time. In the Berlin office, an initial backup of 10 GB might well take ~1 hour. But all following backups will be much faster and a matter of a few minutes.
The first command creates a backup. You will see some progress and information about the running and finished backup. If it's finished, the last lines of the output will indicate clearly whether it has been successful or not. Afterwards, the second command cleans older backups according to the retention policy in order to save worthy backup space. This will keep daily backups for the last 7 days, a weekly backup for 4 weeks, a monthly backup for 12 months, and a yearly backup for 2 years. This is a well-known policy in system administration. Please run this command:
borgmatic --create --progress --stats borgmatic --prune --stats
= Getting help and Error messages (Terminal method)=
If after the first try to run a backup there was an error message and the backup process did not start you can try the following:
- Open the config.yaml in the editor of your choice (e.g. vim, gedit, nano, ...)
- Replace the ~ in the source_directories with the folder you want to backup like /home/name_of_your_folder
If you would like to learn more about borg, the different commands and useful parameters, please have a look at the borg documentation. It is well written and continuously maintained. Other resources might be the wikis of your GNU/Linux distributions, so for instance UbuntuUsers or the Arch Linux Wiki.
The perspective of the System Hackers is explained here.