|
Size: 7831
Comment:
|
Size: 8866
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 12: | Line 12: |
| The default setting for the backups is saving your home directory, so for example `/home/LOCALUSER/` on your GNU/Linux computer. This is the place where all personal data and application configuration should be stored. Please refrain from storing important information outside of this directory – if you decide otherwise, make sure to include it in your backup schema. | The default setting for the backups is saving your home directory, so for example `/home/LOCALUSER/` on your GNU/Linux computer, which is equivalent with `~`. This is the place where all personal data and application configuration should be stored. Please refrain from storing important information outside of this directory – if you decide otherwise, make sure to include it in your backup schema. |
| Line 14: | Line 14: |
| = Setup and Use = | = Preparation = |
| Line 16: | Line 16: |
| First, you will have to install the necessary software packages. Then, you need to configure access to the remote destination your backups will be stored in. Afterwards, you will setup the script which makes the backup really simple. | First, you will have to install the necessary software packages and think of a strong password. Then, you need to configure access to the remote destination your backups will be stored in. Afterwards, you will setup the script which makes the backup really simple. |
| Line 18: | Line 18: |
| In this guide we assume your FSFE username is '''FSFEUSER'''. We also assume that '''LOCALUSER''' is the username on your computer. When following the guide please exchange them accordingly | In this guide we assume that your FSFE user name is ''FSFEUSER''. Make sure to replace this throughout the guide accordingly. |
| Line 20: | Line 20: |
| == Software Requirements == | == Software Requirements (Graphical User Interface method - easy) == |
| Line 24: | Line 24: |
| * ssh * borg * pass |
* `ssh` * `borgbackup` or `borg` (depending on your operating system it may have different names, see [[https://borgbackup.readthedocs.io/en/latest/installation.html#distribution-package|this overview]]) * `vorta` (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use) |
| Line 28: | Line 28: |
| == Access remote backup storage == | == Software Requirements (Terminal method - advanced) == |
| Line 30: | Line 30: |
| You will backup your files to a remote place. At the moment, this is a hard disk in the Berlin office but this might be subject to change in the future. In order to gain access to this space, you will have to set up a SSH key. Like a GnuPG key, this is a secure way to authenticate yourself. | Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions. |
| Line 32: | Line 32: |
| 1. Check whether you already have an SSH key: Type `ls -al ~/.ssh` in your GNU/Linux terminal. If `id_rsa` and `id_rsa.pub` appears, you already have a SSH key. If not, continue with step 2. 1. Generate a new SSH key by typing `ssh-keygen -t rsa -b 4096 -C "FSFEUSER@fsfe.org"`. Omit typing in a password if you are sure that you can protect your SSH private key (it's like a house key!). If you set a password, consider using `ssh-agent` to avoid having to type in the SSH key's password each time you use it. 1. Try to log in via your terminal with `ssh FSFEUSER@fsfe-backup.fritz.box`. Your SSH client will ask you to verify the server's SSH host key if it hasn't connected to it before. You should type 'yes' if you are sure you are in the FSFE's office network and copied everything correctly. After accepting and if no error is triggered, you will see that the terminal prompt (the text in front of the cursor) has changed – you are connected to the server! Type `exit` once to disconnect. |
* `ssh` * `borgbackup` or `borg` (depending on your operating system it may have different names, see [[https://borgbackup.readthedocs.io/en/latest/installation.html#distribution-package|this overview]]) * `borgmatic` (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use) |
| Line 40: | Line 40: |
| Finding a good password is hard, and there are many strategies. [[https://xkcd.com/936/|This famous XKCD comic]] suggests using combined words, but you can also use a password manager like pass or KeePass to auto-generate lots of loooong passwords protected with only one strong master password you have to remember. Whatever you choose for your backups, make sure you only use it for this purpose only and have a safe place to store it, even if your computer is stolen, burns to death, explodes or is kidnapped by evil aliens. Ask your colleagues and/or the internet for tipps. | Finding a good password is not trivial, and there are many strategies. [[https://xkcd.com/936/|This famous XKCD comic]] suggests using combined words, but you can also use a password manager like ''pass'', ''{{{KeePass}}}'', or ''KeePassXC'' to auto-generate lots of loooong passwords protected with only one strong master password you have to remember. Whatever you choose for your backups, make sure you only use it for this purpose only and have a safe place to store it, even if your computer is stolen, burns to death, explodes or is kidnapped by evil aliens. Ask your colleagues and/or the internet for tipps. |
| Line 42: | Line 42: |
| For borg, it is recommended to store the password with a small command line application called `pass` on your computer. It encrypts passwords with your GnuPG key which you already should have set up when working for the FSFE. The commands below show a short way to set everything up, but please also consult [[https://www.passwordstore.org/|pass' website]] for more information. | Again, it is crucial that your password is securely stored somewhere else then on your work computer. You can only restore the backup with this password! So if you use a password manager, please make sure that you will be able to access its content if your computer, apartment, local backup disk explodes, is stolen, or kidnapped by aliens. |
| Line 44: | Line 44: |
| 1. In your terminal, run `pass init FSFEUSER@fsfe.org`, using your @fsfe.org email address which is connected to your GPG key. 1. Run `pass insert borg_backup`. This will open a prompt asking for the password you have chose. Type/copy it and follow the instructions on the screen. |
== Create SSH key == |
| Line 47: | Line 46: |
| Afterwards, you can run `pass` to show all passwords in your local password store. `pass show borg_backup` should print the password you have chosen for borg. If this is the case, you can safely continue. | You will backup your files to a remote place. In order to gain access to this space, you will have to set up a SSH key. Like a GnuPG key, this is a secure way to authenticate yourself. |
| Line 49: | Line 48: |
| == Create the borg repository == | 1. Check whether you already have an SSH key: Type `ls -al ~/.ssh` in your GNU/Linux terminal. If `id_rsa` and `id_rsa.pub` appears, you already have a SSH key. If not, continue with step 2. 1. Generate a new SSH key by typing `ssh-keygen -t rsa -b 4096 -C "FSFEUSER@fsfe.org"`. Omit typing in a password if you are sure that you can protect your SSH private key (it's like a house key!). If you set a password, consider using `ssh-agent` to avoid having to type in the SSH key's password each time you use it. |
| Line 51: | Line 51: |
| Now, you can initiate the borg repository. This only has to be done once. | = Request backup storage creation = |
| Line 53: | Line 53: |
| 1. In your terminal, type `export BORG_PASSCOMMAND="pass show borg_backup`. This will tell borg to use the password from pass and not ask you for it each time. 1. Run `borg init --encryption=repokey ssh://FSFEUSER@fsfe-backup.fritz.box:22/~/Borg`. The last part (starting with `ssh://`) will look different depending on your username. Ask the person controlling backups in the office for assistance here. |
Now it is time to create the remote backup storage. This is handled by our System Hackers. |
| Line 56: | Line 55: |
| == Download and run backup script == | Send an email to contact@fsfe.org asking for the backup store. Provide the following information: |
| Line 58: | Line 57: |
| We have a script which makes it easy to semi-automate backups. Follow the instructions on [[TechDocs/OfficeBackUp/Script|this page]] to download and set it up. | * Which folders would you like to have stored in your backup? By default, we save the whole home directory, but you can provide more if necessary * Attach your public SSH key `id_rsa.pub`. You will find it in `~/.ssh/`. Make sure to send the file suffixed with `.pub`, the other one is your private key! |
| Line 60: | Line 60: |
| == Run a backup == | '''You can only continue with the next step as soon as you have received a reply.''' |
| Line 62: | Line 62: |
| If the script is set up, you can run the backup. | = Configure the backup (Graphical User Interface method) = |
| Line 64: | Line 64: |
| {{{#!wiki blue/solid The process for all following backups is the same but they will take significantly less time. Depending on the speed of your network and the size of the your home directory, this could well take 1 hour or more, but perhaps also much less. Just plan enough time in advance. }}} |
It's probably easiest, if you simply watch the video that details all the steps using the graphical client called Vorta. You can find it on Nextcloud. Simple navigate to Staff > Tech Sessions > Tech Session 3 - Backups. There, you will find a short version (~ 20 minutes) in which everything you need to do in order to start making backups is shown. |
| Line 68: | Line 66: |
| In your terminal, run `~/bin/borg_backup.sh` if you followed the recommendations of [[TechDocs/OfficeBackUp/Script|script setup]]. If you set up everything correctly, borg will not ask for a password and find the remote backup storage. Otherwise, please check everything carefull again. | = Configure the backup (Terminal method) = |
| Line 70: | Line 68: |
| After the backup has been finished, you can check the whole repository by running `borg list ssh://FSFEUSER@fsfe-backup.fritz.box:22/~/Borg`. In order to see more information about a single backup, run `borg info ssh://FSFEUSER@fsfe-backup.fritz.box:22/~/Borg::computername-2019-02-12T14:50:23`, while the backup name will be different. Consult the borg documentation for more helpful commands. | In the reply, you will receive a configuration file for borgmatic. Borgmatic is a so-called wrapper for borg which makes initialisation and creation of backups easier for you. Let us configure borgmatic once: |
| Line 72: | Line 70: |
| = Getting help = | 1. Save the file `config.yaml` in the directory `~/.config/borgmatic/`. If this directory does not exist, please create it. 1. Open the file and replace `<REPLACE_WITH_PASSWORD>` with the strong password you have chosen before Now, you can initiate the borg repository. This only has to be done once. 1. Open a terminal and run the following command: `borgmatic -I -e repokey-blake2` 1. You will see something like ''The authenticity of host 'a3p79bh3.repo.borgbase.com (148.251.50.177)' can't be established.''. Type ''yes'' to confirm. 1. Then, you will see some information about your borg version and security considerations. If you see something completely different, reply back to the System Hacker you had contact with. = Run a backup (Terminal method) = Now you are set to run your first backup. Since you will send it to a remote server, it can take quite some time. In the Berlin office, an initial backup of 10 GB might well take ~1 hour. But all following backups will be much faster and a matter of a few minutes. The first command creates a backup. You will see some progress and information about the running and finished backup. If it's finished, the last lines of the output will indicate clearly whether it has been successful or not. Afterwards, the second command cleans older backups according to the retention policy in order to save worthy backup space. This will keep daily backups for the last 7 days, a weekly backup for 4 weeks, a monthly backup for 12 months, and a yearly backup for 2 years. This is a well-known policy in system administration. Please run this command: {{{ borgmatic --create --progress --stats borgmatic --prune --stats }} = Getting help and Error messages (Terminal method) = If after the first try to run a backup there was an error message and the backup process did not start you can try the following: 1. Open the config.yaml in the editor of your choice (e.g. vim, gedit, nano, ...) 2. Replace the ~ in the source_directories with the folder you want to backup like /home/name_of_your_folder |
| Line 76: | Line 100: |
= Setup on Backup Server = The backup server must provide ssh public key logins for each user. The backup server in the Berlin office is a !RaspberryPi model 1B+ running raspbian stretch. As storage device we use an external hard disk attached to the computer and running 24/7. It is mounted to /srv, and /home is a symlink to /srv. Backup users currently have full shell access. So if they store files in their remote home directory, it's stored automatically on this disk. == Creating a new user == 1. Run `useradd -m FSFEUSER` 1. Copy the new user's public SSH key to `/home/FSFEUSER/.ssh/authorized_keys` |
The perspective of the System Hackers is explained [[TechDocs/TechnicalProcesses/Backup/BorgBase|here]]. |
Contents
Backup Office Computers
This guide explains the recommended way to do backups for people working in the FSFE's office, so staff and interns. It can be useful for other groups as well but may require some modifications.
Currently, the programme we use to create backups is BorgBackup, often simply called borg. Its main features: great backup speed (even when using weak internet connections), space-efficiency, encryption by default, and clever ways for restoring data. It however lacks a stable graphical user interface so users are required to work on the command line for now. But don't be afraid: it is not hard!
The default setting for the backups is saving your home directory, so for example /home/LOCALUSER/ on your GNU/Linux computer, which is equivalent with ~. This is the place where all personal data and application configuration should be stored. Please refrain from storing important information outside of this directory – if you decide otherwise, make sure to include it in your backup schema.
Preparation
First, you will have to install the necessary software packages and think of a strong password. Then, you need to configure access to the remote destination your backups will be stored in. Afterwards, you will setup the script which makes the backup really simple.
In this guide we assume that your FSFE user name is FSFEUSER. Make sure to replace this throughout the guide accordingly.
Software Requirements (Graphical User Interface method - easy)
Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions.
ssh
borgbackup or borg (depending on your operating system it may have different names, see this overview)
vorta (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use)
Software Requirements (Terminal method - advanced)
Make sure to install the software in the list below using your package manager. It should be existent in most GNU/Linux distributions.
ssh
borgbackup or borg (depending on your operating system it may have different names, see this overview)
borgmatic (either available directly in your system's normal package repositories. If not, head over the installation instructions or ask your colleagues for help by mentioning which operating system + version you use)
Set a backup password
Borg encrypts all your backups securely to protect your sensitive data. Therefore, their security heavily depends on the strength of your password and how well you protect it. If you lose the password, there is no way to recover your backups! This section will help you find a good password for borg, and how you can save it on your computer for (semi)-automated backups.
Finding a good password is not trivial, and there are many strategies. This famous XKCD comic suggests using combined words, but you can also use a password manager like pass, KeePass, or KeePassXC to auto-generate lots of loooong passwords protected with only one strong master password you have to remember. Whatever you choose for your backups, make sure you only use it for this purpose only and have a safe place to store it, even if your computer is stolen, burns to death, explodes or is kidnapped by evil aliens. Ask your colleagues and/or the internet for tipps.
Again, it is crucial that your password is securely stored somewhere else then on your work computer. You can only restore the backup with this password! So if you use a password manager, please make sure that you will be able to access its content if your computer, apartment, local backup disk explodes, is stolen, or kidnapped by aliens.
Create SSH key
You will backup your files to a remote place. In order to gain access to this space, you will have to set up a SSH key. Like a GnuPG key, this is a secure way to authenticate yourself.
Check whether you already have an SSH key: Type ls -al ~/.ssh in your GNU/Linux terminal. If id_rsa and id_rsa.pub appears, you already have a SSH key. If not, continue with step 2.
Generate a new SSH key by typing ssh-keygen -t rsa -b 4096 -C "FSFEUSER@fsfe.org". Omit typing in a password if you are sure that you can protect your SSH private key (it's like a house key!). If you set a password, consider using ssh-agent to avoid having to type in the SSH key's password each time you use it.
Request backup storage creation
Now it is time to create the remote backup storage. This is handled by our System Hackers.
Send an email to contact@fsfe.org asking for the backup store. Provide the following information:
- Which folders would you like to have stored in your backup? By default, we save the whole home directory, but you can provide more if necessary
Attach your public SSH key id_rsa.pub. You will find it in ~/.ssh/. Make sure to send the file suffixed with .pub, the other one is your private key!
You can only continue with the next step as soon as you have received a reply.
Configure the backup (Graphical User Interface method)
It's probably easiest, if you simply watch the video that details all the steps using the graphical client called Vorta. You can find it on Nextcloud. Simple navigate to Staff > Tech Sessions > Tech Session 3 - Backups. There, you will find a short version (~ 20 minutes) in which everything you need to do in order to start making backups is shown.
Configure the backup (Terminal method)
In the reply, you will receive a configuration file for borgmatic. Borgmatic is a so-called wrapper for borg which makes initialisation and creation of backups easier for you. Let us configure borgmatic once:
Save the file config.yaml in the directory ~/.config/borgmatic/. If this directory does not exist, please create it.
Open the file and replace <REPLACE_WITH_PASSWORD> with the strong password you have chosen before
Now, you can initiate the borg repository. This only has to be done once.
Open a terminal and run the following command: borgmatic -I -e repokey-blake2
You will see something like The authenticity of host 'a3p79bh3.repo.borgbase.com (148.251.50.177)' can't be established.. Type yes to confirm.
- Then, you will see some information about your borg version and security considerations. If you see something completely different, reply back to the System Hacker you had contact with.
Run a backup (Terminal method)
Now you are set to run your first backup. Since you will send it to a remote server, it can take quite some time. In the Berlin office, an initial backup of 10 GB might well take ~1 hour. But all following backups will be much faster and a matter of a few minutes.
The first command creates a backup. You will see some progress and information about the running and finished backup. If it's finished, the last lines of the output will indicate clearly whether it has been successful or not. Afterwards, the second command cleans older backups according to the retention policy in order to save worthy backup space. This will keep daily backups for the last 7 days, a weekly backup for 4 weeks, a monthly backup for 12 months, and a yearly backup for 2 years. This is a well-known policy in system administration. Please run this command: