This document is guiding for both system-hackers team (as responsible for the main infrastructure and critical parts), as well as service teams (responsible for individual services).
What teams are there?
You can find a list of teams here: https://wiki.fsfe.org/Teams This list is so far incomplete in that it doesn't mention all FSFE teams. It does however list all teams subject to this service agreement.
How is a service team formed?
A service team is formed by two or more people sending a proposal to system-hackers to establish a service team. The information which the system-hackers will need is:
- Who is coordinating/co-coordinating the team
- What resources are seen to be required (disk space, bandwidth, CPU utilization)
- What service is supposed to run and for whom
- Whether the service is installed from Debian repositories or separately
What does the system-hackers provide?
For a team which do not require running root access for running their service, the system-hackers team can provide a light weight hosting option: a user account on a machine where the entire service and its data must reside in the user's home directory, and any startup/shutdown be done through cronjobs. The system-hackers team will do regular (and automated!) upgrades of the operating system and security. When doing so, only the user account home directory and crontabs will be retained. If the service needs to be accessible on a standard port, such as 80 and 443, tthis will be handled via a DNAT translation (ie., the service will run on a non-privileged port such as 1280, and the network will be configured such that access to port 80 on the IP number of the service is mapped to the real port 1280).
For a team which require or would like to run the entire system themselves, the system-hackers team can provide:
- A container or virtual machine running the latest Debian stable release on a Proxmox cluster
- One IPv4 and one IPv6 address for that container
- A daily backup (not accessible by the team, see later)
How are backups managed?
The FSFE has a backup server which pulls data from each virtual server. This happens once a day, generally during the night. Each virtual server to be included in the backup should include an SSH key (provided by system-hackers and installed by default on new virtual servers) which allow connections as root with SFTP and for running the script /root/bin/backup.sh
Before each backup is run, the backup server will run /root/bin/backup.sh on each virtual server. This script should prepare the server for backup, which usually means dumping any databases to flat text files and similar to ensure the backups contain all relevant information in an easily manageable way.
Restoring files from backup require access to the FSFE's backup server, and a GnuPG passphrase which is available only to the system-hackers. Any restore must therefore be coordinated with the system-hackers.
How are major OS upgrades deployed?
Major OS upgrades should be performed well in time before security releases stop for existing distributions. Teams are asked to perform a dist-upgrade when relevant, or, if this is deemed more suitable, to ask system-hackers for a new virtual machine or container on which the service can be installed with a new stable Debian. The switch and migration between old and new server is then managed in coordination with system-hackers.
How are security updates deployed?
Containers should automatically install any security upgrades on a daily basis, with a report sent to the service team. If some packages are deemed too sensitive for automatic upgrades, they should be pinned in dpkg and security upgrades should then be installed manually as quickly as possible, but latest within one month after the security upgrade was made available.