Free Software Audit
Our expectations for a Free Software security audit
In general “try to keep it simple”: Don't use too complex methods, so more people understand it, and can reproduce auditing. There are established and very formalised methods for auditing software creation processes and a software design, which are covered, for instance, in the Common Criteria https://en.wikipedia.org/wiki/Common_Criteria or the ISO 27000 family https://en.wikipedia.org/wiki/ISO/IEC_27000. But in many cases these methods are a total overkill and we suggest to use more lightweight approaches, e.g. the method like the one descried in 1.2 - 1.4 of http://web.archive.org/web/20071201015448/http://dud.inf.tu-dresden.de/~pfitza/SecCryptI_II.pdf.
- Release Early, Release Often: Do not produce a huge report and publish this at the end of the project. Publish ideas, criteria, results, and next steps as soon as possible. Thereby you enable others to give you feedback.
- For the Code Review: We don't know what you mean by code review. In our terminology it is just one part of an audit. You have to audit: processes, the software architecture, and the code itself (see other document we attached). And those three are equally important. Just suppose you have a 100% secure software at the moment, but the processes of its creation are totally messed up (in reality this would not happen...). So an attacker can then easily introduce a vulnerability into the next version of the software due to the bad quality of the processes.
- Do not focus on one time activities, as they will be outdated very soon, but focus on setting up procedures. For example instead of having one big audit, implement procedures that make sure the software is regularly audited.
- Participation: One of the best (highly non-formal) practical methods for a deep source code review is actually contributing to the project. (Side note: You might remember this from math class: The teacher explained something and you finally thought you got it, but when you were to calculate an assignment on your own, you met the devil in the details. The point I am trying to make: You get a *way* deeper insight into the source code of a software if you actually work with it, touch it and play around with it.) For example, as public administration you can either participate yourself, or pay additional companies to participate in the development. In most cases, this method is not possible with proprietary software.
- You will have/had a call with Mirko Böhm; in preparation in the last weeks we also talked with him, and we agree that there should be a threat assessment to identify potential risks, an impact analysis based on which critical projects would be identified, and then a security audit that focuses on high-risk-high-impact systems.
- Vendor Lock-In/Monopolies: One or few actors can control how you can use the software, who is allowed to study how it works, who is allowed to modify, and with whom the software can be shared. You might not be allowed to do changes you need. It is always better if there are several independent companies who can offer services for the software than just one.
Complexity: for software as well as for standards. Security Experts see complexity as the biggest threat to software security. This is why many of them are critical or even hostile towards standards. Applying a minimal principle to standards is crucial here https://fsfe.org/activities/os/minimalisticstandards.en.html.
- Money flow: When you do not have to pay a license fee, you should try to pay for the software nevertheless to influence and secure its further development (Remember the bounty hunter example from the phone call). If there is no money flow, the project might not be sustainable for professional needs.
- Software Patents: They are always a thread to developers and users of software. There are too many, they are too broad, and the patent can be used to take away rights people receive through copyright. So nowadays you can always be sued for software patent infringement, and there is almost nothing you can do about that except changing it on a political level.
How to get more Free Software in the public administration
Public bodies have to implement concrete measures as soon as possible. Standard decisions dominate public bodies choice of software for several years, possibly decades. The most important factors for this are:
1. use minimalistic standards which can also be implemented by Free Software. First it needs to be a real Open Standards. Second it needs to be minimalistic; the two central questions hereby are (see minimalistic standards):
- How well does the data-format solve the problem?
- Is there a simpler format that could solve the problem just as well?
2. enforce that whenever software is written with public money, that this software has to be published as Free Software.
Misc concerns and suggestions
Comparative study: difficult to compare the EU institutions with Free Software communities. Maybe also compare it to big distributions who are more hierarchical themselves (RedHat, SuSE, Canoncial...) but who cooperate with Free Software communities.
- Inventory: don't focus to have one complete inventory at one time, but instead implement procedures so they will be constantly updated in future.
Have a had a look at the French "Ayrault Memorandum" or the work of other EU countries rules about Free Software (e.g. Italy, Sweden, or the UK).
Three parts of an audit (additional information)
Processes (long term)
What processes were used to develop the software (traditional waterfall model, SCRUM,...)? How is a requirement / user story identified, managed and tracked? How is a requirement / user story transformed into a software design and finally into code?
How does the way of a patch to the mainline code base look like? What quality gates are there before a patch can enter the next stage (on its way to the code base)? How do the processes look like at the different quality gates? Who decides whether a patch is allowed to pass a quality gate and how?
How formalised are these processes? How transparent are they?
Free Software licence and the use of well defined minimalist Open Standards per se are already a necessary precondition for having transparent processes in a security relevant project.
A transparent and open development model with well-defined processes (bug tracker, easy ways to submit patches,...) and a good quality assurance is really helpful. (It is not always the case in Free Software that an open development method is used, see Point 1 in <https://fsfe.org/freesoftware/enterprise/freesoftwarecompany.en.html>)
Software architecture (middle term)
Does the software have a carefully thought out software design (at best, it is even specified and documented)? Are there well-defined interfaces? Does the software make use of well-defined minimalistic Open Standards?
Especially when it comes to cryptography: Does the software make use of cryptographic methods, algorithms and formats that are well-analysed and recommended by the world-wide cryptographic community (with recommended and strong enough cryptographic parameters)?
Does the software design include a security concept with a well-defined thread model? (In general, a non-security related software can even have a nice, but highly insecure software design if security is simply not considered.)
Apart from a good software design in general, in a security related software project, it is important to a have good security concept with well-defined attacker and threat models. If the software project does not specify a security concept, the reviewers should consider formulating one.
Code (short term)
How does the code look like? Is it well-structured and clearly written? Does it make use the of the standard methods of the programming language or does it over and over re-invent the wheel (which is much more error- prone)?
Does it comply to standard security coding guidelines (e.g. always validate the input, correctly initialise variables, always use secure variants of standard library functions, if you do bash scripts, unset the PATH variable and explicitly specify all the binaries you execute, ...)?
This area is highly language specific and there are many really good guidelines, such as (Herb Sutter, Andrei Alexandrescu: C++ Coding Standards: 101 Rules, Guidelines, and Best Practices. Addison-Wesley.) or the CERT Secure Coding Standards.
One rule of thumb to apply when doing code reviews or when writing software yourself is: If there are conflicting goals, go for
- explicitness, clarity and readability first, then comes
- code elegance, and only then follows
- performance.