diff options
Diffstat (limited to 'glep-0058.rst')
| -rw-r--r-- | glep-0058.rst | 326 |
1 files changed, 326 insertions, 0 deletions
diff --git a/glep-0058.rst b/glep-0058.rst new file mode 100644 index 0000000..1fd993b --- /dev/null +++ b/glep-0058.rst @@ -0,0 +1,326 @@ +GLEP: 58 +Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest +Version: $Revision$ +Last-Modified: $Date$ +Author: Robin Hugh Johnson <robbat2@gentoo.org>, +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Requires: 44, 60 +Created: October 2006 +Updated: November 2007, June 2008, July 2008, October 2008, January 2010 +Post-History: December 2009, January 2010 + +======== +Abstract +======== +MetaManifest provides a means of verifiable distribution from Gentoo +Infrastructure to a user system, while data is conveyed over completely +untrusted networks and system, by extending the Manifest2 specification, +and adding a top-level Manifest file, with support for other nested +Manifests. + +========== +Motivation +========== +As part of a comprehensive security plan, we need a way to prove that +something originating from Gentoo as an organization (read Gentoo-owned +hardware, run by infrastructure), has not been tampered with. This +allows the usage of third-party rsync mirrors, without worrying that +they have modified something critical (e.g. eclasses, which are still +unsigned). + +Securing the untrusted distribution is one of the easier tasks in the +security plan - in short, all that is required is having a hash of every +item in the tree, and signing that hash to prove it came from Gentoo. + +Ironically we have a hashed and signed distribution (it's just not used +by most users, due to it's drawbacks): Our tree snapshot tarballs have +hashes and signatures. + +So now we want to add the same verification to our material that is +distributed by rsync. We already provide hashes of subsets of the tree - +our Manifests protect individual packages. However metadata, eclasses +and profiles are not protected at this time. The directories of +packages and distfiles are NOT covered by this, as they are not +distributed by rsync. + +This portion of the tree-signing work provides only the following +guarantee: A user can prove that the tree from the Gentoo infrastructure +has not been tampered with since leaving the Gentoo infrastructure. +No other guarantees, either implicit or explicit are made. + +Additionally, distributing a set of the most recent MetaManifests from a +trusted source allows validation of trees that come from community +mirrors, and allows detection of all cases of malicious mirrors (either +by deliberate delay, replay [C08a, C08b] or alteration). + +============= +Specification +============= +For lack of a better name, the following solution should be known as the +MetaManifest. Those responsible for the name have already been sacked. + +MetaManifest basically contains hashes of every file in the tree, either +directly or indirectly. The direct case applies to ANY file that does +not appear in an existing Manifest file (e.g. eclasses, Manifest files +themselves). The indirect case is covered by the CONTENTS of existing +Manifest files. If the Manifest itself is correct, we know that by +tracking the hash of the Manifest, we can be assured that the contents +are protected. + +In the following, the MetaManifest file is a file named 'Manifest', +located at the root of a repository. + +--------------------------------------------- +Procedure for creating the MetaManifest file: +--------------------------------------------- +Summary: +======== +The objective of creating the MetaManifest file(s) is to ensure that +every single file in the tree occurs in at least one Manifest. + +Process: +======== +1. Start at the root of the Gentoo Portage tree (gentoo-x86, although + this procedure applies to overlays as well). + +2. Initialize two unordered sets: COVERED, ALL. + + 1. 'ALL' shall contain every file that exists in the present tree. + 2. 'COVERED' shall contain EVERY file that is mentioned in an existing + Manifest2. If a file is mentioned in a Manifest2, but does not + exist, it must still be included. No files should be excluded. + +3. Traverse the tree, depth-first. + + 1. At the top level only, ignore the following directories: distfiles, + packages, local. + 2. If a directory contains a Manifest file, extract all relevant local + files from it (presently: AUX, MISC, EBUILD; but should follow the + evolution of Manifest2 entry types per [GLEP60]), and place them + into the COVERED set. + 3. Recursively add every file in the directory to the ALL set, + pursuant to the exclusion list as mentioned in [GLEP60]. + +4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED). + This is every item that is not covered by another Manifest, or part + of an exclusion list. + +5. If an existing MetaManifest file is present, remove it. + +6. For each file in UNCOVERED, assign a Manifest2 type, produce the + hashes, and add with the filetype to the MetaManifest file. + +7. For unique identification of the MetaManifest, a header line should + be included, using the exact contents of the metadata/timestamp.x + file, so that a MetaManifest may be tied back to a tree as + distributed by the rsync mirror system. The string of + 'metadata/timestamp.x' should be included to identify this revision + of MetaManifest generation. e.g.: + "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC" + The package manager MUST not use the identifying string as a filename. + +8. The MetaManifest must ultimately be GnuPG-signed. + + 1. For the initial implementation, the same key as used for snapshot + tarball signing is sufficient. + 2. For the future, the key used for fully automated signing by infra + should not be on the same keyring as developer keys. See + [GLEPxx3] for further notes. + +Notes: +====== +The above does not conflict the proposal contained in [GLEP33], which +restructure eclasses to include subdirectories and Manifest files, as +the Manifest rules above still provide indirect verification for all +files after the [GLEP33] restructuring if it comes to pass. + +Additional levels of Manifests are required, such as per-category, and +in the eclasses, profiles and metadata directories. This ensures that a +change to a singular file causes the smallest possible overall change in +the Manifests as propagated. Creation of the additional levels of +Manifests uses the same process as described above, simply starting at a +different root point. + +MetaManifest generation will take place as part of the existing process +by infrastructure that takes the contents of CVS and prepares it for +distribution via rsync, which includes generating metadata. In-tree +Manifest files are not validated at this point, as they are assumed to +be correct. + +-------------------------------------------------------- +Verification of one or more items from the MetaManifest: +-------------------------------------------------------- +There are two times that this may happen: firstly, immediately after the +rsync has completed - this has the advantage that the kernel file cache +is hot, and checking the entire tree can be accomplished quickly. +Secondly, the MetaManifest should be checked during installation of a +package. + +---------------------------------------------------- +Procedure for verifying an item in the MetaManifest: +---------------------------------------------------- +In the following, I've used term 'M2-verify' to note following the hash +verification procedures as defined by the Manifest2 format - which +compromise checking the file length, and that the hashes match. Which +filetypes may be ignored on missing is discussed in [GLEP60]. + +1. Check the GnuPG signature on the MetaManifest against the keyring of + automated Gentoo keys. See [GLEPxx3] for full details regarding + verification of GnuPG signatures. + 1. Abort if the signature check fails. + +2. Check the Timestamp header. If it is significantly out of date + compared to the local clock or a trusted source, halt or require + manual intervention from the user. + +3. For a verification of the tree following an rsync: + + 1. Build a set 'ALL' of every file covered by the rsync. (exclude + distfiles/, packages/, local/) + 2. M2-verify every entry in the MetaManifest, descending into inferior + Manifests as needed. Place the relative path of every checked item + into a set 'COVERED'. + 3. Construct the set 'UNCOVERED' by set-difference between the ALL and + COVERED sets. + 4. For each file in the UNCOVERED set, assign a Manifest2 filetype. + 5. If the filetype for any file in the UNCOVERED set requires a halt + on error, abort and display a suitable error. + 6. Completed verification + +4. If checking at the installation of a package: + + 1. M2-verify the entry in MetaManifest for the Manifest + 2. M2-verify all relevant metadata/ contents if metadata/ is being + used in any way (optionally done before dependency checking). + 3. M2-verifying the contents of the Manifest. + 4. Perform M2-verification of all eclasses and profiles used (both + directly and indirectly) by the ebuild. + +Notes: +====== +1. For initial implementations, it is acceptable to check EVERY item in + the eclass and profiles directory, rather than tracking the exact + files used by every eclass (see note #2). Later implementations + should strive to only verify individual eclasses and profiles as + needed. +2. Tracking of exact files is of specific significance to the libtool + eclass, as it stores patches under eclass/ELT-patches, and as such + that would not be picked up by any tracing of the inherit function. + This may be alleviated by a later eclass and ebuild variable that + explicitly declares what files from the tree are used by a package. + +==================== +Implementation Notes +==================== +For this portion of the tree-signing work, no actions are required of +the individual Gentoo developers. They will continue to develop and +commit as they do presently, and the MetaManifest is added by +Infrastructure during the tree generation process, and distributed to +users. + +Any scripts generating Manifests and the MetaManifest may find it useful +to generate multiple levels of Manifests in parallel, and this is +explicitly permitted, provided that every file in the tree is covered by +at least one Manifest or the MetaManifest file. The uppermost +Manifest (MetaManifest) is the only item that does not occur in any +other Manifest file, but is instead GPG-signed to enable its +validation. + +-------------------------------------------- +MetaManifest and the new Manifest2 filetypes +-------------------------------------------- +While [GLEP60] describes the addition of new filetypes, these are NOT +needed for implementation of the MetaManifest proposal. Without the new +filetypes, all entries in the MetaManifest would be of type 'MISC'. + +---------------------------------------------------- +Timestamps & Additional distribution of MetaManifest +---------------------------------------------------- +As discussed by [C08a,C08b], malicious third-party mirrors may use the +principles of exclusion and replay to deny an update to clients, while +at the same time recording the identity of clients to attack. + +This should be guarded against by including a timestamp in the header of +the MetaManifest, as well as distributing the latest MetaManifests by a +trusted channel. + +On all rsync mirrors directly maintained by the Gentoo infrastructure, +and not on community mirrors, there should be a new module +'gentoo-portage-metamanifests'. Within this module, all MetaManifests +for a recent time frame (e.g. one week) should be kept, named as +"MetaManifest.$TS", where $TS is the timestamp from inside the file. +The most recent MetaManifest should always be symlinked as +MetaManifest.current. The possibility of serving the recent +MetaManifests via HTTPS should also be explored to mitigate +man-in-the-middle attacks. + +The package manager should obtain MetaManifest.current and use it to +decide is the tree is too out of date per operation #2 of the +verification process. The decision about freshness should be a +user-configuration setting, with the ability to override. + +-------------------------------- +MetaManifest size considerations +-------------------------------- +With only two levels of Manifests (per-package and top-level), every +rsync will cause a lot of traffic transferring the modified top-level +MetaManifest. To reduce this, first-level directory Manifests are +required. Alternatively, if the distribution method efficiently handles +small patch-like changes in an existing file, using an uncompressed +MetaManifest may be acceptable (this would primarily be distributed +version control systems). Other suggestions in reducing this traffic are +welcomed. + +======================= +Backwards Compatibility +======================= +- There are no backwards compatibility issues, as old versions of + Portage do not look for a Manifest file at the top level of the tree. +- Manifest2-aware versions of Portage ignore all entries that they are + not certain how to handle. Enabling headers and PGP signing to be + conducted easily. + +====== +Thanks +====== +I'd like to thank the following people for input on this GLEP. + +- Patrick Lauer (patrick): Prodding me to get all of the tree-signing + work finished, and helping to edit. +- Ciaran McCreesh (ciaranm): Paludis Manifest2 +- Brian Harring (ferringb): pkgcore Manifest2 +- Marius Mauch (genone) & Zac Medico (zmedico): Portage Manifest2 +- Ned Ludd (solar) - Security concept review + +========== +References +========== + +.. [C08a] Cappos, J et al. (2008). "Package Management Security". + University of Arizona Technical Report TR08-02. Available online + from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf + +.. [C08b] Cappos, J et al. (2008). "Attacks on Package Managers" + Available online at: + http://www.cs.arizona.edu/people/justin/packagemanagersecurity/ + +.. [GLEP33] Eclass Restructure/Redesign + http://www.gentoo.org/proj/en/glep/glep-0033.html + +.. [GLEP60] Manifest2 filetypes + http://www.gentoo.org/proj/en/glep/glep-0060.html + +.. [GLEPxx2] Future GLEP on Developer Process security. + +.. [GLEPxx3] Future GLEP on GnuPG Policies and Handling. + +========= +Copyright +========= +Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be +distributed only subject to the terms and conditions set forth in the +Open Publication License, v1.0. + +.. vim: tw=72 ts=2 expandtab: |