From 08f68c829bcdecd71ad52f3f7d2dfbb1a175dd42 Mon Sep 17 00:00:00 2001 From: Michael Prokop Date: Fri, 2 Sep 2016 14:46:24 +0200 Subject: [PATCH] MT#22361 Remember file permissions through pre-commit hook + restore-permissions helper script MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Git doesn't track file permissions (except for the executable flag). For sensitive data (like the 'ssl' directory and file 'constants.yml' with passwords included) we've to prevent non-root users from accessing those files. hooks/pre-commit is inspired and based on the implementation as present in etckeeper (and luckily we're license compatible) and takes care of storing the file permissions inside file /etc/ngcp-config/.ngcpcfg_perms. The restore-permissions helper script takes care of restoring the permissions after cloning the ngcpcfg repository via ngcpcfg itself (being actions decrypt, pull (PRO-only) + initialise (PRO-only)). It can be executed manually as well via `usr/share/ngcp-ngcpcfg/helper/restore-permissions /etc/ngcp-config/` (or wherever the according ngcpcfg repository is placed at). Regarding the commit integration: git(1) itself doesn't track file permissions, so we can't detect changes to file permissions using git itself. Our new pre-commit hook records file permissions via the .ngcpcfg_perms file. Now by just invoking it during 'ngcpcfg commit' time we can ensure that even if there have been any file permission changes in the working directory the file .ngcpcfg_perms is then up2date and committed. JFTR: The solution via the git pre-commit hook ensures that no matter whether you're using 'ngcpcfg commit …' or 'git commit …' you always get the file permissions handled via .ngcpcfg_perms. Now if you want to change file permissions in a clean working directory and commit *without* using 'ngcpcfg commit' but directly via git itself then you've to use 'git commit --allow-empty ...' and thanks to the pre-commit hook the file .ngcpcfg_perms will still be up2date. Change-Id: I84d608585c626b52112ff649893e232e441c59d8 --- debian/copyright | 22 +++++++ debian/ngcp-ngcpcfg.install | 2 + docs/ngcpcfg.txt | 29 +++++++++- functions/main | 29 +++++++++- helper/restore-permissions | 44 ++++++++++++++ hooks/pre-commit | 111 ++++++++++++++++++++++++++++++++++++ scripts/commit | 7 +++ scripts/decrypt | 2 + scripts/initialise | 2 + 9 files changed, 246 insertions(+), 2 deletions(-) create mode 100755 helper/restore-permissions create mode 100755 hooks/pre-commit diff --git a/debian/copyright b/debian/copyright index 8856bd13..bee9e732 100644 --- a/debian/copyright +++ b/debian/copyright @@ -2,6 +2,28 @@ Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ Source: https://www.sipwise.com/ Upstream-Contact: Sipwise Development Team +Files: hooks/pre-commit +Copyright: © 2007-2016 Joey Hess + © 2014 Pim van den Berg + © 2013 Zdenek Crha + © 2008 Scott Bronson +License: GPL-2+ + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 2 of the License, or + (at your option) any later version. + . + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + . + You should have received a copy of the GNU General Public License + along with this program. If not, see . +Comment: + On Debian systems, the full text of the GNU General Public License + version 2 can be found in the file '/usr/share/common-licenses/GPL-2'. + Files: * Copyright: Copyright © 2007-2015, Sipwise GmbH, Austria diff --git a/debian/ngcp-ngcpcfg.install b/debian/ngcp-ngcpcfg.install index 58e1051f..dcbd6323 100644 --- a/debian/ngcp-ngcpcfg.install +++ b/debian/ngcp-ngcpcfg.install @@ -1,6 +1,7 @@ etc/ngcp-config/ngcpcfg.cfg etc/ngcp-config/ functions/main usr/share/ngcp-ngcpcfg/functions/ helper/build_config usr/share/ngcp-ngcpcfg/helper/ +helper/restore-permissions usr/share/ngcp-ngcpcfg/helper/ helper/check-for-mysql usr/share/ngcp-ngcpcfg/helper/ helper/fileformat_version usr/share/ngcp-ngcpcfg/helper/ helper/sort-yml usr/share/ngcp-ngcpcfg/helper/ @@ -8,6 +9,7 @@ helper/sync-db usr/share/ngcp-ngcpcfg/helper/ helper/tt2-daemon usr/share/ngcp-ngcpcfg/helper/ helper/tt2-wrapper usr/share/ngcp-ngcpcfg/helper/ helper/validate-yml usr/share/ngcp-ngcpcfg/helper/ +hooks/ usr/share/ngcp-ngcpcfg/ lib/* usr/lib/ngcp-ngcpcfg/ sbin/ngcp-network usr/sbin/ sbin/ngcp-sync-constants usr/sbin/ diff --git a/docs/ngcpcfg.txt b/docs/ngcpcfg.txt index dce1f70c..0e6146ab 100644 --- a/docs/ngcpcfg.txt +++ b/docs/ngcpcfg.txt @@ -390,11 +390,38 @@ functionality (which is highly recommended in a HA Setup). How does the configuration management system work? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -ngcpcfg takes the central yml files (/etc/ngcp-config/*.yml) as input for +ngcpcfg takes the central yml files (/etc/ngcp-config/\*.yml) as input for configuration, searches for supported templates files (*.tt2) in the template pool (_/etc/ngcp-config/templates/etc/_) and builds the resulting configuration files in _/etc_. +Does ngcpcfg also track file permissions? +----------------------------------------- + +When using ngcpcfg in a HA setup (PRO/CARRIER only) you want to ensure that +changing file permissions inside the ngcpcfg repository on one node propogates +to the other node(s) as well. ngcpcfg is using git underneath, though git +itself doesn't track file permissions (except for the execution bit). + +To be able to track file permissions ngcpcfg includes a pre-commit hook which +records file permissions via the _.ngcpcfg_perms_ file inside the ngcpcfg +repository (being _/etc/ngcp-config/.ngcpcfg_perms_ by default). Whenever you +run 'ngcpcfg commit' this pre-commit hook looks at the existing file permissions +and records them through the _.ngcpcfg_perms_ file. After pushing the changes to +the shared storage and when pulling then from the git repository on the foreign +node(s) via 'ngcpcfg pull' the file permissions get restored automatically (via +_/usr/share/ngcp-ngcpcfg/helper/restore-permissions_ which can be used also +manually if needed). + +The solution via the git pre-commit hook ensures that no matter whether you're +using 'ngcpcfg commit …' or 'git commit …' directly (for whatever reason) you +always get the file permissions handled via .ngcpcfg_perms. + +If you want to change file permissions in a *clean* working directory and commit +*without* using 'ngcpcfg commit' but directly via git itself then you've to use +'git commit --allow-empty ...'. Thanks to the pre-commit hook the file +_.ngcpcfg_perms_ will get updated accordingly. + [[errorhandling]] Error handling - what does this error message mean? --------------------------------------------------- diff --git a/functions/main b/functions/main index d629ee4c..bab04dff 100644 --- a/functions/main +++ b/functions/main @@ -47,6 +47,34 @@ log_debug() { console_output "DEBUG: $*\n" >&2 fi } +## }}} + +hook_setup() { + log_debug "hook_setup ${1:-}" + + if ! [ -d /usr/share/ngcp-ngcpcfg/hooks ] ; then + log_debug "Directory /usr/share/ngcp-ngcpcfg/hooks doesn't exist." + return 0 + fi + + local target_directory="$1" + + if [ -z "${1:-}" ] ; then + log_error "Missing argument for target directory in hook_setup. Exiting." + exit 1 + fi + + if ! [ -d "$target_directory" ] ; then + log_error "Hook target directory $target_directory not a directory. Exiting." + exit 1 + fi + + for hook in /usr/share/ngcp-ngcpcfg/hooks/* ; do + [ -r "$hook" ] || continue + log_debug "Creating symlink for $hook in $target_directory" + ln -sf "$hook" "$target_directory"/ + done +} compare_active_branch() { log_debug "get_active_branch ${1:-}" @@ -87,7 +115,6 @@ get_branch_status() { return 3 fi } -## }}} ## important variables we depend on to operate successfully {{{ # support test suite which requires system independent configuration diff --git a/helper/restore-permissions b/helper/restore-permissions new file mode 100755 index 00000000..f8fe4741 --- /dev/null +++ b/helper/restore-permissions @@ -0,0 +1,44 @@ +#!/bin/bash +# Filename: /usr/share/ngcp-ngcpcfg/helper/restore-permissions +# Purpose: restore file/directory permissions after git clone +################################################################################ + +set -e +set -u + +if [ "${#:-}" -ne 1 ] ; then + echo "Usage: /usr/share/ngcp-ngcpcfg/helper/restore-permissions " >&2 + exit 1 +fi + +# support for testsuite, assume defaults if unset +CONFIG_POOL="${CONFIG_POOL:-/etc}" +FUNCTIONS="${FUNCTIONS:-/usr/share/ngcp-ngcpcfg/functions/}" +HELPER="${HELPER:-/usr/share/ngcp-ngcpcfg/helper/}" + +. ${FUNCTIONS}/main + +# used to run a command if the file it acts on (the last parameter) exists. +maybe() { + local command="$1" + shift 1 + + if eval [ -e "\"\$$#\"" ] ; then + "$command" "$@" + fi +} + +target_directory="$1" +if ! [ -d "${target_directory}" ] ; then + log_errror "Directory ${target_directory} doesn't exist. Exiting." + exit 1 +fi + +# should only be run on repositories you trust +if ! [ -e "$target_directory"/.ngcpcfg_perms ]; then + log_warn "Permission file ${target_directory}/.ngcpcfg_perms doesn't exist." +else + . "${target_directory}"/.ngcpcfg_perms +fi + +## END OF FILE ################################################################# diff --git a/hooks/pre-commit b/hooks/pre-commit new file mode 100755 index 00000000..ca9ddc1c --- /dev/null +++ b/hooks/pre-commit @@ -0,0 +1,111 @@ +#!/bin/sh +# pre-commit hook for ngcpcfg, to store metadata +# implementation heavily based on etckeeper's pre-commit.d/30store-metadata + +set -e + +# ensure we're in the appropriate git directory when invoked directly +# e.g. via `/etc/ngcp-config/.git/hooks/pre-commit` inside ngcpcfg +if ! git rev-parse --git-dir >/dev/null 2>&1 ; then + cd "$(dirname "${0}")" + if git rev-parse --git-dir >/dev/null 2>&1 ; then + git_dir="$(git rev-parse --git-dir)" + cd "$(dirname "${git_dir}")" + fi +fi + +if ! git rev-parse --git-dir >/dev/null 2>&1 ; then + echo "Warning: $(pwd) is not a git repository, ignoring $0" + exit 0 +fi + +filter_ignore() { + if [ -e ".gitignore" ]; then + listfile="$( mktemp -t ngcpcfg-git.XXXXXXXXXX )" + (git ls-files -oi --exclude-standard; git ls-files -oi --exclude-standard --directory) | sort | uniq > "$listfile" || true + sed 's/^\.\///' | grep -xFvf "$listfile" + rm -f "$listfile" + unset listfile + else + cat - + fi +} + +shellquote() { + # Single quotes text, escaping existing single quotes. + sed -e "s/'/'\"'\"'/g" -e "s/^/'/" -e "s/$/'/" +} + +generate_metadata() { + # This function generates the script commands to fix any file + # ownerships that aren't owner=root, group=root, as well as to + # store the permissions of files. + # The script is produced on stdout. Errors go to stderr. + # + # The script can use a 'maybe' function, which only runs a command + # if the file in its last argument exists. + + # We want files in the directory containing VCS data + # but we want find to ignore the VCS files themselves. + # + # (Note that when using this, the find expression must end with + # -print or -exec, else the excluded directories will actually be + # printed!) + + # Keep the sort order the same at all times. + LC_COLLATE=C + export LC_COLLATE + + # git does not track directories, + # so empty directories must be stored specially. + find . -path ./.git -prune -o -type d -empty -print | + sort | shellquote | sed -e "s/^/mkdir -p /" + + # Store things that don't have the default user or group. + # Store all file modes, in case the user has an unusual umask. + find . -path ./.git -prune -o \( -type f -or -type d \) -print | filter_ignore | sort | perl -ne ' + BEGIN { $q=chr(39) } + sub uidname { + my $want=shift; + if (exists $uidcache{$want}) { + return $uidcache{$want}; + } + my $name=scalar getpwuid($want); + return $uidcache{$want}=defined $name ? $name : $want; + } + sub gidname { + my $want=shift; + if (exists $gidcache{$want}) { + return $gidcache{$want}; + } + my $name=scalar getgrgid($want); + return $gidcache{$want}=defined $name ? $name : $want; + } + chomp; + my @stat=stat($_); + my $mode = $stat[2]; + my $uid = $stat[4]; + my $gid = $stat[5]; + s/$q/$q"$q"$q/g; # escape single quotes + s/^/$q/; + s/$/$q/; + if ($uid != $>) { + printf "maybe chown $q%s$q %s\n", uidname($uid), $_; + } + if ($gid != $)) { + printf "maybe chgrp $q%s$q %s\n", gidname($gid), $_; + } + printf "maybe chmod %04o %s\n", $mode & 07777, $_; + ' +} + +echo "# Generated by ngcpcfg. Do not edit." > .ngcpcfg_perms +echo >> .ngcpcfg_perms + +# Make sure the file is not readable by others, since it can leak +# information about contents of non-readable directories +chmod 700 .ngcpcfg_perms + +generate_metadata >> .ngcpcfg_perms + +git add .ngcpcfg_perms diff --git a/scripts/commit b/scripts/commit index a0e3cbcb..411cfb1c 100755 --- a/scripts/commit +++ b/scripts/commit @@ -21,6 +21,13 @@ timestamp_replacementchars='' # main script +# ensure that existing hooks are up2date +hook_setup "${NGCPCTL_MAIN}/.git/hooks" + +if [ -x "${NGCPCTL_MAIN}/.git/hooks/pre-commit" ] ; then + "${NGCPCTL_MAIN}/.git/hooks/pre-commit" +fi + cd "$NGCPCTL_MAIN" if [ -z "${NO_DB_SYNC:-}" ] ; then diff --git a/scripts/decrypt b/scripts/decrypt index 49a67b9c..bd7a0d15 100755 --- a/scripts/decrypt +++ b/scripts/decrypt @@ -90,6 +90,7 @@ Please execute the following command on one node as soon as glusterfs share is mounted again: git clone --bare /etc/ngcp-config /mnt/glusterfs/ngcpcfg-share + /usr/share/ngcp-ngcpcfg/helper/restore-permissions /mnt/glusterfs/ngcpcfg-share " @@ -99,6 +100,7 @@ else else log_info "Copying git repository to shared storage." git clone --bare /etc/ngcp-config /mnt/glusterfs/ngcpcfg-share | sed "s/^/$timestamp_replacementchars/" + /usr/share/ngcp-ngcpcfg/helper/restore-permissions /mnt/glusterfs/ngcpcfg-share fi fi diff --git a/scripts/initialise b/scripts/initialise index 7b18920d..578d5090 100755 --- a/scripts/initialise +++ b/scripts/initialise @@ -25,6 +25,8 @@ if [ -d .git ] ; then else log_debug "git init" git init >/dev/null + chmod 0700 .git + hook_setup "${NGCPCTL_MAIN}/.git/hooks" fi # ignore files we do not consider as valid template files