SVN+SSH-path-based-authorisation, version 1

SVN+SSH-path-based-authorisation.md

@@ -0,0 +1,374 @@
+SVN+SSH path-based authorisation
+================================
+
+This is an article about using SVN as a secure replacement for FTP to
+facilitate partners collaborations. If you are not interested in the
+rationale/testing part you ca go straight to the
+[SVN+SSH
+path-based authorisation How-to](SVN+SSH-path-based-authorisation\#\#4-Implementation).
+
+1. Problem
+----------
+
+### 1.1. Rationale
+
+As we are hosting plenty of websites, we have to work with different
+partners (web designer, integrators, website developers etc.). All using
+different operating systems and browsers and all demanding a plain old
+FTP access to upload their work. We tried to get them to use chrooted
+SFTP access but it seems they couldn’t get their head around it.
+
+Internally we were using SVN for all our projects. It’s running on a
+webdav enabled Apache with SSL linked with a password based LDAP
+authentication. This server was only accessible through our VPN (yes we
+are a bit paranoid).
+
+The idea was to allow our partners/customers to have read/write access
+to our web servers without the need for an insecure password based FTP
+access (yes even if it is running through SSL). A solution we are
+contemplating is to give a secure access to our SVN server. Our web
+servers would then have working copies of the part of the repository
+they are concerned. Auto-update can be done through a cron jobs that
+calls a simple svn update. Partners would then be able to work on same
+files instead of having lengthy discussion of whose turns it is to
+upload to the FTP.
+
+In short the goal was :
+
+-   non-password authentication
+-   encrypted
+-   path-based authorisation (to avoid to have to create a SSH
+    repository per-project)
+-   if possible try to avoid another Apache instance (all our servers
+    are openVZ containers and I don’t want to add yet another Apache
+    server)
+
+### 1.2. Existing methods
+
+There is 3 way of running a SVN server :
+
+-   On a Webdav share. It is only available for Apache with
+    mod\_dav\_svn. Lighttpd’s webdav does not support SVN’s specifics.
+    Nginx and lighttpd advise to only use them as caching/proxy
+    mechanism for an Apache webdav server.
+-   svnserve as a daemon. There is no encryption available for svnserve
+    which means that all data are transiting in clear except the
+    authentication that can be through SASL. SASL provides lots of
+    authentication methods and notably LDAP.
+-   svnserve ran standalone from a SSH connection.
+
+According to the [SVN book](http://svnbook.red-bean.com/) it is only
+possible to do path based authorisation with Apache and svnserve daemon
+(See [Table
+6.1](http://svnbook.red-bean.com/nightly/en/svn.serverconfig.overview.html#svn.serverconfig.overview.tbl-1))  
+>svnserve over SSH : Read/write access only grant-able over the
+whole repository
+
+### 1.3. Requirements
+
+Path base authorisation is necessary if we want to provide access to
+partners on website devs. The best solution would be to keep the current
+repository structure :
+
+     - Websites
+      |-trunk
+        |-html
+          |-customer1website1.com
+          |-customer2website2.fr
+        |-drupal6
+          |-customer1website3.net
+          |-customer1website4.com
+          |...
+
+  
+and give only access to the customer’s website folder. Else we would
+have to completely re-structure the SVN and do some script magic to
+create/delete/link lots of separate SVN to redmine.
+
+### 1.4. Choice
+
+-   We cannot use webdav as only Apache has a webdav SVN module and we
+    do not want to waste resources on it.
+-   svnserve is not an option as data won’t be crypted (even if login
+    can be)
+-   SSH is a great option except that when connecting to the SVN server
+    through SSH, it spawns a svnserve standalone process which doesn’t
+    support path based authorisation, according to the documentation.
+
+2. Testing
+----------
+
+### 2.1. Initialise a SVN repository :
+
+    mkdir /var/lib/svn
+    cd /var/lib/svn/
+    svnadmin create test
+
+### 2.2. Set up basic authorisation
+
+    vi /var/lib/svn/test/conf/svnserve.conf
+    -> [general]
+       anon-access = none
+       auth-access = write
+
+       password-db = passwd
+
+       authz-db = authz
+    vi /var/lib/svn/test/conf/authz
+    -> [groups]
+       testwrite = user1
+       testread = svn
+
+       [test:/]
+       @testwrite = rw
+       @testread = r
+
+       [test:/public]
+       @testread = rw
+    # passwd is only required for the svnserve daemon
+    vi /var/lib/svn/test/conf/passwd
+    -> [users]
+       user1 = test
+       svn = test
+    svnserve -d -r /var/lib/svn
+    svn co --no-auth-cache svn://127.0.0.1/test
+
+  
+Committing things with user svn only works in the ‘public’ folder.
+whereas user1 can commit everywhere.
+
+### 2.3. SVN+SSH configuration :
+
+    adduser svn --disabled-password
+    cp -ax /root/.ssh/ /home/svn/
+    chown -R svn:svn /home/svn/
+    chown -R svn:svn /var/lib/svn/
+    vi /home/svn/.ssh/authorized_keys 
+    -> command="svnserve -t -r /var/lib/svn --config-file=/var/lib/svn/haven/conf/svnserve.conf --log-file=/tmp/svnserver.log --tunnel-user=USERNAME",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty USERKEY
+    # On Client
+    svn co svn+ssh://svn@svnserver/test
+
+According to the [SVN book’s SSH
+tricks](http://svnbook.red-bean.com/nightly/en/svn.serverconfig.svnserve.html#svn.serverconfig.svnserve.sshtricks.fixedcmd).
+This configuration is very secure but does not respect above
+configuration regarding path based authorisation.
+
+As said in the [SSHauth
+section](http://svnbook.red-bean.com/nightly/en/svn.serverconfig.svnserve.html#svn.serverconfig.svnserve.sshauth)
+of the SVN book :  
+>When running over a tunnel, authorization is primarily controlled
+by operating system permissions to the repository’s database files; it’s
+very much the same as if Harry were accessing the repository directly
+via a file:// URL. If multiple system users are going to be accessing
+the repository directly, you may want to place them into a common group,
+and you’ll need to be careful about umasks (be sure to read the section
+called “Supporting Multiple Repository Access Methods” later in this
+chapter). But even in the case of tunneling, you can still use the
+svnserve.conf file to block access, by simply setting auth-access = read
+or auth-access = none.
+
+3. Solution
+-----------
+
+After some testing/debugging I manage to come up with 2 solutions that
+should work with our set goals :
+
+### 3.1. svnserve daemon
+
+We can simply override SVN+SSH behaviour by using netcat in the
+commands= section of the authorized\_keys file. So for example we can do
+the following :
+
+    vi /home/svn/.ssh/authorized_keys 
+    -> command="netcat localhost 3690"
+    svnserve -d -r /var/lib/svn/
+    # On the client
+    svn co svn+ssh://svn@svnserver/test
+
+  
+This will use 2 authentications process. First it will use the SSH keys,
+then it will ask for a regular username/password. That’s where you could
+use SASL to link this with your authentication of choice (LDAP or
+others). This isn’t ideal but according to the SVN documentation this is
+the only way of having path based authorisation with SSH, which as you
+will see below is NOT TRUE.
+
+### 3.2. svnserve tunnel
+
+-   From the start of my debugging I found out that svnserve.conf, authz
+    and passwd ARE indeed loaded by svnserve (even in tunnel mode -t).
+    So I was pretty sure that path based authorisation should work with
+    little tinkering.
+-   After debugging the whole application I found that path based
+    authorisation SHOULD work by modifying the configuration. Simply
+    revoking all authorisation in the authz file before allowing them
+    works like a charm !
+        vi /var/lib/svn/test/conf/authz
+        -> [groups]
+           testwrite = user1
+           testread = svn
+
+           #Revoke all rights to the SVN folder
+           [/]
+           * = 
+
+           [test:/]
+           @testwrite = rw
+
+           [test:/public]
+           @testread = rw
+
+      
+    In that case svn can’t check out /test but has full permissions in
+    /test/public !
+
+<!-- -->
+
+-   Beware of the default permissions in the svnserve.conf that cannot
+    be overridden. Setting auth-access to read will prevent EVERY user
+    from writing despite anything that can be written in the authz file,
+    as stated above :  
+    >But even in the case of tunneling, you can still use the
+    svnserve.conf file to block access, by simply setting auth-access =
+    read or auth-access = none.\_
+
+4. Implementation
+-----------------
+
+Now that we found out that the SVN documentation is misleading (probably
+from lack of update), here is a simple procedure to setup path based
+authorisation with SVN+SSH.
+
+### 4.1. Give all permissions to svn users :
+
+    chown -R svn:svn /var/lib/svn/
+
+### 4.2. Add public SVN user keys to its authorized\_keys following the below format :
+
+    vi /home/svn/.ssh/authorized_keys
+    -> command="svnserve -t -r /var/lib/svn --log-file=/tmp/svnserver.log --tunnel-user=USERNAME",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-dss USERKEY
+
+### 4.3. Configure a generic svnserve.conf
+
+    /var/lib/svn/svnserve.conf
+    -> [general]
+       anon-access = none
+       auth-access = write
+       # password-db = passwd
+       authz-db = authz
+    rm */conf/svnserve.conf .
+    # Our example repository for all websites :
+    ln -s /var/lib/svn/svnserve.conf websites/conf/
+    # All you other projects
+    ln -s /var/lib/svn/svnserve.conf project2/conf/
+    ...
+    ln -s /var/lib/svn/svnserve.conf projectn/conf/
+
+### 4.4. Per repository permissions :
+
+    vi /var/lib/svn/websites/conf/authz
+    -> [aliases]
+       # Could be used with LDAP i.e. :
+       # joe = /C=XZ/ST=Dessert/L=Snake City/O=Snake Oil, Ltd./OU=Research Institute/CN=Joe Average
+
+       [groups]
+       websitesread = webserver1, webserver2
+       websiteswrite = adminuser
+       customer1site1read = customer1
+       customer1site1write = designer1, webdeveloper1
+
+       # VERY IMPORTANT
+       [/]
+       * =
+
+       [websites:/]
+       @websitesread = r
+       @websiteswrite = rw
+
+       [websites:/trunk/html/customer1/site1.com]
+       @customer1site1read = r
+       @customer1site1write = rw
+
+### 4.5. User checkouts
+
+SVN+SSH makes it very easy to test permissions of your repository. You
+just have to change your USERNAME in SVN’s authorized\_keys.
+
+As customer1 :
+
+    svn co svn+ssh://svn@svnserver/websites
+    svn: Authorization failed
+    svn co svn+ssh://svn@svnserver/websites/trunk/html/customer1/site1.com
+    At revision 5.
+
+As adminuser :
+
+    svn co svn+ssh://svn@svnserver/websites
+    At revision 5.
+
+### 4.6. Deploy website on web server
+
+-   Generate ssh key for webserver1
+        su - www-data
+        ssh-keygen -t rsa -b 2048
+
+<!-- -->
+
+-   On the SVN server add webserver1’s public key to svn’s
+    authorized\_keys and give it the username webserver1
+-   Back on webserver1
+        cd /var/www/customer1/site1/web/
+        svn co svn+ssh://svn@svnserver/websites/trunk/html/customer1/site1.com
+        crontab -e
+        */5 * * * * /usr/bin/svn --non-interactive update /var/www/customer1/site1.com/web 2>&1 > /dev/null
+
+5. Related topics
+-----------------
+
+### 5.1. SVN migration
+
+See [Migration-trac-redmine\#6-SVN-migration](Migration-trac-redmine\#6-SVN-migration).
+
+### 5.2. Clean up a SVN repository from bogus commits
+
+While migrating our SVN to a new server we jump on the occasion to clean
+up some bogus commits. Here is an example on how to modify the author
+property, that can be used for any revision properties.
+
+-   List all committer
+        svn log --quiet file:///var/lib/svn/project1/ | awk '/^r/ {print $3}' | sort -u
+
+-   Change bogus committer (requires the
+    [pre-revprop-change
+    script](Migration-trac-redmine\#71-Add-pre-revprop-change-hooks-to-allow-revision-modifications))
+        for rev in `svn log --quiet file:///var/lib/svn/project1/ | awk '/bogususer/ {print substr($1,2)}'`; do 
+          svn propset --revprop -r $rev svn:author correctuser file:///var/lib/svn/project1/ ; 
+        done
+        # If done after the import in redmine we need to modify this in Redmine's DB as well :
+        sqlite3 /var/lib/dbconfig-common/sqlite3/redmine/instances/default/redmine_default
+        -> update changesets set user_id = X where committer="bogususer" and user_id is null;
+           update changesets set committer="correctuser" where committer = "bogususer";
+
+### 5.3. Reference for Debugging
+
+Here are some links to tools and documentation that were useful for my
+debugging :
+
+-   The [SVN daemon
+    protocol](http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_ra_svn/protocol).
+-   If you require to sniff the full SVN protocol you can use the
+    following :
+        aptitude install simpleproxy
+        simpleproxy -L 3333 -R 127.0.0.1:3690 -t trace
+        # Locally do 
+        svn co svn://127.0.0.1:3333/test
+
+-   For svnserve step by step debugging, I compiled from source and used
+    gdb, nemiver and netbeans.
+
+See Also
+--------
+
+-   The [migration from trac to Redmine](Migration-trac-redmine)
+    that was performed in conjunction with the above.
+