Mercurial Web with FastCGI & Nginx

So I’ve finally decided to make the switch to a distributed source control system.  The benefits are well-documented, and I’ve grown weary of Subversion.  After some research, I decided Mercurial would be best for me.  Since I have OCD, and I wanted to push via HTTP to my remote repository, I did some homework and figured out how to get everything running on my VPS.  If you’d like to see how I did it, read on.

Step 1:  Prerequisites

Since my VPS runs Ubuntu Linux 9.10, there are a few packages to install before we get started.  Run the following command to install them:

sudo apt-get install python-setuptools acl python-dev spawn-fcgi

Now you need to make sure your main partition has the acl system enabled in /etc/fstab.  Modify the partition options to look something like this:

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/xvda       /               ext3    noatime,errors=remount-ro,acl 0       1
/dev/xvdb       none            swap    sw              0       0

Pay attention to that acl option in the default partition above.  Now you can just remount the partition:

sudo mount -o remount,acl /

Step 2:  Install Mercurial and flup

Mercurial is obvious, but flup might be new to you.  Flup is a python module that allows us to create a WSGI server to host the repositories.

sudo easy_install mercurial
sudo easy_install flup

Step 3:  Create a user account to hold your repositories

I prefer to place my repositories underneath a normal user account and then add www-data and myself to the group.  Here are the commands to set it all up:

sudo /usr/sbin/groupadd hg
sudo /usr/sbin/useradd -g hg -s /bin/false hg
sudo mkdir /var/hg
sudo chown hg:hg /var/hg
sudo /usr/sbin/usermod -G hg www-data
sudo /usr/sbin/usermod -G hg scott

The next part deserves a little bit of explaining.  Basically the whole /var/hg folder should be owned by the group hg that we just created.  But what we need to ensure three things.  First, the hg group should have write permissions to the root folder.  Second, we need to make sure that any new files created under this folder will belong to the group hg also.  And third, we need to make sure that the default permissions for this folder are read/write for the owner and the group.  These two commands will ensure each of these rules is followed, which is why we added the acl package earlier and remounted the main filesystem.

I’m sure there are other ways to accomplish this (like running Nginx and the FastCGI process as the hg user), but this is the method I prefer, and it saves a lot of headaches down the road dealing with why a push won’t go.  So here are the three commands you need to execute:

sudo chmod -R g+rwxs,o+rx /var/hg
sudo setfacl -R -m d:u::rwx,d:g::rwx,d:m:rwx,d:o:r-x /var/hg

Step 4:  Create the hgwebdir.cgi script

This script is used to serve the incoming requests for the FastCGI interface, mine is /opt/hg-fastcgi/hgwebdir.fcgi

#!/usr/bin/env python
#
# An example CGI script to export multiple hgweb repos, edit as necessary

# adjust python path if not a system-wide install:
#import sys
#sys.path.insert(0, "/path/to/python/lib")

# enable demandloading to reduce startup time
from mercurial import demandimport; demandimport.enable()

# Uncomment to send python tracebacks to the browser if an error occurs:
#import cgitb
#cgitb.enable()

# If you'd like to serve pages with UTF-8 instead of your default
# locale charset, you can do so by uncommenting the following lines.
# Note that this will cause your .hgrc files to be interpreted in
# UTF-8 and all your repo files to be displayed using UTF-8.
#
#import os
#os.environ["HGENCODING"] = "UTF-8"

from mercurial.hgweb.hgwebdir_mod import hgwebdir
from flup.server.fcgi import WSGIServer

# The config file looks like this.  You can have paths to individual
# repos, collections of repos in a directory tree, or both.
#
# [paths]
# virtual/path1 = /real/path1
# virtual/path2 = /real/path2
# virtual/root = /real/root/*
# / = /real/root2/*
#
# [collections]
# /prefix/to/strip/off = /root/of/tree/full/of/repos
#
# paths example:
#
# * First two lines mount one repository into one virtual path, like
# '/real/path1' into 'virtual/path1'.
#
# * The third entry tells every mercurial repository found in
# '/real/root', recursively, should be mounted in 'virtual/root'. This
# format is preferred over the [collections] one, using absolute paths
# as configuration keys is not supported on every platform (including
# Windows).
#
# * The last entry is a special case mounting all repositories in
# '/real/root2' in the root of the virtual directory.
#
# collections example: say directory tree /foo contains repos /foo/bar,
# /foo/quux/baz.  Give this config section:
#   [collections]
#   /foo = /foo
# Then repos will list as bar and quux/baz.
#
# Alternatively you can pass a list of ('virtual/path', '/real/path') tuples
# or use a dictionary with entries like 'virtual/path': '/real/path'

WSGIServer(hgwebdir('/var/hg/hgweb.config')).run()

Make sure you set this script as executable with the following command:

chmod +x /opt/hg-fastcgi/hgwebdir.fcgi

Step 5:  Create the Mercurial hgweb.config

There’s a simple INI-style config file we have to create.  I called mine /var/hg/hgweb.config.  If you decide to place yours elsewhere, you need to modify the last line of the hgwebdir.fcgi script you created in step 4.

[web]
baseurl = /
allow_push = *
push_ssl = false

[collections]
/var/hg = /var/hg

Step 6:  Create the FastCGI init.d script

This script will not only handle starting and stopping your FastCGI process for Mercurial; it will also handle restarting the service on reboot.  Make sure you save this as /etc/init.d/fcgi-hg.

#! /bin/sh
#
# fcgi-hg     Startup script for the nginx HTTP Server
#
# chkconfig: - 84 15
# description: Loading php-cgi using spawn-cgi
#	       HTML files and CGI.
#
# Author:  Ryan Norbauer <ryan.norbauer@gmail.com>
# Modified:     Geoffrey Grosenbach http://topfunky.com
# Modified:     David Krmpotic http://davidhq.com
# Modified:	Kun Xi http://kunxi.org
PATH=/opt/python/bin:$PATH
DAEMON=/usr/bin/spawn-fcgi
FCGIHOST=127.0.0.1
FCGIPORT=9003
FCGIUSER=www-data
FCGIGROUP=www-data
FCGIAPP=/opt/hg-fastcgi/hgwebdir.fcgi
PIDFILE=/var/run/fcgi-hg.pid
DESC="HG in FastCGI mode"

# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0
test -x $FCGIAPP || exit 0

start() {
	$DAEMON -a $FCGIHOST -p $FCGIPORT -u $FCGIUSER -g $FCGIGROUP -f $FCGIAPP -P $PIDFILE 2> /dev/null || echo -en "\n already running"
}

stop() {
	kill -QUIT `cat $PIDFILE` || echo -en "\n not running"
}

restart() {
	kill -HUP `cat $PIDFILE` || echo -en "\n can't reload"
}

case "$1" in
  start)
    echo -n "Starting $DESC: "
    start
  ;;
  stop)
    echo -n "Stopping $DESC: "
    stop
  ;;
  restart|reload)
    echo -n "Restarting $DESC: "
    stop
    # One second might not be time enough for a daemon to stop,
    # if this happens, d_start will fail (and dpkg will break if
    # the package is being upgraded). Change the timeout if needed
    # be, or change d_stop to have start-stop-daemon use --retry.
    # Notice that using --retry slows down the shutdown process somewhat.
    sleep 1
    start
  ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop|restart|reload}" >&2
    exit 3
  ;;
esac

exit $?

Now you need to update the rc.d directories so this service will start when you reboot your box.  This simple command will do it for you:

update-rc.d fcgi-hg defaults

Now you’ll be able to start and stop your service like this:

/etc/init.d/fcgi-hg start
/etc/init.d/fcgi-hg stop
/etc/init.d/fcgi-hg restart

Step 7:  Create hguser.config for user setup

I prefer not to keep my repositories open to everyone (for obvious reasons) so we need to create an htpasswd-style user configuration file.  Note that since this will be used for basic authentication, credentials will be sent in plain text.  If that matters to you, then you should configure your Nginx install to use SSL, but this is outside the scope of this article.  Anyway, onward.  You should put this file at /var/hg/hguser.config

# Format <user>:<encrypted-password>:<comment>
scott:myencryptedpassword:Scott Anderson

Obviously you need to replace the username with your username, and the password with an appropriately encrypted value.  If you have Ruby installed, this is easy, just launch irb and type the following command:

"password".crypt("salt")

Copy the encrypted value into the file and you’re all set.

Step 8:  Modify your Nginx configuration

There are about 100 ways to setup websites in Nginx, so I’ll assume you know where to place server sections.  If you followed my previous tutorial, this will be in a file something like /usr/local/nginx/sites-enabled/geeksharp.com.  The block should look something like this:

server {
    listen 80;
    server_name dev.geeksharp.com;
    root /var/hg/;
    gzip on;

    location / {
        include fastcgi_params;
        fastcgi_pass 127.0.0.1:9003;
        fastcgi_param DOCUMENT_ROOT /var/hg/;
        fastcgi_param QUERY_STRING $query_string;
        fastcgi_param REQUEST_METHOD $request_method;
        fastcgi_param CONTENT_TYPE $content_type;
        fastcgi_param CONTENT_LENGTH $content_length;
        fastcgi_param SCRIPT_FILENAME /opt/hg-fastcgi/hgwebdir.fcgi;
        auth_basic 'geek# Source Control';
        auth_basic_user_file /var/hg/hgusers.config;
    }
}

That’s it!  You can edit all these miscellaneous files to suit your needs.  If you did everything correctly, all you need to do is start the fcgi-hg script and then restart nginx.  When you want to create a new repository, you should create it in /var/hg like so:

hg init /var/hg/project

Once you get everything running and you create a few repositories, you should see something like this:

Hope this helps someone out there.  If you have questions, or something doesn’t work, let me know and I try and help you out.  Good luck!