Git-svn bare mirrors
I’ve been trying to automate the updating of my git-svn repos on my server via cron and I’ve finally succeeded in getting something working so I thought I’d share how I’ve got things setup since there seems to be a lack of information about how to do this.
First hurdle was git-svn doesn’t understand about bare repositories - it always looks for a .git directory. Fortunately this is easily solved since git-svn does support the $GIT_DIR environment variable. So I simply set this to . and run the git-svn commands from within the repository directory. So I initialise the repository with the appropriate git-svn init command and then call fetch to retrieve the svn history.
The next problem is that the refs fetched using git-svn are placed in refs/remotes which isn’t cloned when you do a regular git clone. So we need to plugin to the hooks to update the refs after we update the repository; an example script of how to this is provided below (Taken from dscho.git):
#!/bin/sh
git for-each-ref –format=”%(refname)” refs/remotes |
sed “s/refs\/remotes\///” |
while read ref
do
git update-ref refs/heads/svn/$ref refs/remotes/$ref
done
git for-each-ref –format=”%(refname)” refs/heads/svn |
sed “s/refs\/heads\/svn\///” |
while read ref
do
if ! git-rev-parse refs/remotes/$ref > /dev/null 2>&1
then
git update-ref -d refs/heads/svn/$ref refs/heads/svn/$ref
fi
done
I call the script from the post-update script (which you should make executable by running chmod +x hooks/post-update). This script suffices but doesn’t quite do what I’d like; I’d like the tags to be placed in refs/tags/svn rather than refs/heads/svn, but the script linked will do for me for now.
My cloned repositories are available at http://git.jsharpe.net/, unfortunately I can’t offer the git protocol since my host doesn’t allow it. As I get time I’ll convert more of the Gnome repositories; I’ve only converted the ones I know I’m going to be working on for the moment.
June 5th, 2008 at 7:33 pm
Hi James, thanks for blogging about this. By any chance, did you encounter repositories that use svn:externals?
June 5th, 2008 at 8:27 pm
Just wanted to add my thanks to you for taking on this project. You probably already know this, but this idea has over 1700 votes for it on the Ubuntu Brainstorm website:
http://brainstorm.ubuntu.com/idea/93/
Note that if one enables compiz and disables the nautilus desktop, this can be accomplished by compiz (I have been running like this for 6 months). I assume however, your solution will not require compiz.
You may also want to check out this video as it is a decent proof-of-concept of this idea along with the idea of different icons per workspace.
http://www.youtube.com/watch?v=oGXYLdZEf2c
Do you plan to support different icons per workspace? In your previous blog you mentioned a directory for each workspace. Perhaps this is the way to accomplish this. Different icons in each workspace is a gnome bug that goes back to 2006.
http://bugzilla.gnome.org/show_bug.cgi?id=356922
In any event, there are many thousands of us who look forward to your work! Thank you!
June 5th, 2008 at 8:52 pm
RubenV:
Yes I have come across repositories with svn:externals. I’m currently ignoring these as jhbuild does the right thing when fetching from a git-svn conversion by parsing the unhandled.log created by git-svn. I don’t think that reconstructing the svn metadata recreates the unhandled.log file though, but I can’t really think of a nice solution to the problem of externals. Even submodules don’t really help as they require committing to the repository each update which breaks the ability to pull from the tracking repository.
Any ideas are welcome!
June 5th, 2008 at 9:02 pm
Hi James
Thanks for blogging about this. I’m in the process of mirroring the whole of svn.gnome.org - you can see the progress at http://gnomegit.unrouted.co.uk. Its a bit slow right now with the gimp mirror taking an age to run. It may well be saturday before the initial run is complete and the box isn’t dog slow. Once its all converted and i’ve applied your suggestions i’m hoping to have it run along side bzr-mirror.gnome.org (which is on the same box). With git-daemon, and hopefully some GNOME colouring.
When I get this going, the mirror should never be more than 5 minutes out of date, as I use the svn-commits-list to pull/fetch a module as it changes. For bzr-mirror, it even automatically clones new modules when they are added. It seems this will be doable for git-mirror too.
June 5th, 2008 at 9:26 pm
DaveV: yes I have seen that you can do this via compiz. When I mentioned a directory per workspace I was actually referring to gconf keys and not the actual Desktop directory. AFAIK the Desktop directory is one of those things standardised by freedesktop.org and so its not a good idea to go changing the location of the desktop, and so no I’m not planning to implement different icons on each desktop.
June 6th, 2008 at 1:55 am
There’s a simpler solution than having a hook to update a ref. Instead, just make refs/heads/svn/foo a symbolic ref of refs/remotes/foo: git symbolic-ref refs/heads/svn/foo refs/remotes/foo
June 6th, 2008 at 10:08 am
Steven Walter: That’s what I originally did, but it doesn’t work when git packs the refs - they no longer track each other once they’ve been placed in packed-refs and I also want the refs to automatically track the branches in svn - if a branch is deleted or created I want that to automatically follow in the git repo.
November 3rd, 2008 at 3:37 pm
I’ve made a post-update script to handle both tags and branches here:
http://www.shatow.net/fix-svn-refs.sh
March 1st, 2009 at 10:01 pm
thanks for the info on the scripts.