I recently realized that when I changed my WordPress permalink structure from /%year%/%month%/%day%/%title/ to just /%title%/ that I had some old links floating around on various websites like StackOverflow. While normally I wouldn’t care too terribly much, one in particular was quite popular which I can only assume and hope meant that it was particularly helpful. I didn’t want that to be lost to the community.

To address this, I added a new location directive with a rewrite rule to my nginx config, like so:

# For locations matching /0000/00/00/some-title-of-a-post
location ~ "^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+)" {
     # Store the post title in a variable, because the rewrite resets the backreferences from the regular expression
     set $post_title $1;
     # Use the stored title to rewrite the address
     rewrite ^(.*)$ http://blog.sdbarker.com/$post_title permanent;
}

This allowed me to not break any old WordPress permalinks that were floating around while keeping the new, current style.

Both links now work as expected. Unforunately, though, the poor StackOverflow visitors who needed the help that article provided will not be able to find it. The StackOverflow moderators oh so very helpfully deleted not only the original answer that contained the link, but all three of my attempts to re-answer the question. And all without any explanation. Awesome.

I recently migrated my Arch Linux VMs from sysvinit to systemd as the newer Arch install media are all using systemd and support for sysvinit is discontinued. The migration itself was an incredibly trivial and straight-forward process. I had only one hang up with the migration in general; every time I’d reboot the machine, eth0 wouldn’t come back up. That was fixed with a simple systemctl enable dhcpcd@eth0.

After I had the migration complete, sshd (and everything else) enabled, and eth0 finally coming up, I had two services (or as systemd refers to them, units) that I needed to configure; Chiliproject and git-daemon. Adding these custom units to systemd was incredibly easy.

Your custom service files live in /etc/systemd/system and end with the extension .service. Here the contents of the two that I created:

sdbarker.com-chiliproject.service:

[Unit]
Description=sdbarker.com Chiliproject
Requires=mysqld.service nginx.service
Wants=mysqld.service nginx.service

[Service]
User=www-data
WorkingDirectory=/path/to/chiliproject/install
ExecStart=/usr/bin/bundle exec /usr/bin/unicorn_rails -E production -c config/unicorn.rb
PIDFile=/path/to/chiliproject/install/tmp/pids/server.pid

[Install]
WantedBy=multi-user.target

git-daemon.service:

[Unit]
Description=Git daemon

[Service]
Type=forking
User=git
WorkingDirectory=/path/to/git/home
ExecStart=/usr/lib/git-core/git-daemon --pid-file=/path/to/git/home/temp/pids/git-daemon.pid --detach --syslog --verbose --base-path=/path/to/git/home/repositories
PIDFile=/path/to/git/home/pids/git-daemon.pid

[Install]
WantedBy=multi-user.target

Overall, it was an incredibly easy migration, and my startup time (however rare) is greatly reduced, service management is much, much easier, journalctl for viewing logs is very convenient, and removing all of the boilerplate initscript code that was always a pain in the ass and honestly always a bit flakey was very welcome.

I noticed the other day that my Archlinux logs were all zero bytes. Empty logs don’t do much good, so I spent a while trolling around the internets and had a difficult time finding anything valuable.

I did find one valuable thing in my boot log at /var/log/boot:

Starting Syslog-NG [BUSY] Error binding socket; addr='AF_UNIX(/run/systemd/journal/syslog)', error='No such file or directory (2)'

Which was enough to lead me to this post, https://bbs.archlinux.org/viewtopic.php?id=151132 (which had an unrelated title). The solutioon is to update /etc/syslog-ng/syslog-ng.conf and change the line that sets unix-dgram to read unix-dgram("/dev/log");, and then restart (or start, since it probably never started in the first place) syslog-ng with rc.d start syslog-ng. If you log out and log back in after that, you should see at the very least auth.log will have content.

Of note, I also had a permissions problem with a lot of the log files, so I had to do a quick chmod --recursive g+w /var/log/* to give the log group permissions to write to the logs.

Not so long ago, I had a post about Completely removing all traces of files and history from Git repositories. That title was somewhat misleading, since it was more about migrating from TFS to Git, and in the process pruning out files that you didn’t want to leave in your repository. So, if you’re looking to migrate from TFS to Git, and/or you’re looking to prune out history, have a look at that post.

I’ve recently started using Google+ Hangouts. One problem that I noticed out of the gate is that the Google Talk Plugin (which is required for Hangouts to work) thinks that it’s smarter than I am about adjusting my microphone level, an constantly adjusts it. This leads to the unfortunate side effect of everybody listening to my keystrokes. A little bit of registry hackery fixes this problem.

[HKEY_CURRENT_USER\Software\Google\Google Talk Plugin] 
"audio-flags"="1"

For the record, my default value for that key was 15.

Update: This is no longer necessary, check out the update here.

In my last post (PRO-TIP: Recursively updating all git repositories in a directory) I made mention of using Git’s new credential caching to improve your Git experience.

A couple things of note:

• The API for credential management in Git is fairly newish (as of 1.7.9) so you’ll a fairly newish Git to make use of it.

• This doesn’t work for Windows systems as git-credential-cache communicates through a Unix socket. Windows users can find joy here at https://github.com/anurse/git-credential-winstore (specifically for the download the joy is located at https://github.com/anurse/git-credential-winstore/downloads). Just make sure that the binary is in your path (which is likely C:\Program Files (x86)\Git\libexec\git-core if you’ve installed msysgit and didn’t mess with the defaults). The integration is with Windows Credential Manager which you can pull up via the Windows Control Panel.

Once you’ve got it installed, from a command prompt:

git config --global credential.helper winstore

Or you can edit your .gitconfig manually:

[credential]
     helper = winstore

I have a directory, let’s call it ./src (because it is). This directory has several other directories of which some subset are git repositories. Updating them all by hand is tedious at best and I am lazy. Here’s a one-liner that will do all of the work for you.

W=`pwd`;for i in $(find . -name .git);do D=$i;D=${D%/*};cd $W/$D;pwd;git pull;done

This has the potential to be a bit taxing as well though if you need to enter credentials for every pull. We can solve that one as well.

git config --global credential.helper 'cache --timeout=[S]'

The --timeout option has a parameter, S, which is the timeout in seconds. This can be omitted and a default timeout of 15 minutes will be used.

Being lazy + saving time + making things easy = Happy Panda!

At my company, we recently made the decision to migrate our source repositories from TFS to Git. While new projects have been going in to Git for quite some time, and some projects had already been migrated, the largest still remained. Let’s call this project Website.

The Website repository in TFS contained not only history for the last several years but also all of that history for our marketing content for the site, largely images. We determined that this shouldn’t be part of the repository at all (at least this one, it would continue to live in its own TFS repository until final arrangements could be made) and thus removing it would be part of our migration process. The process boiled down to this:

1. Branch out the Marketing content in to its own TFS repository using a build step for Website to perform the check out and get it where it needs to be.
2. Clone the TFS repository in to a new Git repository.
3. Clean out all of the old history of the Marketing content from Website.

I had no real involvement with the first step, so I spent some time with steps #2 and #3. I ran through this process on my development machine (an eight core Xeon 3.33ghz with 12gb of ram, Windows 7 Professional) a few times to make sure that I had the process down and to get an idea of timing so that we could better plan. Both steps took about 4 days each to complete. Woof. As it turns out, a co-worker spun up a large Ubuntu EC2 instance to do the heavy lifting after the process was down and brought the execution time down to about 2.5 hours total.

Here’s what we did to actually make this happen.

Clone the TFS repository in to a new Git repository.

This becomes a very straight forward process using git-tfs (http://git-tfs.com/, https://github.com/spraints/git-tfs). I won’t go in to the details of getting that pulled down and running. Once you have it though, you’re going to issue this command from the root of the directory that you want to clone the repository in to:

git tfs clone -d --username <username> http://server:8080/tfs/collection <repository> .

Note: You’ll have to do at least this initial clone on a Windows machine as the git-tfs binaries are Windows-only. Anything after that can run anywhere that Git can run.

Make sure to replace the placeholders (<username>, the URL, and <repository>) with values that are relevant for your system.

After that’s done spinning for a while, you’ll have a Git repository that mirrors your TFS repository.

Clean out all of the old history of the Marketing content from Website.

Now we want to get rid of all of the enormous Marketing content. This is where the Git voodoo comes in.

From the root directory of your Git repository, you’ll want to issue these commands:

git filter-branch --index-filter "git rm -rf --cached --ignore-unmatch Website/Marketing" --prune-empty HEAD
rm –rf ./.git/refs/original
git reflog expire --expire=0 --all
git gc --aggressive --prune=0

The filter-branch command will go through every commit in your repository removing any files and history in the Website/Marketing path and, if any commits would be left empty after that, removing the commit in its entirety. The next command removes the original references that were in the repository; these would be pointing to the bits of history that we removed earlier. After that, expire all of the references in the repository. Finally, garbage collect your repository.

The final result is going to be a commit-by-commit clone of your TFS repository, less any of the data you told it to remove and potentially less a few commits if they only contained files that you removed.

In the end, we had a TFS repository weighing in at about 1.2gb which become a Git repository sitting around 330mb. That’s a pretty substantial win.

Potential pitfall

You might find that the size of your repository didn’t decrease at all after you do this. My previously mentioned coworker had to actually clone the repository to another git repository to see the result and end up with a smaller repository.

cd ..
mkdir WebsiteClone
cd WebsiteClone
git clone ../Website

After seeing that the project that I’m working on had 48 branches hanging out, 40 of which were completely merged and abandoned, I threw together a quick bit of one-liner (it’s still one line even if it’s more than 80 columns, right?) to delete all of the fully merged branches.

Full disclosure; this works on my machine, which happens to be a Win7 machine with some release of Cygwin on it (which I loathe):

for k in $(git branch -a --merged) | grep -v "\->\|master" | sed s/remotes\\/origin\\///);\
do $(git branch -d $k; git push origin :$k); done

And when you’re done, make sure you alert others to git remote prune origin.

The breakdown

To elaborate on this a little bit, for each branch $k in every completely merged branch:

for k in $(git branch -a --merged)

Make sure that it isn’t pointing somewhere (like HEAD would be) and isn’t master:

 | grep -v "\->\|master"

Remove the remotes/origin/ part of the branch name:

| sed s/remotes\\/origin\\///);\

Delete the local branch (if it’s there) and push the delete to the repository:

do $(git branch -d $k; git push origin :$k); done

Inspiration taken from Graham King at http://www.darkcoding.net/software/cleaning-up-old-git-branches/

One of the great features of Github (and there are many) is the ability to fork a project to do your own development and experimentation on. I don’t, however, use Github, as I prefer to keep everything on my own server and not pay for something that I’m capable of hosting myself. I recently started a new project that had a base of the framework from an old project, so this was an obvious place to do some pseudo-refactoring (I’m leaving the old, original project intact) and giving me a good starting bed for future new projects. I had a few goals:

• Break the framework out of the Old Project into its own New Framework repository
• Fork New Framework and start development of New Project
• Easily fold changes to New Framework into New Project without tainting New Framework
• Easily fold changes to New Project that related explicitly to New Framework back in to New Framework

I started off by cloning Old Project in to New Framework and stripping out everything that had anything to do with Old Project.

git clone OldProject NewFramework
cd NewFramework
rm -rf .git/
git init
# *hack hack hack to remove all OldProject cruft from NewFramework*
git remote add origin [new-framework repository url]
git push origin master

At this point, we have our NewFramework directory that’s full of just NewFramework, and it’s all happy and pushed off to the NewFramework repository. This is the starting point that we want for future projects that will be using this framework which is perfect because that’s exactly the situation that we’re in. So, let’s start NewProject using NewFramework as a base.

git clone NewFramework NewProject
cd NewProject
git remote -v
git remote add upstream [url of the 'origin' remote from the git remote -v command]
git remote remove origin
git remote add origin [url of the repository for NewProject]
git push origin master
git branch upstream

And now we have our NewProject repository created with our bare copy of NewFramework committed and pushed up. When we have changes in NewFramework that we want to fold back in to NewProject, we can do this:

git checkout upstream
git pull upstream master
git checkout master
git merge upstream

And if we have some changes to the framework that we want to go back to NewFramework:

git log
# *find the sha of the commit we want*
git checkout upstream
git cherrypick [sha of the commit we want]
git push upstream upstream:master