rsync Issues After Upgrading to Ubuntu 14.04 LTS

I use Fabric to automate deployments to my personal website,  The underlying mechanism that Fabric uses to send code from my personal computer to my web server is rsync, and after upgrading my WWW server VM to Ubuntu 14.04 LTS I started getting a strange rsync error from rsync when trying to push changes to my website:

Pushing files remotely...
[] rsync_project: rsync --delete --exclude ".git*" --exclude "settings_local*" --exclude "*.jar" --exclude "*.pyc" --exclude "*.sqlite" --exclude "*.tar.gz" --exclude "*.zip" -pthrvz  --rsh='ssh  -p 22 ' ../tehranian
[localhost] local: rsync --delete --exclude ".git*" --exclude "settings_local*" --exclude "*.jar" --exclude "*.pyc" --exclude "*.sqlite" --exclude "*.tar.gz" --exclude "*.zip" -pthrvz  --rsh='ssh  -p 22 ' ../tehranian
protocol version mismatch -- is your shell clean?
(see the rsync man page for an explanation)
rsync error: protocol incompatibility (code 2) at compat.c(181) [sender=3.0.9]

Fatal error: local() encountered an error (return code 2) while executing 'rsync --delete --exclude ".git*" --exclude "settings_local*" --exclude "*.jar" --exclude "*.pyc" --exclude "*.sqlite" --exclude "*.tar.gz" --exclude "*.zip" -pthrvz  --rsh='ssh  -p 22 ' ../tehranian'

The highlighted text made me think that there was some sort of incompatibility between the rsync 2.6.9 client on my Mac and the rsync 3.1.0 that my shiny new Ubuntu disto came with.  Unfortunately upgrading rsync on my Mac via Homebrew did not to help.

The real issue at hand was that the local user on my WWW server that I use to rsync had its login shell switched to “/usr/sbin/nologin” by the Ubuntu upgrader.  How strange.

After editing “/etc/passwd” and changing the login shell for my user back to “/bin/bash” rsync is working again without cryptic & misleading errors about protocol incompatibilities.


RELENG 2014 Wrap-Up

I was a speaker at the 2nd International Workshop on Release Engineering @ Google HQ last week.  I enjoyed meeting up with other release engineers and sharing ideas with them.  Here are some talks I enjoyed:

Finally, my own presentation went quite well.  There were a lot of people in the audience that came up to chat with me afterwards and it seemed that the message really resonated with the audience (confirmation bias, much? : ).  From the sounds of it, I may have an opportunity to give an extended-version of this talk at Google and VMware in the near future which is great because there were several ROI examples and software industry anecdotes around quality and time-to-market that I had to cut out to meet the time requirements.

Here are the slides I used:

Jenkins Job DSL Plugin Saves the Day

At Virtual Instruments we have some ~50 Jenkins jobs that generate the RPMs from each of our GIT repos as well as the appliance images of our product (ISO, OVA, PXE, Vagrant, etc).

Over the past few months my team had put a lot of time & effort into taking the configuration of all of those pre-existing, manually created jobs and codifying them via the Jenkins Job DSL Plugin.  This week all of that hidden “plumbing” work finally paid its first dividends for us when we had to cut the branch for our upcoming release, and the benefit was enormous.  Since the Job DSL Plugin puts all of our configuration into source control and gives us the ability to programmatically create all of our Jenkins jobs, it saved us many painful hours of manual job configuration via the Jenkins UI.  Instead of having to copy each of those existing 50 jobs and modify them by hand, we only had to do some editing of our Groovy files, send for review, submit the changes to our Git repo, and watch the new jobs get created automatically.

Given that this was our first time doing this type of release branching we had to work out a few kinks and do extra validation, but by 5pm we had all of the continuous builds working for each repo as well as images for our product being successfully created.

Even better is that as we were working out a few obscure kinks with the developers, those developers quickly realized that they too could edit the Groovy files in Git to make simple changes to the job configuration.  In other words, the developers realized that they were not gated upon our DevOps team to make job configuration changes for them.  I’m hoping that we can expand on this in the coming weeks and have the developers take even more ownership of the build job configurations.

The big benefit with the Job DSL Plugin is that it takes job configuration that was previously done by hand in a UI and puts it into a documented format in source control where the changes can be reviewed via Review Board and tracked over time.  With that we can confidently provide access to making job configuration changes to the whole development team which ultimately makes for a quicker configuration change process and a lighter workload for my team.