Paying for cloud services? Damnit, there is a cheaper way.
I don’t get why people pay so much for online services for developers. Don’t get me wrong: it is great for the companies that people are, but I don’t get it.
These days you can do it for much cheaper if you are willing to run a server.
Look, for instance, at Github, Lighthouse and Kiln. Now, I use Git and I have private projects, some of which require somewhere to stash my code and my issues.
Github is pretty good: for $7, you get 5 private repositories and 600Mb of storage space. And each of those projects gets a lightweight issue tracker. I like Github and I use it for public projects.
Before Github did issue tracking, I used Lighthouse for issue tracking. They do free public projects and then have a freemium model that starts at $25 a month for 10 projects.
Bitbucket is like Github but for Mercurial: they start at $5 a month for 5 private repositories and 2.5Gb of storage. DISREGARD THAT: SINCE ATLASSIAN BOUGHT BITBUCKET, PRIVATE REPOS ARE FREE! WOOOO!
Kiln is a new one and comes from Fog Creek. It is also Mercurial-based and you can either get it for installation on your own (Windows and SQL Server) hardware on a per-head basis, or you can get it on Fog Creek’s servers for a whopping $25 per user per month, or £30 per user per month with FogBugz, Fog Creek’s bug tracking software.
There’s a much cheaper way of doing it.
You go along to Linode, Slicehost, Amazon EC2 or a VPS or cloud hosting service of your choice. Gotta be able to run Linux. You sign up for their cheapest plan. I’m using Linode - $19.95 a month. That gives you 16Gb of storage and 200Gb of transfer, 360Mb of RAM. When you sign up, you get to choose what OS you want on there. Unless you’ve got strong preferences to the contrary, go for Debian.
Then you install the following things: apache2 (or nginx or whatever), mod_ssl, Git (or Mercurial or SVN if you are old skool), PostgreSQL (or MySQL), Ruby, rubygems and Rails (or Python), Phusion Passenger (or mod_wsgi), Redmine (or Trac). If you know your shit with Linux, you can probably do this in about an hour. You also need to upload SSL certificates and so on and maybe set up a git account. This is all you need. You’ve got an issue tracker with Redmine and everyone can commit to your private Git repositories (the public ones? Stick ‘em on Github. Same with Mercurial and Bitbucket, Subversion and Google Code). You can tie Redmine into Git (and SVN, Mercurial, CVS, bzr, darcs) so you can explore your repository through the website. It has got syntax highlighting, per-file history and each change gets a URL, albeit a quite long one. And it scans commit messages for issue references using a reasonably simple syntax to close or refer to issues.
Now, for $20 a month you get 16Gb. If you need more, you can upgrade. You can host as many projects as you like and have as many contributors as you like. The closest Github plan is $22 ‘Medium’ plan which gets you 2.4Gb, 20 private repos and 10 private collaborators. And Bitbucket’s closest is $12 for 5Gb and 15 repos.
If your issue tracker isn’t quite spiffy enough, you can download new plugins, and if you don’t like it, it is open source so you can change it.
You can do other stuff quite easily too - you get a wiki to shove all your documentation and it does some time tracking stuff for the Gantt chart obsessed. And it’ll integrate with e-mail pretty well.
Also, you can host your website on the VPS too. Did I mention that?
Once you go beyond about 30Gb of storage, you should probably switch to an EC2 Reserved Instance. You’ll need to faff around a bit to back-up your stuff (to S3) but you basically purchase a small reserved instance, pay $227.50 up-front and then pay $0.03 an hour - the whole thing works out at $40.85 a month (or $48 to host it in the EU - remember the Data Protection Act if you are in the UK). For that, you’ll get 160Gb of storage. To get an equivalent amount from Github? $200 a month for their 60Gb plan.
Really. These services are very convenient, but we are in a recession. Every penny counts. If you know what you are doing, it shouldn’t take more than about an hour to set this stuff up and it can really reduce the amount you are paying. I hate to write this post because I really do like and wish the best to the entrepreneurs who have created Github and Bitbucket and other similar services. They’ve brought vital competition to the aging Sourceforge and - in the case of Github - brought many people into open source programming (I remember sending in a patch by e-mail to Dr. Nic’s Ruby on Rails TextMate plugin in order to earn a Github beta invite!) by making the whole forking and contributing thing simple.
But hopefully, if people realise how it isn’t necessarily that hard to host this stuff for yourself with free, open source software on commodity VPS hosting, that becomes competition which should benefit users of these services.
Going even cheaper: S3 for hosting your repositories and even your bug tracker
So, to carry on from running your own Git and issue tracking stuff on a VPS, we can go even further.
Who needs that stuff? You can do it even cheaper.
The first step is to ditch your issue tracker. We’ll make one, but it’ll be decentralised.
Where do you put your repository? On Amazon S3, of course. 60Gb on Github is $200 a month. 60Gb on S3 is $9 a month.
How do you get your stuff up onto S3? That’s a bit of a difficult one. The easiest way is to use jgit. Once I found the download for jgit, it took me about ten minutes to setup. I can now stow my private projects on Amazon S3. The Github $12/month account gives you 1.2Gb of storage. Well, remember, Amazon S3 is 15 cents per gigabyte per month plus transfers.
The only difference I have now is that to push to S3, I do jgit push s3 master rather than git push origin master
The setup is easy for UNIX: you download jgit here, as the .sh file, then rename that to jgit, stick it in ~/bin or somewhere else on your path (/usr/local/bin maybe), chmod it to 755. And to setup, follow the instructions here.
Okay, so we’ve got cheap repository hosting for private repos. But what about our Redmine install? Well, we have a distributed version control so why not have a distributed bug tracker to go with it? Kind of makes sense. Especially as one of the great advantages of Git and other DVCSes is cheap branching and merging. If you let ten thousand flowers bloom on hard drives across the land, why bother still having one canonical bug tracker? People start up branches to fix issues. As they make changes in those branches, they’ll undoubtedly uncover more bugs.
This is where distributed issue trackers come in. There’s a bunch to choose from. Now, some of these seem to operate on the basis of sticking stuff in a separate branch - git-issues and ticgit put stuff in an issues branch. They are also both Git-centric. I basically ruled them out without trying.
There are a few left: Ditz is Ruby-based and is VCS agnostic - you can use it with both DVCSes like Git and with old-fashioned VCSes like Subversion. And it’ll work for the tarballs-and-patches crowd too. It also does static site generation. cil is similar. Ditz is written in Ruby and cil is written in Perl.
A slightly more promising approach is the Perl programme Simple Defects - or SD for short. It can be used without a VCS, it is compatible with all the VCSes, but has integration plugins with Git and Darcs. But it also syncs up with existing bug trackers: RT, Hiveminder, Trac, Github, Google Code and Redmine. Huzzah! (My excitement is tempered slightly by the fact that Redmine support is read-only.) The command line interface seems pretty simple, and lets you setup aliases - if you are used to saveable filters in Trac and Redmine, this will let you setup handy shortcuts to common filters or commands. Simple Defects is based on another Perl project called Prophet which is a grounded, semirelational, peer to peer replicated, disconnected, versioned, property database with self-healing conflict resolution
. The developers have a philosophical basis for this: they like the Web 2.0 stuff but boy is it centralized. Preach on.
This is all good, but because it is Perl, you get CPAN, which means you spend half an hour hitting the return key while CPAN installs a few more thousand modules and then it still all breaks. Pathologically Eclectic Rubbish Lister indeed.
So, there is Python project called Bugs Everywhere which is in the same category as Simple Defects, but without all the nice integration with the existing bug trackers.
I eventually went with ditz because Bugs Everywhere just failed to compile and Simple Defects - as I said - sent me into merry CPAN hell. ditz was as simple as sudo gem install ditz and it Just Worked, Damnit. Despite not having all the nice features that SD has, it is pretty nice. The only thing I’m not fond of is that it uses the shell rather than my text editor to let me enter bugs. There is apparently a way around that - ditz.vim exists (as does something for the Emacs crowd).
Just as an interesting data point, this e-mail on ditz-talk says a lot of stuff I agree with: it’d be nice if ditz were to sync up with bug trackers (he uses JIRA - I’ve heard the horror stories and I’ve experienced the searing pain that is Atlassian Confluence, and if JIRA is anything like that, I don’t envy anyone who has to use it).
Ditz has some nice stuff: you can get it to render up static HTML listings of all the bugs. The data format is nice: it stores everything in a folder which you can nominate during project setup (default is ‘bugs’). Inside there are lots of YAML files: starts with project.yaml and then creates an issue-[hash].yaml file for each issue. The YAML format is human-readable, and you can tweak it with a text editor without much fuss. It should merge nicely when you merge or rebase branches.
If you aren’t wild about paying a monthly credit card subscription for Github or Bitbucket (etc.) and you don’t want to go the VPS route, then jgit will help you get your private projects up onto S3 and ditz means you can cut the tie to Redmine or Trac (etc.) and work through your bugs with the sort of workflow that Git/Mercurial has gotten you used to with version control.
