I’m broadcasting this on all channels at the moment, just so I can get the attention of the guys in Carbonite and hopefully get this feature added to their (otherwise excellent!) software. So if you’re a Carbonite user and you find this feature I’m asking for is worth anything, would love some feedback on this post — I’m not gonna ask you to sign a petition 🙂 but safety is in numbers sometimes, so if it’s a few of us hopefully it will make a convincing case for Carbonite to slot this in.

As a user of Carbonite, hopefully you know what this is all about (I know this is providing a bit of advertising for them, but I feel I do need to provide a bit of context here too): it’s a backup/restore solution; you select which directories to back up and the software copies them transparently for you “in the cloud” when files are changed.

So once set up, and you indicated which directories you want to back up, you don’t have to touch a button — it’s all done for you behind the scene. Great!

There is though one scenario which is the bit that I feel needs some lovin’ from Carbonite: as you might have spotted in the above screenshot, I keep all of my coding projects under my Documents/workspace directory. There is reason behind my madness: from my point of view it makes it easier to find things — as Documents is always present in my shortcuts bar, and also, because it’s under my home directory, rather than in the root folder, I don’t have to concern myself with sudo-ing or changing file permissions etc when editing these files. Either way, even if you keep your source files outside the Documents directory, my suggestion still stands.

So, I’ve set my ~/Documents/workspace/ folder structured in all the various projects I work on — some for Cognitive Match, some for Apache Software Foundation, some just randomly off SourceForge or github and some just pieces of code that I’m trying out. While some of the sources are pulled from repositories (subversion, cvs, github etc) — and as such I can assume that they are backed up at origin — some other sources as I said exist entirely on my Mac. Also, even with some of the sources from repositories, I find myself hacking away at them for 1-2 weeks (sometimes less, sometimes more) until (I think) I get it right. During this period, I won’t commit anything to the repository until I’m pretty damn sure that my code is correct and won’t break the build. As such, during this time, should anything happen to my Mac (touch wood!) I can lose a few days worth of work on that project. Now, I don’t know about you, but I prefer for that not to happen 🙂 which is why I set up my projects directory to be backed up by Carbonite — so I get the piece of mind of my sources being safe even in between commits to repositories. And this is where my pain starts 🙂

Let’s look at an example: the workspace/functor directory — this is pulled from Apache Commons SVN and it is the code for the Functor component which I’m committing to (occasionally). Simone Tripodi did a great job of mavenizing and organising this project, as such the structure of the folder looks like this:

-rw-r--r--   1 liviutudor  staff    598 13 Nov 08:44 .classpath
-rw-r--r--   1 liviutudor  staff    584 13 Nov 08:44 .project
drwxr-xr-x   6 liviutudor  staff    204 13 Nov 08:44 .settings
drwxr-xr-x   9 liviutudor  staff    306 20 Feb 20:47 .svn
-rw-r--r--   1 liviutudor  staff  11358 13 Nov 08:43 LICENSE.txt
-rw-r--r--   1 liviutudor  staff    187 20 Feb 20:47 NOTICE.txt
-rw-r--r--   1 liviutudor  staff   1570 13 Nov 08:43 RELEASE-NOTES.txt
-rw-r--r--   1 liviutudor  staff   1370 13 Nov 08:43 checkstyle-suppressions.xml
-rw-r--r--   1 liviutudor  staff   8045 13 Nov 08:43 checkstyle.xml
-rw-r--r--   1 liviutudor  staff   2411 13 Nov 08:43 doap_functor.rdf
-rw-r--r--   1 liviutudor  staff   1372 13 Nov 08:43 fb-excludes.xml
-rw-r--r--   1 liviutudor  staff  11496 20 Feb 20:47 pom.xml
drwxr-xr-x   8 liviutudor  staff    272 13 Nov 08:43 src
drwxr-xr-x  32 liviutudor  staff   1088 25 Feb 19:38 target

As you would expect, there is a src and a target directory — the src directory is where the sources reside (see below) and the target directoy is where compilation takes place. Below the src directory structure — though really that’s a bit irrelevant here:

drwxr-xr-x   8 liviutudor  staff  272 13 Nov 08:43 .
drwxr-xr-x  16 liviutudor  staff  544 25 Feb 19:38 ..
drwxr-xr-x   8 liviutudor  staff  272 20 Feb 20:47 .svn
drwxr-xr-x   4 liviutudor  staff  136 13 Nov 08:43 changes
drwxr-xr-x   5 liviutudor  staff  170 13 Nov 08:43 conf
drwxr-xr-x   5 liviutudor  staff  170 13 Nov 08:43 main
drwxr-xr-x   6 liviutudor  staff  204 13 Nov 08:43 site
drwxr-xr-x   5 liviutudor  staff  170 13 Nov 08:43 test

Now, as I said, I obviously want these sources backed up — I typically work on Functor in the evenings and weekends, and as such I find it takes me sometimes 1-2 weeks to get a full set of code changes ready: I’ll do bits and pieces one night, then continue the next night, pause it for a couple of days and think about stuff then continue again over the weekend. With the functor directory being under Documents/workspace, obviously Carbonite will pick up on any source code changes I might operate and safely back those up for me. The trouble is, like any other coder, when I make changes to the source code, I will run a full compile to make sure it works, probably run the full set of unit tests too and do a mvn site too and go through the Checkstyle/Findbugs/PMD/etc reports too.

The compilation though will wipe out my functor/target directory and re-create all the .class files, jar them up, and in some cases, will also produce a lib/ directory with all the dependencies too (all the jars/libraries used by the project) — if the maven dependency plugin is configured in the project for instance. If I run a mvn site too, this will typically create a target/site directory where it generates a bunch of html files, copies even images and whatever else I might have under src/main/site — and depending on whatever I’ve configured under Project Reports, there could be quite a few of these html files being generated. If I have enabled the JavaDoc Maven plugin too, the list of html files can significantly grow bigger! So at the end of this compile run, I have got myself a pretty big target directory, with every single file “changed” (from last run) — as files will be deleted and re-created (so different time stamp, but quite likely content changes too).

At which point, Carbonite signals this internally and decides to back up again the whole directory — so I got a whole lot of CPU time being taken and bandwidth for the purpose of backing this folder up. As I have just explained above, this folder can be re-created at any point from the source files, and as such I don’t care if it gets backed up or not; in fact I would prefer if it doesn’t get backed up, as it’s subject to changing entirely on every compilation and since I run a lot of compilations during my code changes, I’d rather not have Carbonite kick off a backup every 5-10 minutes or so for this.

All good, you’d say, I can go in my Carbonite Preferences dialog and add ~/Documents/workspace/functor/target as a “do not backup this” item. And that would solve the problem with my Functor directory. But my workspace directory contains about 20 projects! I can go of course through every single one of them and do the same for the target subdirectory under each one of these folders — but this can be a lengthy process 🙁 Also, if at some point I decide to pull another project in from SVN or github etc, I need to remember to go to Carbonite Preferences and add the target directory each time — and if you knew me, you’d know I’m unlikely to remember this shite every time 🙂

So what I would like Carbonite to implement (and it looks like a tiny feature, but it will save me lots of time and lots of CPU and lots of bandwidth wasted!) is a feature where I can go and say: backup everything under this directory apart from files or directories which match this pattern/regular expression/whatever. If they do that, then I can point Carbonite to my ~/Documents/workspace directory and ask it to back it up — but then just say “though exclude anything named target/*“ and I’m done! Any compilation I run, will happen under a target directory — which will then be caught by Carbonite as a “no go” and ignored every time my compilations run. And every time I pull in a new project in my workspace directory, I don’t have to remember to add its target directory to the “do not backup” list, I don’t have to set any flags anywhere, I’ll just pull the project and work on it, with Carbonite still backing up my sources but not worried about the changed compiled classes.

Do you not think this would be such an awesome feature for us, techies? Carbonite guys, come on, get your arse in gear and give me this feature — please! 🙂