Steve's blog

A blog about what's going on in Analysis UK...

MSBuild S3 Publisher

Recently a few of us got together in a local pub to enjoy a brief glimpse of the British sunshine, some code and the nice beer served at the Tram Depot.

The Goal: (Other than to drink beer)

We wanted to learn about the AWS Api and create a MSBuild target that could copy files to S3 as part of a build. I'm currently using S3, CloudFront and EC2 for hosting and storage to support Dollars2Pounds and my various other websites but much of the deployment still requires a variety of tools and manual steps so a automated build for this would make life simpler.

After a lot of flapping around trying to get the Three portable WiFi hotspot to work, some food and a drop of beer the coding began.

Creating a MSBuild task is really simple, Bart De Smet has a great custom MSBuild task cookbook blog entry which helped us get the basics in. Next we fired up the AWS SDK for .Net, that again made things very simple.

A few lines of code (and another beer!) later we had the basics in place, a MSBuild task that would create a S3 bucket and add an object to it. Sadly we had to stop coding at that point so we could catch the first part of Dr Who's weeping angles (did I mention we are all geeks?)

I finished the S3 publisher off latter that evening, so here's how how to use it.

Download the code from snowcode.google.com

svn checkout http://snowcode.googlecode.com/svn/trunk/ snowcode-read-only

Note that this also contains a stack of code from the recent SnowCode event.

Navigate to the 2010-04-TramDepot/Snowcode.S3BuildPublisher folder and either open and build the Snowcode.S3BuildPublisher.sln solution or use MSBuild:

MSBuild Build.proj /p:Configuration=Release

You can then use the binary file in your own MSBuild scritps with:

<UsingTask TaskName="S3BuildPublisher" AssemblyFile="Snowcode.S3BuildPublisher.dll"/>

<UsingTask TaskName="StoreClientDetailsTask" AssemblyFile="Snowcode.S3BuildPublisher.dll"/>

 

The task library currently consists of two tasks:

StoreClientDetailsTask

Use this task to store your Aws credentials on the machine. They get stored in the registry and the secret access key gets encrypted on the way using the EncryptionContainerName. This saves you having to embed your aws credentials in build scripts.

S3BuildPublisher

This is the main task which will copy the files to the appropriate S3 bucket, and takes the following parameters:

EncryptionContainerName – this is the container name you used when storing the client details, without this the secret key cannot be decrypted.

SourceFiles – single filename or a MSBuild array (i.e. @(files) ) of source files from a ItemGroup.

DestinationBucket – the bucket to copy the files to, will be created if it does not exist.

PublicRead – if set to “true” the files will be marked as public readable, otherwise they will be private.

The source contain two examples of using these tasks, the PublishData.proj file is used for debugging the tasks and the Publish.proj file is used during a CCNET build to publish the binaries to S3, using the task it's self.

It currently has some limitations, sub folders are not supported, error handling is very light weight, no choice of aws region.

We have released this under the MS-PL license so do as you please with it, if you like it why not join us for some more CAMDUG pub coding sessions, perhaps next time in a pub with WiFi.

Thanks to Alan for his help and driving the keyboard and Alastair for the photos and input.