Steve's blog

A blog about what's going on in Analysis UK...

MSBuild AWS Task Library

Previously myself, Alan and Alastair had pitched in down the pub to create a simple MSBuild to S3 publisher task, mainly as a way for us to learn the Amazon Web Services (AWS) API. It turns out I wasn't the only one wanting a MSBuild task for S3/AWS, the week that followed saw Roy Osherove tweet that he was looking for something similar and Neil Robbins was looking for EC2 automation.

So with that, and a bank holiday weekend at hand the S3 publisher task got extended. It now supports:

  • More S3 commands

    • CreateS3BucketTask

    • DeleteS3BucketTask

    • PutS3FileObjectTask

    • PutS3TextObjectTask

    • DeleteS3ObjectTask

    • SetS3ObjectAclTask

    • S3BuildPublisher

  • EC2 Automation

    • AssociateIpAddressTask

    • AttachVolumeTask

    • CreateSnapShotTask

    • CreateVolumeFromSnapshotTask

    • CreateVolumeTask

    • DeleteSnapShotTask

    • DeleteVolumeTask

    • DetachVolumeTask

    • DisassociateIpAddressTask

    • RebootEC2InstancesTask

    • RunEC2InstancesTask

    • StartEC2InstancesTask

    • StopEC2InstancesTask

    • TerminateEC2InstancesTask

  • SimpleDB

    • CreateSimpleDBDomainTask

    • DeleteSimpleDBDomainTask

    • PutSimpleDBAttributeTask

    • GetSimpleDBAttributeTask

    • DeleteSimpleDBAttributesTask

  • Simple Notification Service

    • AddSNSPermissionsTask

    • CreateSNSTopicTask

    • DeleteSNSTopicTask

    • PublishSNSNotificationTask

    • SubscribeToSNSTopicTask

    • UnsubscribeFromSNSTopicTask

  • Simple Queue Service.

    • CreateSQSQueueTask

    • DeleteSQSQueueTask

    • SendSQSMessageTask

    • DeleteSQSMessageTask

    • ReceiveSQSMessageTask

    • WaitForSQSMessageTask

If you want to see how to use the .NET SDK for AWS have a loot at the code, it gives a really simple insight and the code for the MSBuild task library is licensed under MS-PL so you can take what you want.

Since the last blog post I've changed the build script so the files are now zipped and distributed to S3 so you can download the latest MSBuild AWS Task Library build, I am also uploading builds (not automated yet) to the download section of the SnowCode Google code repository.

Alan pointed out theirs a much better Url to svn the code from: https://snowcode.googlecode.com/svn/trunk/2010-04-TramDepot/Snowcode.S3BuildPublisher without the rest of the SnowCode.

Lasest Builds here: http://s3.amazonaws.com/MSBuild-S3-Publisher/MSBuild.AWS.Tasks.Release.zip

Official Downloads here: http://code.google.com/p/snowcode/downloads/list

 

 

Getting CCNET working with Git

Getting started with Git has to be the hardest part but it's well worth it as the result is a great distributed source control system.

If you've not tried it yet go get your self a free GitHub account and start playing, they have some excellent introductory articles to help you get started.  After that you might want to check out TekPubs Git series.

Anyway, once your up and running with Git you'll be wanting to get your CI system using it to do your continuous integration builds from, this is where it gets complicated again.

I'm currently running CCNET 1.4.4 which doesn't have Git support baked in, I beleive 1.5 should have but I've not tried that yet.

I had a post ready to go with details on setting up CCNET 1.4 with Git, but never got around to posting it, so to save me the time here's another great post for setting up CCNET with Git which says almost everything I was going to say.

However... after doing all that you might still find CCNET failing to build for no apparent reason.  If you look into the log file you will probably see that the Git task has timed out.

Whats happening behind the scenes is one of two things.

  •   Git is wanting you to accept the remote site as authentic and wishes to remember the key.
  •   Git is unable to use your private/public key pair to authenticate.

However if you try to connect from the Git Bash command line (on Windows) you will probably be fine.

Here's the hack/fix:

Your public/private key pair and the known hosts file are stored in your user folder (i.e. on WinXP C:\Documents and Settings\<UserName>\.ssh), which when you access Git Bash it uses these files. 

When you run CCNET as either the command line or service it doesn't use those files, instead it uses the files from Gits Program Files folder (i.e. C:\Program Files\Git\.ssh). 

Make sure you can access the remote Git from Bash first as the known_hosts file gets updated.

Backup any files you are about to overwrite in the Program Files\Git\.ssh folder.

Now copy the contents for the Documents and Settings .ssh folder over to the Program Files .ssh folder (remember to back up first!) and your away.

 

Once nice thing I did discover in this process, is that you give out your public key, so you can give it out to numerous hosting sites, such as GitHub and Unfuddle and be able to use just the one key for them all as you sign the requests with your private key.

 

MSBuild Item List Iteration

Repeating a task for each item in a MSBuild item list is really easy if the task supports ; seperated lists, if not, well it's still easy but trying to remember the syntax is another matter!

This keeps catching me out and I've seen few examples of its use around so if for no other reason than as a reminder to myself here's how to iterate a list of items with MSBuild.

Define your item list as something like:

<ItemGroup>
<InstallerFiles Include="$(MSBuildProjectDirectory)\Setup\**\Release\*.msi" />
</ItemGroup>

In this example I have a folder .\Setup\ which contains multiple subfolders of different setup projects which generate msi files.

Method 1:

<Target Name="CopyOutput">
<Message Text="Copying installers to Build Output folder. Source = @(InstallerFiles)"/>
<Copy SourceFiles="@(InstallerFiles)" DestinationFolder="$(BuildOutputFolder)" />
</Target>

This example copies all of the msi files into the BuildOutputFolder.

This method relies on the Copy target to accept a “;” separated list of values. Sadly if you have a task that only takes a single value then your a little stuffed with the @ operator.

Method 2:

<Target Name="CopyOutput">
<Message Text="Copying installers to Build Output folder. Source = %(InstallerFiles.FullPath)"/>
<Copy SourceFiles="%(InstallerFiles.FullPath)" DestinationFolder="$(BuildOutputFolder)" />
</Target>

This method will call the Copy task once for each item in the InstallerFiles list. But that's not really a fair example as you can just use the @ operator for Copy.

Here's another example that's more appropriate.

<PropertyGroup>
<DeploymentLocalPath>$(CCNetWorkingDirectory)\Setup\</DeploymentLocalPath>
<VersionUpdater>VersionVDProj.exe</VersionUpdater>
<VerisonFile>$(CCNetWorkingDirectory)\Version.txt</VerisonFile>
</PropertyGroup>

<ItemGroup>
<SetupProjects Include="$(DeploymentLocalPath)**\*.vdproj" />
</ItemGroup>

<Target Name="Version">
<!-- Some bits to set MSIVersion normally go here and have been removed for clarity →

<Exec Command='$(VersionUpdater) -msi "%(SetupProjects.FullPath)" version=$(MSIVersion)'/>
</Target>

Here I use the VS.NET Deployment Project Version Updater (http://www.codeproject.com/KB/install/VersionVDProj.aspx) to update each of the setup projects with the MSIVersion number, but this could be any colsole application that took a filename.

The .FullPath is a “Wellknown Item Metadata” property. Theirs more listed on the MSDN page

Sara Ford also has an example of a good use of metadata for recursive copying (http://blogs.msdn.com/saraford/archive/2005/11/08/490445.aspx).

Theirs also an interesting MSBuild wiki over on Channel 9.