Steve's blog

A blog about what's going on in Analysis UK...

I Met Jeff Atwood And All I Got Was This Sticker

Coding Horror sticker

What a week last week was, my brain is seriously hurting for two, no wait, three reasons.



So naturally I have to say a big thank you to everyone that presented at both the London and Cambridge conferences. If you didn't go you missed out, huge amounts of quality content from great speakers, all jam packed into one day. Also thanks to Carsonified for organising the events and to Neil Davidson Red Gate's joint CEO for pushing to bring DevDays to Cambridge – fingers crossed he does the same for the Business of Software Conference one day!

Unfortunately by the time the Cambridge DevDays was announced I had already purchased the London ticket, but at only £85 it wasn't such a problem, sadly their was a lot of duplicate content but enough not duplicate to make both days worth the money. I was pleased to see the Android and iPhone talks in London, as well as Jon Skeets talk (turns out he is real and not a robot designed to test the limits of the Stack Overflow scoring system).

The real bonus for the Cambridge event was the post DevDays party, how often can you say “I spent Friday evening in a pub with Jeff Atwood, Joel Spolsky and Ryan Carson”, and yes, free beers from Red Gate! - apparently their a great company to work for (did I mention free beers?). I got to speak very briefly with Jeff as he was trying to escape, now most people swap business cards, Jeff on the other hand gave out stickers – an awesome Coding Horror sticker, a testament to Code Complete and Jeff's Coding Horror blog that I've been following for years. The only question I have now is where is worthy enough to stick it – probably on the front of my Dev box (I tend to keep the case and upgrade the components as it's a great case – so it will stick with me for years – sorry about the pun!).

Apparently this was Jeff's first time in the UK, so Jeff if your reading this I hope you had a good experience and got to see a slightly different side to the Austin Powers version of the UK. It was great that you decided to come over.

If you didn't make this year it sounds like theirs a really good chance they will do the same again next year, and thanks to the careers.stackoverflow.com special offer they announced that was only available during the coarse of the DevDays talks, you get to sign up for $29 for 3 years rather than the normal $99/year, that alone pays for the ticket very quickly (assuming your serious about taking control of your job hunting and are looking to work for companies who love their devs rather than certain others that don't appreciate the value a good developer brings) – it will be really interesting to see how the careers site changes the face of recruiting.

DevDays badge

Write ups of the DevDays events are slowly starting to appear so I won't boor you with my half remembered version of the actual content, for London check out http://www.horsesizepills.com/2009/10/dev-days-london.htm or http://johntopley.com/2009/10/30/stack-overflow-dev-days-london-2009 and for Cambridge http://thom.org.uk/2009/10/31/diary-of-a-schwag-hag/ possibly the only thing missing was that in London Joel entered wearing a union jack cap and hideous London T-Shirt – very tourist, I was hoping he was going to wear a cap and gown for Cambridge, but sadly not. I should mention he changed out of the tourist outfit very quickly after his first talk!

Probably the biggest let down of the two days was my HTC Magic battery not really making it past lunch. Ouch! That's what you get for trying to use Wi-Fi on your phone.

Well I'm off to fry my brain some more with the excellent TekPub NHibernate videos, I really want to watch the GitHub ones as well but I don't think my grey cells can cope with that on top of everything else. Huge thanks again to everyone involved in DevDays and for bringing it to Cambridge.

Typemock Racer Evaluation (Part 2)

The Blue Screen Of Death was traditionally how us developers demonstrated to customers that they had reached parts of our code that we hadn't tested all that well, however that is so 1990's and with the break down of Moores Law we are focusing on multicore processors to give improved performance.

What does this mean for developers, well it means that we will have to start to use more and more multi-threading in our applications to take advantage of this. Writing multi-threaded applications is hard, very hard (to do well!). It's all to easy to introduce race conditions and thread locking, try updating the UI on the wrong thread in a WinForms application and you will be punished.

This is where we introduce the White Screen Of Death or the White Box of Doom. When an application enters a dead lock and is unable to respond and give feedback to the user Windows creates a white ghost like image covering to it. The White Screen Of Death is our chance to show customers that we got it wrong again.

Don't let your customers suffer from a long loch! Loch Long: (Sorry, it's the best I could do!)
Autumn in Loch Long

Not only is it hard to create multi threaded applications it's also exceptionally hard to test them for scenarios where thread locking may occur. I'm sure their will be plenty of opportunities for us to show the user a WSOD or WBOD.

I'm a huge fan of tools, and having the write tool for the job. I recently put a curve on the edge of my kitchen worktop to make a nicer entrance to the kitchen.

There were a few ways to do this badly, I could have used a jigsaw, I could have done it by hand with a coping saw, I could have used my small router which would have gone through bits like crazy and burnt most of the wood. However for me their was only 1 way. A decent powerful big yellow router, some good sharp bits and a MDF template made with my smaller router – again right tool for the job. I'm really pleased with the result even if it did cost me about £300 (twice what all 3 worktops in the kitchen cost!), and I now have a really nice router for my next job.

Curve to worktop

Enter Typemock Racer to the rescue, the right tool for multi-thread deadlock testing of your classes.

In the past I've been involved a lot with connecting up instrumentation and lab automation. As soon as you have hardware you need to communicate with and data to collect as well as the need for user control and real time data display you create a situation crying out for multi-threads with locks, mutexs and correct invocation onto the UI.

When I started work at Akubio(1) on the RAPid 4 I picked up some code that had all sorts of threading going on with complex locking scenarios, combine this with a need to get the product out of the door quickly, real time data display and last minute instrument control specification changes the chances of a thread lock were all too high, at that time their was little we could do in the way of testing other than manually exercising the software for long periods of time trying all sorts of evil things.

Fortunately Typemock have recently released a product to help reduce the need for so much manual testing called Racer. Now developers can write unit and integration tests with test frameworks like NUnit, apply a [ParallelInspection] attribute or use the ThreadTest class and let Racer go and run through various scenarios weaving through the code to try and catch a dead lock. This can be run from within Visual Studio using test runners such as Resharper.

When I found out about the release of Racer I was really interested because of previous issues and being fully aware of how easy it is to get into a dead lock situation and how difficult it is to test for this so I've thrown together a very simple application that illustrates just how easy it is with a misplaced event to get into a thread lock.

I've included 4 ways to test for the thread lock.


  • A NUnit test using the ParallelInspection test.

  • A NUnit test using the ThreadTest class.

  • A Nunit test that creates 2 threads and starts these together and tests for timeout on Joins.

  • A WinForm UI where buttons can be clicked to test.



The application is available for download from GitHub. The current version is really just a shell with some basic functionality with the sole purpose of inducing a thread lock.

The application is intended to demonstrate a situation of having one or more external hardware devices that communicate with the application, sending temperature data in and allowing the PC to control cooling devices (i.e. fans) via the external hardware. A simple example of this would be a Arduino controller with a relay or driver to control a fan, a thermistor on an analog input and then communicating with the PC via the USB serial port.

I have introduced an interface IExternalDevice which allows the communication class for the hardware to be faked.

public interface IExternalDevice

    {

        event EventHandler DataReceived;

        void Open();

        void Close();

        float ReadTemperature();

        void EnableCooling();

        void DisableCooling();

        bool CheckStatus();

    }



The class FakeExternalDevice simulates an external device communication class and provides a way to simulate the data being received in tests.

namespace ThreadLockSimulator.Model

{

    /// <summary>

    /// Represents an external device that reads temperature

    /// and reports this back, leaving the manager to

    /// decide if heater should be enabled or disabled.

    /// </summary>

    public class FakeExternalDevice : IExternalDevice

    {

        public event EventHandler DataReceived;

 

        private object hardwareLockObject = new object();

 

        protected void RaiseDataReceived()

        {

            // Raise an event to pretend data has been received.

            if (DataReceived != null)

            {

                DataReceived(this, EventArgs.Empty);

            }

        }

 

        public void Open()

        {

            ////// Create a new thread to poll the hardware on.

            ////ThreadStart threadStart = new ThreadStart(PollHardware);

            ////Thread thread = new Thread(threadStart);

            ////thread.Start();

        }

 

        public void Close()

        {

            // TODO: Disable ticker.   

        }

 

        public void TriggerDataReceived()

        {

            // Simulate hardware event where data has been received.

            // in an ideal world the event needs to be outside of the lock

            // as this is what will cause our thread lock.

            lock (hardwareLockObject)

            {

                RaiseDataReceived();

            }

        }

 

        public float ReadTemperature()

        {

            Debug.WriteLine("Read Temperature");

 

            lock (hardwareLockObject)

            {

                return 100F;

            }

        }

 

        /// <summary>

        /// Turns the fan on.

        /// </summary>

        public void EnableCooling()

        {

            lock (hardwareLockObject)

            {

                // Write to hardware

                Debug.WriteLine("Enable cooling cooling", "Fake Device");

            }

        }

 

        /// <summary>

        /// Turns the fan off.

        /// </summary>

        public void DisableCooling()

        {

            lock (hardwareLockObject)

            {

                Debug.WriteLine("Disable cooling", "Fake Device");

            }

        }

 

        /// <summary>

        /// Checks the status of the hardware connection

        /// </summary>

        /// <returns>Returns true if the device is connected</returns>

        public bool CheckStatus()

        {

            lock (hardwareLockObject)

            {

                Debug.WriteLine("Check Status", "Fake Device");

                return true;

            }

        }

    }

}



The class DeviceManager is responsible for managing the external devices (1+), holding data received and passing through commands to the hardware communication classes.

namespace ThreadLockSimulator

{

    /// <summary>

    /// Responsible for hooking up the devices and reading the data

    /// </summary>

    public class DeviceManager

    {

        #region Events

 

        public event EventHandler DataReceived;

        public event EventHandler CurrentTemperatureChanged;

 

        #endregion

 

        #region fields

 

        private float currentTemperature;

        private List<IExternalDevice> externalDevices = new List<IExternalDevice>();

        private List<float> temperatures = new List<float>();

        private ITemperatureController temperatureController;

 

        /// <summary>

        /// Syncronisation object.

        /// </summary>

        private object syncObject = new object();

 

        #endregion

 

        #region Constructors

 

        public DeviceManager(IEnumerable<IExternalDevice> devices)

            : this(new SimpleTemperatureController(), devices)

        {

        }

 

        public DeviceManager(ITemperatureController temperatureController, IEnumerable<IExternalDevice> devices)

        {

            this.temperatureController = temperatureController;

            AddExternalDevices(devices);

        }

 

        #endregion

 

        #region Event Raisers

 

        protected void RaiseDataReceivedEvent(object sender, EventArgs empty)

        {

            System.Diagnostics.Debug.WriteLine("RaiseDataReceived, Thread:" + Thread.CurrentThread.GetHashCode(), "DeviceManager");

 

            EventHandler handler = DataReceived;

            if (handler != null)

            {

                handler(this, EventArgs.Empty);

            }

        }

 

        #endregion

 

        #region Public Properties

 

        public List<float> Temperatures

        {

            get

            {

                // Lock whilst getting the temperatures to prevent alteration.

                lock (syncObject)

                {

                    return temperatures;

                }

            }

        }

 

        public float CurrentTemperature

        {

            get

            {

                lock (syncObject)

                {

                    return currentTemperature;

                }

            }

            private set

            {

                lock (syncObject)

                {

                    currentTemperature = value;

                }

            }

        }

 

        #endregion

 

        #region Public Methods

 

        public void Start()

        {

            // Lock to prevent the colleciton being altered whilst itterating.

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.Open();

                }

            }

        }

 

        #endregion

 

        #region Private implementation

 

        void device_DataReceived(object sender, EventArgs e)

        {

            System.Diagnostics.Debug.WriteLine("Data Received", "DeviceManager");

 

            // Lock whilst reading the data to prevent

            // other communications with the hardware

            // and internal data being read whilst it's being updated.

            lock (syncObject)

            {

                IExternalDevice device = (IExternalDevice)sender;

 

                GetTemperature(device);

 

                //CheckTemperature(device, CurrentTemperature);

            }

 

            RaiseDataReceivedEvent(this, EventArgs.Empty);

        }

 

        private void GetTemperature(IExternalDevice device)

        {

            float temperature = device.ReadTemperature();

            CurrentTemperature = temperature;

            temperatures.Add(temperature);

        }

 

        private void CheckTemperature(IExternalDevice device, float temperature)

        {

            if (temperatureController.ShouldEnableCooling(temperature))

            {

                EnableCooling();

            }

            else

            {

                DisableCooling();

            }

        }

 

        private void AddExternalDevices(IEnumerable<IExternalDevice> enumerable)

        {

            // Wire up the data received event and add to the collection of known devices.

            foreach (IExternalDevice externalDevice in enumerable)

            {

                AddExternalDevice(externalDevice);

            }

        }

 

        #endregion

 

        public void AddExternalDevice(IExternalDevice externalDevice)

        {

            lock (syncObject)

            {

                externalDevice.DataReceived += new EventHandler(device_DataReceived);

                externalDevices.Add(externalDevice);

            }

        }

 

        public void EnableCooling()

        {

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.EnableCooling();

                }

            }

        }

 

        public void DisableCooling()

        {

            Debug.WriteLine("Disable Cooling", "Manager");

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.DisableCooling();

                }

            }

        }

 

        public bool CheckDeviceStatus()

        {

            // Lock whilst accessing hardware, check all the external decivices

            // are ok.

            Debug.WriteLine("Check Status", "Manager");

            lock(syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    if (!externalDevice.CheckStatus())

                    {

                        return false;

                    }                  

                }

 

                return true;

            }

        }

    }

}



You will notice there are 2 flows going on, one the flow of data from the external device into the DeviceManager (and then ultimately to the UI), and the second the flow of control going from the DeviceManager class to the external device's.

I've been a little OTT with the thread locking, but not unrealistic of a real world class to protect the data and instrument from multiple threads accessing it all at once, I've not used anything special in the way of mutex's or other locking techniques.

I have intentionally induced a situation that's easy to do accidentally. I've raised an event from within a lock, this is a recipe for disaster and whilst it's really obvious it's happening here, there are all sorts of ways to do this accidentally (imaging a Dirty event being raised on a data object that you populate from within a lock, a side effect of populating it which you might not be expecting).

So what happens:

If you call EnableCooling on the manager you will get a lock from the manager and then a second lock from the external hardware class.

If on a separate thread the hardware has some data to push to the device manager this will lock on the external hardware class and then try to gain a lock in the device manager.

If these two happen at once you can get a dead lock between the two threads. Hello WSOD!

Here's the test for this:

[Test]

        public void TestUsingThreadTest()

        {

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            ThreadTest

                .AddThreadAction(device.TriggerDataReceived)

                .AddThreadAction(manager.EnableCooling)

                .Start();

        }



Here's how Racer highlights this. Clear? Certainly not!.
ThreadLock Visualization

I also tried a test with the ParallelInspection attribute applied to the test:

        [Test]

        [ParallelInspection]

        public void TestSwitchOnCoolingWhilstReceivingData()

        {

            // Create an instance of a faked external hardware device

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            // Force a data received from the device.

            // Locks external device lock then try to lock on syncObject in manager.

            device.TriggerDataReceived();

 

            // This should lock on Manager syncObject, try to lock on the external device lock.

            manager.EnableCooling();

 

            // Either the TriggerDataReceived call should fail to lock on the managers syncObject

            // or EnableCooling should fail to lock on the external Device's hardware lock as the other thread should

            // already have acquired the lock.

        }



Here's the result.

Parallel Inspection Test

That's right, Racer failed to detect the deadlock!

Creating 2 threads and running them at the same time actually worked detecting the deadlock, although it's unreliable and would probably fail on a single core processors (I'm running this on a quad core box).

/// <summary>

        /// This test should fail with thread timeouts due to locking but it's

        /// not reliable.

        /// </summary>

        [Test]

        public void TestManualThreadLockTest()

        {

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            ThreadStart start1 = device.TriggerDataReceived;

            ThreadStart start2 = manager.EnableCooling;

 

            Thread thread1 = new Thread(start1);

            thread1.Name = "Thread 1";

 

            Thread thread2 = new Thread(start2);

            thread2.Name = "Thread 2";

 

            // Start both the threads.

            // Expect thread blocking as external device locks internally then calls manager which locks internally

            // call to manager locks internally then calles external device which tries to lock internally.

            thread2.Start();

            thread1.Start();

 

            if (!thread1.Join(5000))

            {

                Debug.WriteLine("t1 join");

                Assert.Fail("Thread 1 timeout");

            }

 

            if (!thread2.Join(5000))

            {

                Debug.WriteLine("t2 join");

                Assert.Fail("Thread 2 timeout");

            }

        }



What have I learnt from this:


  • Dead lock detection is difficult, really difficult. Even with good tooling it's still possible to not spot it. Just because your test passes doesn't mean you don't have a dead lock problem.

  • You need to understand what your application is doing and where the locking is likely to occur and write tests around that. It's very possible that you will end up with a lot of tests trying out the combinations.

  • I think Racer tests should be in their own project and run as a separate category, probably along side integration tests. They can be slow running numerous scenarios, the license cost is expensive so theirs a good chance that not every developer on the team would have a license, not having Racer enabled when running the tests will show a lot of incorrect failures.

  • Having a separate project is a visual reminder to add Racer tests which helps highlight that multi thread testing is important and should be included.

  • It's worth testing your classes for deadlocks in both a unit and integration test, just because you test your class in isolation for thread locking doesn't mean that it wont play a part in creating a situation where deadlock can occur.



Problems with Racer.

As you probably noticed from my Racer Part 1 post I'm all to happy to point out problems. I found a few more whilst learning how to use Racer.

Racer has 3 menu options. I think they are all broken.
Racer Menu


  • Racer is disabled by default when you start Visual Studio. I really don't understand why this should be. If you have tests that use Racer and you hit your “Run All Tests” button you get failures from your Racer tests because it's disabled, it's annoying.

  • In part 1 I covered the problems with "Configure Typemock Racer"

  • When I installed the real license Typemock were kind enough to send me (because I wrote the Part 1 blog entry about Racer) is tells me “Evaluation license expires in 3 day(s)”. Erm, I thought that was supposed to be a full license. (What happens in 3 days?)
    License expires

  • Visualize Threads doesn't do anything on my installation. Nothing! Roy Osherove's excellent Racer screencast highlights that the visualization is still experimental. BTW – if you are trying to learn Racer watch that screen cast! Also, if you value unit testing follow Roy's blog and read his book! - it's on my list.



The visualization is terrible, on so many grounds.

Here's what it looks like:
ThreadLock Visualization

It's a really small image, theirs no need for that, theirs no indication that the box's are threads, theirs no clue where in the code the locks/dead locks are. I appreciate that Racer is still very Version 1 and this really shows.

When using Resharper the output “link” for the image is plain text, so you need to copy and paste into something that can show images.

Take a look at CodeRush, the visualization in that is fantastic, they even do an Xpress version which is free, even the CodeRush and Refactor! Pro only costs $249, significantly less than Racer.

I would love to see some of the Code Rush visualization effects make their way into Racer. A big arrow for each thread sat at a lock or a bubble floating above the lock with the thread count in it as well as another big arrow pointing the the dead lock highlighting where it's all gone wrong.

I realise that Racer has two responsibilities, one to integrate with the IDE and provide developers feedback and the other to run as an automated NUnit test where the Visual Studio visualization wont help!

You can't run Racer tests in the NUnit IDE – not every body uses Resharper or TestDriven.NET - seriously some people still like to use NUnit IDE!

NUnit Runner with Racer Test

I've not tried to get these tests running as part of a CI build, but clearly theirs a few extra steps that need to be taken to get this running from MSBuild or CCNET rather than your regular NUnit test running – is this all down to Racer being disabled? – if so it's a real letdown.

I've got a few more things to try Racer with and the nice people at Typemock suggested I try Isolator in combination so I'm going to give that a try when time allows. As it happens I want to give Isolator a try on the .NET micro framework stuff - it's not on the supported list which probably means I won't get very far, just like what happened when I tried redgate's Ants performace and memory profilers with the micro framework which was really sad.


(1) Akubio are no longer in business and the RAPid 4 has been taken over by TTP Labtech.








Typemock Racer – Part 1a

I thought it only fair to add a post to stay thank you to Typemock who true to their word have been kind enough to send me a license for Typemock Racer because I blogged about it – even though its somewhat negative.

I wouldn't have bothered posting anything had I though the product was duff, I do think it's going to be one of those must have tools – I already wish it had been around a few years ago as I could have done with it for some evil thread locking I managed to get myself into, going forward with multi-core processors being the norm, more and more of us developers will consider it an essential part of our tool kit, just like NUnit, Resharper, Ants Profiler and many other killer tools.

Typemock Racer Evaluation (Part 1) - FAIL

It's taken me a lot of time to get Typemock Racer up and running to the point I can actually evaluate it and I have to say that if it wasn't for the carrot being dangled in front of me (the chance of a free copy for blogging about Racer) I would have given up long before now, sadly all because of a stupid error and a misleading error message.

Error

Now I know it's not very nice to publicly trash a product but Typemock have asked for us to blog about it ("All reviews are welcome") and having a reference on the net that others can find when they have similar issues is always a help, plus I do like to have a little moan about stuff as you might notice from my @BookSwapSteve Twitter account.

One important thing to remember is this product costs a lot of money €890.00 (US $ 1241 / GBP 765) - seriously big money for a tool – even one that works smoothly and sadly no sign or a personal edition. I would expect a much better experience than I've had so far. If this was $30 shareware Id still be disappointed but for non beta $1200 commercial offering it's terrible. Actually that's a little unfair as I've seen some great stuff from the videos and this looks to be a really promising product that I'm sure took a lot of bright people some serious hard work to pull off.

I also believe it's important to provide a balanced view, because Typemock are giving away an expensive license just because we write a bit of a review about it means the quality of the write ups may not be so in depth, for example, here's one that @Typemock mentioned earlier Review of Typemock Racer which whilst it sure is a review it looks like someone threw together some marketing talk and published it hardly touching the product (heck, even the price is quoted in $ not Euros so it's very wrong! - at the time of writing!) – given we have limited time to get the review done that's understandable (they may well have played with the product as far as I know and they do raise some good points about the Racer).

So here's part 1 of my experience of Typemock Racer:


  • Find download page after discovering the download link was broken on the Typemock website.

  • Download the evaluation (x64 version).

  • Install as per normal (Vista Ultimate box).

  • Install opens a web browser so you can request a evaluation license (21 days) and the help file.

  • Fire up Visual Studio.

  • FAIL!



First error:

Typemock failed to load error

The Addin 'Typemock Racer for VS2008' failed to load or cause an exception.

That's a REALLY bad start for a product, and for a product that's testing focused, erm, you'd hope their had been some testing done on it!

So I clicked No as I wanted Typemock Racer to load.
Then I get another error:

Can not open Typemock.Racer Registry key

It's also interesting to note how inconsistent they are in the use of "TypeMock/Typemock" is it M or m? And for added value theirs even a \n in their to try and get a new line - FAIL!.

So I removed the product, installed again (and repeated that a few times over a number of days).

Eventually deciding that today one way or the other I'm going to get Racer going (because I really do think it's a great idea and is something I would have found really useful in the past) I decided to battle on, I've written myself a little mock application simulating reading a hardware device with opportunities for thread locking and got to the point when I wanted to try out Racer.

The clue, as you would expect is in the error, Typemock Racer couldn't access the registry key.

This registry key:

Typemock Racer registry key that it can not open.

Then I realised, the configuration tool would be writing my license code as well as trying to read it so it would need write access to HKLM (local machine) – hence I needed to be an administrator!

Starting the Racer configuration utility from the Start menu by right clicking and using "Run as administrator" soon got the tool loaded up.

It wasn't long before the next problem. Copy and paste my "Company Name" and License from the email and the Set License button was still disabled. WTF!

Typemock Racer Configuration fail.

It turns out that in the copy and paste their was a space at the end of my email address (as you can see – it's not obvious!) and also the License.

So:


  • The application should request admin access through the application manifest thingy

  • The application needs a better error message or the option to install into the current user registry not local machine – “Can not write”, not “Can not open” – because you can open that registry key for read!

  • We should not be developing application with admin access unless we really have to, clearly the Typemock configuration from Visual Studio expects to be in admin mode.

  • The dialog box should do a trim of the entered values to remove spaces either side of the values.



Now back to my application...

Interestingly I can now enable Typemock Racer from the VS2008 Typemock menu when not an admin – I guess that doesn't write to the HKLM key.

Well my first "simple" test turned out to be a lot more impressive than I thought. All I wanted to do was to ensure that my manager class raised a DataReceived event when a (fake) hardware device it is managing raised it's DataReceived event and the manager had processed the data.

Here's the resultant output:

DeviceManagerTest.TestDataReceivedEventRaisedFromManagerWhenDeviceDataReceived : Passed
Typemock Racer Version 1.0.0.23
------------------------------
Total scenarios run: 1

No Issues found running test in single thread
Started running test on multiple threads
Starting new Test Run
..Total scenarios run: 252

Total scenarios run 252 – WOW, 252! where did they all come from.

Time for me to investigate and play with Racer some more. I'm intending to do a (less negative) part 2 post and make my code available so check back in a week or two (13 days of my trial license left!).

Interestingly running Racer didn't require Visual Studio to be run as an administrator (as you would hope) – BUT the stupid license configuration utility required admin access (and could easily have used HKCU – or given the option).

Times like this reminds me as a developer how important it is to get the licensing experience right for the user! as it's all too easy to do down the wrong path and bite the hand that feeds you.

Success of Failure

As developers it's important for us to realise that a step we would do once early on in development and probably not visit again can have a serious implication for the end user, it's all to easy for us to focus on parts of the code we run over and over again – but that one of step can have big implications.

Home Network

I just finished off some updates to my home network so I thought I'd share a few pics of the main switch rack.

Here it is:
Home Network patch panel

The big thing you will probably notice is that most of the switches are not used and some only very sightly, I do have more switches than I really need at present!

Here's what the box contains, working down from the top.

1) APC PDU – Power switcher.
2) Netgear POE switch
3) 3 COM POE switch
4) Netgear 10/100 Switch
5) Patch panel for the 22 outlets in my flat.
6) Netgear Gigabit Switch
7) HP 10/100 Switch.

How my flats split:

Kitchen:
2 Ports over cupboard on outside wall.
2 Ports over extractor hood/cupboards on inside wall.
4 Ports under worktop.

Living room:
2 Ports by TV.
2 Ports by Rear wall.

Bedroom:
2 Ports by door.

Office (Bedroom 2):
2 Ports by door.
4 Ports by Server/workstation PCs.

Airing cupboard:
2 Ports, yes I really do have 2 network ports in the airing cupboard!

Most of the ports come from the main Netgear Gigabit switch. Currently I have 3 ports connected to the 3COM POE switch.

I only intended to buy 1 POE switch but apparently their like buses. Having waited many months to get a good price on eBay the Netgear POE switch came up as a BIN so I immediately purchased it, then a few days latter I realised that I had one the 3COM POE switch I put a low-ball bid on, net result is 2 POE switches.

This has the advantage that I can have one POE switch powering devices that I want on 24x7 and the other powering devices that I might not want on all the time, so giving me better control over power usage.

If you want to know why I wanted POE switches you'll have to wait for another blog post!

The 10/100 Netgear switch was the original switch I purchased to service the flat before it ran out of ports and I wanted something faster than 100M. I'm leaving it in the rack as it might well stay with the flat when I eventually move out and it's filling a hole I'd have to buy a blanking plate for otherwise.

I was lucky enough to be given the gigabit switch and HP switch, currently the HP switch is also not used.

At the top of the rack is my latest purchase, a UPC PDU (Power Distribution Unit), this means that I can switch off all the unneeded network switches and should I need to I can hard reboot any as well – I needed to do that a few months ago before the UPS was connected when a very brief brownout put the Netgear gigabit switch in a funny state and took down my network.

If you look closely enough you will see the obvious problem – the PDU network comes from the gigabit switch, so if I have a dippy moment and power off that switch I can't switch it on again over the network (it would have been nice to see the APC PDU have 2x network ports!), so I will be sticking my head in the loft if that happens. Should the power fail and the UPS battery run out then the device will switch off, but it's set-up to automatically power on the Netgear switch when power is returned so hopefully I should regain control in that situation.

I've got a few little side projects under way and wanted the APC PDU to play with as it was, but it's turned out to be useful as the POE switches can be noisy when switched on (although I've hacked them and replaced the little 40mm fans for quiet ones which makes a big difference), having 3+ switches all running little 40mm fans can be really noisy when it's in the loft and your trying to sleep!

The whole lot is powered through a little APC UPS, I don't care much for the device, mainly because it screams when the power goes off and that's not amusing when it's in the loft – I've got a network monitor card for a SmartUPS so now all I need is to get a SmartUPS to replace the little one and I'll be happy.

The cabinet it's self is a plastic 19” 7U cabinet I got off eBay, sadly most of the Netgear and 3COM switches were to deep for the original configuration of the cabinet – hence the reason the door is not fitted as it won't close – fortunately their were 2 sets of mounts for the mounting brackets and I was able to bring them forward and the switches just about fit now (about 300mm) otherwise I had them resting on top of the case which was a bit naff.

Here's the switch depths of the 2 POE switches, the maximum I can get in my cabinet is 300mm and that has to include the power lead coming out the back of the switches.
uploads/CAT5 exiting network patch

At the back of the case theirs a bit snake of CAT5 that heads off all over the loft:
CAT5 exiting network patch
This was before I got the PDU, hence the mass of mains leads.

Wiring up the patch panel has been interesting:
CAT5 exiting network patch
You can see that was before I rearranged the switches.

An unhandled exception has occurred. The image is not found.

A while ago I updated my exchange rate sites (Dollars2Pounds et al) to use the "new" ASP.NET Chart control, after some initial testing every thing appeared to be going well and the live websites were updated with the chart control.

Shortly after deploying the servers event log filled up with messages along these lines:

Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 5/19/2009 1:17:59 AM
Event time (UTC): 5/19/2009 12:17:59 AM

Exception information:
Exception type: ArgumentException
Exception message: The image is not found.

Request information:
Request URL: http://www.yuan2dollars.com/ChartImg.axd?i=chart_6_6.png&g=e036ff271be14badb39473c8d04f8609
Request path: /ChartImg.axd

T Stack trace: at System.Web.UI.DataVisualization.Charting.ChartHttpHandler.ProcessSavedChartImage(HttpContext context)
at System.Web.UI.DataVisualization.Charting.ChartHttpHandler.System.Web.IHttpHandler.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)


I've been trying to get to the root cause for some time concerned that visitors to the websites were seeing blank images and I followed various suggestions based on image cache lifetime, temp folder permissions, phase of the moon and so on.

It wasn't until just recently when I followed Scott Hanselman's top tip of implementing elmah that the real cause of the problem became obvious.

It's Google causing all the exceptions! Well googlebot and other search engines.

Their right in front of me under the “ALL_RAW” heading of the server variables was the answer "Connection: Keep-alive Accept: / Accept-Encoding: gzip,deflate From: googlebot(at)googlebot.com"

I should have guessed that!

What happens is googlebot request the page, so the charting control creates the image and renders a image tag for it, caching the image for a predefined period of time. Naturally googlebot does what it does best, it saves that link to the image and tries to re-index it time and time again at a much latter date (weeks), by the time googlebot comes back for the image a second time the cache has long since been updated and naturally the image no longer exists and the naming convention of the image means that it's not going to get the replacement.

Recently I also ran some performance test with the charts in place using LoadImpact, they provide some limited and basic performance testing for free, but its more than enough for simple sites like mine to see that I had serious performance issues on my sites when more than a couple of visitors were hitting the site at the same time.

The charts on each pages were taking about one second to render (it's not surprising as I'm trying to render something like 26,000 data points for the 3 month USD/GBP chart and 7,000 for the other 3 month charts on a 2.4GHz P4). With the CPU fully pegged for a second trying to produce those charts it doesn't take long before the delay hits in if more than a couple of users access the page at the same time.

Three month GBP/USD chart.

My fix for this has been fairly simple, the charts are now updated in the background by a cache expired event and are written to a static .png file with a fixed name. The charts are stored in a “Charts” folder and existing files are replaced with fresh images, the page is rendered with a normal img tag so when googlebot visits it can get the latest chart using the correct name and page rendering now happens in about 6ms.

Interestingly this performance fix has also seen a 10% increase in traffic.


Tips:


  • Watch your servers event logs.


  • Implement ELMAH, it's crazy simple and very useful.


  • Take advantage of the free load testing from LoadImpact, even if you only try it the once.


  • If you are running IIS/Server 2k3/2k8 have Perfmon run in the background as you run the Load Impact tests and look at:

    • CPU usage

    • Error Events Raised

    • Requests Current

    • Requests Queued

    • Request Execution Time

    • and others that might be relevant.


  • In perfmon save the setup somewhere easy to get to (i.e. the desktop) so you can just double click it to get your perfmon setup back quickly


Dinner Timer Lite 1.2.0 Released

I've just uploaded the new version of Dinner Timer Lite (1.2.0) and updated the DinnerTimer.com website to add the new version and update the screen shots.

Some new additions this time are:


  • Exec notifier to start .exe files or open web pages so custom notifications can be run on timer events.

  • The Timer Event selection for notifiers has been simplified with some help text thrown in as well.

  • The timer can now be started with a command line /start hh:mm:ss switch.



As well as many more improvements which are listed on the Download page.

Open Plan Offices – What a HUGE Mistake.

I read earlier today on news.com.au that Open-plan offices are making knowledge workers sick, say Australian scientists - No kidding!

Headphones and missing developer

Problems with open plan offices are hardly a secret. Joel has promoted office spaces for developers for a long time, it's also covered in PeopleWare - although sadly I don't think many managers actually read that book, Jeff Atwood of Coding Horror covered it and I'm sure there are many more instances.

I think the whole office thing is back to front, if you insist on having an open plan office then the managers should be in the open plan parts and the knowledge workers should have the separate quiet offices – managers have to communicate more and know what's happening much more than the workers. Knowledge workers need to be productive and granted communications is really important but not as important as being able to focus on the work.

Now I'm sure most companies would want better value for money out of their knowledge workers and to have a better product on the market. I see a quiet working environment as a major step towards that.

So employers next time you are wondering why your software project cost so much money, why it was so late, why it's so buggy and not as good as the competitors and why there appears to be a skills shortage start by think a bit closer to home and how the business uses it's resources (sorry employees).

Sadly having an office has been seen as a status symbol. More fool the companies that make their knowledge workers work in a noisy office and tuck their managers away in nice quiet offices – You are wasting a fortune in employee productive. With the exception of a few, generally managers are only their to support the workers – without the workers the manager is pointless – the workers are the important bit! (Hint: Think about small companies, they do really well given the lack of managers).

I've had the miss-fortune to work in an open plan office situation for my last 5 proper jobs (I don't count my own business) over the previous 8 years and I'm one of those people who is really sensitive to the noise around me so I can totally agree with the findings.

Noisy environments are stressful, frustrating and prevent you from doing your job to the best of your abilities.

Most of my previous employers have been biotech, pharma or life science instrumentation but my current employer is a large IT focused software company and of all the places I expected to understand about the damage noise does I would expect my current employer to understand(*), sadly this is not the case.

Now the sound of developers working hard is quietness with a few keyboard strokes and the occasional conversation. If like my current position your do code reviews prior to checking in code then this adds to the background noise. If like my current position you have more than 1 person who loves the sound of their own voice and is loud then you are doomed.

Loud person.

Take the bug fixing part of the project when your trying to get the application out of the door (and probably didn't budget much time for bug fixing so it's critical to get it done asap). All the developers are fixing bugs, if you have an office of 20 developers, each fixing 1 bug/day which results in a 10 minute code review. That's 20 * 10 minutes of conversation about code reviews per day. Or put it another way 3 1/3 hours worth of background conversations happing every day.

If those code reviews are about 10 minutes apart then you get a full day of distractions for all of the 20 developer in that one office.

Guess what, it's so easy to introduce side effects and bugs whilst fixing a bug, especially if you are not concentrating, so whilst everybody is trying hard to concentrate to fix the bug and not introduce more they are subject to 50% of the time being a distraction caused by a code review happening in the office (and most likely once the quietness is broken other people will start chatting as well which makes it even noisier). Any then we wonder why so many extra problems get introduced during the bug fix cycle!

Personally I'm a late starter, I like working latter in the day when everybody has gone home and the office is quiet – I generally get more done during the last 1-2 hours than I'm able to do for the bulk of the day because of the background distractions. Many friends criticise me for being late, having the pleasure of missing the rush hour traffic to and from work, but you know what, it lowers my stress levels, I get more done and I'm happier and I guess whilst I'm not stuck in traffic that helps lower my carbon footprint.

Here's some hints to employers looking to get better value for money from developers (I'm sure this or variations of this apply to a lot of developers):


  • I will work for a lower salary at a place that I am able to be in late.

  • I will work for a lower salary at a place that's quiet.

  • I work better latter in the day so you get better value for money if I'm working latter.

  • I'm not a morning person and mornings == stress for me.

  • I get better work done in a quiet office so if you can't offer a quiet office then make sure I can work from home or work late when everybody else has gone.

  • I happy when I'm able to give my best. Background noise causing me to work below par stresses me out, I'm more likely to leave if I'm stressed and unhappy with my work.

  • I'm much more productive and happier given reasonable decent hardware and a few extra dev tools (R# etc).


Here's what doesn't work:


  • Listen to music to drown out the background voices:

    • This doesn't drown out the low pitch part of the voice.

    • It is also uncomfortable to listen to headphones all day long.

    • It discourages communication.

    • If we are listening to internet radio your not going to like the bandwidth usage.

    • To get a good cancelling effect you have the music loud, that annoys others and damages hearing(**).


  • Noise cancelling headphones to drown out the noise in the office.

    • These just don't work for voices.

    • They also suffer from the same problems as music (except the bandwidth usage!)

  • Lots of people in one big office trying to work with more than 1 loud person.

  • Never ever ever combine sales or marketing people in the same office as developers.

  • When employing someone think how loud and chatty they are and what effect that will have on the rest of your team.

  • Please don't reward the loud chatty people by putting them in an office, loud and chatty people are having a negative effect on the rest of your team!

    Loud person distracting worker.

    If you have 2 loud chatty people and they chat for about an hour over the course of a day and this disrupts just 6 other people close by guess what: that's a full day's worth of 1 persons work LOST, you've managed to employ 2 people but only get the benefit of one, and with it irritate the other 6, and guess who's likely to leave first – the chatty ones or the ones trying to work and getting irritated by the others? Guess who feels punished? Not the people chatting!


Not long ago I changed jobs because my previous employer went into administration and at the end of the day I had 3 offers to choose from.

The first of these was offering £5k more than the one I accepted. Here's why I turned it down:


  • It was an 8 am start along with everybody else, no flexibility (despite being a 24 hour operating plant), so if I did have a problem with noise their was no chance of shifting my hours to get a quieter time of day to work in.

  • They also wanted me to work late to communicate with the US – now they said I could occasionally start latter after the initial probation period if I was working a lot with the US and going home late. That just ends up confusing your body as to what's going on.

  • The office didn't look particularly pleasant to work in from the bit I saw (and actually they didn't show me the actual office area during the interview)

  • The coding part of the interview was done in a horrible meeting room and I was given a small screen that wobbled like crazy (I was nearly sea sick) – hardly a good sign.


So how many employers would like to get someone at a discount of £5,000/year? I'm guessing quit a lot. Share holders – how happy would you be that the company you own part of is paying a £5,000 premium? That little office space is going to pay for it's self really quickly and it's highly likely that the cost of flexible working hours is nothing – sounds like a bargain to me.

So improved productive, cheaper staff, less stress and most likely lower employee turn over can all be achieved for free, all that needs to be changed is the mindset, so it's never going to happen is it.

Not listening.

If like me you are a developer and struggling in a noisy office for a company that doesn't get it and your kicking your self for taking a duff job and your now stuck finding something you prefer better because the economy fell apart then check out ChatterBlocker it works much better than music at drowning out background voices, it's not distracting, you can put it up loud if you need to without effecting others and it also works great in the background when listening to podcasts to drown out the noisy people in the office!

(*) although having said that they also think that paying for an employees eye test means paying the legal minimum and saving about £10 on the cost of a normal test, which given us developers spend the day staring at the monitor you'd think they would care about out eyes – they pay for Bupa, provide tea, coffee, fruit but save a few quid on the eye test – go figure! (Guess which bit HR are responsible for!)

(**) I wonder if in a few years time we will see a new set of health and safety litigation where knowledge works sue employers for damaged hearing caused by the constant use of headphones + music used on recommendation by their manager to drown out the background noise of the open plan office.