Remember The Bin - Preview

by Stephen Harrison 3. January 2010 16:18

I thought I would share a preview of my x-mas/new year holiday work. 

RememberTheBin is a site I've put together to help me remember which bin I'm supposed to be putting out and when.  For some reason I get confused about putting my bins out for Wednesday collections.  It should be simple, Tuesday evening I put them out, but if I'm not in the routine of returning home from work and seeing everybody elses bin then I forget to do my own. 

Just to help matters Cambridge have recently introduced blue recycling bins, they look very much like the black bins in streetlights so trying to figure out what bin goes out from the others on the street is also a nightmare.

I figured I'm not the only person who will forget or get confused, and missing a bin day can result in a months worth of garbage overflowing the bin which isn't pleasant - so I figured I'd release RememberTheBin to the world.

Now I've got a few bugs to iron out, a few style issues to resolve, some spell checking and typos to sort and the small issue of buying some sms credits(1), but for now here's a brief preview.

 

It's an off the shelf template from TemplateWorld.com, it took a bit of re-work to make it go with ASP.NET MVC, sadly hardly any template providers make their templates for ASP.NET masterpages which is a shame and they generally need a fair bit of work to clean them up.

I've got a twitter feed set up for RememberTheBin so it can send you tweets to remind you to put out the bin, depending on which operator your with you can get those sent as sms to your phone, failing that you can have RememberTheBin txt you directly, however for now we only supply 10 credits, you have to request a top up when they get low - but thats a simple button click on the website.

You might notice I'm signed in using OpenId, RememberTheBin will also support other authentication providers such as FaceBook so you won't need to create a userid/password and I don't have to store your password which is a huge help.

<Geeky Bit>

The underlying application is based on ASP.NET MVC V2 which rocks.  I'm not using too much of the new stuff in V2, but validation through the view model alone has been a huge bonus and the MVC pattern is a must.  Under the covers RememberTheBin is using StructureMap for IoC, NHibernate for data access going against a MySQL database hosted by Amazon RDS, authentication is done through RpxNow, Twitter integration uses TweetSharp, most of these are open source projects so a huge thanks to every one involved in them - it makes building something like RememberTheBin.com so much simpler and quicker and a huge amount of fun.

</Geeky Bit>

Hopefully the site should be live in a few days, please leave feedback if you like/hate it or any ideas you have.  I've got a few feature's yet to be implemented as well.

 

(1) Would you believe, I have to wait for someone to go into the office on Monday before I can buy some sms credits from intellisms.  I found that out on Saturday.  It's like they don't want my money.  I also signed up with BulkSms who did manage to figure out that by sending a code via sms they could verify me, however their SDK for .Net is rubbish so I haven't implemented the code to use them, if intellisms havn't sorted it out on Monday then I'll go with BulkSms, and if all that fails, well I've got a GSM modem on order and 3 sims on their way but that's a lot more work to implement.

Tags: ,

Remember The Bin

Introducing NukeThemPeeps.comm

by Stephen Harrison 22. December 2009 22:40

I've just launched a new website, NukeThemPeeps.com, it's based on microwaving those tasty little marshmallow Peeps(R).

When I first found out about putting a Peeps in a microwave to blow them up the idea appealed to me, it's taken some time for me to actually get this togther, but now I think it's the perfect time, launching with Snowman Peeps having just had some snow here in the UK, together with technology being on my side (You Tube, cheap home video camera, open source blog softare). So finally NukeThemPeeps.com has been launched!

Head on over to NukeThemPeeps.com and see what a Marshmallow Peeps looks like when it's nuked!

 

Peeps is a registered trademark of JustBorn Inc.

Tags:

General

Remember The Bin Graphics

by Stephen Harrison 21. December 2009 00:04

Sometimes my graphics skills amaze me, they are like my spelling, terrible, truly terrible!

So, here for your amusement is the first pass graphic for RememberTheBin.com, a website I'm currently working on, it will be a reminder service for those of us that forget to put the bin out, or can't remember which bin we are supposed to be putting out.

Now if any of you readers know of a way to get dustbin collection details from the council in a automated style (I.e. not scraping the various council websites) and especially how to get the list of changes that occur because of bank holidays please get in touch.

You can fine me on Twitter @BookSwapSteve or the new RememberTheBin.com twitter account @RememberTheBin

Failing that give me a call on 0800 612 4699 (0800 61 Binzz) – which by the way is my new business number.

Tags:

I Have A Dream

by Stephen Harrison 15. December 2009 00:54

Tonight, after a horrible drive I returned home to a cold flat, my central heating had broken. I had no heat and no hot water.

Fortunately I'm a bit of a DIY nut, a few quick checks on the boiler, it had electric, I tried switching it off and on again, still nothing. Turning on the hot tap, the boiler made a noise, life! - but no heat. I checked the pressure gauge, low, very low but still green and no warning lights. I was all out of ideas until I decided to try and put some more pressure in the system. The pressure gauge rose and then the boiler fired up – yay, heat, lovely heat and hot water a bonus!

So how many (be honest now) readers would check the pressure and put more water in the system? I'm guessing very few, and that's fair enough, personally I'd have been pissed if I'd spent the night in the cold and then an emergency visit from the boiler repair person resulted in a 2 minute fix to add some pressure. Wouldn't it be better if you could point your computer at the boiler, have it return an error code which linked to a page on the manufactures website, with suggestions on how to fix the error.

This hasn't been the only time I've returned home to a problem. A few weeks ago I returned home to find the fridge door open. Everything was fine, but I was lucky. Now how many people have returned home to a flood? Or some other situation that you would like to have known about earlier on.

I have a dream, a crazy one, and one that's very unlikely going to happen. I've been thinking about many of these things for a while now, so maybe, just maybe if I tell someone it will come true, failing that we might just take a step closer, so here goes...

Every appliance in the house should have a network connection. There. Simple hey!

Now I know it's added cost, and manufacturing is kept as low as possible, but I would buy the next model up if it had a good network port and wasn't a blatant rip-off for what it was like so often happens.

So what should be networked?

  • Central heating? Yes
  • Microwave? Yes
  • Cooker? Yes
  • Fridge? Yes
  • Freezer? Yes
  • Washing machine? Yes
  • Tuble dryer? Yes
  • Light switch? Yes (I'll explain latter).
  • Air con? Yes.
  • TV? Yes
  • HiFi? Yes
  • Satellite receiver? Yes
  • Wheelie Bin? Well, maybe a stand for it, waste by weight? Sneaky neighbour using your bin?
  • Any more? Yes – everything (except the kettle and kitchen sink)
  • heck, maybe even the kettle and kitchen sink (how much water do you use to wash up?)

 

I'm a big fan of home automation, you've probably figured that out from my dream, but the market sector is a horrible mess and the consumer devices are generally cheap (yet expensive) and naff.

I've got lots of X-10 devices laying around the flat that are no longer user (a very disappointing and expensive hobby in the UK).

Now I've got some Home-Easy devices kicking around, and for the most they work well, but the range is disappointingly limited. A few simple changes and Id be over the moon, but for now, it's still naff.

So what should this network connection onto these devices do?

I'd love to see two things, first and most importantly reporting. Lets have some reasonable sensors wired into the network, then secondly control of the devices as an added extra where possible.

A few examples:

  • Central Heating:
  • Reporting - internal water pressure/flow, water supply pressure/flow, gas pressure/flow, flame temperature, last on, last off, send and return temperatures for radiator system, system age, firmware version and importantly error code.
  • Control – on/off times if settable. Manual override of on/off.
  • The Fridge:
  • Reporting – internal temperature, door status, light bulb status(1), alarms, last on, last off, power usage.
  • Control – operating temperature, alarm hi and lo points.
  • The Washing Machine:
  • Reporting – wash progress, efficiency (load/cost), age.
  • Control – On time, Off time. I have a Home-Easy device on a timer so my washing machine will start automatically in the morning and shut off late at night.

Now a lot of these values are currently measured one way or the other but not reported, and some are not measured, such as power usage. But something like the fridges temperature are so core to the operating of that device it would be hard to find one without it.

How about control, well a lot of device still use mechanical systems, and maybe they should stay that way, although I wonder if it might not be cheaper and easier to provide the control through the network port and some solid state electronics.

Now I'm not suggesting every device implements a full blown web site with Ajax styled web pages and all that, that's a recipe for disaster! (2), instead what I would want to see is a REST based API interface, you point something at the device, it returns a list of capabilities and endpoints that you can connect to and query the devices sensor values.

Why REST, well it's very simple, just hit the endpoint, maybe with a parameter or two and it returns you an lump of xml, fantastic for automation, and if you want to make a nice UI to go with it, no problem – the device manufacturers could then have a skin-able application that the consumer runs on their PC or Mac (even if it's just javascript downloaded from the devices web server) and you've got a nice UI where you can provide much better feedback.

Where does this lead us, well you end up being able to monitor what's happening in your home, make changes based on the results, prevent systems failing, or catch them earlier – wouldn't it have been better for me to find out my fridge was open as I was leaving the house, or that the heating had failed at lunchtime when I could have been able to get it seen to in the afternoon? How about a warning when I go to bed that I've left the oven on or that the front door is unlocked?

Lets talk more about the fridge because theirs two other aspects that are interesting. What happens when the power fails? The fridge and freezer start warming up, and with no power (perhaps a blown fuse), chances are theirs no way for the device to warn you until it's too late.

Well, the network port can provide a low level of power, enough to ensure the monitor and reporting system can be kept alive. Currently corporations everywhere are rolling out Power-Over-Ethernet for use by VOIP phones, how about consumers, well POE devices are coming onto the market, many of us have decent broadband, so how long before we want a nice desk phone connected to the internet giving us free calls?

So lets leverage that, a POE network switch, combined with a small UPS to keep the power going to your fridge, freezer the little magic box that monitors them, and your broadband router, so you can get a SMS message or email when something goes wrong and the devices get to keep power when it's out, and get to tell you theirs a problem.

If were going to implement Power-Over-Ethernet we can even go a step further. This is where the light switches come in. Lets do away with the old light switch, it's got problems, pull out that twin and earth cable feeding it and drop in a Cat 5 cable, attached to a POE switch at one end and the other, to a new lighting switch, that has a motion sensor, a light level sensor, a microphone and possibly speaker, and to top it off it's a touch screen like your iPhone.

Where does that take us? We get much better control over the lighting, the ability to control the lights in another room (left the downstairs light on after going to bed?) The ability to automatically switch off lights in rooms when the daylight is enough, or theirs no activity in the room, and maybe even that star trek intercom system where we can page another room (kitchen to bedroom page for the kids anyone?)

I have a Home-Easy sensor in the kitchen and some small cabinet lights hooked up to a Home-Easy power switch, when I go in the kitchen the lights go on, their usually enough for my needs, when I leave the lights go off after about a minute. With the X-10 set-up I was able to turn any other lights in the kitchen off as well, but I can't do that (3) with the Home-Easy set-up.

Did I mention the idea of a UPS on the POE system to maintain power? Well lets push that a step further as well. All the rage now days are LED lighting, low power bulbs with long lives. What happens if it's dark and the power fails? People fall over, light candles and set fire to the house? How about using LED lighting, either the odd lamp or having a nice arrangement of ceiling lamps like we see with the halogen ones now days. And how about if they were networked as well. When the power fails one or two of these lamps could be driven in low power mode from the POE switch, the back-lights on the light switches can come on to help the occupants find the door, the sensors can figure out of theirs anyone in the room and save emergency power by switching off the lights in that room.

 

How about if theirs a fire? You can get smoke alarms that send out RF signals, you could monitor that and in the event of a fire you could switch on lights to help the occupants find their way out. Maybe the light switches can determine what rooms have people in, or excess heat to help out the fire brigade.

So I've diverged from the fridge haven't I. How many times have we heard about the fridge that figures your out of milk and orders it for you, we see photo's of the fridge with a LCD monitor built in – now that's silly as them things get hot, it adds a lot of expense and will no doubt provide a poor user experience because hardware people just don't get software (4)

You know, it would be great if out food had RFID tags and the fridge and freezer could work out what's in them, oh, and the bin could also (5), but really those devices should just make this information available via the network port so we can get it from where ever we happen to be parked with out laptop, or iPhone, perhaps even in the supermarket, query your fridge to see what's missing whilst in Tesco's anyone?

If theirs ever a time to start pulling all this together it's now, the technologies there, the green movement is on us to save energy, electricity suppliers are going the way of smart meters so we can take advantage of cheaper electricity, or if were not careful, use more expensive electricity, broadband adoption is huge, and people are embracing the web more and more.

Infact some of this is already happing, you can DIY your own home sensor network with ioBridge however that's more for techy geeks like me and I believe it still needs a server, I want to see that stuff build into appliances so all you have to do is connect up a network cable.

Which brings me onto another issue, the network cabling, not everyone's like me and got 8 network outlets in the kitchen Cat 5 cable is really easy to install, it's a lot less dangerous than mains cable and it's fairly cheep, the only difficult bit is making a nice connection at either end, and that's just a case of punching 8 colour coded wires into the 8 colour coded terminals on a patch panel or outlet.

I'm not the first to wire up my pad for networking, and I sure won't be the last, Scott Hanselman has a great series of blog posts Wiring the new house for a Home Network althought I think it should be said both our set-ups are perhaps a little OTT (I could make do with 1 patch panel, 1 switch and 1 UPS, that would remove 5 of the devices in the rack!, but then I'd have nothing to play with).

Scott's set-up is quite large, personally I've got a small 19 inch case in the loft which is descreet and easy to manage, but only does the cat 5 network cables. Now not everybody wants to have network cables so you could argue, that like my Topfield 6000 PVR it should be wireless. If you've ever tried setting up a Wi-Fi network on a device other than a PC you will know it's a hideous job, best bet is to have a simple network connector on the fridge/freezer etc., have it default to DHCP to get it's IP address and then use a Netgear wireless link to do the wireless bit. Putting a little LCD screen on the fridge just because you want Wi-Fi is going to make the costs even worse.

Hopefully in the future part of the first fit of a house will include dropping some network wiring into each room, TVs now come fitted with Ethernet network connectors and hopefully more and more appliances will, so hopefully new build houses will improve on that.

Why do I think this is never going to happen? Well theirs a few reasons:

  • The general aim of manufacturing an appliance is to keep costs as low as possible, throwing in sensors and a network port add to that, and the cost of that extra stuff can easily be multiplied by 4 by the time the various middlemen take their cut.
  • Hardware manufactures generally don't get software, and putting high tech stuff into a low tech fridge is probably something the manufacture is not that confident at doing.
  • Software support , hardware people are generally rubbish at software, which means that if we do get the network port it's likely to be a poorly implemented website rather like the one offered on my toppy.
  • Does anyone other than me and a few random geeks actually want this stuff? Now the basics maybe, and I'm sure a few people wished they could get an alert when the freezer is defrosting so they could save the food, but the overhead of networking up the appliances? Too much maybe. Perhaps we need a simple short range wireless set-up rather than ethernet?

Anyway, there you have it, my crazy idea for a tech filled house.

What am I doing to move along that route and why am I sharing this dream?

Well I've got a little Arm micro controller running .net micro framework and a network adaptor for it, plans include the light switch I talked about, light control using LED lamps and networked temperature monitors. Oh, and somehow I want a device to track what's in the fridge and what gets put in the rubbish bin for a new website I'm working on.

Making a hardware platform to sell is really tough and expensive, so theirs no hope of me doing that, so sharing my ideas hopefully will help those that can and do.

How about you? What do you see in the future house? What tech would make your life better.


  1. Finally I will be able to see if the light stays on when the fridge door is closed!

  2. My Topfield satellite receiver has a web server with nice little web pages in, but it's limited to what you can do with those pages and little automation is possible via that interface.

  3. I have a small Arduino project in progress to fix that issue.

  4. And that's one of the main reasons I say we won't see this, because the hardware manufacturers don't get software, and when they think they do the result is usually poor the only good combination I've seen around is Apple.

  5. One of my wishes for ThreeTomatoes.co.uk is to detect what's going in the bin.

Tags:

General | MicroISV

This blog's moved

by Stephen Harrison 18. November 2009 20:21

Well if your reading this you've probably managed to work that out as it is! With a great deal of pain I've ported my old S9Y blog to BlogEngine.Net, mainly so it would run on a windows Server – yes I know S9Y in php should run on a Win Server and it was, but in a push to move to a new server I just haven't managed to get S9Y working on php 5.2+, to be fair I think it's the php install not S9Y, but still.

So, the rants and random bits which are my blog is now here at blog.analysisuk.com

Some interesting points come out of this.

  1. It's really difficult to transfer from one blog platform to another. Choose wisely!

    1. Now GoDaddy appear to have finally got their S9Y import working having wasted $9 before finding out it wasn't I found a free way (when you've got a domain with them) to try this. I still wasn't happy so didn't do that.

    2. Wordpress have a reasonable import, but that wouldn't so S9Y but I was able to export from GoDaddy's blogcast thing. In the end I still wasn't happy with a hosted wordpress blog.

    3. Eventually I went with BlogEngine.Net which has a tool to import blogs from RSS feeds, as it turns out that doesn't work to well with S9Y either (misses out the entire blog content and also the BlogEngine.net webservice has a bug or two), but as it's all .net I was easily able to hack a few lines of code to get it working – hopefully I'll tidy it up and send the diff to the team as it's on codeplex.

  2. Choose your blog's location carefully.

    1. Now one of the problems I had was that I chose AnalysisUK.com/blog/ originally for the blog location, when I moved to ASP.NET for the AnalysisUK.com site that meant I needed php and ASP.NET (MVC) working smoothly together for url lookup, it all worked but everytime I use a web installer to update the site it would break the php settings in IIS 6.

    2. using the subdomain blog.* appears to be much nicer as it seperates off the blog and you can even point it to a hosted service.

So, the long and the short is, new url, new rss feed and even a new physical location in the cloud for this blog, I've also been shuffling a load of my other sites around and now a lot of them are running on an EC2 instance which has been really interesting to do, that warrants a separate blog post so I'll put one together soon for that.

 

 

Tags:

Blog | General

Ready for Winter?

by Stephen Harrison 12. November 2009 03:18

Tags:

I Met Jeff Atwood And All I Got Was This Sticker

by Stephen Harrison 1. November 2009 18:08

Coding Horror sticker

What a week last week was, my brain is seriously hurting for two, no wait, three reasons.



So naturally I have to say a big thank you to everyone that presented at both the London and Cambridge conferences. If you didn't go you missed out, huge amounts of quality content from great speakers, all jam packed into one day. Also thanks to Carsonified for organising the events and to Neil Davidson Red Gate's joint CEO for pushing to bring DevDays to Cambridge – fingers crossed he does the same for the Business of Software Conference one day!

Unfortunately by the time the Cambridge DevDays was announced I had already purchased the London ticket, but at only £85 it wasn't such a problem, sadly their was a lot of duplicate content but enough not duplicate to make both days worth the money. I was pleased to see the Android and iPhone talks in London, as well as Jon Skeets talk (turns out he is real and not a robot designed to test the limits of the Stack Overflow scoring system).

The real bonus for the Cambridge event was the post DevDays party, how often can you say “I spent Friday evening in a pub with Jeff Atwood, Joel Spolsky and Ryan Carson”, and yes, free beers from Red Gate! - apparently their a great company to work for (did I mention free beers?). I got to speak very briefly with Jeff as he was trying to escape, now most people swap business cards, Jeff on the other hand gave out stickers – an awesome Coding Horror sticker, a testament to Code Complete and Jeff's Coding Horror blog that I've been following for years. The only question I have now is where is worthy enough to stick it – probably on the front of my Dev box (I tend to keep the case and upgrade the components as it's a great case – so it will stick with me for years – sorry about the pun!).

Apparently this was Jeff's first time in the UK, so Jeff if your reading this I hope you had a good experience and got to see a slightly different side to the Austin Powers version of the UK. It was great that you decided to come over.

If you didn't make this year it sounds like theirs a really good chance they will do the same again next year, and thanks to the careers.stackoverflow.com special offer they announced that was only available during the coarse of the DevDays talks, you get to sign up for $29 for 3 years rather than the normal $99/year, that alone pays for the ticket very quickly (assuming your serious about taking control of your job hunting and are looking to work for companies who love their devs rather than certain others that don't appreciate the value a good developer brings) – it will be really interesting to see how the careers site changes the face of recruiting.

DevDays badge

Write ups of the DevDays events are slowly starting to appear so I won't boor you with my half remembered version of the actual content, for London check out http://www.horsesizepills.com/2009/10/dev-days-london.htm or http://johntopley.com/2009/10/30/stack-overflow-dev-days-london-2009 and for Cambridge http://thom.org.uk/2009/10/31/diary-of-a-schwag-hag/ possibly the only thing missing was that in London Joel entered wearing a union jack cap and hideous London T-Shirt – very tourist, I was hoping he was going to wear a cap and gown for Cambridge, but sadly not. I should mention he changed out of the tourist outfit very quickly after his first talk!

Probably the biggest let down of the two days was my HTC Magic battery not really making it past lunch. Ouch! That's what you get for trying to use Wi-Fi on your phone.

Well I'm off to fry my brain some more with the excellent TekPub NHibernate videos, I really want to watch the GitHub ones as well but I don't think my grey cells can cope with that on top of everything else. Huge thanks again to everyone involved in DevDays and for bringing it to Cambridge.

Tags:

General

Typemock Racer Evaluation (Part 2)

by Stephen Harrison 26. July 2009 23:50

The Blue Screen Of Death was traditionally how us developers demonstrated to customers that they had reached parts of our code that we hadn't tested all that well, however that is so 1990's and with the break down of Moores Law we are focusing on multicore processors to give improved performance.

What does this mean for developers, well it means that we will have to start to use more and more multi-threading in our applications to take advantage of this. Writing multi-threaded applications is hard, very hard (to do well!). It's all to easy to introduce race conditions and thread locking, try updating the UI on the wrong thread in a WinForms application and you will be punished.

This is where we introduce the White Screen Of Death or the White Box of Doom. When an application enters a dead lock and is unable to respond and give feedback to the user Windows creates a white ghost like image covering to it. The White Screen Of Death is our chance to show customers that we got it wrong again.

Don't let your customers suffer from a long loch! Loch Long: (Sorry, it's the best I could do!)
Autumn in Loch Long

Not only is it hard to create multi threaded applications it's also exceptionally hard to test them for scenarios where thread locking may occur. I'm sure their will be plenty of opportunities for us to show the user a WSOD or WBOD.

I'm a huge fan of tools, and having the write tool for the job. I recently put a curve on the edge of my kitchen worktop to make a nicer entrance to the kitchen.

There were a few ways to do this badly, I could have used a jigsaw, I could have done it by hand with a coping saw, I could have used my small router which would have gone through bits like crazy and burnt most of the wood. However for me their was only 1 way. A decent powerful big yellow router, some good sharp bits and a MDF template made with my smaller router – again right tool for the job. I'm really pleased with the result even if it did cost me about £300 (twice what all 3 worktops in the kitchen cost!), and I now have a really nice router for my next job.

Curve to worktop

Enter Typemock Racer to the rescue, the right tool for multi-thread deadlock testing of your classes.

In the past I've been involved a lot with connecting up instrumentation and lab automation. As soon as you have hardware you need to communicate with and data to collect as well as the need for user control and real time data display you create a situation crying out for multi-threads with locks, mutexs and correct invocation onto the UI.

When I started work at Akubio(1) on the RAPid 4 I picked up some code that had all sorts of threading going on with complex locking scenarios, combine this with a need to get the product out of the door quickly, real time data display and last minute instrument control specification changes the chances of a thread lock were all too high, at that time their was little we could do in the way of testing other than manually exercising the software for long periods of time trying all sorts of evil things.

Fortunately Typemock have recently released a product to help reduce the need for so much manual testing called Racer. Now developers can write unit and integration tests with test frameworks like NUnit, apply a [ParallelInspection] attribute or use the ThreadTest class and let Racer go and run through various scenarios weaving through the code to try and catch a dead lock. This can be run from within Visual Studio using test runners such as Resharper.

When I found out about the release of Racer I was really interested because of previous issues and being fully aware of how easy it is to get into a dead lock situation and how difficult it is to test for this so I've thrown together a very simple application that illustrates just how easy it is with a misplaced event to get into a thread lock.

I've included 4 ways to test for the thread lock.


  • A NUnit test using the ParallelInspection test.

  • A NUnit test using the ThreadTest class.

  • A Nunit test that creates 2 threads and starts these together and tests for timeout on Joins.

  • A WinForm UI where buttons can be clicked to test.



The application is available for download from GitHub. The current version is really just a shell with some basic functionality with the sole purpose of inducing a thread lock.

The application is intended to demonstrate a situation of having one or more external hardware devices that communicate with the application, sending temperature data in and allowing the PC to control cooling devices (i.e. fans) via the external hardware. A simple example of this would be a Arduino controller with a relay or driver to control a fan, a thermistor on an analog input and then communicating with the PC via the USB serial port.

I have introduced an interface IExternalDevice which allows the communication class for the hardware to be faked.

public interface IExternalDevice

    {

        event EventHandler DataReceived;

        void Open();

        void Close();

        float ReadTemperature();

        void EnableCooling();

        void DisableCooling();

        bool CheckStatus();

    }



The class FakeExternalDevice simulates an external device communication class and provides a way to simulate the data being received in tests.

namespace ThreadLockSimulator.Model

{

    /// <summary>

    /// Represents an external device that reads temperature

    /// and reports this back, leaving the manager to

    /// decide if heater should be enabled or disabled.

    /// </summary>

    public class FakeExternalDevice : IExternalDevice

    {

        public event EventHandler DataReceived;

 

        private object hardwareLockObject = new object();

 

        protected void RaiseDataReceived()

        {

            // Raise an event to pretend data has been received.

            if (DataReceived != null)

            {

                DataReceived(this, EventArgs.Empty);

            }

        }

 

        public void Open()

        {

            ////// Create a new thread to poll the hardware on.

            ////ThreadStart threadStart = new ThreadStart(PollHardware);

            ////Thread thread = new Thread(threadStart);

            ////thread.Start();

        }

 

        public void Close()

        {

            // TODO: Disable ticker.   

        }

 

        public void TriggerDataReceived()

        {

            // Simulate hardware event where data has been received.

            // in an ideal world the event needs to be outside of the lock

            // as this is what will cause our thread lock.

            lock (hardwareLockObject)

            {

                RaiseDataReceived();

            }

        }

 

        public float ReadTemperature()

        {

            Debug.WriteLine("Read Temperature");

 

            lock (hardwareLockObject)

            {

                return 100F;

            }

        }

 

        /// <summary>

        /// Turns the fan on.

        /// </summary>

        public void EnableCooling()

        {

            lock (hardwareLockObject)

            {

                // Write to hardware

                Debug.WriteLine("Enable cooling cooling", "Fake Device");

            }

        }

 

        /// <summary>

        /// Turns the fan off.

        /// </summary>

        public void DisableCooling()

        {

            lock (hardwareLockObject)

            {

                Debug.WriteLine("Disable cooling", "Fake Device");

            }

        }

 

        /// <summary>

        /// Checks the status of the hardware connection

        /// </summary>

        /// <returns>Returns true if the device is connected</returns>

        public bool CheckStatus()

        {

            lock (hardwareLockObject)

            {

                Debug.WriteLine("Check Status", "Fake Device");

                return true;

            }

        }

    }

}



The class DeviceManager is responsible for managing the external devices (1+), holding data received and passing through commands to the hardware communication classes.

namespace ThreadLockSimulator

{

    /// <summary>

    /// Responsible for hooking up the devices and reading the data

    /// </summary>

    public class DeviceManager

    {

        #region Events

 

        public event EventHandler DataReceived;

        public event EventHandler CurrentTemperatureChanged;

 

        #endregion

 

        #region fields

 

        private float currentTemperature;

        private List<IExternalDevice> externalDevices = new List<IExternalDevice>();

        private List<float> temperatures = new List<float>();

        private ITemperatureController temperatureController;

 

        /// <summary>

        /// Syncronisation object.

        /// </summary>

        private object syncObject = new object();

 

        #endregion

 

        #region Constructors

 

        public DeviceManager(IEnumerable<IExternalDevice> devices)

            : this(new SimpleTemperatureController(), devices)

        {

        }

 

        public DeviceManager(ITemperatureController temperatureController, IEnumerable<IExternalDevice> devices)

        {

            this.temperatureController = temperatureController;

            AddExternalDevices(devices);

        }

 

        #endregion

 

        #region Event Raisers

 

        protected void RaiseDataReceivedEvent(object sender, EventArgs empty)

        {

            System.Diagnostics.Debug.WriteLine("RaiseDataReceived, Thread:" + Thread.CurrentThread.GetHashCode(), "DeviceManager");

 

            EventHandler handler = DataReceived;

            if (handler != null)

            {

                handler(this, EventArgs.Empty);

            }

        }

 

        #endregion

 

        #region Public Properties

 

        public List<float> Temperatures

        {

            get

            {

                // Lock whilst getting the temperatures to prevent alteration.

                lock (syncObject)

                {

                    return temperatures;

                }

            }

        }

 

        public float CurrentTemperature

        {

            get

            {

                lock (syncObject)

                {

                    return currentTemperature;

                }

            }

            private set

            {

                lock (syncObject)

                {

                    currentTemperature = value;

                }

            }

        }

 

        #endregion

 

        #region Public Methods

 

        public void Start()

        {

            // Lock to prevent the colleciton being altered whilst itterating.

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.Open();

                }

            }

        }

 

        #endregion

 

        #region Private implementation

 

        void device_DataReceived(object sender, EventArgs e)

        {

            System.Diagnostics.Debug.WriteLine("Data Received", "DeviceManager");

 

            // Lock whilst reading the data to prevent

            // other communications with the hardware

            // and internal data being read whilst it's being updated.

            lock (syncObject)

            {

                IExternalDevice device = (IExternalDevice)sender;

 

                GetTemperature(device);

 

                //CheckTemperature(device, CurrentTemperature);

            }

 

            RaiseDataReceivedEvent(this, EventArgs.Empty);

        }

 

        private void GetTemperature(IExternalDevice device)

        {

            float temperature = device.ReadTemperature();

            CurrentTemperature = temperature;

            temperatures.Add(temperature);

        }

 

        private void CheckTemperature(IExternalDevice device, float temperature)

        {

            if (temperatureController.ShouldEnableCooling(temperature))

            {

                EnableCooling();

            }

            else

            {

                DisableCooling();

            }

        }

 

        private void AddExternalDevices(IEnumerable<IExternalDevice> enumerable)

        {

            // Wire up the data received event and add to the collection of known devices.

            foreach (IExternalDevice externalDevice in enumerable)

            {

                AddExternalDevice(externalDevice);

            }

        }

 

        #endregion

 

        public void AddExternalDevice(IExternalDevice externalDevice)

        {

            lock (syncObject)

            {

                externalDevice.DataReceived += new EventHandler(device_DataReceived);

                externalDevices.Add(externalDevice);

            }

        }

 

        public void EnableCooling()

        {

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.EnableCooling();

                }

            }

        }

 

        public void DisableCooling()

        {

            Debug.WriteLine("Disable Cooling", "Manager");

            lock (syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    externalDevice.DisableCooling();

                }

            }

        }

 

        public bool CheckDeviceStatus()

        {

            // Lock whilst accessing hardware, check all the external decivices

            // are ok.

            Debug.WriteLine("Check Status", "Manager");

            lock(syncObject)

            {

                foreach (IExternalDevice externalDevice in externalDevices)

                {

                    if (!externalDevice.CheckStatus())

                    {

                        return false;

                    }                  

                }

 

                return true;

            }

        }

    }

}



You will notice there are 2 flows going on, one the flow of data from the external device into the DeviceManager (and then ultimately to the UI), and the second the flow of control going from the DeviceManager class to the external device's.

I've been a little OTT with the thread locking, but not unrealistic of a real world class to protect the data and instrument from multiple threads accessing it all at once, I've not used anything special in the way of mutex's or other locking techniques.

I have intentionally induced a situation that's easy to do accidentally. I've raised an event from within a lock, this is a recipe for disaster and whilst it's really obvious it's happening here, there are all sorts of ways to do this accidentally (imaging a Dirty event being raised on a data object that you populate from within a lock, a side effect of populating it which you might not be expecting).

So what happens:

If you call EnableCooling on the manager you will get a lock from the manager and then a second lock from the external hardware class.

If on a separate thread the hardware has some data to push to the device manager this will lock on the external hardware class and then try to gain a lock in the device manager.

If these two happen at once you can get a dead lock between the two threads. Hello WSOD!

Here's the test for this:

[Test]

        public void TestUsingThreadTest()

        {

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            ThreadTest

                .AddThreadAction(device.TriggerDataReceived)

                .AddThreadAction(manager.EnableCooling)

                .Start();

        }



Here's how Racer highlights this. Clear? Certainly not!.
ThreadLock Visualization

I also tried a test with the ParallelInspection attribute applied to the test:

        [Test]

        [ParallelInspection]

        public void TestSwitchOnCoolingWhilstReceivingData()

        {

            // Create an instance of a faked external hardware device

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            // Force a data received from the device.

            // Locks external device lock then try to lock on syncObject in manager.

            device.TriggerDataReceived();

 

            // This should lock on Manager syncObject, try to lock on the external device lock.

            manager.EnableCooling();

 

            // Either the TriggerDataReceived call should fail to lock on the managers syncObject

            // or EnableCooling should fail to lock on the external Device's hardware lock as the other thread should

            // already have acquired the lock.

        }



Here's the result.

Parallel Inspection Test

That's right, Racer failed to detect the deadlock!

Creating 2 threads and running them at the same time actually worked detecting the deadlock, although it's unreliable and would probably fail on a single core processors (I'm running this on a quad core box).

/// <summary>

        /// This test should fail with thread timeouts due to locking but it's

        /// not reliable.

        /// </summary>

        [Test]

        public void TestManualThreadLockTest()

        {

            FakeExternalDevice device = new FakeExternalDevice();

            device.Open();

 

            DeviceManager manager = new DeviceManager(new[] { device });

 

            ThreadStart start1 = device.TriggerDataReceived;

            ThreadStart start2 = manager.EnableCooling;

 

            Thread thread1 = new Thread(start1);

            thread1.Name = "Thread 1";

 

            Thread thread2 = new Thread(start2);

            thread2.Name = "Thread 2";

 

            // Start both the threads.

            // Expect thread blocking as external device locks internally then calls manager which locks internally

            // call to manager locks internally then calles external device which tries to lock internally.

            thread2.Start();

            thread1.Start();

 

            if (!thread1.Join(5000))

            {

                Debug.WriteLine("t1 join");

                Assert.Fail("Thread 1 timeout");

            }

 

            if (!thread2.Join(5000))

            {

                Debug.WriteLine("t2 join");

                Assert.Fail("Thread 2 timeout");

            }

        }



What have I learnt from this:


  • Dead lock detection is difficult, really difficult. Even with good tooling it's still possible to not spot it. Just because your test passes doesn't mean you don't have a dead lock problem.

  • You need to understand what your application is doing and where the locking is likely to occur and write tests around that. It's very possible that you will end up with a lot of tests trying out the combinations.

  • I think Racer tests should be in their own project and run as a separate category, probably along side integration tests. They can be slow running numerous scenarios, the license cost is expensive so theirs a good chance that not every developer on the team would have a license, not having Racer enabled when running the tests will show a lot of incorrect failures.

  • Having a separate project is a visual reminder to add Racer tests which helps highlight that multi thread testing is important and should be included.

  • It's worth testing your classes for deadlocks in both a unit and integration test, just because you test your class in isolation for thread locking doesn't mean that it wont play a part in creating a situation where deadlock can occur.



Problems with Racer.

As you probably noticed from my Racer Part 1 post I'm all to happy to point out problems. I found a few more whilst learning how to use Racer.

Racer has 3 menu options. I think they are all broken.
Racer Menu


  • Racer is disabled by default when you start Visual Studio. I really don't understand why this should be. If you have tests that use Racer and you hit your “Run All Tests” button you get failures from your Racer tests because it's disabled, it's annoying.

  • In part 1 I covered the problems with "Configure Typemock Racer"

  • When I installed the real license Typemock were kind enough to send me (because I wrote the Part 1 blog entry about Racer) is tells me “Evaluation license expires in 3 day(s)”. Erm, I thought that was supposed to be a full license. (What happens in 3 days?)
    License expires

  • Visualize Threads doesn't do anything on my installation. Nothing! Roy Osherove's excellent Racer screencast highlights that the visualization is still experimental. BTW – if you are trying to learn Racer watch that screen cast! Also, if you value unit testing follow Roy's blog and read his book! - it's on my list.



The visualization is terrible, on so many grounds.

Here's what it looks like:
ThreadLock Visualization

It's a really small image, theirs no need for that, theirs no indication that the box's are threads, theirs no clue where in the code the locks/dead locks are. I appreciate that Racer is still very Version 1 and this really shows.

When using Resharper the output “link” for the image is plain text, so you need to copy and paste into something that can show images.

Take a look at CodeRush, the visualization in that is fantastic, they even do an Xpress version which is free, even the CodeRush and Refactor! Pro only costs $249, significantly less than Racer.

I would love to see some of the Code Rush visualization effects make their way into Racer. A big arrow for each thread sat at a lock or a bubble floating above the lock with the thread count in it as well as another big arrow pointing the the dead lock highlighting where it's all gone wrong.

I realise that Racer has two responsibilities, one to integrate with the IDE and provide developers feedback and the other to run as an automated NUnit test where the Visual Studio visualization wont help!

You can't run Racer tests in the NUnit IDE – not every body uses Resharper or TestDriven.NET - seriously some people still like to use NUnit IDE!

NUnit Runner with Racer Test

I've not tried to get these tests running as part of a CI build, but clearly theirs a few extra steps that need to be taken to get this running from MSBuild or CCNET rather than your regular NUnit test running – is this all down to Racer being disabled? – if so it's a real letdown.

I've got a few more things to try Racer with and the nice people at Typemock suggested I try Isolator in combination so I'm going to give that a try when time allows. As it happens I want to give Isolator a try on the .NET micro framework stuff - it's not on the supported list which probably means I won't get very far, just like what happened when I tried redgate's Ants performace and memory profilers with the micro framework which was really sad.


(1) Akubio are no longer in business and the RAPid 4 has been taken over by TTP Labtech.








Tags:

Evaluations

Typemock Racer – Part 1a

by Stephen Harrison 15. July 2009 21:09

I thought it only fair to add a post to stay thank you to Typemock who true to their word have been kind enough to send me a license for Typemock Racer because I blogged about it – even though its somewhat negative.

I wouldn't have bothered posting anything had I though the product was duff, I do think it's going to be one of those must have tools – I already wish it had been around a few years ago as I could have done with it for some evil thread locking I managed to get myself into, going forward with multi-core processors being the norm, more and more of us developers will consider it an essential part of our tool kit, just like NUnit, Resharper, Ants Profiler and many other killer tools.

Tags:

Evaluations

Typemock Racer Evaluation (Part 1) - FAIL

by Stephen Harrison 12. July 2009 20:03

It's taken me a lot of time to get Typemock Racer up and running to the point I can actually evaluate it and I have to say that if it wasn't for the carrot being dangled in front of me (the chance of a free copy for blogging about Racer) I would have given up long before now, sadly all because of a stupid error and a misleading error message.

Error

Now I know it's not very nice to publicly trash a product but Typemock have asked for us to blog about it ("All reviews are welcome") and having a reference on the net that others can find when they have similar issues is always a help, plus I do like to have a little moan about stuff as you might notice from my @BookSwapSteve Twitter account.

One important thing to remember is this product costs a lot of money €890.00 (US $ 1241 / GBP 765) - seriously big money for a tool – even one that works smoothly and sadly no sign or a personal edition. I would expect a much better experience than I've had so far. If this was $30 shareware Id still be disappointed but for non beta $1200 commercial offering it's terrible. Actually that's a little unfair as I've seen some great stuff from the videos and this looks to be a really promising product that I'm sure took a lot of bright people some serious hard work to pull off.

I also believe it's important to provide a balanced view, because Typemock are giving away an expensive license just because we write a bit of a review about it means the quality of the write ups may not be so in depth, for example, here's one that @Typemock mentioned earlier Review of Typemock Racer which whilst it sure is a review it looks like someone threw together some marketing talk and published it hardly touching the product (heck, even the price is quoted in $ not Euros so it's very wrong! - at the time of writing!) – given we have limited time to get the review done that's understandable (they may well have played with the product as far as I know and they do raise some good points about the Racer).

So here's part 1 of my experience of Typemock Racer:


  • Find download page after discovering the download link was broken on the Typemock website.

  • Download the evaluation (x64 version).

  • Install as per normal (Vista Ultimate box).

  • Install opens a web browser so you can request a evaluation license (21 days) and the help file.

  • Fire up Visual Studio.

  • FAIL!



First error:

Typemock failed to load error

The Addin 'Typemock Racer for VS2008' failed to load or cause an exception.

That's a REALLY bad start for a product, and for a product that's testing focused, erm, you'd hope their had been some testing done on it!

So I clicked No as I wanted Typemock Racer to load.
Then I get another error:

Can not open Typemock.Racer Registry key

It's also interesting to note how inconsistent they are in the use of "TypeMock/Typemock" is it M or m? And for added value theirs even a \n in their to try and get a new line - FAIL!.

So I removed the product, installed again (and repeated that a few times over a number of days).

Eventually deciding that today one way or the other I'm going to get Racer going (because I really do think it's a great idea and is something I would have found really useful in the past) I decided to battle on, I've written myself a little mock application simulating reading a hardware device with opportunities for thread locking and got to the point when I wanted to try out Racer.

The clue, as you would expect is in the error, Typemock Racer couldn't access the registry key.

This registry key:

Typemock Racer registry key that it can not open.

Then I realised, the configuration tool would be writing my license code as well as trying to read it so it would need write access to HKLM (local machine) – hence I needed to be an administrator!

Starting the Racer configuration utility from the Start menu by right clicking and using "Run as administrator" soon got the tool loaded up.

It wasn't long before the next problem. Copy and paste my "Company Name" and License from the email and the Set License button was still disabled. WTF!

Typemock Racer Configuration fail.

It turns out that in the copy and paste their was a space at the end of my email address (as you can see – it's not obvious!) and also the License.

So:


  • The application should request admin access through the application manifest thingy

  • The application needs a better error message or the option to install into the current user registry not local machine – “Can not write”, not “Can not open” – because you can open that registry key for read!

  • We should not be developing application with admin access unless we really have to, clearly the Typemock configuration from Visual Studio expects to be in admin mode.

  • The dialog box should do a trim of the entered values to remove spaces either side of the values.



Now back to my application...

Interestingly I can now enable Typemock Racer from the VS2008 Typemock menu when not an admin – I guess that doesn't write to the HKLM key.

Well my first "simple" test turned out to be a lot more impressive than I thought. All I wanted to do was to ensure that my manager class raised a DataReceived event when a (fake) hardware device it is managing raised it's DataReceived event and the manager had processed the data.

Here's the resultant output:

DeviceManagerTest.TestDataReceivedEventRaisedFromManagerWhenDeviceDataReceived : Passed
Typemock Racer Version 1.0.0.23
------------------------------
Total scenarios run: 1

No Issues found running test in single thread
Started running test on multiple threads
Starting new Test Run
..Total scenarios run: 252

Total scenarios run 252 – WOW, 252! where did they all come from.

Time for me to investigate and play with Racer some more. I'm intending to do a (less negative) part 2 post and make my code available so check back in a week or two (13 days of my trial license left!).

Interestingly running Racer didn't require Visual Studio to be run as an administrator (as you would hope) – BUT the stupid license configuration utility required admin access (and could easily have used HKCU – or given the option).

Times like this reminds me as a developer how important it is to get the licensing experience right for the user! as it's all too easy to do down the wrong path and bite the hand that feeds you.

Success of Failure

As developers it's important for us to realise that a step we would do once early on in development and probably not visit again can have a serious implication for the end user, it's all to easy for us to focus on parts of the code we run over and over again – but that one of step can have big implications.

Tags:

Evaluations