It's taken me a lot of time to get Typemock Racer up and running to the point I can actually evaluate it and I have to say that if it wasn't for the carrot being dangled in front of me (the chance of a free copy for blogging about Racer) I would have given up long before now, sadly all because of a stupid error and a misleading error message.
Now I know it's not very nice to publicly trash a product but Typemock have asked for us to blog about it ("All reviews are welcome") and having a reference on the net that others can find when they have similar issues is always a help, plus I do like to have a little moan about stuff as you might notice from my @BookSwapSteve Twitter account.
One important thing to remember is this product costs a lot of money 890.00 (US $ 1241 / GBP 765) - seriously big money for a tool even one that works smoothly and sadly no sign or a personal edition. I would expect a much better experience than I've had so far. If this was $30 shareware Id still be disappointed but for non beta $1200 commercial offering it's terrible. Actually that's a little unfair as I've seen some great stuff from the videos and this looks to be a really promising product that I'm sure took a lot of bright people some serious hard work to pull off.
I also believe it's important to provide a balanced view, because Typemock are giving away an expensive license just because we write a bit of a review about it means the quality of the write ups may not be so in depth, for example, here's one that @Typemock mentioned earlier Review of Typemock Racer which whilst it sure is a review it looks like someone threw together some marketing talk and published it hardly touching the product (heck, even the price is quoted in $ not Euros so it's very wrong! - at the time of writing!) given we have limited time to get the review done that's understandable (they may well have played with the product as far as I know and they do raise some good points about the Racer).
So here's part 1 of my experience of Typemock Racer:
- Find download page after discovering the download link was broken on the Typemock website.
- Download the evaluation (x64 version).
- Install as per normal (Vista Ultimate box).
- Install opens a web browser so you can request a evaluation license (21 days) and the help file.
- Fire up Visual Studio.
- FAIL!
First error:
The Addin 'Typemock Racer for VS2008' failed to load or cause an exception.
That's a REALLY bad start for a product, and for a product that's testing focused, erm, you'd hope their had been some testing done on it!
So I clicked No as I wanted Typemock Racer to load.
Then I get another error:
It's also interesting to note how inconsistent they are in the use of "TypeMock/Typemock" is it M or m? And for added value theirs even a \n in their to try and get a new line - FAIL!.
So I removed the product, installed again (and repeated that a few times over a number of days).
Eventually deciding that today one way or the other I'm going to get Racer going (because I really do think it's a great idea and is something I would have found really useful in the past) I decided to battle on, I've written myself a little mock application simulating reading a hardware device with opportunities for thread locking and got to the point when I wanted to try out Racer.
The clue, as you would expect is in the error, Typemock Racer couldn't access the registry key.
This registry key:
Then I realised, the configuration tool would be writing my license code as well as trying to read it so it would need write access to HKLM (local machine) hence I needed to be an administrator!
Starting the Racer configuration utility from the Start menu by right clicking and using "Run as administrator" soon got the tool loaded up.
It wasn't long before the next problem. Copy and paste my "Company Name" and License from the email and the Set License button was still disabled. WTF!
It turns out that in the copy and paste their was a space at the end of my email address (as you can see it's not obvious!) and also the License.
So:
- The application should request admin access through the application manifest thingy
- The application needs a better error message or the option to install into the current user registry not local machine Can not write, not Can not open because you can open that registry key for read!
- We should not be developing application with admin access unless we really have to, clearly the Typemock configuration from Visual Studio expects to be in admin mode.
- The dialog box should do a trim of the entered values to remove spaces either side of the values.
Now back to my application...
Interestingly I can now enable Typemock Racer from the VS2008 Typemock menu when not an admin I guess that doesn't write to the HKLM key.
Well my first "simple" test turned out to be a lot more impressive than I thought. All I wanted to do was to ensure that my manager class raised a DataReceived event when a (fake) hardware device it is managing raised it's DataReceived event and the manager had processed the data.
Here's the resultant output:
DeviceManagerTest.TestDataReceivedEventRaisedFromManagerWhenDeviceDataReceived : Passed
Typemock Racer Version 1.0.0.23
------------------------------
Total scenarios run: 1
No Issues found running test in single thread
Started running test on multiple threads
Starting new Test Run
..Total scenarios run: 252
Total scenarios run 252 WOW, 252! where did they all come from.
Time for me to investigate and play with Racer some more. I'm intending to do a (less negative) part 2 post and make my code available so check back in a week or two (13 days of my trial license left!).
Interestingly running Racer didn't require Visual Studio to be run as an administrator (as you would hope) BUT the stupid license configuration utility required admin access (and could easily have used HKCU or given the option).
Times like this reminds me as a developer how important it is to get the licensing experience right for the user! as it's all too easy to do down the wrong path and bite the hand that feeds you.
As developers it's important for us to realise that a step we would do once early on in development and probably not visit again can have a serious implication for the end user, it's all to easy for us to focus on parts of the code we run over and over again but that one of step can have big implications.