Steve's blog

A blog about what's going on in Analysis UK...

Hello and Goodbye signalR it was fun!

Over the last weekend I decided that it would be worth trying out signalR for Dollars2Pounds as it appears to be an ideal technology for pushing updated exchange rates down the the client.

Behind the scenes I had my RabbitMQ client watching for a broadcast exchange rate message, when this was received it updates the local cache and then pushed a message out through a signalR client group based on the currency pair to those clients subscribed to the group/pair. Nice and simple! I was even able to re-use the view model I had for the ajax service which turned nicely to a json object on the client.

Sadly this evening I had to role back to my previous implementation. The main issue came as the site was taking the strain of the busy part of the day at a steady 800 connections, when suddenly it turned to 2000 with a 20,000 backlog, I'm not sure exactly what happened, it wasn't a sudden burst of traffic as far as I can tell. The net result of this was the exchange rate sites being unavailable with IIS being overloaded.

Their are a few other issues that mean signalR isn't suited to Dollars2Pounds and the network of sites, particularity when running the sites load balanced across more than one server, subscribing to a group, which is how the client gets only the exchange rate they are interested in, becomes difficult with the current version.

Below are a few graphs from EC2 monitoring, the final two days (6 and 7 of Feb) are with singalR running, the previous days I had a 10s poll to the server implemented.

CPU usage as you can see with the signalR implementation the average load was a bit more.

Network out is about the same, a bit more noisy. It's not so easy to seperate out the sending of a small data packets for the exchange rate when compared to the html, javascript, css and images that are served. Most visitors stay on the site for about 3-4 minutes so their would typically be 18-24 exchange rate packets (100's of bytes/packet) to every page, css files, image (ca. 100k) and chart data (300k) combination (400-800k).

Network In is one area you might expect a big difference going from polling to the pish model, but is difficult to see the change, however given that the exchange rates refresh every 10s so signalR would get the response and then open an new connection at about the same interval the 10s polling worked out it's not a suprise.

Whilst it's possible to look on this experience as a failure of signalR, I think it's more of a compliment that I could spend a Sunday afternoon starting with no knowledge to putting in a server side hub and a little bit of JavaScript and have a push driven exchange rate update live by the evening. I had already put a lot of effort into the polling and caching so for signalR to be comparable out of the box with no tuning says lots.

I've been really impressed with how simple it was to get the solution up and running and it's a great technology which is being actively developed over on GitHub so I'm going to be keeping an eye on it with a view to re-enabling it.

Currently Dollars2Pounds uses regular polling to get the latest rate from the server. The ASP.NET MVC endpoint is cached for a short time so that it can server the rates quickly, the maximum frequency that I update the rate is every 10s for USD to GBP, are refreshed between every 60s or daily dependent on the currency, and now the browser polls for the rate between 10s and 30s intervals again based on the currency and this appears to be working nicely, so in the tradition of if it ain't broke don't fix it, I'm, erm, un-breaking the fix I put in for the thing that wasn't broken in the first place.

So, no real change for Dollars2Pounds but some good fun with signalR along the way.

Some useful SignalR resources:

Scot Hanselman has a typically excellent write up of signalR.

Follow SignalR on Twitter.

SignalR on GitHub.

Great article on SignalR and Knockout in an MVC 3 application

 

 

Happy Birthday Dollars2Pounds.com

Wow, apparently it's 10 years ago today that I first registered Dollars2Pounds.com, I'm not sure how long after that it took me to get a proper website deployed, probably a month or two.

Since then traffic has grown nicely, I've had some great feedback via email and the UserVoice community and I've expanded the general theme to a variety of popular currencies such as Dollars2Euro.com and more recently Dollars2Dollars.com to cope with conversion between Canadian Dollars/Australian Dollars/US Dollars and so on.

Initially I was able to get daily updates from the federal reserve but recently they stopped that so the data is now from XIgnite in either 15 minute or 60 minute up dates and I'm looking to improve on that as well.

The server has gone from being a classic asp IIS server back in 2000, then to a LAMP setup for many years, and now back to Windows IIS using ASP.NET, and with a recent transition from a rented server into Amazon's AWS EC2 cloud.

It's been an amazing experience, theirs so many things you just don't appreciate as a developer, the best has to be feedback from customers, so I'd like to say a huge thank you to everybody who uses Dollars2Pounds.com or one of my currency sites (and even my other non-currency based websites!). 

If you are a developer and don't have your own website then your missing out.  Go do something simple, look at Dollars2Pounds.com, how much more simple could it be!, get the experience of trying to host a site, keeping a database and web servers running at 99% uptime, backing up the website, dealing with customer requests, trying to generate revenue, generating analytics from log files and Google Analytics, try buying adwords, and most importantly something us developers always put down, try your hand at sales and marketing your website - I promise you it will be a shock, chances are if you don't get sales and marketing right your website will be nothing.

It's also been interesting to see the UKs take on the Euro, I was concerned early on that I had wasted my time with Dollars2Pounds.com and that the pound would be no more, but it's still going strong.

I've got some more updates in the pipeline for the website, as well as some new websites almost ready to do, so 2010 should be another interesting year.

To celebrate I'm off to the 14th Cambridge Winter Ale Festival!

Introducing Dollars2Dollars.com

OK, I put my hand up, I should have done this site a LONG time ago, anyway Dollars2Dollars.com is now live, at this time it's fixed to US Dollars against one other currency from Canadian Dollars (CAD), Australian Dollar (AUD), New Zealand Dollars (NZD) or Hong Kong Dollar (HKD).

 

It's the same usual easy format, a nice simple calculator and some history charts. 

Exchange rates are updated hourly at present.

I'm going to work on the UI to support more conversion options (i.e. Australian Dollars to New Zealand Dollars).

 

 

 

An unhandled exception has occurred. The image is not found.

A while ago I updated my exchange rate sites (Dollars2Pounds et al) to use the "new" ASP.NET Chart control, after some initial testing every thing appeared to be going well and the live websites were updated with the chart control.

Shortly after deploying the servers event log filled up with messages along these lines:

Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 5/19/2009 1:17:59 AM
Event time (UTC): 5/19/2009 12:17:59 AM

Exception information:
Exception type: ArgumentException
Exception message: The image is not found.

Request information:
Request URL: http://www.yuan2dollars.com/ChartImg.axd?i=chart_6_6.png&g=e036ff271be14badb39473c8d04f8609
Request path: /ChartImg.axd

T Stack trace: at System.Web.UI.DataVisualization.Charting.ChartHttpHandler.ProcessSavedChartImage(HttpContext context)
at System.Web.UI.DataVisualization.Charting.ChartHttpHandler.System.Web.IHttpHandler.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)


I've been trying to get to the root cause for some time concerned that visitors to the websites were seeing blank images and I followed various suggestions based on image cache lifetime, temp folder permissions, phase of the moon and so on.

It wasn't until just recently when I followed Scott Hanselman's top tip of implementing elmah that the real cause of the problem became obvious.

It's Google causing all the exceptions! Well googlebot and other search engines.

Their right in front of me under the “ALL_RAW” heading of the server variables was the answer "Connection: Keep-alive Accept: / Accept-Encoding: gzip,deflate From: googlebot(at)googlebot.com"

I should have guessed that!

What happens is googlebot request the page, so the charting control creates the image and renders a image tag for it, caching the image for a predefined period of time. Naturally googlebot does what it does best, it saves that link to the image and tries to re-index it time and time again at a much latter date (weeks), by the time googlebot comes back for the image a second time the cache has long since been updated and naturally the image no longer exists and the naming convention of the image means that it's not going to get the replacement.

Recently I also ran some performance test with the charts in place using LoadImpact, they provide some limited and basic performance testing for free, but its more than enough for simple sites like mine to see that I had serious performance issues on my sites when more than a couple of visitors were hitting the site at the same time.

The charts on each pages were taking about one second to render (it's not surprising as I'm trying to render something like 26,000 data points for the 3 month USD/GBP chart and 7,000 for the other 3 month charts on a 2.4GHz P4). With the CPU fully pegged for a second trying to produce those charts it doesn't take long before the delay hits in if more than a couple of users access the page at the same time.

Three month GBP/USD chart.

My fix for this has been fairly simple, the charts are now updated in the background by a cache expired event and are written to a static .png file with a fixed name. The charts are stored in a “Charts” folder and existing files are replaced with fresh images, the page is rendered with a normal img tag so when googlebot visits it can get the latest chart using the correct name and page rendering now happens in about 6ms.

Interestingly this performance fix has also seen a 10% increase in traffic.


Tips:


  • Watch your servers event logs.


  • Implement ELMAH, it's crazy simple and very useful.


  • Take advantage of the free load testing from LoadImpact, even if you only try it the once.


  • If you are running IIS/Server 2k3/2k8 have Perfmon run in the background as you run the Load Impact tests and look at:

    • CPU usage

    • Error Events Raised

    • Requests Current

    • Requests Queued

    • Request Execution Time

    • and others that might be relevant.


  • In perfmon save the setup somewhere easy to get to (i.e. the desktop) so you can just double click it to get your perfmon setup back quickly


DNS Problems

Apologies for the recent outage of Dollars2Pounds and the other exchange rate sites. You may not have noticed if you or your ISP had the DNS values in cache as the site was actually up and running.

Just a couple of days ago I updated Dollars2Pounds to use the DNS servers of the host I have the server with, thinking that it would reduce the load on my servers and that they would be less likely to go down than my servers.

How very wrong I was, earlier this evening I got an email from the site monitoring service I use (site up time) telling me that Dollars2Pounds was unreachable.

A quick check and the site was up and running but the DNS servers were not returning anything and the host providers main web site was down as well.

Hopefully this is a rare occurrence, however since EV1Servers got taken over things have gone badly, not least the 3 day outage recently when a transformer blew up.

Lets hope things improve as it's really painful to change hosts and you don't really know what will happen at a new hosts.

I've also managed to accidentally knock the odd site off line over the past week whilst I've been setting up the accounts on the new server.

The good news is that all my exchange rates sites (Dollars2Pounds, Pounds2Euro etc.) are now on a more powerful server and using a more up to date exchange rate source which is updated every 15 or 60 mins. dependent on the currency pair.

Pound breaks the 2 Dollar point.

Today the Pound has broken the 2 Dollar exchange rate, this is apparently a 15 year high for the Pound against the US Dollar.

Don’t forget if you want to check out the exchange rate on your mobile phone or other mobile devices you can browse to D2P.mobi, Dollars 2 Pounds for your mobile.

Screen shot from Dollars2Pounds showing todays rate above the 2 dollar point:

Pound touches 14 year high against US Dollar

The BBC is reporting that the Pound reached a 14 year high against the US Dollar during Monday.

I don’t have 14 years worth of data in the Dollars2Pounds exchange rate database, but here’s a chart of what the last 5 years looks like.

(Apologies for the Y axis label problem, Excel has decided not to allow me to show the full label – I have no idea why, I’ve tried allsorts).



I'm currently working on adding charting to Dollars2Pounds.com, however this is in a big queue with lots of other things I need to be working on so isn’t happening as quickly as I would like – it’s also reliant on me finding a charting package that I like, and I’ve not had much luck so far.

If you want to play with a chart in Excel your self you can use Dollars 2 Pounds exchange rate history page and Excel’s Web Query.

From Excel select :
Data -> Import External Data -> New Web Query

In the Window that opens enter the Rates History URL.
(http://www.dollars2pounds.com/RatesHistory.php?Base=USD&To=GBP&Days=60)



Select the arrow pointing to the date title of the rates table, it should turn to a tick



Click on Import, select where to import the data to in the Import Data window and wait whilst Excel gets the data.

Your spreadsheet should now look something like:




Now, if like me your in the UK, it’s a great time to buy software, components, domain names and the like from US sources as everything’s practically half price!.

Please remember the actual exchange rates you receive from your credit card, money transfer or other currency conversion will differ from the rates published, please check the rate offered by the company converting your currency first and remember that conversions are an approximate indication only and cannot be guaranteed by Analysis UK.