Over the last weekend I decided that it would be worth trying out signalR for Dollars2Pounds as it appears to be an ideal technology for pushing updated exchange rates down the the client.
Behind the scenes I had my RabbitMQ client watching for a broadcast exchange rate message, when this was received it updates the local cache and then pushed a message out through a signalR client group based on the currency pair to those clients subscribed to the group/pair. Nice and simple! I was even able to re-use the view model I had for the ajax service which turned nicely to a json object on the client.
Sadly this evening I had to role back to my previous implementation. The main issue came as the site was taking the strain of the busy part of the day at a steady 800 connections, when suddenly it turned to 2000 with a 20,000 backlog, I'm not sure exactly what happened, it wasn't a sudden burst of traffic as far as I can tell. The net result of this was the exchange rate sites being unavailable with IIS being overloaded.
Their are a few other issues that mean signalR isn't suited to Dollars2Pounds and the network of sites, particularity when running the sites load balanced across more than one server, subscribing to a group, which is how the client gets only the exchange rate they are interested in, becomes difficult with the current version.
Below are a few graphs from EC2 monitoring, the final two days (6 and 7 of Feb) are with singalR running, the previous days I had a 10s poll to the server implemented.
CPU usage as you can see with the signalR implementation the average load was a bit more.
Network out is about the same, a bit more noisy. It's not so easy to seperate out the sending of a small data packets for the exchange rate when compared to the html, javascript, css and images that are served. Most visitors stay on the site for about 3-4 minutes so their would typically be 18-24 exchange rate packets (100's of bytes/packet) to every page, css files, image (ca. 100k) and chart data (300k) combination (400-800k).
Network In is one area you might expect a big difference going from polling to the pish model, but is difficult to see the change, however given that the exchange rates refresh every 10s so signalR would get the response and then open an new connection at about the same interval the 10s polling worked out it's not a suprise.
Whilst it's possible to look on this experience as a failure of signalR, I think it's more of a compliment that I could spend a Sunday afternoon starting with no knowledge to putting in a server side hub and a little bit of JavaScript and have a push driven exchange rate update live by the evening. I had already put a lot of effort into the polling and caching so for signalR to be comparable out of the box with no tuning says lots.
I've been really impressed with how simple it was to get the solution up and running and it's a great technology which is being actively developed over on GitHub so I'm going to be keeping an eye on it with a view to re-enabling it.
Currently Dollars2Pounds uses regular polling to get the latest rate from the server. The ASP.NET MVC endpoint is cached for a short time so that it can server the rates quickly, the maximum frequency that I update the rate is every 10s for USD to GBP, are refreshed between every 60s or daily dependent on the currency, and now the browser polls for the rate between 10s and 30s intervals again based on the currency and this appears to be working nicely, so in the tradition of if it ain't broke don't fix it, I'm, erm, un-breaking the fix I put in for the thing that wasn't broken in the first place.
So, no real change for Dollars2Pounds but some good fun with signalR along the way.
Some useful SignalR resources:
Scot Hanselman has a typically excellent write up of signalR.
Follow SignalR on Twitter.
SignalR on GitHub.
Great article on SignalR and Knockout in an MVC 3 application