Steve's blog

A blog about what's going on in Analysis UK...

Why you should do other stuff...

As a software developer it's all too easy to turn up to work, do what's required, go home in evening and once a month pick up your pay. Just where does that get you?

Well maybe your happy doing that, then 5-10 years down the line you find you've been doing the same thing, the same way, never really pushing the boundaries or questioning if what you are doing is the best way to do it. Net result - you've failed your self and your employer, and if you end up needing a new job, well we all know how picky the tech industry is about having a perfect match of acronyms on your CV.

 

 

Ever since I had my first computer I've always had a personal project, be it software or hardware. These projects are for me to solve problems I have, every now and then one of them makes it out the door and I share it with the rest of the internet.

Ten years ago I was a Lab Rat, working as a scientist testing blood gas analyzers and pH meters. In my spair time I had I started buying bits from the US (I'm in the UK) and trying to figure out the cost in pounds wasn't that easy (open calculator, figure out if it's divide or multiply, press buttons, hit equals). So I started Dollars2Pounds.com, mainly for myself, and the 10 other people who initially visited the site every month. That's now grown a lot, but is still true to it's roots, an easy way to calculate the cost of something in your local currency.

If I hadn't started Dollars2Pounds.com I wouldn't have touched any web programming. In 2002 I went a step further and built a beast of a site in php, now that's taught me one thing, I'm not a fan of lots of php, it's great to get started with but once things get busy, ohh boy it's not so great.

The thing is, during the time I was building those sites my day job involved connecting up scientific instruments to computers, one way or another. I'd never have touched web based programming if I only did my day job. And you know what, that would have been a huge mistake. I've learnt so much from escaping the binds of the day to day work and trying new things.

I can't emphasise enough as a developer how eye opening it is to release a product or service your self. It's all to easy to sit in our cubical wearing our Dilbert curled up ties and laugh about marketing and sales, but until you've actually tried it and seen the results you will never understand how frustrating it can be, or how amazing the feeling is to see people using YOUR product and how strange the things that your users do truly are, and the actual challenges your product faces on the internet, not just the perceived problems where you think you need to shave a microsecond off of some routine that it turns out people don't use anyway.

Fast forward 8 years, I've recently transitioned Dollars2Pounds.com and the network of other exchange rate sites from a dedicated server host onto Amazon's AWS/EC2(1).

Over the years my websites have grown, from the original Dollars2Pounds.com, shortly after followed Pounds2Euro.com, Dollars2Euro.com, Dollars2Yen.com, as well as Yuan, Won and Rupees, these combined result in a network of around 23 websites.

When these sites were in php I was maintaining a core set of functionality and then some individual domain specific pages, over the years I've put a bit of work in here and there (for the most the sites just run themselves), and moved over to ASP.NET (thanks to listening to DotNetRocks), I've consolidated the code base into one single codebase with just a seperate configuration file, database and installer for each site.

That's been the story for about a year now, my builds were taking >25 minutes to build just the installers (3 mins for the main app), the web msi installer that comes as part of VS2008 has problems, I've found installing upgrades on the site would often not update files, so upgrading the websites involved running un-install on 23 msi's, installing 23 new msi's and then saving them away to be able to automate the un-install next time around.

I also have a CCNET project that goes and runs some basic tests on each of the 23 websites to ensure that the correct configuration got installed, the site is performing ok and is not down.

 

So here's the problem:

I was kind of happy with that, it wasn't ideal, I didn't like the 30ish minutes between me checking in a change and being able to start a roll out, not to mention the trouble in updating 23 different websites, but it worked.

More and more I see company specific sub-domains (i.e. analysisuk.fogbugz.com for my bug tracking) on websites and I'm a big fan of that style, it makes me feel special, unique and I know from Spolsk's wramblings that I have my own little database for my bugs so a bug in the fogbugz code base isn't likely to show another company or competitor my bugs.

I had always wondered how sub-domaining website was done. Was a new IIS/Apache website created and the appropriate files copied over and a new database created in the background? (which by the sounds of it is how StackExchange is currently working). How about DNS updates?

 

The Soultion:

I was investigating another project thats been on my mind, this project would greatly benefit from sub-domains like the analysisuk.fogbugz.com one. So I did a bit of research. Ideally it would be one website that connected to a different database, or just had an additional identifier in the database based on the subdomain (the analysisuk bit).

Well it turns out thats all fairly easy.

  • On IIS you can add bindings each domain you want to resolve to the website. (e.g. bindings of CompanyA.AnalysisUK.com, CompanyB.AnalysisUK.com for a single website will send both CompanyA and CompanyB to the one site, or you can have a single IP based site and have the DNS records point to that IP address so no need to setup bindings).
  • In ASP.NET you can get the Host server variable to tell you the full host name (possibly including the port). So you might get something like “CompanyA.AnalysisUK.com:80” (HttpContext.Current.Request.Headers["HOST"])
  • it's important to also really include the VartByHeader=”host” in any page caching you do, otherwise CompanyA might see the page for CompanyB and you really don't want that happening!

Bingo, multiple sub domains on one IIS website with a master database giving a host lookup to get some kind of unique database or identifier.

Shortly after finding this out it occurred to me that this solution would work well for Dollars2Pounds.com it's network of sites. The host variable gives the domain name not just the subdomain, I can't believe I didn't think of that before.  This was the missing link, between one way of working and another.

And so the result of a Saturdays worth of coding is a new table to the master database to give domain name resolution (e.g. Dollars2Pounds.com and Pounds2Dollars.com both resolve to the Dollars2Pounds website). Configuration was shifted from web.config files to a simple database table keyed by the website, strings for the website are now keyed by the website and string key.

Twenty three separate databases merged into one database, twenty three IIS websites merged into one website, a 30 minute build became 3 minutes, a 30 minute deploy became a 5 minute deploy, the web server CPU baseline load went from 20%+ to about 10% just by removing those 22 extra websites.

And whilst the benefit of a quicker build and deploy sound nice what's missing from that picture is that it also means I'm more likely to add a small feature to Dollars2Pounds and roll it out quickly which is a massive benefit. I'd tinkers with advertising changes some time ago but never released the code.

The downside? Well there is a small downside, I use to like to roll out changes to one of the quieter websites first (Dollars2Yen.com or something like that) to check that I hadn't missed something in testing that when deployed on the production server wouldn't stuff up the website. Now if I stuff it up I stuff them all up!

 

What's my point?

Do something different, it might just pay off big time for your main work.

If your job is writing embedded code, go build a website in your spare time, it doesn't matter if it's a failure, you will learn and that means it's not a failure - failure is not doing it. If you job is writing websites go grab an Arduino and play, it's fun and again, you might just learn something.

If you are an employer take Googles advice. Let your developers have 20% time to try something totally different and radical, you might just find your main product benefits. How many big companies do we see spending a fortune to buy a small company with some technology thats different from the big companies. Maybe if the devs had 20% time that would have grown in house and saved a fortune?

And if you are a job seeker, avoid like the plague companies that don't like you having projects in your own time, they are welcome to the 9-5 developers who go home and watch TV or who go down the pub in the evening and don't touch a computer until the next morning.

 

(1) AWS in its self backs up what I'm trying to say here, look at Amazon, look at AWS, the book seller who's now selling compute and bytes by the hour, who'd have thought it, but it's massive. Again, Amazon tried something different outside of their day to day work. If Microsoft had done it, well no one would be surprised, the surprise is they didn't and now they are playing catch up with Azure.

 

 

 

Comments are closed