Steve's blog

A blog about what's going on in Analysis UK...

NetFlix announces $1 Million prize.

In case you haven’t already heard, NetFlix has announced a competition with the grand price of $1 M USD (Convert) for a system that will improve it’s prediction of movies it’s customers would like to rent. (or as NetFlix put it “promote progress in recommendation systems”).

To help contestants NexFlix is providing a huge 100 million anonymous movie ratings (it’s a 665M download) training data set from its archives and they have very carefully updated the data to remove privacy issues (e.g. you can’t find out who actually borrowed what).

If you want a little help understanding why NetFlix would put up so much cash for a small (10%) improvement in their prediction have a read of Why better DVD recommendations are worth a million bucks from Chris Anderson over on the Long Tail (and don’t forget to have a look at his book The Long Tail: Why the Future of Business Is Selling Less of More).

Now if you want to enter the competition or learn some more about machine learning then you may want to have a check out of a recent Channel 9 video MSR Cambridge Tour: Machine Learning Group, Computer Vision and F# from Microsoft Research down the road here in Cambridge. If you are interested in what the future holds for games, video conferences, machine vision and learning then it’s well worth a watch. Also you may want to check out Pattern Recognition and Machine Learning by Christopher Bishop for a read to help you when your wading through all that NexFlix data.

BTW: Anybody having issues with dumprep.exe, I’ve had a few crashes recently and this thing takes forever to work and steals all my CPU. IE 6 just crashed when I was hunting round Channel 9 for a link to the video and now dumprep.exe has 133M of my memory and 92% of my CPU, and it’s been that way for a while.

Whilst I’m on a Channel 9 note could they make it any more horrible to navigate, they have some fantastic video’s on there, just you have to go hunting – and you should as there are some really interesting ones. Mind you theirs not even a nice link from MSDN to Channel 9 like their used to be so I guess we just aren’t meant to be watching them.

OK, rant over!

Anyway, the NetFlix competition looks like it’s going to be interesting, good luck to all those that enter.

DVD Downloads from

Robert Scoble yesterday talked about Amazon becoming the new Google (along with one heck of a busy schedule - I can’t wait to see the results of Scobel’s work, if its anything like the Channel 9 videos and his normal bloging they should be fantastic). Interestingly his comments come a year to the day after the Scoble goes to Amazon and Learns All About Web Services Channel 9 video was posted.

Amazon have also been very busy on the development interface over the last few years, AWS as it was, now ECS is a market leading initiative that has lead to lots of great sites consuming and mashing up Amazons data. Over at I have been able to include results from Amazon searches and product details with ease whilst keeping the style of, this is great for people looking for more details about a book they are interested in, with a single click they can get the product details, reviews and the product value information, as well as being able to purchase the product or similar products from Amazon or third party sellers if they wish.

The most recent offering from is UnBox a DVD download service. Along a similar line to Napster but for video rather than music.

Dave Taylor has an excellent review of unbox, a few things struck me about MP3 and video download services like these is that at the end of the day the products are still comparably priced to the CD or DVD based products, but the one thing you don’t get is a tangible product, hence it has no resale value which has the hidden effect of elevating the cost of the goods.

If you go and buy a DVD from your local retailer or online from or Amazon if you then decide you don’t like it, or just grow bored of it you can always sell or swap the DVD, however with downloads this becomes so much more difficult.

I would never pay for a second-hand downloaded DVD movie. If it’s DRM protected then chances are it’s not going to work on my PC, if it’s not protected then how do I know it’s original and the person selling it is legit? At least with DVD disk based products I have something that is easy to tell from a cheap rip-off.

The thought of buying a DRM protected DVD (a new one) leaves me wondering that when it comes time to upgrade/rebuild my PC will I still be able to watch this DVD or will I have thrown my money down the drain. I have one eBook (21 dog years) which I find really annoying to read off the PC and I just know one day I won’t be able to read it anymore because I will have broken something on the PC.

Personally I like to watch DVDs on my TV, music wise I enjoy the normal Hi-Fi, I’m a big fan of WinAmp on my PC, but I hate watching DVDs on my PC, you just know an IM window or email message will arrive at a key point of the movie popping into the middle of the screen and messing up the whole scene or the audio will get all messed up as the disk drives become busy for some other reason and sit their for 10 seconds making lots of noise for no apparent reason.

So for now I’m sticking with buying boxed DVD’s, selling them as I feel fit and swapping my DVD’s with my friends through The same goes for music, if I’m buying an album then I will buy the cased thing, selling and swapping with friends through

On a similar note I recently realised a new use for that hadn’t even crossed my mind, last week I went around a friends for a few Friday evening beers, before leaving I thought I would take a DVD with me, I couldn’t find one on my shelf’s that I thought he would enjoy or hadn’t already seen. It wasn’t until we were talking that I realised he should have used his DVDSwap.wsaccount to have a browse through my DVD collection and then ask me to bring over the one he would like. Look out for a new "Bring it over" style feature on to help friends get together over a mutually enjoyable DVD soon – when I figure out a good name and how to implement the service!!!

BTW, as I’m talking about WinAmp and listening to music on the PC I have to mention .977 the 80s channel a shoutcast radio station I listen to a lot. You can enjoy classic 80s tracks with WinAmp and [Club] .977.