Simon Squared – We have Multi-player: Days 4, 5 and (ahem!) 6

OK. So you let me have one more day, and I took 3. But I got it working. After 6 days of work, I can show you Simon Squared in all its Windows Azure powered multi-player glory:

Simon Squared Multi-player, nicely showcasing my Multiple-instances-of-the-WP7-emulator hack

So what took me so long? Surely after getting the Windows Phone 7 emulator to talk to the Windows Azure emulator, finding a UI library for XNA and enabling multiple instances of the WP7 emulator to run at once, everything else should have been easy? If only!

Here’s what happened.

Day 4

9:00 Start by setting up a Windows Azure subscription, making use of my MSDN subscription benefits: it's pretty generous: 750 hours/month of Compute Time, 10 GB storage, 7/14 GB traffic in/out.

I get a little worried when the sign up screen tells me that I'll receive an activation email "within 24 hours" - but I'm relieved to see that within a couple of minutes of being redirected to the Azure management console (very swish Silverlight app) my subscription appears, and I can start creating deployments.

10:11 Got side-tracked into a discussion with my boss about possibilities for future products and offerings based on the work we've been doing here

12:45 Implemented the phone side behaviour for running the count down to the games beginning. Making heavy use of the Reactive Framework (which comes built into WP7) for hiding away the messiness of asynchronous calls.

13:45 Problem with WCF REST: if I return a derived type of message from my operation, the client only receives the properties defined on the base type. This is the problem that the KnownType attribute solves elsewhere in WCF, but it doesn’t seem to be working for WCF REST.

15:22 Discovered that WCF REST uses XmlSerializer by default rather than DataContractSerializer - so KnownTypes woudn't work. Also discovered that RestSharp is using XmlSerializer when sending data to the server, so not serializing inherited types in the way that DCS is expecting.

15:39 Now got inherited message types passing correctly: WCF Rest and RestSharp are both pretty pluggable, so I was able to implement a custom ISerializer in RestSharp, and a custom MediaTypeProcessor in WCF Rest, both using DataContractSerializer instead of XmlSerializer

17:30 Caught out by Http response caching. The symptom was that I’d launch a fresh HttpWebRequest,  but get back stale data. Investigation showed that the requests weren’t ever leaving the phone – it never showed up in Fiddler. I spent ages prodding my Reactive Framework query, convinced that it wasn’t issuing the requests properly. Then I remembered Http Caching. In my haste, and newbie-ness, I solved the problem by putting a NoCache on all the responses on the server side. A better way would probably have been to set the WebRequest.CachePolicy.

Day 5

09:00 Thought occurs to me that I might get on faster if I don't have to keep waiting for the Azure Emulator to start up and shut down every time I build my server. Why not just run my server project directly in IIS? I try it – and WCF Http stops working. I eventually discover that the CTP bits I’m using don’t yet support HTTPS, and I had a HTTPs binding configured on my default Website in IIS.

10:30 Fortunately Cassini works - why did it take me an hour and half to discover that!

10:43 Start work on refactoring puzzle generation, so I can do it on the server side. The goal is that the server will generate rounds of 5 puzzles, and send them out to all the players in advance. Within a round, the server will send out a message saying “start puzzle X at Time T” – and thus synchronize the experience of each of the players

12:00 Working on updating the Phone side to play the puzzles that server sends out.

13:16 Finished refactoring the Game screen so that it is backed by a GameController - allowing me to implement a SinglePlayerGameController and MultiPlayerGameController.

17:20 Got some simple state machines working on the client and server. It looks something like this:

17:40 With two instances of the emulator running, I can now see the same puzzles displayed on each. The problem is, they’re horribly out of sync – player 1 might get the puzzle two seconds ahead of player 2, which for small puzzles means he’s certain to win!

Day 6

09:21 Implemented Christian’s algorithm to estimate the clock skew between server and phone. The implementation is pretty straight-forward using the Reactive Framework, and quite elegant, if I do say so myself. Remind me to show you, sometime.

10:33 Couldn’t figure out why changes I was making in the code weren’t having any effect when I ran the game. Then discovered that Visual Studio had some how set itself to “Never Build on Run”.

12:00 It’s working! Multiple players now receive the puzzles beautifully in sync, one after another. As soon as one player completes a puzzle, it vanishes from all the other players screens, and the count-down to the next puzzle begins

13:20 Implemented a scoreboard to show the position at the end of each round

14:30 Having deployed the game to my phone, I challenge Vinod to a dual – me on the emulator, him on the phone. I now discover a serious downside to being a games developer: once a game reaches playability, you inevitably get sucked into playing when you should just be testing.

15:35 Use Expression Design 4 to recreate a game logo that Neil designed using Word. What do you think?

17:05 Spitting and polishing. Tidying up the various screens. I might not have time to make them pretty, but at least I can make the buttons big enough to touch. That’s the downside of testing with an emulator using a mouse – it doesn’t alert you to the imprecision of fat fingers!

17:30 Waiting for Windows Azure to deploy the server. Visual Studio 2010 is supposed to be able to package and deploy your project straight into the cloud, but I couldn’t get that to work (something about it not being able to resolve the URL of my storage account). Instead, I uploaded the package manually.

17:45 Still waiting for Windows Azure to start up my role instance. Give up and go home.

21:00 Wife gives me permission to interrupt our traditional Thursday evening film (The Prestige last night, – recommended) to see whether the Azure role has started up yet. It hasn’t, so in the time-honoured way I tell it to reboot – and that fixes it. A few minutes later, she’s thrashing me – mainly because she’s using the phone, and I’m using the emulator over Remote Desktop (my laptop is old, and doesn’t support XNA games in the emulator) – in that setup the emulator doesn’t respond to my mouse moves!

What I need is another phone.

Ah, what’s this I see – the Windows Phone Marketplace Developer Newsletter, announcing Windows Phone 7 Handsets for Developers. The Newsletter says that local evangelists have codes for free phones to give away. I think Mike Ormond is the WP7 evangelist for the UK …

Multiplayer-enabling my Windows Phone 7 game: Day 1

I’m building a game to enter in Red Gate’s Windows Phone 7 App competition. The first challenge I set myself was to build the core of the game in 3 days from a standing start. Then Red Gate moved the goal posts. So I’ve set myself a new challenge: make the game multi-player in 3 days, using Windows Azure for the server side. Here’s Day 1:

9:00 Started researching networking capabilities in WP7. MSDN has a useful summary. In short, HttpWebRequest and WebClient are supported, as are parts of WCF. One notable absence are Duplex Services – so it looks like I’ll be doing my own client side polling. ChannelFactory.CreateChannel is not supported, so service proxies cannot be generated dynamically - they have to be created using Add Service Reference, or slsvcutil on the command line. Since I’m not a great fan of all the boilerplate that the Add Service wizard spits out, I think I’ll give WCF on the client side a miss. This gives me the idea of creating a RESTful service that I can call using plain old HttpWebRequests.

9:16 Given the recent hoo-ha  over Windows Phone 7 data leaks, I think I should do my bit to save bytes and minimise data usage. I'Since I’m quite keen on Google’s Protocol Buffers format, I’m pleased to discovered that ProtoBuf.Net, Marc Gravell's implementation of the Google Protocol Buffers format supports WP7.

9:22 Starting work on a Windows Azure project for testing communication between client and server. It’s neat how the Cloud project in VS 2010 has a Windows Azure emulator.

10:38 Downloaded and built Microsoft's WCF Web APIs Preview 3. This extends the REST support that WCF has, as of .Net 4.0. Using this, and helpful blog post from Jesus Rodriguez, got a test service sending back responses using Protocol buffers. One thing that caught me out was failing to decorate my data contract classes with the [ProtoContract] and [ProtoMember] attributes. The serializer just returned an empty stream without that. Now to get the phone talking to the server.

11:05 An interruption: the boss arrives back in the office with an HTC HD7 for me to test the game on!

Signed up for a developer account to allow the phone to be unlocked for testing my apps. To my dismay, I discovered that I can't unlock the phone for deploying test apps until the publisher verification process has been completed. What?! I can understand why Microsoft would want to check that our company is who it claims it is before it lets us submit Apps to the market place; but to my own phone? I think I know who I am!

11:39 Encountered my first real problem: Windows Azure compute emulator only listens on the loopback address, 127.0.0.1; this means that the Windows Phone 7 emulator can't talk to it. Seems I’m not the first to have encountered this. I tried a number of solutions:

• Using NCat, a Port fowarding tool. No joy with this at all.
• TcpTrace, another kind of port forwarding tool, worked, but not when Fiddler was listening in.

13:00 Thanks to some suggestions from Phil Haack I discovered that Fiddler will do the job on its own (is there anything it can’t do?). Following these instructions, I set up Fiddler as a reverse proxy. This means my WP7 app can connect to MyMachine:8888, and Fiddler will shuttle the traffic to and from the Azure emulator.

13:10 Discovered RestSharp, a library that builds on top of HttpWebRequest to simplify making calls to RESTful services. In particular, it can deserialize the Response bytes into objects for you.

13:17 Trying to write a ProtoBuf deserializer for RestSharp, but Protobuf is throwing  MissingMethodException. It looks like I might have to use pre-built Serialization assemblies

14:40: Following Mark Gravell's post on using Protobuf-net with WP7, I mange to get deserialization of messages working on WP7 by pre-building a serialization assembly. I have to resort to subterfuge to get the resulting Serializer assembly added to my WP7 project: The Add Reference dialog refuses to do it, claiming that the assembly is not a bona-fide WP7 assembly, so have to add it to the .csproj file by hand.

15:00 In order to allow users to setup multiplayer games, I’m going to need to show some kind of UI. WP7 doesn’t allow you to mix and match Silverlight and XNA within one App. I don’t really fancy rolling my own UI library, so I start casting around for one. A helpful post on the XNA forums provides a list libraries for XNA in general, though only a couple of them support WP7.

15:35 Start investigating Nuclex UserInterface library, a UI library for XNA that claims support for WP7. Initially it looks quite promising: I can get it to render a couple of buttons. But I can’t make the buttons respond when I tap them.

17:15. I give up on Nuclex, and look for something else. The most promising looking framework is XPF, from Red Badger. It’s ambitious: it aims to replicate some of the core features of Silverlight for the XNA framework, including layout, data binding and animation. They have nightly builds available now, but they’re only valid for 30 days from the build date, and there’s no word yet on a release date, or indeed what the cost of a license will be. Do I take a chance on it?

What are your thoughts? Have I missed a framework that will save me hours? Get in touch: @samuel_d_jack on Twitter, or by email.

(Simon Squared) x n: The Multi-player Challenge

A new year, a new challenge.

The end of 2010 found me scrambling to build a Windows Phone 7 game in 3 days using the XNA framework, spurred on by a competition deadline and the potential reward of a $10,000 prize. The challenge was to “build something that developers will love”, and who doesn’t love reading of a race against the clock?

Well, I got the game done (Ok, maybe not done done – it will still stand some polishing) only to find the deadline moved by two months, and thus my unique selling point rendered an unlovely shade of void. So I need a new challenge, a bigger, a bolder challenge.

How about making a multi-player version of the game?

Using GPS to locate players nearby?

In 3 days?

Oh, and I nearly forgot the geeky dessert topping: I’ll be using Windows Azure to pull it all together.

Now, just to be clear, my calendar doesn’t contain three consecutive day-sized gaps, so these 3 days will have to be spread out over the next couple of weeks. In betweenwhiles, I can’t guarantee that my eye won’t stray to the MSDN pages about Windows Azure, or that my brain won’t allocate a background thread to designing message protocols. But I will promise to keep my Visual Studio solution firmly shut whilst the clock is not ticking.

Microsoft PDC 2010 Speculations

We’re now less than a week away from Microsoft PDC 2010. The session list has just been published so I think it’s time to indulge in some speculation as to this years announcements.

C# 5

For me the highlight of recent PDCs has been discovering what Anders and his sous-chefs have been cooking in the C# kitchen. Of all the changes likely to be announced, it is changes to the C# language that are likely to make the most day-to-day difference to my work. In 2005 it was LINQ, in 2008 it was dynamic programming. What will it be in 2010? Well, Anders has been given a slot, and we already have some clues as to what he might say. At his presentation on the future of C# and VB.Net at PDC 2009, Luca Bolognese trailed some of the likely features, including:

• Compiler as a Service – the C# compiler completely rewritten in C#, and available to use programmatically, allowing all kinds of interesting meta-programming, including the ability to write refactorings.
• Better support for immutability built into the language
• Resumable methods.

It was this last one that interested me most. Luca showed an example involving asynchronous programming. Today we do this by calling a Begin* method, and passing in a callback that will be executed when the asynchronous invocation has finished (and we never, ever, forget to call the corresponding End* method!). This can get quite difficult to wrap your head around if you have a whole chain of methods that need to be invoked asynchronously – say in the callback of the second one you kick off another asynchronous method, passing in a callback that runs a third – and so on.

This style of programming actually has a name – Continuation Passing Style (or CPS) – and, from recent developments on Eric Lippert’s blog it seems they’re planning to add support for CPS to the C# language to make it easier to write this kind of code without blowing a fuse in your brain (in typical Eric Lippert style, he is neither confirming or denying what hypothetical features might or might not hypothetically be added to a hypothetical future version of C#, hypothetically speaking).

No doubt Anders will reveal all.

LINQ

Since it was released in 2007, LINQ has colonised .Net code bases at an incredible pace. Parallel LINQ was another great innovation in .Net 4. And Microsoft aren’t done yet. The Reactive Framework (aka Linq-to-Events) is coming on nicely. And this year Bart de Smet has a session tantalisingly entitled “Linq, Take Two: Realizing the LINQ to Everything Dream”. Linq-to-Everything sounds pretty ambitious, so I’m looking forward to hearing what he has to say.

Silverlight

It looks like we can expect some big announcements for Silverlight this year – check the session list for proof: there’s a note against Joe Stegman’s session on Silverlight saying that the abstract is embargoed until after the PDC keynote.

What would I like to see? Well, one of the biggest pain-points I’ve found in my Silverlight programming is that Continuation-Passing-Style programming is thrust upon you whenever you want to show a dialog, or make a call to the server: because Silverlight runs in a browser, you aren’t allowed to block the UI thread, so every potential blocking operation has to be done asynchronously. It would be really nice if C# 5 does simplify CPS, and if that also applied to Silverlight.

Azure

I’ve already made my PDC prediction for Windows Azure: I’m guessing that Microsoft will announce free, enthusiast-scale hosting for Web Applications in Azure. This is looking even more likely now that Amazon have announced a free entry-level version of their EC2 cloud hosting platform.

Thursday 28th October will reveal all. Stay tuned – once again my company has graciously given me time to follow along with the PDC sessions (and this year they’re being transmitted live, as well as being available for download within 24 hours). As on previous occasions, I’m hoping to find time to share with you all what I learn.

Need Virtual Server hosting in the UK or USA? Try ElasticHosts

If you’re looking to host Virtual Servers in the cloud with good response times from the Europe or the USA, you should take a look at ElasticHosts. Here’s why:

• By my calculations they are just about the cheapest of the options available in the UK. I compared them against RackSpace, CloudHosts (part of UKFast), Amazon and FlexiScale. Only FlexiScale came close.
• Incredible flexibility in sizing your server. Forget about Small, Large and Extra Large: with ElasticHosts you get a slider control for CPU, Memory and Hardisk allowing you to resize your machine as your needs vary.
• You can either choose pay-as-you-go pricing, with no minimum commitment; or if demand is predictable you can take out monthly subscriptions and save money. You can even mix and match: if say, you typically use 5GB bandwidth in a month, but sometimes burst up to 7GB, you can subscribe for 5GB a month, and use pay-as-you-go pricing for the extra.
• They have a REST API giving complete programmatic control over your pool of servers, allowing you to do everything from stopping and starting servers, through provisioning completely new servers on the fly, to copying drives from the cloud to your machine.
• They offer a free, no commitment, five day trial to get you hooked started.

I’ve saved the best till last: ElasticHosts Customer Service is amazing. Fronted by Elastigirl herself (aka Anna Griffiths, her cover as Helen Parr now being thoroughly blown) they’ve been quick to respond whether I’ve called or emailed.

The first time I got in touch, I was rather cheeky: the server they offer for the free trial is clocked at 2GHz with 1GB RAM. This rather struggled as I loaded it with Windows Server 2008, so I called asking if there was anything they could do to make the trial server more representative of the system we might be looking to subscribe to.

“Sure”, said Anna, “What would you like?”

I was stunned! And continued to be so as she upped the memory, hard disk capacity, bandwidth availability and gave me the use of a static IP address.

If I had one suggestion to make, it’s that they set up a support forum to supplement their email and telephone support. I’m sure they’ve already answered several of the questions I asked many times over, and I always prefer to bother Google before I bother a support person. Added to that, it would be nice to have a place to share some of the titbits that I discovered – like how to call the ElasticHosts REST API from .Net (you have to force WebRequest to include the Basic Authentication header).

So give ElasticHosts a try. I’m sure you’ll be pleasantly surprised.

My PDC 2010 Prediction: Free Enthusiast App Hosting on Windows Azure

So Microsoft have announced that there will, after all, be a Platform Developers Conference this year. It will however, be an austerity PDC, chopped down to two days, held on Microsoft’s campus rather than in a big conference city, and with a much narrower focus. This is billed to be a Cloud Conference. And I think I know what Mr Steve “Developers, Developers, Developers” Balmer is going to be announcing.

Think about what Microsoft have revealed over the last couple of weeks:

• WebMatrix, a new streamlined IDE aimed at enthusiasts wanting to produce web applications. And skipping hand-in-hand with WebMatrix comes Razor, a new syntax for creating web views.
• SQL Server Compact Edition 4, a new version of the embeddable, in-process edition of SQL Server that has been enabled to run inside ASP.Net
• IIS Express, a version of IIS 7.5 with full support for the new IIS Integrated pipeline, SSL, etc. but capable of running in a locked down environment with no admin privileges.

Then there are the rumours about KittyHawk, a tool aimed a tech-savvy business users wanting to produce Silverlight applications without any coding.

Add all this up, and what do you get? Hundreds of enthusiasts with shiny new web applications, eager to share them with the world – but not wanting to go through the pain of ftp-ing their creation to a web host.

Hence my prediction: At PDC 2010 Microsoft will announce free hosting for enthusiast-scale web applications within Windows Azure. And they’ll throw in tools to publish web applications from WebMatrix and Visual Studio Express to Azure at the click of a button.

Take a look at the top two feature requests on mygreatwindowsazureidea.com (the official feature request list for Windows Azure): Make it less expensive to run my very small service on Windows Azure and Continue Azure offering free for Developers. Microsoft seem pretty serious about addressing issues on such lists just recently.

Google have a head-start in this arena: they’ve been offering this kind of free hosting in their AppEngine for years. But with their Visual Studio tooling, I think Microsoft could clean up.

PDC 2009 @ Home

Can you believe it? A year flown by already since I jetted off to LA, reclined to the soothing sound of Scott Gu's Keynote, and all-round indulged in a general geek-out at Microsoft PDC 2008! And already Microsoft PDC 2009 is upon us.

Windows 7, which we gawped at for the first time twelve months ago, is now gracing our desktops; Windows Azure is almost ready to go live; and after a good 52 weeks of head-scratching developers are just beginning to work out what Oslo is and isn't good for.

What with our baby being born and Bankers busting our economy, I'm not going to be present in person at the PDC this year. But the Directors at Paragon have very kindly granted us all a couple of days to be present in spirit by taking advantage of the session videos that Microsoft generously post online within 24 hours of the event.

So which sessions will I be watching?

Top of my list are the PDC specialties, the sessions usually flagged up with the magic word "Future" in their title. What developer doesn't go to PDC with an ear greedy for announcements of shiny new features for their favourite platform?

• The ever-entertaining Luca Bolognese will be telling us all about Future Directions for C# and Visual Basic. Maybe he'll have news on Ander's Compiler as a Service?
• Karen Corby will be on hand to show us the Microsoft Silverlight Roadmap and Futures. How many items will Microsoft have checked off the suggestion list? There are at least indications that Data Binding is being enhanced. It looks like we’ve heard all that Microsoft are ready to say about WPF for the time being, but there is at least a session on XAML Futures in Microsoft .NET Framework, Microsoft Silverlight and Tools.
• Windows Azure might be almost ready for prime-time, but Amazon's Cloud has been going for years. How is Microsoft going to compete? I'm guessing that Manuvir Das (Windows Azure Present and Future) Yousef Khalidi (Bridging the Gap from On-Premises to the Cloud) and David Robinson (The Future of Database Development with SQL Azure) will be showing us the view through their crystal ball.
• Microsoft still have a lot of explaining to do when it comes to Oslo - some of the Oslo developers confess to only recently getting to grips with what it is. Earlier this week Doug Purdy caused consternation in the blogosphere when he announced that Oslo is going to be renamed “SQL Server Modelling”, and will be delivered as part of a future SQL Server release. He has since gone on to clarify that the DSL tools are alive and well, and that Oslo will be part of the broader SQL Server platform, not the core engine, and that this will be no means limit its availability.

  All of which is no doubt calculated to stir up interest in the many Oslo sessions at PDC. The dynamic duo, Chris Anderson and Don Box will be kicking off with Data Programming and Modeling for the Microsoft .NET Developer. Then, in Evolving ADO.NET Entity Framework in Microsoft .NET Framework 4 and Beyond, Anderson will be teaming up with Shyam Pather to explain how Oslo is making connections with the Entity Framework. Chris Sells and Doug Purdy are doing a session the following day on “Quadrant”, the friendly graphical face of Oslo, explaining the art of Building Data-Driven Applications Using Microsoft Project Code Name "Quadrant" and Microsoft Project Code Name "M". Digging a bit deeper, Don Box and Jeff “Pinky” Pinkston will be explaining Microsoft Project Code Name “M”: The Data and Modeling Language.

The other thing PDC is famous for are “deep-dives”: talks by the architects and developers of the various technologies on show, laying bare the inner workings of their creations. This year I’ll probably focus on WPF and Silverlight.

• Blake Stone, presenter of Windows Presentation Foundation 4 Plumbing and Internals warns us to bring a hard-hat for his session
• Any nay-sayers avoiding WPF because Microsoft aren’t using it for any significant applications should be silenced after Paul Harringtons talk, How Microsoft Visual Studio 2010 Was Built with Windows Presentation Foundation 4
• .Net RIA Services (RIA = Rich Internet Application) holds a lot of promise as a framework that provides a measure of RAD without compromising on all of the principles that make the Alt.Netters hot under the collar. Make sure to watch Building Amazing Business Applications with Microsoft Silverlight and Microsoft .NET RIA Services for an introduction, and Mastering Microsoft .NET RIA Services for more depth.
• One thing that has impressed me with Silverlight is how WCF “just works”. Networking and Web Services in Microsoft Silverlight promises to explore that whole area in some depth

As well as watching these sessions, I hope to find some time over the next week for blogging about my discoveries. So why don’t you follow along?

Cloudy Azure Skies at the PDC

Microsoft are clearly getting in touch with their Arty side. The most recent evidence came today when they announced Windows Azure; earlier hints can be found in “Silverlight” and Project Indigo (which later became WCF).

Windows Azure is Microsoft’s new Cloud Operating System. Not an Operating System in the sense of something that you install  on your PC, but an Operating System in that it provides a layer of abstraction upon which interesting things can be built. Just as Windows XP or Windows Vista abstract away the messiness of interacting with keyboard, mouse, graphics card, disk drives, etc. to provide a nice API with which developers can do interesting things, so Windows Azure abstracts away the difficulties of purchasing hardware for data-centres, provisioning machines with identical copies of code, configuring load-balancers, backing up databases etc. and provides a nice programming model and user interface to streamline all those things.

We were shown a demo of a standard ASP.Net application being packaged up and tagged with a config file. This package was then submitted to the Azure portal, where it was given a url and then published. It was instantly available at that address. The magic came when they pretended that load on the application had increased, and more computing power was needed: just changing a number in the config file was sufficient to scale the application out across another couple of machines – Windows Azure taking care of the details.

Another component is the cloud storage. They have an API that allows Blobs, Queues and simple Tables to be managed in the cloud, with space being provided elastically. All the data is available through REST APIs, with a .Net object model on top for easier access through managed code.

And of course we developers need our tools, so there is a Windows Azure SDK which you can download today. This provides templates for Visual Studio, but more importantly, it provides a mini-version of the Azure platform for your desktop, so that you can build and test applications without needing to upload them.

In the CTP that they released today only managed code is supported, but the intention is to allow native code, and other languages like PHP in the near future. Also in the future will be a “Raw” mode that will allow Virtual Machines to be hosted in the cloud a la Amazon’s EC2.

The intention is to release this commercially in 2009, though Microsoft apparently will be guided by the community as to when they think it is ready.