Building a massively-multiplayer game on Microsoft Azure
Every now and again, a really intriguing opportunity comes along that’s just impossible to resist. Age of Ascents is one such case. In the spring of 2013, Illyriad, a small UK games company with an existing massively-multiplayer real-time strategy game, approached Microsoft and said:
“We want to build a game that could host the world’s biggest on-line space battle. We think Microsoft Azure is the right platform to do this, but we aren’t sure how to design it. Can you help? ”
Of course we said “yes”.
Illyriad was planning to build a game of a size and scope never before seen and Microsoft Azure is a great fit for online games such as this because it offers game developers a low cost, highly scalable, distributed computing platform, combined with a set of rich services to build upon that reduce development costs and on-going software maintenance work.
During the early stages they worked closely with Microsoft through a number of architectural design sessions to flesh out some of the key architectural aspects of the project, specifically around the scale out design that the game needed and how to implement it in a robust and predictable fashion.
The final stage of the development process took place at the Microsoft UK offices, where the code was refined, the system was put through stress testing and a TED program manager and Microsoft Technology Center team leader helped cut the code.
The server side logic was developed in C# and deployed into Microsoft Azure Platform as a Service VMs to enable fast and easy deployment. It was then stored on Microsoft Azure to persist the data.
The game needed to be highly responsive, so it was critical that this game could be partitioned across a number of data centres around the globe, with high bandwidth and low latency network access. Microsoft Azure Traffic Manager provided geo-routing to the closest data center and the Microsoft Azure Content Delivery Network (CDN) cached data at edge nodes so players had the best gaming experience possible.
The final design consisted of five core services running on the Microsoft Azure platform:
- A gatekeeper service to check for an available free slot in the game, securing a web socket upon entry
- A routing service to manage data flowing to and from the browser via a web socket, as well as handling SSL off loading
- An interest management service to perform the physics engine calculations on spaceships and any other users interaction objects
- A high latency service to consolidate data across the whole universe and update users about far distant events
- A backplane to connect each of the routing/interest management clusters
Unlike most other systems, where the designers have some idea up front of how many users they need to support, with Age of Ascent the focus was on understanding how many users a particular configuration could support. The team focused on the concept of “scale units” as the smallest unit of deployment and tested how many users a given scale unit could support.
During a large scale public beta, when all the scale units were stood up around the globe on 748 cores, the system coped with an amazing 267 million application messages per second across the system. This would have handled in excess of 100,000 concurrent users in a single contiguous battle.
What started with a great idea became reality with Microsoft Azure and WebGL: a multi-massive online game realized with a small team in just a few months, and demonstrated through the scalability of Microsoft Azure. Illyriad CEO James Niesewand recently shared his company’s experience at Cloud Gaming USA but the architecture used can also be applied to non-gaming scenarios.