Concurrency and the Games Industry

Ever since I played my first Massively-Multiplayer game I have been addicted. Not to the games, but to know how they can possibly work. Managing thousands, sometimes tens of thousands of simultaneous players is no small feat.

When I moved to the games industry from the (rather less interesting) business sector, I was therefore very excited to be working on a large-scale MMO as my very first project. With the success of games like World of Warcraft, and even services such as Xbox Live, much attention has been paid to massively scalable server solutions. The number of people specializing in development and technology for multiplayer games such as myself has expanded rapidly in the last few years. Games which have been traditionally single-player are now expected to implement multiplayer as standard.
There are more third-parties offering middleware for networking than ever before and services such as Microsoft’s Xbox Live offer a framework that developers targeting that platform must implement, this simplifies both the development process and the end-user experience. Networking and developing across a large-scale environment is complicated, many developers find the subject tedious at best.

Most programmers are used to writing applications which run in-order on a single thread. Even in the age of multi-threaded games, most developers only work with a single thread in their day to day tasks. Typically, different systems such as rendering, AI and networking will run on their own thread or will be split logically across a number of threads, in many cases this is hidden away from most of the developers by the engine and code is written as if it were a single-threaded environment.
Most modern programming languages such as C++ are designed in a way which makes their core functionality inherently single-threaded. Understanding code which is running in a multi-thread or multi-core environment is difficult. Debuggers have a hard time presenting the information in a form which can be easily understood by programmers, much multi-threaded code is powered by more abstract concepts such as messaging and queuing systems; it’s simply not possible for the debugger to deduce exactly how the systems hang together in order to present it in a way which would be easily understood.

In many ways, networking in a large-scale environment is similar to developing for a multi-threaded environment. Work is typically spread across multiple peers with each peer doing different bits of work concurrently with other peers.
There are numerous paradigms and implementations for these large-scale systems and going into each one and its advantages and disadvantages would be another post at the very least. In general these solutions fit into one of two categories:

1. Each peer has a specific purpose, proxy or coordinator servers dish out work to the relevant peers as requests are received.

2. All peers share the same functionality; work is spread out between the peers based on available resources.

It is not atypical to have a solution which fits into category two but also derives from category one. A website might have numerous web servers which can all deal with the same requests, but also a second layer of servers for database management.

You might, at around this point, wonder what the point of this post actually is.
As consumers expect more and more from online games and services, the games industry going to need more programmers who are familiar with the abstract and concurrent nature of large-scale network solutions. With the rising popularity of CELL, GPGPU and Intel’s Larrabee on the horizon the games industry is also going to need more and more programmers who have a firm grasp of multi-threaded development as consoles and PCs become more and more concurrent. I think we’re at an exciting time of technological development and in many cases games are one of the first places new technology is leveraged.
Most of all, the industry is going to need people who are interested and passionate about this stuff.

When people decide they want to be games programmers, very few of them think “I want to be a multiplayer programmer” or “I want to work on core tech libraries” such as those for multi-threading. Understandably people are more interested in the far more obvious areas of games such as rendering and that's a real shame.
It is my hope that the games industry will be able to get new developers interested in what is still one of the least explored areas of software development which is helping to drive some of the biggest changes the games industry has seen in recent years.