Dev blog 30 Dec 2016

Some data on matchmaking and the steam sale

What we've been doing:

  • Released party matchmaking

  • Had a sale

  • Ran facebook and youtube advertising

  • Customer support for the new influx of players

We are towards the end of the steam sale period. A few thousand new people have picked up the game and we are seeing 50-70 matches played each day (in the preceding weeks this was ~10 per day). In terms of income we've probably bought another month of development time on top of what we already had banked.

What's been really nice is seeing parties of new people playing repeatedly. We also now have good data on the behaviour of the system. We know when people joined, what options they picked, how long it took to find a game, how well balanced the games were etc. Fundamentally we think the matchmaking system is working. When there are players, matches happen quickly, and where possible games are being balanced well.

What has been less nice is that we now have the data to see how long people wait in the queue before either finding a match or giving up. We haven't crunched the numbers in detail yet, but of the players who left the queue before getting a match we can see that the largest group of players (42%) wait between 1-15 seconds before giving up. Only half wait more than a minute.

Of those who joined the queue and did eventually play, 37% got a match within 15 seconds, 49% within 30 seconds, 58% within a minute, and 65% within 2 minutes.

So what does this mean? It means that if 42% of people are only going to hang around for 15 seconds before giving up, and of those people, only 37% will actually get that match, we've got a large group who will not get a match.

Let's put some real numbers on it: Let's say that 42% who will only stay for 15 seconds is approximately 2500 people over the last two weeks. 37% get a match (925), leaving 1575 who do not. So how do we cater for the 1575 instances where we couldn't find a match for someone in that crucial time window?

This is partly down to player numbers, but not entirely. Imagine 100 people online all playing matches. Another 9 join the matchmaking queue. Until either someone else joins, or one of the matches ends, the 9 will be stuck waiting. The more people you have online the more likely there will be games ending, but as we've seen in the matchmaking data, some players will only hang around for 15 seconds. For matches to be finishing every 15 seconds would need roughly 40 simultaneous games (and that's ignoring different match types and skill levels).

The consensus seems to be leaning towards having bots to fill in the gaps and get matches started, and then as new players join the queue, they can swap in immediately. This is no small job, but is likely to be the next big thing we'll be looking to add over the next couple of months. We also need to think carefully and discuss how we do this as there are various complications such as ranked matches (we probably don't want to add bots to these), different skill levels, substitutes, player preferences (some may prefer to wait for all-human games) etc.

We also still have lots of stuff remaining on matchmaking, but in the realm of enhancements rather than any fundamental changes. Once everyone is back from various holiday commitments we'll be putting together a release plan and publishing it.