Unlike eBook readers, CD players, MP3 players and BluRay drives, video games consoles are in the unique position of not featuring standardized hardware. Where any cassette player would play any cassette, video games need to be tailored to the unique rendering hardware of their machines, which incentivizes hardware manufacturers to create their own first party exclusives. You buy an Xbox One over a PlayStation 4 if you want to play Gears of War more than God of War, which makes first party exclusives a vital deciding factor between similar gaming hardware.
So… we’ve got a lot of data about this. But because video game fans can be silly, we’re going to explain where it’s from, caveat it appropriately, and right now remind you that this is just an interesting and partial snapshot of reality – rather than any kind of truth-cudgel to be swung mercilessly at your foes.
The questions are obvious: which publisher has the best first party game software, which platform holder produces the most consistently high-quality games, and which console is most worth getting based on the quality of its exclusives? The answers might surprise you…
How We Did It
Our dataset was taken from Metacritic, the review aggregator that collects game reviews. We downloaded data on over 350,000 reviews covering over 18,000 different games for analysis (yes, it took ages to do). Obviously, given that there are a number of important caveats that mean you should take this data with as many pinches of salt as you deem necessary: For example, the data will only be as accurate as Metacritic’s database was when we took our snapshot.
What The Numbers Say
So here’s the results of all of that data mashing. Of the three current major systems on the market – the PS4, Xbox One and Switch — Nintendo comes out most favourably if you decide to look at which publisher’s first party games have the highest average review score. Nintendo’s first party Switch games have had an average (mean) Metacritic score of 79.24 across the 15 games produced for the Switch.
This average score beats Microsoft Games Studios’ average score of 73.09 and Sony Interactive Entertainment’s average score of 69.09. Nintendo does have the fewest first party exclusives, and has also been on the market the shortest amount of time, so it may be the case that after three or four years Switch trends downwards – though to be fair, Nintendo is also on a blistering run of form.
Nintendo’s review scores also share almost the exact same level of consistency as those from Sony, with a standard deviation (a measure of how far scores tend to vary above or below the average) of 11.58. This means that, while they have the highest average score, Nintendo’s scores tend to fluctuate between approximately 68/100 and 90/100.
Sony Interactive Entertainment produce the most ‘consistent’ first party exclusives, with a standard deviation of 11.55 (a lower number is better, as it indicates less fluctuation) which compares to Nintendo’s 11.58 and Microsoft Game Studios’ 12.16. Sony’s average review score is slightly below that of Microsoft, 69.09 compared to 73.09, but this is something of a Pyrrhic victory – because it’s reflecting the sheer number of PlayStation exclusives in this generation, 89, compared to Xbox’s 53.
A Nerdy Diversion
What’s especially interesting about these stats is they paint a picture of three platform holders that have distinct appeals when it comes to first party software. None is unanimously ‘better’ than the others, they each win out in different categories depending on how you slice the data, and each has their weaknesses. People may love online arguments about which company has the best exclusives but, as this shows, answering that question depends on what’s important to a given player: number of exclusives, consistent quality, or highest average score.
So: Nintendo gets the highest average scores, albeit with fewer games, while Sony’s sheer quantity of titles pulls down their average. The important caveat is that PlayStation manages to produce the greatest number of first-party games with consistent levels of quality. Microsoft’s average beats out Sony’s average, but their standard deviation is also a little higher (12.16 plays 11.55), so Microsoft Games Studios is the most inconsistent of the major platform holders.
You could make a decent claim from this that Nintendo makes the best games overall. They have the fewest exclusives right now, but that’s down to being on the market for less time than the competition. They’re actually on pace to catch up with Sony, if their production remains as consistent as it has been in Switch’s first year.
Sony has the largest library of exclusives, and those exclusives are consistently good, but the average isn’t so good. This is probably one of the areas where statistics don’t show the whole picture – what’s impressive about PlayStation is the range and number of titles, including many unusual games that may not score as highly as traditional genres, and looking at an average score doesn’t really communicate that. Microsoft has a slightly higher average score than Sony, but their scores also fluctuate more wildly. There’s a slight overall bump in quality, but the bad games are worse too.
Of the three areas we looked at, each platform holder excelled in exactly one. I love it when the numbers do nice things like this.
James O’Malley (@Psythor) helped us crunch all this data.