I have a bit of a sore throat so I wasn’t up for doing a Diecast. But as a way of giving your your usual dose of content, I thought I’d share a strange meeting I had the other day on the internet…
EXT – The Internet – Night
Shamus is minding his own business, scrolling Reddit and generally wasting time, when Don Data enters. He’s an exaggerated stereotype of Jersey-area Italian-Americans. Maybe he’s played by the late James Gandolfini, but maybe not. It’s hard to tell in this lighting. He looks around, and once he sees the coast is clear he invades Shamus’ personal space.
Hey. You’re that internet guy, right? You do programming and stuff. You know about databases, right?
(Visibly uncomfortable.) Yeah. I mean, sort of. I’m kinda… I more of a graphics…
Yeah, yeah. Close enough. Whatever. Check this out… (Don pulls a DATABASE out of his coat and hands it over.) I figured you might want to have a look at this.
(Eyes the database suspiciously.) I don’t get it. It’s just a list of video games and… what is all this?
(Shrugs.) I heard it was, like, a list of all the Metacritic scores for the last 20 years.
(Raises his eyebrows.) Where did you get this?
(Shrugs again.) I know a guy.
Okay, but where did THAT guy get it?
It fell off a truck.
A database fell off a truck?
(Chucks him on the shoulder and nods.) Have fun, kid.
Kid? I’m nearly 50… (Shamus looks up from the database to see that Don has vanished.)
So that was a little weird. Anyway, let’s open up this database and have a look.
The first thing to note about this data is that it’s only PC games, and it seems to only include games that scored over 30-ish. That works out to a little over 4,000 games. The data set is pretty small in the… hang on. I’ve got the data, so why don’t I just show you?
So the data is obviously very thin before 2000. This makes sense. Metacritic was launched in 2001. Do try to keep this in mind as we go forward. This sample size isn’t that big to begin with, and it’s microscopic in the 90s.
We always hear people say that reviewers are less critical today than they were 20 years ago. I sort of instinctively lean towards this assumption, but let’s see what the data says. Here is an average of all scores given by critics, by year:
Note that this graph is re-scaled to show the 60-100 range. Still, that’s interesting. Obviously we need to ignore the outliers in the 90s. If I’d had more time to redo these charts I probably would have made a version of the spreadsheet without the pre-2001 data. In any case, we can clearly see a sawtooth pattern there in the middle. That low spot in 2006 is the launch of the PS3 / Xbox 360 generation. We tend to get really horrendous PC ports in the early days of a new console. Plus, this was the dawn of phone-home DRM.
I see things improved until 2011, when it took a dive and we had a few bad years. I’d love to blame that on GFWL, but I honestly have no idea what that is.
So that’s what the critics had to say about PC gaming. Let’s see what the Metacritic user scores look like:
Again, note that the scale starts at 60 here. You can see that users rated games much lower than critics pretty much across the board. That makes sense. Critic reviews are gathered automatically, but user reviews are self-reported, and people generally only bother when they feel strongly about something. For users, the mid aughts low point was in 2005, not 2006.
So let’s see how big the delta is between the critics and users…
This chart is basically the green chart minus the red one. Or, how many points higher are critic scores than user scores? We could see this as an indication of the diverging opinions between the people and the press, but this spike at the end could also be the result of games that didn’t turn on their aggressive monetization systems until after the review scores were set. This pissed people off and led to review bombing. So maybe it’s not a measure of the difference between critics and users, but the difference in quality between launch day, and the day the cash shop opens.
As always, take everything here with a grain of salt. I haven’t spent a lot of time with the data yet and I might be missing something. Also, this was my first time creating a pivot table in Google Sheets and it’s possible I bungled it.
This project needed a little more time, but right now I really need to take a break and drink some tea. And maybe play some games from before 2016.
The Best of 2013
My picks for what was important, awesome, or worth talking about in 2013.
In Defense of Crunch
Crunch-mode game development isn't good, but sometimes it happens for good reasons.
Video Compression Gone Wrong
How does image compression work, and why does it create those ugly spots all over some videos and not others?
The plot of this game isn't just dumb, it's actively hostile to the player. This game hates you and thinks you are stupid.
Good Robot Dev Blog
An ongoing series where I work on making a 2D action game from scratch.