{"id":21653,"date":"2013-11-27T15:40:30","date_gmt":"2013-11-27T20:40:30","guid":{"rendered":"http:\/\/www.shamusyoung.com\/twentysidedtale\/?p=21653"},"modified":"2013-11-28T07:54:18","modified_gmt":"2013-11-28T12:54:18","slug":"user-reviews","status":"publish","type":"post","link":"https:\/\/www.shamusyoung.com\/twentysidedtale\/?p=21653","title":{"rendered":"User Reviews"},"content":{"rendered":"<p>The question was put to me on Twitter:<\/p>\n<blockquote align=\"center\" class=\"twitter-tweet\" data-conversation=\"none\" lang=\"en\">\n<p><a href=\"https:\/\/twitter.com\/JimSterling\">@JimSterling<\/a> <a href=\"https:\/\/twitter.com\/DanStapleton\">@DanStapleton<\/a> <a href=\"https:\/\/twitter.com\/shamusyoung\">@shamusyoung<\/a> what do you guys think of the &quot;user&quot; section of Metacritic?<\/p>\n<p>&mdash; Shri (@ResonanceSD) <a href=\"https:\/\/twitter.com\/ResonanceSD\/statuses\/405234284572577792\">November 26, 2013<\/a><\/p><\/blockquote>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<p>Interesting question. I don&#8217;t go to Metacritic very often, but assuming by &#8220;user&#8221; section we&#8217;re talking about the place where users submit reviews and scores, then I guess it depends on what you value in a review. Some people think review scores should converge on some hypothetical One True Score that accurately reflects the value and quality of a game. Some people (me kinda people) aren&#8217;t as interested in scores, but instead value the opinions of specific critics. Other people think of game critics as corporate shills and sellouts and only &#8220;true&#8221; gamers can be trusted.<\/p>\n<p>On one hand, users often write little text blurbs to go with their ratings, and they often come out like this: <\/p>\n<p><!--more--><\/p>\n<blockquote><p>Best game ever<br \/>\nmuch better then GTA IV<br \/>\ni played it last night and loved it<br \/>\nlooking forward to the online<br \/>\nThe driving physics are worlds better than they were in GTA IV. I keep finding myself driving around aimlessly because it&#8217;s so fun to drive\n<\/p><\/blockquote>\n<p>(Actual review, as it appeared. Score given was (of course) 10\/10. I&#8217;ve stripped the name to protect the incoherent.)<\/p>\n<p>I&#8217;m pretty sure the formatting is a result of ineptitude and not some kind of avant-garde attempt to present a review in free verse. In any case, this is a pretty good example of what users have to say.  It&#8217;s easy to look at stuff like this and conclude that this person&#8217;s post is worthless because they can&#8217;t think or communicate clearly and their criteria for appraising a game are murky and somewhat suspect. <\/p>\n<p>User reviews are also prone to shenanigans. People who hate a franchise will tend to score titles at ZERO to &#8220;balance out&#8221; all those brainwashed sheeple who like the game. And people who like the game will give it a perfect 10 to balance out all those haters. And then there are review-bomb protests. And then there are occasions of ballot-stuffing by marketing. User scores are &#8211; by necessity &#8211; scores by anonymous people, which means that they&#8217;re every bit as accurate as any other online poll.<\/p>\n<p>So user reviews are simplistic, barely literate, and completely unreliable. I guess we&#8217;ll have to trust in professional reviews then?<\/p>\n<p>The thing is, most professional reviews are created by people who aren&#8217;t playing games the way consumers do.  They play more games, they have less control over what they play, they&#8217;re obligated to finish uninteresting games, and they&#8217;re often tasked with playing stuff outside of their area of interest. <\/p>\n<p>Example: I can&#8217;t stand modern military shooters. Call of Warfare et al. seem to be engineered to repel and offend me on every level. The cartoon machismo. The lack of player agency. The mixture of &#8220;realistic&#8221; geopolitical conflicts with childish black-and-white morality. The horrible dialog. The hyperactive roar of continuous gunfights with little variation in mood or pacing. I wouldn&#8217;t spend my free time playing these things, and I certainly wouldn&#8217;t pay AAA dollars for them.<\/p>\n<p>But if I was a professional reviewer, I&#8217;d probably have to. And if I wanted to keep my job I couldn&#8217;t just eviscerate the genre when the latest release of Sound &#038; Fury was dropped on my desk. I&#8217;d be expected to play the damn thing and construct a review that would be useful to the public at large. I&#8217;d be obligated to review the game for <em>other people<\/em>. And I&#8217;d have to do so in a week. And if I did like it I&#8217;d still be obliged to move on next week and play something else, so any further play would need to be on my own time, at home. (Which would probably mean starting over, depending on platform and how feasible it is to port saves.) <\/p>\n<p>If you put me in that situation, I&#8217;d probably fall into the habit of reviewing games the way a lot of journalists do: I&#8217;d value spectacle and accessibility over depth. I wouldn&#8217;t want a game to take too long and I wouldn&#8217;t want it to ask too much of me skill-wise unless it was in one of my favorite genres. How would I assign scores? Pffft.  I don&#8217;t know what military shooter fans like in a game, so I&#8217;ll rate on polish and novelty instead of getting tangled up in discussing tricky stuff like themes or tone. <\/p>\n<p>Some time ago I saw a chart (I can&#8217;t find it now but I&#8217;ll add it if someone will link it in the comments) showing how user Metacritic scores lined up reasonably with professional reviews until a few years ago, at which point they began to diverge sharply. Big-name publications continued to give top marks, while users rated them lower and lower.  You can read this either way:<\/p>\n<ol><LI>Reviewers continue to rate based on gloss and spectacle while existing fans insist on narrative quality. OR&#8230;<\/p>\n<li>Professionals continue to be even-handed while users become increasingly entitled and demanding.\n<\/ol>\n<p>Does it matter? BioWare&#8217;s recent transformation from building games focused on lore and theme to games focused on gameplay and action was not accidental and it had the intended effect of bringing in lots of new fans.  Who is the professional reviewer supposed to review for? Should they guess how hard-core, old-school fans will feel about a game, or should they try to guess how the general gaming public will respond? You likely know which camp you&#8217;re in already. <\/p>\n<p>Getting back to the question of user reviews: Pros reliably rate games on a very narrow set of criteria, while users tend to review from the gut. The latter channel is a noisy one, but I&#8217;m of the opinion that more data is always better. <\/p>\n<p>One user review is probably useless, but given enough data points we can construct a very rough but useful picture of how well a game is connecting with the hearts of gamers. <\/p>\n<p>EDIT: Originally I said the example reviewer had a &#8220;worthless&#8221; opinion, when I was trying to say their <em>review<\/em> is worthless. (Their opinion is just as valid as anyone&#8217;s.) <\/p>\n","protected":false},"excerpt":{"rendered":"<p>The question was put to me on Twitter: @JimSterling @DanStapleton @shamusyoung what do you guys think of the &quot;user&quot; section of Metacritic? &mdash; Shri (@ResonanceSD) November 26, 2013 Interesting question. I don&#8217;t go to Metacritic very often, but assuming by &#8220;user&#8221; section we&#8217;re talking about the place where users submit reviews and scores, then I [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[120],"tags":[],"class_list":["post-21653","post","type-post","status-publish","format-standard","hentry","category-videogames"],"_links":{"self":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/21653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=21653"}],"version-history":[{"count":0,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/21653\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=21653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=21653"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=21653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}