Sessler’s Soapbox: Innovation Vs. Numeration

By Shamus
on Dec 5, 2008
Filed under:
Movies

Adam Sessler of G4TV has a regular video op-ed titled “Sessler’s Soapbox”. He’s a reviewer who is obliged to give numeric scores to games, and here he clearly comes out against them.


G4TV.com

But what really caught my attention was the part at the beginning where he talks about the difficulty of a game and how it does (or doesn’t) figure in to a review score. This is one of those subjects that calls for a thousand words or none, and I don’t have time for a thousand words. The best I can manage right now is to gesture wildly in Adam’s direction and say, “What he has said is significant.”

Enjoyed this post? Please share!


is a programmer, an author, and nearly a composer. He works on this site full time. If you’d like to support him, you can do so via Patreon or PayPal.

201131 comments. Hurry up and add yours before it becomes passé.

From the Archives:

  1. mark says:

    I don’t have time for 1000 words either, so I AGREE.

  2. qrter says:

    He isn’t really saying that the difficulty of a game figures in a review, he’s saying how it’s hard for reviewers to seperate where their own ineptitude ends and bad gameplay begins, how it’s hard to distinguish between the two. It’s a subtle difference, I’ll grant you, but a difference nonetheless.

    It’s an interesting point, though and it (again) shows how much growing up some parts of games journalism have to do – that said, I always used to be part of the group of gamers who say they want to see more indepth reviews (critiques, some would say..) but I’m not so sure anymore. I mean, what’s the point of having sophisticated reviewers when most games still are stuck at the Hollywood action blockbuster level? The whole industry could do with a good hiking up of its pants..

    (And yes, review scores are ridiculous but I’m guessing most readers of your blog will say that.)

  3. Matt says:

    I agree that it shouldn’t factor into a review unless the difficulty makes it unplayable, I feel difficulty should be mentioned at least. I’ve become somewhat of a casual gamer, and I don’t feel like wasting my time retrying that same damn level over again for the dozenth time just so I can see where the story goes. I don’t want my games to be a complete pushover, but I hate completely frustrating games.

  4. TickledBlue says:

    You could have done a picture… or do they take even longer?

    On a more serious note though, I agree that the current format for numerical scores for game reviews is ineffective at best. The problem is that there is no common ‘ruleset’ for scoring in reviews – not to mention all the background noise of ad revenue for game review sites and the 7 – 9 scoring principle that still boggles my mind (why have a 10 point scale if you’re only ever going to use 3 numbers – and don’t give me that decimal point crud). So I have no way of knowing if a 9 for one game means its as good as another game with the same score.

    I disagree that removing scores will force people to read the review, it might help them get past the first paragraph but it is well established that people (and I’m referring to the general internet population and not specific communities within it) just don’t read that much online. They want their information to be quickly accessed and easily digestible, just what scores give them.

    Personally I long for a more ‘if you liked this game you’ll also like..’ style of system. One where the reviewers preferences are clearly indicated as part of their profile and the game itself is linked to other games in a web of similarities. I also like the games being played right now sections a few of the podcasts I listen to have. You get a round table discussion of the game, often with enough of a focus on what is good and bad from multiple points of view for you to be able to make up your mind.

    But you know what? I’m glad that reviews are the complex mess that they are. I’m glad that we still don’t rely on them to the exclusion of all else, because if we did the only games we would ever play are those that everyone else likes. We wouldn’t take a chance on the poor scorers or even those that were never reviewed. Sure I’ve bought a few duds but there are also one or two gems that I would probably never have picked up otherwise.

  5. Ben says:

    That guy’s accent was strange and annoying but what he was saying was sooooo good. I really respect him for his views. I suddenly realised how ridiculous it is to rate a game with a numerical score. People don’t rate paintings with scores do they?

  6. DavyRam says:

    A related point to that of a game’s “quality” varying according to the player’s skill is that there are a lot of factors that will affect the quality of the game experience based on the circumstances the game is being played in. This is most obvious with PC games. Lets say you have a game like NWN2 which requires massive system specs relative to the graphics. How much does this ‘cost’ a game? Take it from 80% to 70%? To someone who meets the specs, assuming the quality of the graphics is already factored into the score before the technical difficulties, it is an irrelevancy, they enjoy the game fully, it’s an 80%. To someone who doesn’t, the game is unplayable, it’s not a 70%, because it’s virtually unplayable. The 10% off isn’t going to reflect anyone’s actual game experience. Or the fact that the PC Fifa 2009 was incompatible with my specific laptop graphics card line, a fact that was not mentioned in any review because it applied to so few, but was crippling to this specific buyer. Or for a console example, when Halo 1’s LAN multiplayer was highly praised in reviews, since LAN facilities are readily available to pro game reviewers – but was a less accessible to most gamers.

    Essentially, game reviewers rank a game based not only on its’ “artistic” content, but how well it interacts with the medium it is on or the capabilities of those it is being reviewed for, like blasting Hamlet because the quatro editions were badly printed or because it is inaccessible to German speakers. The fact these factors vary so much make numeric ratings far less appropriate for games than a finished, singular piece of art like a movie – and even then they’re controversial enough.

  7. Sashas says:

    Is there any way to read the text of this review? I would love to watch the video, but I have no sound on my computer.

  8. Ozy says:

    The key difference between good difficulty and bad difficulty is that good difficulty makes it feel like it was your fault when you die. A game can be incredibly deep difficulty wise and still have good difficulty, but nowadays legitimately hard games are usually indie titles which harken back to when game makers sold to arcades, and had to balance the desires of players, who did not want to die, and arcade owners, who wanted players to die, a compromise they achieved with quality skill-based gameplay.

    Now people act as though they are entitled to beat the games they buy, but don’t actually want it to be easy. Games makers achieve this compromise with fake difficulty that makes the game “hard” but which any incompetent player can overcome if they just stick with it long enough and have a high tolerance for stupid design.

    This is why I’ve mostly stopped playing mainstream games and spend what time I do use for gaming on the Touhou series of bullet-hell shooters.

  9. Robyrt says:

    @Ozy: Yes, I do expect to be able to beat the games I buy. Being too easy is less of a sin than being so hard I give up. Otherwise, I feel I wasted my money – I didn’t get to finish the story, and I probably spent a couple hours getting myself angry instead of having fun.

    The proper solution is to have varying difficulty levels, and to provide an answer to every situation so that it always seems “fair” when the player dies. God of War does this very well, with an “easy” setting that can be completed without the triangle or circle buttons, and a “very hard” setting that gives you less XP for killing tougher and smarter bad guys.

  10. Nutter says:

    It’s only really recently that I’ve started to actually listen to reviewers again. When I was younger I got hooked up over the scores and went off reviewers altogether when I realised that when they score a game it’s based on how they felt, which probably isn’t going to be how I feel. Now being a little bit wiser I’ve realised that a good reviewer isn’t one who I even have to agree with. It’s one that tells it how they see it in enough detail to allow me to make an informed decision.

    The likes of Shamus and Zero Punctuation, are good reviewers not because they avoid scores all together but because they give the game play elements enough detail to give you a good idea of what your going to find when you play the game.

    So when Zero Punctuation complains about the way a game seems to be shovelled full of long drawn out dialogs. I know that the game is going to at least have some kind of background story, which is something I like and enjoy so where he complains and dislikes, I know I’m in for something more than a run around senselessly blowing stuff up, not that I’m adverse to blowing stuff up I just like have a little reason to my madness.

    And when Shamus complains about the interface, I can just about ignore what he has to say on the matter, I’ve never had any problems with adapting to controls. Occasionally they’ll seem weird but it’s to the same degree of weird as playing Command and Conquer all day and then switiching to Warcraft and I’ve yet to see anyone complain about the differences there.

    I’m also not total adverse to scores though, scores can help give a general feel of how something going to go. For example, if Shamus was to start giving numbers to reviews and published them at the top of the articles. If he reviewed a game that I wasn’t particularly interested in and he gave it a low score I could skip over it without much cause for concern, go on and do something that I am interested in.

    However if he gave a game that I thought I wasn’t going to be interested in a very high score I’d be intrigued and read the whole review and make my judgement from there. The same could be said about games I’m interested in and if he gave it a very low score.

    It’s not that scoring is bad, it’s readers that allow themselves to make snap judgements based on them with a blatant disregard for all other information and that, which I can appreciate this, it’s difficult to give a number to something, especially if its one number that is supposed to cover all for all aspects.

  11. Kevin says:

    I wasn’t even going to watch that, but I went to the G4 website and saw that the review had been given 5 stars by the viewers. Hey! It was the best review evar!

  12. Yahzi says:

    Kevin: I see your irony and raise you two oxymorons! :D :D :D

  13. Eric says:

    I figured you would like this.

  14. krellen says:

    Sessler’s difficulty question is basically “Is this game too hard or do I just suck?”

    The basic problem here is expectations. If you’re the “God of War”, some badass Spartan murder machine, you’d expect to perform like a badass Spartan murder machine, even if the player personally sucks at the game. Most games nowadays have “professionals” as your character, so you expect to be good at things you are asked to do, so when the game difficulty makes you unable to do those things, there is a mental disconnect and the game’s difficulty is frustrating.

    If, on the other hand, you are playing as, say, a plumber trying to save a princess in some magical fantasy world, a bit of trial-and-error mistakes in the beginning is to be expected. After all, your character is new to all this as well, so the expectation of competence and excellence isn’t there.

    Sessler is also the experienced and principled part of X-Play. He’s been there for its entire run, and I’m not surprised to hear that he’s less than happy with the direction game reviewing has gone in.

  15. Kdansky says:

    I give him a 12 ouf of 13.

    what?

  16. ArcoJedi says:

    The best option is to skip reviews altogether. Change the industry around.

    For instance, if you want to purchase a video game, you simply send a blood sample in a self-addressed stamped envelope with your payment and they send you back a disc with games on it that are matched with you based on your genetic structure.

    Why doesn’t anyone like my idea? I may have gone too far with that example…

  17. DaveMc says:

    I have to admit that I found this commentary a big step up from his work on X-Play, where I found the writing pretty silly most of the time (which may or may not have been his choice, to be fair). On the other hand, I thought the *format* of the reviewing on X-Play (which I haven’t seen in years, so forgive me if I’m out of date) was *perfect*: they did audio commentary over video of the game actually being played. That seems like an ideal way to convey game reviews: when they say something about a gameplay mechanic, you can see it being implemented right there on the screen.

    (Not that reviews based on screen captures with hilarious captions aren’t good, too.)

  18. Telas says:

    Long story short: Read the f’n reviews, and know your reviewers.

    If you’re lazy enough to think that some reviewer will tell you what you like with a single number, you deserve your disappointment.

    Let’s use movies… If I see that Roger Ebert gave “The Matrix” three stars, then I might rent it, or just catch it on cable some day. But if I actually take two and a half minutes to read the review (yes, I timed it), then I discover that Ebert felt it was a beautiful movie that challenged him intellectually, but ultimately disappointed him with what he felt was a clichéd ending.

    I know that Ebert sees everything in terms of ‘films that have come before,’ and that I haven’t seen all of those films or been exposed to all those clichés. Suddenly, it’s a damned sight different than “three stars”, and now I want to watch this stylish and intelligent sci-fi/zen thriller.

    (FWIW, I disagree with Ebert on the ending; “The Matrix” ended exactly as it had to. And the sequels should never have been made.)

  19. Sean McCulloch says:

    While I agree in principle with not liking the numerical numbering systems, let me throw out an opinion that comes from the “I don’t spent tons of time reading reviews and stuff” side of the fence.

    Let’s say that I like RPG’s, especially tactical RPG’s.
    (Well, not all of them. But that’s another issue). Every couple of months I finish whatever game I was playing, find myself with some free time, and say “Gee, I wonder what I should buy next?”

    My strategy has been to go to Metacritic, click “Advanced search”, type in the systems I own, and the genres I like, and see what comes up. Then I go in descending order of score until I find a game that looks cool, and buy it.

    “Find a game that looks cool” for me usually means that I read some of the reviews of some of the games to get a feel for what kind of game it really is, and how it plays. But the fact of the matter is that I’m usually only doing that to a couple of reviews that give the games high scores.

    So my question is this: If I don’t want to spend tons of time reading long reviews continually, what’s the right process to find good games?

  20. Kevin says:

    I take the exact opposite viewpoint… I think all reviews of anything should provide a numerical measure of approval on a ten-point scale. But they should only use the numbers seven through ten.

    If there’s one thing I’ve always wanted to know, it’s how the Mona Lisa compares to The Last Supper numerically.

    Oh, and Telas: I don’t think the Matrix sequels shouldn’t have been made. I just think they should have been well made.

  21. krellen says:

    If I don’t want to spend tons of time reading long reviews continually, what’s the right process to find good games?

    Find a reviewer whose taste in games match yours fairly closely and get games they favourably review. Then you only have to read one person’s reviews.

  22. eloj says:

    Well, Adam Sessler might be good, but he’s uglier than Mogan Webb — just totally unrelatable — so I’ll reject his words out of hand. I give him a three out of five.

  23. Kerin says:

    Man, I do not like Sessler one bit but I can recognize that he’s saying something important here. I-

    hrewesfds. Okay, I can’t believe that the first reviewer I’ve seen making this point is Adam Goddamn Sessler. Now I have to watch and see if he says anything else good.

    Thanks a bunch, Shamus.

  24. JKjoker says:

    i love how gta4pc is getting 9, 9.5 an 10s while ppl are screaming for blood for getting an alpha release, even steam is giving refunds

  25. Magnus says:

    Are reviewers indoctrinated by the hype machine? are they too close to what they review, and cannot see the wood for the trees?

    Using a five star system, or school-grade marks A-F are the most understandable, after all, what seperates a 89/100 game from a 94/100? (except 5 arbitary points of course.)

    Reviewers are giving their opinion, and the majority don’t share my opinion, so if anyone knows a good site to get reviews for PC games, mostly RPGs, then let me know, as this is my preferred genre.

  26. Hal says:

    First, this might be the first thing I’ve ever heard come out of Sessler’s mouth that didn’t make me want to pop him one. As for his points . . .

    1) I agree that difficulty is a hard thing to measure. I’m terrible at sports games, so I wouldn’t know a good football sim if it bit me on the nose.

    2) I think the more interesting point, that he only briefly touched on, was the disincentive for innovation. Mirror’s Edge, was, supposedly, a game that tried to do something different. Given the glut of sequels we see out there anymore, this is something we ought to encourage, but the execution didn’t turn out well. Does the poor review mean that people won’t buy the game and thus tell EA that we want to play endless sequels and remakes?

    This might be a larger problem with paying attention to the metascore: Players don’t end up seeing that there was an interesting game that just didn’t quite get off the ground, while the producers only see that their experimental game didn’t sell nearly as much as Shooter Guy 7.

  27. MadTinkerer says:

    Holy Cow, Adam Sessler has grown up.

    About two years ago, I lost access to G4/TechTV because money was tight and we switched to basic cable. I remember Adam as a funny and entertaining reviewer, but he came off as a bit of a goofball who didn’t have anything serious to say. Kind of like Gabe (the character, not the artist) from Penny Arcade.

    Now I have quite a bit more respect and look forward to more from him.

  28. Felblood says:

    Where praytell am I supposed to find a reviewer, whose tastes match mine?

    And how the goomba is he supposed to play enough games that I can fill out my gaming diet on nothing but the ones he liked?

    I play a lot of games, but in the past few months it’s gotten harder to find interesting reviews that address real concerns and concepts from the game. The average professional review is fast becoming as content-free and blatantly inoffensive as a political campaign.

    Genre doesn’t matter to me much, if it’s well made, I enjoy just about anything, but I have no tolerance at all for buggy games or stupid design decisions. If your game has a single bad mechanic, hamstringing gameplay, I will loathe it almost as I loathe you. (Fie upon Dungeon Lords! For encasing such wonderful combat in such a completely broken quest system, Bradley should be flayed. You can never be sure if it’s just poorly scripted, or if the game is eating it’s own hair again, until you finish the quest, –and if the game is broken, I hope you staggered your saves.)

    In short, perfection is the only thing I ask from a game. It doesn’t have to do anything specific, but it must do it well, and without a single crippling flaw. (He’s a good hound dog, loyal, awesome endurance, well trained, excellent nose, good bay– too bad he ain’t got any legs no more.)

    A game that does that gets a 6/10 from me. For being both passable, and above average.

    I suppose Shamus and Yhatzee are a good starting place, but I need/want information on more games that those two could review between them, if they gave up sleep and human contact.

  29. I still remember Daikatana getting an 18 out of 100, or Swamp Buggy Racing that got somewhere in the 20s as I recall.

    There are some problems. Reviewers often do not have time to actually learn to play a game, and often don’t play it through. I blame those people for the reviews of Hellsgate London that got me to buy the game, and my initial experiences for my mistakes in recommending it. It is a game that plays well for the first five or six levels and seems to have promise.

    Next, if you’ve played games in a sequence, you may very well segue in to the next game much better (e.g. if you played Wing Commander I, Wing Commander II was very intuitive for you). That often gives sequels an advantage.

    But the real issue is the rabidness of the fans. Rabid fans are a real problem, and they, and the marketing force of some companies, form a toxic mesh.

  30. Zaghadka says:

    Judging by his grasp of rhetoric and language, it’s clear that all I will ever want from this man is a number. SURE, he gives us a few possibly salient tidbits in his monologue, but mostly it’s sandwiched in layer upon layer of self-contradictory crap.

    It’s like watching someone chop up your favorite epistemology thesis to better shoot it in your face with the confetti cannon.

    My simple response: If he wants me to read his reviews, he’s going to have to take a writing class.

    Where as a writer can you tell if the job is hard, or you just suck at expressing yourself?

    Sorry for the snark. I’m not feeling charitable.

  31. Tesh says:

    So long as the culture of gaming is steeped in ADHD “tl;dr” instant gratification nonsense, a number is going to be all that processes through the brain mass.

    Even the “sound bite” culture of two sentence reviews does a disservice to customers and developers. The only way through the quagmire is to have educated and intelligent patrons and developers, but the industry has fostered the lowest common denominator for a long time now. “Reap what you sow” and all that.

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>