Study: Gamers are more educated, more social than the people who make fun of them

By Shamus Posted Sunday Sep 7, 2014

Filed under: Random 121 comments

The Washington Post has this story under the headline “Study: Gamers are more educated, more social than the people who make fun of them“. Now, that sounds pretty good to me. I love scientific studies that prove I’m smart and nice. But I also thought the story felt a little ego-stroke-y, which made me suspicious. Who did this study, who paid for it, and what did it (actually) reveal?

The WP talks about the study being done on behalf of Twitch.tv. Shamefully, they neglected to link to the data or even to the research site. Once again, I have to condemn the way news sites write news stories for the web as if they were writing for print. There’s no reason not to link this stuff.

Digging a little deeper, I found that the group behind the study is Life Course Associates. They are not a research institute. They are a marketing firm. The WP story did say that Twitch.tv funded the study, but the crucial context we were missing is that they funded this study as part of a marketing campaign. There’s a huge difference between a company giving a grant to (say) a university to do a study, and a company paying a consulting firm that does research as a form of marketing.

Now, that doesn’t mean that the research is automatically false, but it is crucial context that should be included in the news story. This report is exactly the sort of thing a marketing team is after:

  1. Headline-grabbing conclusion that generates buzz.
  2. An effort to change of the public perception of the demographic they were hired to market to.

My last complaint is that their definition of “gamer” is shockingly broad: “anyone who has played a game on a digital device in the past 60 days”. When I hear “gamer” I think of someone who is engaged with the culture on some level. Maybe you read forums, or reviews, or you network with other players via your friends list, or you watch streamers or Twitch, or Let’s Plays on YouTube, or… something. Calling someone a “gamer” because they play Flappy Bird once every other month is like calling me a NASCAR fan because I drove a car this week. This definition is so broad that it borders on useless. Note that I’m not trying to condemn people who aren’t “real gamers” or whatever. But if the study is supposedly to find out about “gamers” then broadening the definition to include such a massive chunk of the population is going to muddle the results. If I want to study people who live in the city, then including “people have visited a city in the last month” is clearly spreading the net far too wide.

I don’t have a background in science or statistics, so I can’t comment on the veracity of the data or critqique how it was collected, but I’d be surprised if it withstood the normal standards for scientific rigor. In any case, when a marketing firm releases a study that tells you what you want to hear about yourself, you might want to be skeptical.

And if you work at the Washington Post, you really should include these details with your story.

 


From The Archives:
 

121 thoughts on “Study: Gamers are more educated, more social than the people who make fun of them

  1. Pastacat says:

    My favorite way to define gamers is via self-identification. If you consider yourself one then you are one.

    1. Aldowyn says:

      I like this definition, actually. Works for a surprising number of other things as well.

      1. Rick says:

        I mean, as long as you’re fine with the inevitable (label of) GINOs (Gamers In Name Only).

        1. Pastacat says:

          That would happen anyway.

        2. NotDog says:

          We should be fine as long as we don’t gender it like what lead to the “fake geek girls” thing.

    2. Daemian Lucifer says:

      Self identification doesnt work on its own,actions are important as well.

      For example,if I were to identify myself as a japanophile because Ive watched a few anime,one of which was subbed,that wouldnt mean I really am one.

      1. Canthros says:

        That sounds like the Dunning-Kruger effect.

        1. Shirdal says:

          From what I read on Wikipedia, the Dunning-Kruger effect applies to skillfulness, which I don’t think applies when talking about watching anime or performing other unskillful tasks.

          To me it sounds more like simple old-fashioned pretentiousness (it’s so rare that I get the opportunity to use this word properly).

          I don’t put much stock in labels in the first place, but they have their uses.

          1. Chamomile says:

            Dunning-Kruger can also apply to depth of knowledge in a field. Anime isn’t the first thing that comes to mind when you say something like that, but it is a field you can have depth of knowledge in.

      2. Ah, but “-phil-,” used as a suffix or prefix, are derived from the Ancient Greek word philia (φιλία), “love, affection”.

        You can most certainly be a Japanophile if you like or have positive feelings towards the people and/or the culture of Japan. You could more specifically be an Animeophile.

        I think expertise only enters the picture if you put suffixes like “-ologist” on the ends of nouns.

        1. MichaelGC says:

          I’ve decided to re-train as a philologist. :D

    3. Vegedus says:

      Doesn’t work well for statistical analysis, though.

      1. Abnaxis says:

        Depends on what kind of analysis you want to perform.

        I asked my wife–post-graduate in statistics as well as Sociology/Political Science–and this is what I cam away with.

        From a design perspective, the way you would/should classify a “gamer” depends entirely on what types of questions you want to answer. Asking people to self-identify is not good if you want hard demographics, because other social forces besides gaming habits can drive the answer. If your identity as a “gamer” conflicts with another identity (likely true for e.g. females, minorities, or elderly) you won’t self-identify, even if you really should.

        Moreover, your own perceptions of others around you will shape your answer as well. This is why you see the number of people who identify as “middle class” vary even as their economic status doesn’t change–if they feel like they are less prosperous or less economically secure than their neighbors, their actual financial situation takes on a different role in their self identification. This applies to “gamers,” in that you might spend a lot of time playing TF2 than the average self-identified TF2-playing “gamer,” yet not consider yourself a “gamer” if you’re friends with people who play DOTA2 professionally. Sure, you like shooting, but you’re not that into it.

        At the same time, there are situations where self-identification is a more appropriate delineation. Any study interested in “gaming culture”–like, say, if a researcher wanted to study the relationship between the gaming community and games journalism in light of recent events–would certainly want to use self identification as a primary indicator for “gamer-ness”.

        The funny thing is, the measure the study quoted used is probably completely appropriate for the purposes of marketing. Marketing firms are generally interested in “people who would buy our product if it was marketed to them right,” which includes basically everyone who’s played a video game in the last two months.

        The problem is, they aren’t using the study to target marketing, but rather trying to extend it’s findings to a greater social context. That doesn’t work with the way the study was designed. They might as well have summarized their findings with “people who own smartphones more likely to be socially active and have more education than people without smartphones.” Well, duh.

        A more appropriate method to proceed would probably to use some sort of classification analysis–basically, search the data you have for respondents’ gaming habits for clusters, and use those clusters to decide how you are going to categorize the populace (that’s the gist, at least). That’s a lot more complicated, however, with a lot of fiddly mathematical pitfalls, and leaning a lot more heavily into the theoretical side of statistics than a marketing firm is likely to tread. However, it does give you a set numeric cut-off for “this is a gamer, this isn’t” which you need to run demographics, which is primarily what the study is concerned with.

        1. Vegedus says:

          I was more going for the idea that it’s relatively hard to poll a thousand people for whether they’re a gamer and what their gaming habits are, while it’s pretty easy, once you got permission, to simply pull data from millions of smart phones or accounts in specific games. You can make statistical signifigant results with a lot of polling, but it’s simply impossible to work at the scales, and therefore also the details, that you can by simply harvesting the data that’s already out there in some form. But sure, that too.

          1. Abnaxis says:

            I suppose that’s true, but now you’re getting into areas of scientific ethics that I’m not 100% comfortable with. Like, I know people agree to data collection in EULAs with no reading or regard, but I’m not really a huge fan of utilizing that data for a study when the subjects don’t know they’re being studied.

            Plus, there’s plenty of people who play games that don’t do it on a device that tracks their habits automatically, and for those you’re going to have to administer a survey. It’s not all that more difficult to ask if someone considers themselves a gamer than it is to ask when the last time they played a game was. Otherwise, yes, you can get much more resolution in your estimates from the larger data-set, but you are introducing a bias that makes it unreliable as a tool for generalization.

    4. MadTinkerer says:

      So I’m not a gamer, even though I own thousands of games? I demand that I rethink this assertion!

    5. Wide And Nerdy says:

      As Daemian and Mad Thinker point out, there are problems with this definition. Some people might include themselves who don’t really fit anybody’s definition of gamer while others might exclude themselves even though they fit most people’s definitions.

      For a perfect and timely example, Campster just declared on Twitter that he burned his gamer card and renounced his gamer cred. Yet the guy plays tons and tons of games and writes and posts reviews in his spare time (its not even his primary occupation). By most definitions he is a gamer whether he wants to be considered one or not. But he distances himself from the term because it means something specific and ugly to him. Yet there was a time when he would have appreciated the broadness of your definition (he generally argues for inclusive definitions of the term “game” as well).

      If the term gamer should have a broad definition then we could probably use other terms that are more specific for types of gamer (not to exclude people but because they’re useful for discussion.)

  2. Rick says:

    On the bright side, whenever someone says that studies show video games cause violence, I can direct them to this study as being “equally valid.”

  3. urs says:

    Meh. Whatever the article, isn’t it about time that the word and notion “gamer” just goes away? I am not a gamer. I am not a movier, not a seriesr, not a musicr, not a comicr, hell, I’m not even a foodr. Yet, I take an interest in all of these things, I have formed opinions, developed specific likes and dislikes, mostly based on thoughts about them, … Don’t you?
    Why, *still*, “gamer”?

    1. M. says:

      There are lots of words which mean “I’m the sort of person who likes this thing.” Foodie, moviegoer, Trekkie, the list goes on. I don’t see a reason to be opposed to this one in particular.

      1. urs says:

        Yeah, but. Here’s the thing: I happen to like Star Trek. To the point of looking it up in the TV guide while at my parents to make sure I can catch some episodes (no StarTrek at home) (and I always catch the diplomatic missions, never the stuff where Geordi and Data encounter a Transporter mishap). But I am not a Trekkie and noone would label me one. A Trekkie dresses up as Neelix and goes to conventions. In videogames, there is no distinction like that. At least, without bringing in the Hardcore- prefix.
        Our medium should by now have matured past the point where everyone engaged with it – to whatever extent – falls under a label.

        1. I thought a Trekkie was someone who was a fan of the show with a deep knowledge of the canon, the universe, tech manual stuff, etc.

          Dressing up in the uniforms and going to cons falls into cosplay which I think deserves an addition to the label of its own: Trekkie cosplayer.

        2. Klay F. says:

          Except people who watch a few movies a year are not the same thing as cinephiles. People who read maybe a book a year are not bookworms. A person who plays Farmville and Bejeweled exclusively is not a gamer.

          Like it or not, people like to base their identities on their hobbies and passions. You don’t think the term describes you? Nobody is forcing it on you. Don’t use it. That you deem the term unacceptable for other people to use to base their own identities is ludicrous.

          1. I think it also has to do with who you’re measuring against. If the most avid player of games in your group IS the Bejeweled nut, then they’re the “gamer” of the group.

            Every circle might not have a consumer of vast swaths of text, but at least one of them is probably considered the Bookworm when compared to the others. :)

            1. Klay F. says:

              Thats great and all, but its pretty meaningless to say that somebody is a thing without that person also agreeing that they are that thing. And by “pretty meaningless”, I mean “the most meaningless thing I can think of at the present moment.”

              In the realm of pointless exercises, deciding what someone else’s identity is based on ranks right up near the top.

              1. Yet you know what a cinephile is or is not? Isn’t that your assertion?

                1. Klay F. says:

                  That isn’t really what I was getting at, though I obviously used the wrong words. I’m just using these oh-so-fun analogies to draw a comparison. If you want to call yourself a cinephile, you go right ahead. That obviously isn’t going to stop me from thinking you’re full of shit, but then my opinion on your identity is worthless and you should treat it as such. In the end we’re all full of shit.

                  The OP wants the word “Gamer” to go away over the simple fact that he doesn’t self-identify as one, which is ludicrous. He wants gaming to “mature”, whatever that even means, yet ignores the fact that cinephiles and bookworms exist, and that people do indeed self-identify as these things.

                  1. Daemian Lucifer says:

                    ” In the end we're all full of shit.”

                    *flush*

                    At the moment,Im not.

              2. Daemian Lucifer says:

                “Thats great and all, but its pretty meaningless to say that somebody is a thing without that person also agreeing that they are that thing.”

                Except there are plenty of cases when its not.If I say “I dont believe in god,but Im not an atheist,no matter what you say”,then Im just being against the term for the sake of it,and you have every right to call me an atheist,no matter how much I object to it.If someone knows all the minutia of a narrow field,but that is not their job,they are a geek,no matter how much they object to it.And if someone is playing a bunch of video games for hours every day,they are a gamer,no matter what they say.

        3. Daemian Lucifer says:

          ” A Trekkie dresses up as Neelix and goes to conventions. ”

          No self respecting trekkie would dress up as neelix.

          1. They might dress as Neelix with a really huge phaser wound in his chest. Maybe one of those iPad costumes where it looks like there’s an actual hole going through his guts.

            And they’d have their photo taken with Ethan Philips, even if they had to pay Klingons to hold him in place for the picture.

    2. Daemian Lucifer says:

      Oh,so youre a hipster./jk

      1. DIN aDN says:

        But can you blame them with all this fantastic hipst available to us nowadays?

      2. urs says:

        Nah. I don’t dress like me when I was 18.

      3. “Somebody under 40 who wears clothes”

        1. I noticed hipsters before there was an agreed-upon label for them. I just called them “people who dressed like newly-regenerated Time Lords.”

          1. Daemian Lucifer says:

            So what youre saying is that you were spotting hipsters before spotting hipsters was cool?

            1. Was it ever cool, truly?

              It actually was before the word “hipster” became a revived label (and/or epithet). Most people seemed really annoyed with them at the time, even without a name to stick to them. Well, people who didn’t run vintage or secondhand clothing stores or record shops.

    3. BeamSplashX says:

      Someone responded to me when I complained about this with “it’s shorter than ‘a person who plays video games'”.

      It is, but when does anyone use that phrase, outside of responding to “What is a gamer?” You can call people who read a lot “readers”, but people don’t say “I’m a reader!” much.

      Actually, it’s kind of strange that you hear phrases like “readers of Harry Potter” but not “gamers of Call of Duty” (clearly the harry potter of video games).

      1. BeamSplashX says:

        hi i’m sid and i love quotation marks apparently

        1. Daemian Lucifer says:

          So youre a quoter.

          1. Humanoid says:

            Sid the Quoter Kid. Question is, what metric of quoting would be represented as “one Sid” of quote?

            1. Daemian Lucifer says:

              My guess would have to
              Be that it is equal to
              One haiku poem.

      2. tmtvl says:

        Call of Duty the Harry Potter of video games? I don’t like either of those things, so it seems perfect!

        And Half Life is the Twilight of video games.

        1. PopeJupiter says:

          Here lies tmtvl. He/She was a good man/woman. Well-loved by his/her husband/wife, children, and parents. He/she was murdered late Sunday night by a rabid pack of “Half Life” fans after comparing the beloved video game franchise to the young adult book series “Twilight”. He/She will be missed deeply.

          1. 4th Dimension says:

            Correction here he “mostly” lies. Since he was chopped to bits by the same angy mob and his remains were used to paint in blood a lambada 3 sign in wain hopes it would appease the dark deity GabeN.

      3. Akri says:

        “You can call people who read a lot “readers”, but people don't say “I'm a reader!” much.”

        But they might say “I’m a bookworm.”

        1. Timelady says:

          Or ‘avid reader.’ I hear people self-identify as ‘avid readers’ a fair amount. Or ‘moviegoer.’ ‘Film buff.’ ‘Knitter.’ ‘Runner.’ Music lover? I don’t run in music circles so much. But I think it’s fairly safe to say that people self-identify with their hobbies all the time. And I really do think it depends on who you hang around the most, and the type of people you’re likely to run into, which terms you hear the most.

          1. Felblood says:

            Adjectives! That’s what we need!

            –and not useless ones like “hardcore” is means something slightly different to every person. For fun: put a happy hacker and a lifetime COD player in the same room, and ask them to prove which one is more hardcore. Obviously they will both think that the presence/absence of 3D rendered graphics should count for something, but what, for who and how much?

            The thing is, we have perfectly serviceable adjectives with reasonably consensus on their meanings, that can be further modified by degrees (and are therefore fairly quantifiable).

            Old-School Gamers might not agree on why the old ways are best, but you can get an idea of how old-school someone is by finding out when they thought things stopped being good. (I’m 16-bit era old-school).

            Novelty Driven Gamers want to play something new and different. If the thought of playing Call of Madden 2020 causes you physical pain, you’re at least a little novelty driven. This is often seen as a negative trait, but the “Been there; done that” attitude has a strong following in gamer culture, and woe unto the developer who neglects them, for they shall reap small profits.

            Premium gamers value their time. The less patience a player has for “grinds” or part of a game that aren’t fun and just fill out the 40 hour mark, the more Premium they are. The premium gamer doesn’t stand for a lot of BS, because the world is full of better games that deserve his time and money.

            Completionist gamers want to invest their time in a single experience and make that game a regular part of their daily routine. An easy baseline measurement would be the most time you’ve invested in a single game.

            Come up with enough of them and you could use these as chewy personality tests that disgorge horoscope style advice:

            Have you spent 100+ hours on a single game character?

            Where you born after 1990?

            Do you buy less than 5 games a year?

            You’re a Completionist Gamer!

            You want to wring the complete experience out of everything you touch, to the point that the game almost becomes a part of your self. You invest a part of your soul in your love for this world, and you may be a bit aggressive about defending it. You can have deep conversations with perfect strangers about the this world you have studied it so much (Just make sure they share your love before you corner them for a 3 hour lecture. ;)). Console marketing is targeted directly at the most vulnerable part of your soul, and you should be careful of that.

            Daily Fortune: Beware of men selling boxes, before Christmas.
            Lucky Numbers: 777 – 255 – 15 – 100

            Be sure to Share your results on Facebook, Twitter, Reddit, Pinterest and even Myspace. Then like and subscribe so we can show these numbers to our potential advertisers.

            Free business model, and a way to make the world a better place, right here. Go forth and multiply.

        2. BeamSplashX says:

          I suppose so. I thought bookworm was old-fashioned, but then again I don’t really read books anymore.

          1. MichaelGC says:

            I guess these days I’m a kindleworm.

        3. Ranneko says:

          Thing is that gamer has two meanings:

          1) Someone who plays games, or almost everyone these days. This is where you get stories about the average gamer being over 35, and 48% of gamers being women. The debates you run into here are as to if these people count, and of course if what they are playing are really games.
          2) People for whom gaming is a hobby. These are the ones that spend a lot of time/mental effort on games and the games industry. Oddly enough here the demographics are a bit more questionable, but if you go by the larger games like WoW, they do tend to skew heavily towards male and also younger demographics.

          You don’t have that in Books or Movies. You have readers (i.e. everyone actually reading) and bookworms, moviegoers and film buggs, etc.

          1. Daemian Lucifer says:

            Well thats why we have casual and hardcore gamers.We just havent decided which of the two should be the default gamer.

            1. Ranneko says:

              I don’t really think that casual and hardcore are very helpful distinctions, there are people that are really into game marked as casual, that invest a lot of time and thought into playing them.

              The terms are pretty vague and meaningless, I think that really it is better to look at it in terms of those who game as a hobby and those who just play games occasionally.

              I don’t know that either of them are a specific or cohesive enough group that you can target the whole very well for the purposes of game creators or marketers. I have friends who definitely would include gaming as one of their hobbies, but have a completely different taste in games, I simply cannot use them to judge whether or not I would like a game.

          2. Wide And Nerdy says:

            Point 1 has been bugging me for a long time. I don’t mind people saying that, based on a broad inclusive definition of gamer, all that stuff about gamers being older and gender split being about even.

            What bugs me is the equivocation that occurs when we then talk about specific types of games. If you look at just shooters, the data should skew very differently (though the ESA seems to be almost stubborn in its refusal to offer any data about demographics by genre instead just hammering on the “There are more adult female gamers than male gamers under the age of 18” in multiple reports).

            1. Ranneko says:

              They probably haven’t actually been collecting the genre information. It is a lot more complicated to do so since genres in games are both narrative and mechanically based and your data is going to be a bit less reliable.

              I mean it is really easy to ask if someone has spent at least an hour playing games in the last week and be sure that the question was understood and answered accurate. Much hard to be sure of the answer to whether or not they have played a puzzle platformer or a survival game.

              1. Wide And Nerdy says:

                You have a good point. Though they could just ask for a list of games and then decide for themselves what categories they fit in. Its still tricky to make reliable distinctions (for example, there are lots of shooters these days that are also rpgs, though some shooters just have a few rpg elements and some rpgs have only very basic shooter mechanics (is Bastion a shooter?).

              2. Daemian Lucifer says:

                And yet people have no problems in answering what kind of metal they listen to,or if they read mostly young adult fiction,or if they are into found footage movies,all of which are relatively recent genres.

                1. I’d say that’s an easy one to figure out. Those things you list are parts of the broader categories of music, literature, and film, all of which have been sub-categorized quite comfortably by their respective producers and consumers for decades.

                  “Gaming” is still an activity that’s making inroads into the mainstream culture, even though it seems to have deep penetration among the populace. There’s never been a Siskel & Ebert of video games or a New York Times Bestselling Games list. We can also thank our fragmented media delivery for that, in a way. There probably isn’t a chance for a gamer version of Siskel & Ebert to capture the same swath of the viewing public anymore, letting game criticism/praise become a part of the everyday language.

                  1. Daemian Lucifer says:

                    “Those things you list are parts of the broader categories of music, literature, and film, all of which have been sub-categorized quite comfortably by their respective producers and consumers for decades.”

                    Not quite.

                    Then there is the fact that found footage is a method of filming rather than genre,and while its mostly used for horror,it can be used for other genres as well.

                    And lets not even go near the absurdities around fantasy and science fiction.Especially the various steampunk,cyberpunk,teslapunk,punkpunk,punkjazzpunk,spamspampunk&spam.

                    1. Again, those are in the broader categories of film and literature which have been sub-categorized for decades. If I come up with a method for shooting movies that defines its own sub-genre, nobody who enjoys movies will hesitate to slap a label on it and the fans of that sort of filmmaking. The only quibblers will be those who want a different label.

                      Geeks have been arguing about the difference between science fiction and science fantasy since before Star Wars came out, but few if any think media associated with those categories should be put under “romance” or “thriller,” nor are they objecting to an attempt to categorize them in the first place which is my point.

                      If your next thought is to argue “The Phantom Menace” is a romance, you’re intentionally being pedantic for your own pleasure and deserve to enjoy the company of ten caffeinated role-players who won’t rest until they’ve told you every detail of their 5th Edition D&D characters.

                      If widely-accepted broad categories for books and movies didn’t exist, libraries and video rentals would be truly irritating things to cope with.

      4. The Rocketeer says:

        A lot of people don’t like the word “gamer,” or even the word “games” in reference to the medium. But no one has a better suggestion. That’s what it’s going to take, but no one seems to be able to come up with anything.

        1. Shirdal says:

          Even if we found a new word for the medium that isn’t ‘games’, there would still be a word for people who strongly associate themselves with the hobby. People like putting these labels on themselves and each other, and the word ‘gamer’ is not unique in that regard.

          No one seems to question the existence of labels like ‘movie buff’ and ‘bookworm’, so I wonder why ‘gamer’ gets such a harsh treatment.

          1. BeamSplashX says:

            I guess those bother me less because there’s less of a perceived need to state one’s exclusion from those… are they even cultures, really?

            I’ve just noticed a lot of people who play video games, but will preface their experiences with “I’m not a gamer.” They feel like there’s something they can’t lay claim to.

            1. Daemian Lucifer says:

              Novelty of the medium.Im pretty sure that when TV became a thing,there were a bunch of people who were watching movies only in theaters and didnt want to be associated with your average movie watcher.

              1. Ivan says:

                No one has brought up the “stigma” of being a gamer yet. The hobby is still looked down upon as the most recent tool of the devil and even those who laugh at that want to distance themselves from the guy who is 30 and still lives in his parents basement.

                But yeah, I agree, another aspect of this is that the more “hardcore” want to distance themselves from the “casual”. Gaming is just reaching the point where it’s common enough that everyone and their grand-mom has played something, weather it’s Civilization or Bejeweled. Being a gamer used to mean that you had a passion for the hobby and the new technology capable of running it, but now the barrier for entry is almost non-existent. Now that the novelty is wearing off we’re trying to figure out exactly who is a gamer and who isn’t. Maybe once the hobby is fully accepted and just part of normal life we’ll be able to distinguish those with a passion for games from those who just like to play games. But for now there’s a great deal of confusion.

      5. syal says:

        They call them “Call of Duty players”, but calling yourself a “Player” has other connotations.

        1. BeamSplashX says:

          Ah, right. All kinds of terms are slipping my mind these days.

          I suppose I’m just used to seeing things written as “[media name] fans”, even though that’s not entirely accurate.

    4. Nick-B says:

      It could be worse. They could start referring to us as “video” gamers.

      1. Shirdal says:

        Or computer gamers. That would be the worst.

        1. Daemian Lucifer says:

          Well I do like to specify myself as a pc gamer,because I dont own,and probably never will own,a console.And my cell phone is used for calls,text,music and books,but not games.

  4. Daemian Lucifer says:

    Yay,we have a poorly reported news article as well!Rejoice!

  5. A study report which only got a methodology section in its second version? What could possibly have gone wrong?

  6. Peter H. Coffin says:

    While I agree that “Played a game in the past 60 days” is probably far too loose a definition, there’s no lack of people that play hard enough and often enough to easily merit the label, but don’t put outside time into meta-gaming. They buy stuff they see on tv or whatever else catches their eye while at Game Stop buying the thing they saw on tv, but they’re clocking a thousand hours a year on whatever those things are. They’ll show up in sales figures, and Xbox Live memberships, etc, but they’re not posting stuff to forums nor reading gaming press, just playing games.

    1. Tizzy says:

      I barely qualify by that definition of gamer, yet I have a decent knowledge of what are the current games, some sense of how the industry works and what’s on people’s minds, and I spend an inordinate amount of time reading criticisms, listening to conversations about gaming, and watching others play games… And, of course, posting on blogs… Where would I find the time to play, anyway? Not sure what that makes me, though…

      1. Bubble181 says:

        Yup. I spend *far* more time on this blog and on a few fora, reading about and critisising games, than I do actually playing them. I’ve played, maybe, two hours in the past month, but I’ve spent a hundred hours on game-related media. Am I a gamer? Sure, but not the type developers or publishers care about.

      2. Timelady says:

        Yeah, I’m the same way these days, whether I like it or not. Just don’t have that kind of time anymore, but I like to read about them anyway. Maybe ‘game aficionado’ would be a better term for people like us?

      3. There are also precious few games that grab my interest the way sandbox RPGs and games like Minecraft do.

        I’ve got nothing against the cover-based shooter genre per se (beyond what my Doom and Half Life lovin’ self finds weird about regenerating health if you’re behind a wall), but other than maybe HL3 or Fallout 4, there’s not much out there that’s grabbed my attention.

        Maybe if they ported Katamari over to PC?

    2. Dt3r says:

      The really confusing part is that they use that definition, and then one of the graphs is “Percent of respondents who personally use a game console.” Apparently 12% of non-gamers are actually gamers!

      1. C0Mmander says:

        Or they are pc gamers.

        1. Or they don’t see what they do with their console/PC/Mattel Electronics LED device as a big enough part of their lives to call themselves a “gamer,” even in a casual sense (or they weren’t offered the “casual” adjective).

          I’m sure a great many people tune into various events held at stadiums with regularity and don’t really consider themselves “sports fans.”

  7. In fairness, the new version of the report includes a section that splits the gamers into three groups, depending on how much they play in an average week. It’s still not great, but it’s better than the binary definition they had before.

  8. Ancorehraq sis says:

    It seems to me that the problem is with Shamus, and not with the study. He holds some expectations about research that are either entirely fictional, or historical and inapplicable to the modern business of science.

    It’s like complaining about the precise shape of headphone-shaped wooden blocks worn by an air traffic controller in an airstrip simulacrum. Misses the point entirely!

    1. Ivan says:

      It took me way too long to figure out that you were being sarcastic…

      1. Bryan says:

        Is that … huh. Well, that would make slightly more sense than what I was thinking, which ended with “eh, whatever, start ignoring one person”.

        On the other hand, seen through that light, the phrase “business of science” *probably* should have tipped me off. Sigh. Yeah.

        1. C0Mmander says:

          I think he meant the part about the “headphone-shaped wooden blocks” was meant to be sarcastic not the whole comment in and of itself. At least that’s what I hope cause if not I’m going to have to ask why someone who has a different opinion than yourself has to automatically be sarcastic.

          1. Ivan says:

            No, I was talking about the “Business of science” part. Science is not a business. It’s only agenda is to discover the nature of the universe. While science needs money to progress and sometimes has to exist in a business setting, it’s aim is still to learn. Science does not dictate nature, instead it reflects it in a way we can understand.

            You can’t slap the term “research” or “theory” onto any idea and call it science. Trying to do so when there is no data to support your claim, or you have manipulated data to support your claim is either lying to your self or an outright lie to the public.

            None of this is my opinion, all of this is derived from my understanding of science and the scientific method, and in science one’s opinion has very little value when there is no data to back it up.

            Sorry if this all sounded overly aggressive, I guess I got a little ranty there. But yeah, the idea of the “Business of Science” is so absurd because it is the equivalent of trying to dictate the nature of the universe. Wanting something badly enough will not warp reality to your whims.

            Also to respond more directly; holding science to anything less than the highest standards is idiocy at best and corrupt at it’s worst.

            1. syal says:

              “The business of X” is a generic term meaning “We treat X like we’re getting paid for it.” It often also implies the person knows they’re taking things more seriously than they really need to.

              Now, “The modern business of X” implies the old rules have gone out the window, which is absolutely sarcastic when applied to science which still uses rules that pre-date calenders. (Thanks for pointing out the sarcasm BTW, I would have glazed right over it otherwise.)

              1. Ivan says:

                Ah, thanks for that extra context. I was very confused for a while before I finally figured out what he was trying to say. Although ironically I still get the opposite impression from the “Business of Science line” that being that standards are being lowered so as to provide the maximum monetary gain. You know, that whole “profits now!” “cash in. Cash in! CASH IN!” mentality that is apparently so common today.

  9. DaMage says:

    Go to data, scroll through to bottom, very few references, all to other media and marketing companies……conclusion: not research.

    This is little more then a survey, and you cannot call a survey academic research unless you are also prepared to compare it to other studies that have taken place that substantiate your work. There are very strict guidelines in academia to what is defined as research, but like normal, media groups will call anything they do research in order to muddy the waters around how bias it is.

  10. Jexter says:

    There are certain tricks unscrupulous scientific papers can use to fudge statistical results.

    That “study” doesn’t need to use any of them, because it barely even bothers with statistics. I hesitate to even call it a study. They’re just reporting the results of a single online survey. Not even a little statistical significance testing to see how the results compare to chance.

    Worse, they didn’t show the survey itself, so for all we know there were 50 questions on it, and they only reported the 20 or so questions that – as you would expect from chance – cast the gamer group in a positive light. Since every data point mentioned is framed to make gamers look sociable, this seems likely.

    1. Paul Spooner says:

      Also, no control conditions. Most likely, what they actually found was that rich people have more time for games, and are also more likely to be educated and sociable. Which, while an interesting result in itself, is far from the implication they were trying to draw.

  11. Thomas says:

    “Marketing firm is unethical” probably isn’t a great attention grabber as far as headlines go. We live in a world where good journalism actively punishes the people perpetrating it.

    Sure you could talk about how The Washington Post is ruining it’s reputation which will hurt it in the long wrong. But to this I reply with the following fact:

    The Sun and the Daily Mail are the most successful papers in the UK. And it’s not even close.

  12. Cuthalion says:

    Matt DiPietro, Twitch’s vice president of marketing:

    “There’s this perception that [the community] comprises loners and rejects … and that couldn’t be more wrong,” he said. ” We didn’t go in with an idea of what the data would show, but we knew what we thought the data would show, And it showed what we knew to be true.”

    That was a doozy.

    Anyway, looks like this was a marketing survey Twitch is probably using to figure out what sort of people use their site. Doesn’t sound particularly unethical to me. Heck, I took a whole class (mandatory for my major, iirc) on how to write marketing research reports like that (the pdf, I mean) in college. Kinda normal.

    Buy yeah, the WP should’ve linked to the sources and explained a little more what the research actually was, as it comes off a little more science-y and unbiased than it probably is.

  13. Re: Gamers are more educated, more social than the people who make fun of them

    Shamus, since we’ve both professionally made fun of our hobby (“gaming,” for want of a better term), does that mean our targets are smarter than we are?

    Bonus philosophical question: If we were to both start making fun of each other for being gamers in the form of webcomics, would the smarter one be whoever has fewer comics making fun of the other?

    1. MichaelGC says:

      Bonus bonus philosophical question: what happens if one of you was to poke fun at yourself?…

      1. I think it’s one of those “constantly failing to succeed” skill check things. It’s the gamer equivalent of “as you approach zero.” :)

  14. lostclause says:

    Okay let me tear into this quickly. I’m a medical student, we read a few studies in our training.

    First off their sampling is terrible. An online survey? That’s some pretty significant selection bias right there but it also doesn’t say how it was advertised or sent out. Given this selected 63% of people as ‘gamers’ I think it’s pretty obvious their definition of gamer is flawed. The obvious confounding factor would be wealth, if you’re richer you can afford gadgets for gaming, and there is no evidence they even asked that question at all so they cannot have controlled for it. Note they report gamers are more likely to be employed. Of course they are!

    Their definition of ‘more social’ is also terrible. Living with other people and self reporting that friends are important is not a terribly good measure of social interaction. It’s much more common for young people to live with others, either parents or flatmates, and as they point out younger people are more likely to be gamers. I’m not sure what would be a good definition of more social but number of times meeting with friends (either in person or online) would be a much better proxy.

    More educated is the same problem as wealth. If you’ve got a college degree you have higher earnings and so of course you’re more likely to have access to gaming gadgets and the leisure time for them. This is telling in the ‘gamers parents’ statistic. Curiously, there’s no mention as to whether or not they had an age cutoff for answering that question.

    As to all the self reported stuff… that’s just weak. Self-reporting is a pretty terrible way of assessing things that aren’t yes/no facts and ‘natural leader’ is a very hard one to quantify.

    I’m sure someone who better understands sociology could come in and heap more shame on this thing but that’s my quick two cents. Happy to answer any questions.

    1. syal says:

      A pretty good definition of “more social” is how many people you spend time with in a non-mandatory setting (maybe throw ‘routinely’ in there for good measure). Most introverts have one or two friends they spend time with, sometimes a lot of time. And of course if you’re married you spend oodles of time with one person, but that certainly doesn’t mean you’re a social butterfly.

  15. nerdpride says:

    I think the first place I’d ever seen this “gaming makes people smarter” thing was over at some TeamLiquid article specifically about Starcraft: Brood War. Lots of people who played or followed it were college students and the whole English scene had a smart feeling to it. Some universities even offer Starcraft/2 related courses somehow. I think there was a bit of math related to exponential growth of resources. Maybe it was a series of things connected mostly by the game. Sounds like Twitch may have got the idea from there?

    The second place I something like it was at the Escapist. It presented some really disgusting “gaming makes the world better for people” thing which may have included bits about making people smarter. I was a bit incredulous about it (you really think that pumping lots of money into big entertainment is the best for growing technology, education?). I need to find a better way with words on the internet or something.

  16. EmmEnnEff says:

    This probably isn’t by accident, Shamus – but rather, by design. It’s likely that you stumbled into an advertorial [1] – an advertisement written by the advertiser, sent to a newspaper in the form of an editorial.

    Not to get all conspiratorial at you, but between the Twitch.tv-Amazon acquisition, and Jeff Bezos being the owner of the Washington post, it’s not unlikely that this relationship may have played a role in this.

    Or not. Newspapers tend to be quite happy printing advertorials, even without a close ownership structure between them, and their authors.

    [1] http://en.wikipedia.org/wiki/Advertorial

    1. This. There are a lot fewer reporters per newspaper these days than there used to be. Their job is to come up with stuff to fill the pages (except for a few high-profile columnists, whose job is to flog the editorial party line). It is a lot easier to fill the pages with already-written articles you receive from PR firms than it is by finding things out and writing about them. Try that “writing news” stuff too much and your output will shrink way below that of your colleagues–good way to be the next one cut. Especially since you practically can’t write a factual report these days without accidentally stumbling over something that will offend one of the advertisers.
      And hey, chances are that any given PR outfit will, if they’re doing well, also place ads, so you want to be on good terms with them anyway. And they will have done their best to make the thing engaging so your readership will read it. Win-win-win!
      Well, unless you were a newspaper reader under the impression that you were doing this newspaper-reading thing to become informed or something.

  17. Otters34 says:

    If half of what I’ve seen is remotely true about people in the hobby, that’s pretty much impossible.
    (Not saying anyone here is a terrible person or anything, because that’s totally unverifiable and rather unlikely, but the averages are really really skewed!)
    Anyway yeah, that’s a pretty sketchy basis to draw that conclusion on, it’s a shame how in an age when that kind o f information is at billions of people’s fingertips tricks like that are STILL largely successful.

    Makes me worry that, instead of actually improving their research and fact-checking, news outlets will just turn more and more to disguising lack of evidence. Even a lot of people who grew up with the internet(like me) still have the bad habit of just assuming “Oh, they wouldn’t say that unless it was accurate”. Especially if, like the headline above, it says something we’d want to think anyway.

    1. EmmEnnEff says:

      If we wanted accurate reporting, we wouldn’t be trying so hard to drive newspapers out of business. Alas, when we, collectively, don’t even care to drop 30 cents a day for quality reporting, it should come to no surprise that the quality (Or lack thereof) of free newspapers will trend towards that of free dailies. Which are 30% pulp, and 70% paid advertising.

  18. tzeneth says:

    My god, I knew the last few days were bad but Francis Ferdinand was shot? oh dear, that could have terrible consequences. I hope Austria will be able to handle things.

    (I decided to resist the old Shamus is an old man jokes. Then again, I couldn’t come up with anything original there.)

    1. Daemian Lucifer says:

      I wouldnt worry too much about it.Austrohungary is a strong country and will survive for many more centuries.

  19. Phantos says:

    A few weeks ago, I was watching the local news, and they had a segment on how kids in a study who played at least 1 hour of video games a day reported being happier in general than those who either didn’t play them at all, or those who played them way too much at the expense of everything else.

    Which sounds like it could be biased, and even if it’s true, it’s still a fluff-piece to round out a broadcast. And yet, ten years ago, that same report would have been shouting at me until it was red in the face that video games turn kids into cannibal serial killers.

    Progress is slow, but I think I saw society budge an inch that day.

    1. syal says:

      Might have to do with the parenting and attention levels: no games sounds restrictive and possibly implies parental pressure to do more productive things than having fun, and gaming away the evening is quite lax, probably a sign that parents let the kid go unsupervised for long periods. Set time periods imply the parents are both paying attention and allowing for fun stuff.

  20. Decius says:

    The way I would do that study would be to make a giant survey that measured lots of things, including intelligence, sociability, income, family size, and whatever else I want, and also gather lots of questions about games use and frequency.

    Then I would define “gamers” using whatever criteria showed the best results.

    I’m not saying that’s what was done here, but it’s pretty easy to massage data.

    1. Jexter says:

      There’s not even a need for that. Just give multiple response options for “How long has it been since you’ve last played [insert definition of game here]? Say, < 10 days, < 20 days, etc. For added bonus, include multiple definitions of game.

      Now you have (interval * definition) chances to get a result that is statistically significant. Or, in this case, some numbers that are slightly higher than the others. (Because significance testing is for nerds, amiright?)

      1. Decius says:

        That’s what I meant when I said “define ‘gamers’…”

        You have the space dimensioned by of the number of definitions, the number of options for each definition, and the number of personal attributes to search for soundbites of interest:

        “Hardcore mobile game players have better friends, study shows”
        “Moderate PC games have better sex, compared to hardcore and infrequent gamers, study shows”
        “Study methodology found precisely the expected number of ‘interesting’ results, metastudy shows”

        and Obligatory XKCD

        1. Jexter says:

          Heh, I was thinking of referencing that XKCD. Beat me to it.

        2. Digging deeper, one intrepid reporter ran the headline, “SCIENTISTS’ OBSESSION WITH MINECRAFT LEADS TO COPY-PASTING OF DATA ACROSS STUDY.”

  21. Corpital says:

    The part about the last 60days is pretty bad, but at least they didn’t labeled everyone a gamer who owns a lot of games.

    Anyway, going to a restaurant, bar or visiting friends and then spending the majority of the time being silent while fiddling with a smartphone has stopped being socially inept? Thanks, study!

    1. We used to have a term for people who seemed to purchase (or “obtain”) copies of games for the sole purpose of having tons of them: Librarians.

      Before the ‘net was so ubiquitous and torrenting was a thing, some people prided themselves (and still do, being collectors) on having racks of games still with the original packaging and media intact. Others just loved having access to the most obscure titles you could think of. Back in the C-64 days, getting games from the UK was quite a novelty.

  22. Mike says:

    I deeply admire your restraint and adherence to the principle of charity in approaching the study in question, sir.

    Even a man as well-educated in arts of language and literature as Rutskarn (if their twitter stream is any indication) would’ve probably just dropped a few f-words, an m-word and something ending with *it there somewhere, describing the scientific rigor of the thing (and probably would’ve been deeply ashamed of it just a few years down the road).

  23. Cybron says:

    Wow, I did not notice that when the story was making the rounds. That’s pretty shameful.

    Though my first response to the story was that twitch chat is probably the ideal place to conduct a study to contradict this one.

  24. Daemian Lucifer says:

    Ive just watched this episode of mythbusters,and I have to say twitch really shouldve hired them to do this study.

  25. Aaron says:

    well i for one think the Emperor’s new clothes look fantastic

    but yeah that does kind of loudly scream bad science, and the next pet i get will be named science just so i can tell science to go think about what it did or congratulate it ‘who’s a good little science? You are!’

  26. Doug Sundseth says:

    “Calling someone a “gamer” because they play Flappy Bird once every other month is like calling me a NASCAR fan because I drove a car this week. ”

    No. It’s like calling you a driver because you drove a car this week. It’s not dispositive, but it’s moderately statistically likely. Had the interval in the study been “this week”, it would have been fine. Other than the appalling methodology, of course.

    Further, your attempt to restrict “gamer” to mean first, “one who plays video games” and also “one who follows gaming media” is deeply flawed on both parts of the definition:

    1) An avid player of M:tG, or European family board games, or ASL, or 40K, or whatever is every bit as much a gamer as an avid player of computer games (broadly construed, since your phone, your handheld gaming device, your tablet, and your console are all incontrovertibly computers). Your attempt to limit it to only computer games is … very odd.

    2) There is absolutely no need to follow media to be a gamer. Many gamers do, but it’s certainly not a requirement. Just as it isn’t necessary that you read Car & Driver to be considered an avid driver.

    FWIW (not a lot), I’m a past member of the Academy of Adventure Gaming Arts and Design, and the periodical I edited was even nominated for an Origins Award (that it had precisely no chance to win, but there you go). I’ve been doing this gaming thing for a long time, and for that matter computer gaming since the Vic20 was state of the art. I’d be hard pressed to find even the most tortured definition of “gamer” that wouldn’t include me.

    All that said, this “study” isn’t any different than claims that roleplayers are smarter than the population average, or wargamers are smarter than the population average, or any other such self-serving “study”. And, as you note, it was really badly done.

  27. Adam says:

    Hi Shamus,

    I’d really recommend the UK author Ben Goldacre who loves busting junk research like this. It happens for all sorts of things, from the myth of ‘Blue Monday’, to the ‘equation for the perfect night out’.

    As you’ve found they’re flimsy puff pieces which don’t stand up to investigation, but hey they’ve got a good headline and lots of clicks!

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun. Your email address will not be published. Required fields are marked*

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>

Leave a Reply to tzeneth Cancel reply

Your email address will not be published.