The Washington Post has this story under the headline “Study: Gamers are more educated, more social than the people who make fun of them“. Now, that sounds pretty good to me. I love scientific studies that prove I’m smart and nice. But I also thought the story felt a little ego-stroke-y, which made me suspicious. Who did this study, who paid for it, and what did it (actually) reveal?
The WP talks about the study being done on behalf of Twitch.tv. Shamefully, they neglected to link to the data or even to the research site. Once again, I have to condemn the way news sites write news stories for the web as if they were writing for print. There’s no reason not to link this stuff.
Digging a little deeper, I found that the group behind the study is Life Course Associates. They are not a research institute. They are a marketing firm. The WP story did say that Twitch.tv funded the study, but the crucial context we were missing is that they funded this study as part of a marketing campaign. There’s a huge difference between a company giving a grant to (say) a university to do a study, and a company paying a consulting firm that does research as a form of marketing.
Now, that doesn’t mean that the research is automatically false, but it is crucial context that should be included in the news story. This report is exactly the sort of thing a marketing team is after:
- Headline-grabbing conclusion that generates buzz.
- An effort to change of the public perception of the demographic they were hired to market to.
My last complaint is that their definition of “gamer” is shockingly broad: “anyone who has played a game on a digital device in the past 60 days”. When I hear “gamer” I think of someone who is engaged with the culture on some level. Maybe you read forums, or reviews, or you network with other players via your friends list, or you watch streamers or Twitch, or Let’s Plays on YouTube, or… something. Calling someone a “gamer” because they play Flappy Bird once every other month is like calling me a NASCAR fan because I drove a car this week. This definition is so broad that it borders on useless. Note that I’m not trying to condemn people who aren’t “real gamers” or whatever. But if the study is supposedly to find out about “gamers” then broadening the definition to include such a massive chunk of the population is going to muddle the results. If I want to study people who live in the city, then including “people have visited a city in the last month” is clearly spreading the net far too wide.
I don’t have a background in science or statistics, so I can’t comment on the veracity of the data or critqique how it was collected, but I’d be surprised if it withstood the normal standards for scientific rigor. In any case, when a marketing firm releases a study that tells you what you want to hear about yourself, you might want to be skeptical.
And if you work at the Washington Post, you really should include these details with your story.