neils clark

This post is going to speak primarily to researchers who work with games.

Before research is ever conducted, there’s a lot of planning. If the chosen plan calls for the use of human beings, then university research generally has to get the approval of an institutional review board (IRB), or human subjects committee. These guys are trained to make sure that researchers don’t cause harm.

In a nutshell, very few people really understand games, be they these IRB committees, regular folks, professors or researchers. That’s why, in my experience of games research, the researcher is the one responsible for the safety of the people involved. Researchers are also accountable for the research that they publish. I’ll admit that I’ve made mistakes in my research. What’s important with a new area, like games, is that we take a critical eye, and separate the wheat (the useful and representative data) from the chaff (the unrepresentative and potentially harmful data). I want to talk about wheat and chaff in published research as well as the process of conducting research.

It’s because society doesn’t get games

A lot of regular folks don’t get games. Games, especially of the MMO variety (i.e. World of Warcraft), are still relatively new. Beyond that, there are all kinds of different skill levels within these games. Inside a game like World of Warcraft, a player who has logged 30 hours total has a vastly different perspective than somebody who has logged 3000. People who have never picked up a computer game are going to have a really hard time understanding these worlds, because there’s really no acculturation. I love Nick Yee’s example of football, from his The Trouble with Addiction:

…there is a tremendous difference in how people interpret tragedies that occur in these two worlds. High school and college students on football teams regularly die during practice (1, 2, 3), but their deaths are dealt with by the media with a very holistic perspective. The media questions whether the coach set an unreasonably exhausting regimen. The media questions whether the parents saw warning signs. They ask whether the school reviewed the coach’s history thoroughly when the hiring was made. They wonder why the school mandates year-round practice that necessitates training in the hot summers. They ask whether the team physicians condoned the exhausting practices despite the individual’s particular health idiosyncrasies. And in no time during all this does anyone suggest that football is addictive and caused the deaths. This is because that statement would be naïve and simplistic.

Society doesn’t teach everyone about Azeroth, the fictitious Warcraft universe, but society (at least in the United States) forces football down nearly everyone’s throat. When somebody dies while playing football, there’s a vastly different approach to understanding it. If you want more info on why we misjudge game addiction, check out my awesome badger references in my gamasutra article Games & Addiction: Are We There Yet?

“It’s an addiction, and it’s everywhere!”

It’s not just regular folks. Some otherwise brilliant researchers are making game addiction claims that are blatantly ignorant and harmful. A few months ago Maressa Orzack of Harvard University estimated that about 40% of those individuals playing World of Warcraft were addicts/game dependent. While that claim has since been widely rebuffed, even oversimplifications really do have a massive potential to cause harm. If the researchers can’t get a grip, then how are regular people supposed to have a chance?

Even worse, some researchers are taking these oversimplifications to the next level. Instead of understanding technology, and treating it with some modicum of thoughtfulness and/or respect, one researcher in particular, Kimberly Young, copied criteria from gambling addiction, applying it to the Internet and then games. At first, this was helpful in one respect: it recognized that some people were using the internet and playing games to excess. That’s great. Unfortunately, the criteria copied from gambling addiction’s DSM criteria in 1996 have remained stagnant for the last ten years. In 2004, Young once again published her criteria for IAD, “Internet Addiction Disorder,” reinforcing her claims with market research and a rampant use of unsupported anecdotes. These criteria have been criticized by many other professionals and researchers, for instance John Grohol and John Charlton, as having no face validity (i.e. the criteria just don’t make sense) and the potential to grossly overestimate the number of addicts out there.

Kimberly Young is expected to propose IAD criteria for the upcoming 2012 edition of the DSM (psychology’s handbook of disorders). For one example of her criteria, one sub-criterion for ‘withdrawal,’ is “voluntary or involuntary typing movements of the fingers.” I have never come across reliable data suggesting that ‘twitchy fingers,’ are significantly associated with withdrawal from excessive internet use or gaming.

Worse, her research has been employed by researchers in huge, expensive studies intended to determine the prevalence of addiction. Wait, what? Why are people already using it if there’s no evidence that it works? These studies are not only establishing a simplistic criteria for addiction (remember that football deaths receive balanced coverage). When these criteria greatly overestimate the number of “addicts,” they give regular people out there a reason to be afraid of this naïve criteria. Regular folks love simple descriptions. Once they hear Harvard and Stanford research that says, “It’s an addiction, and it’s everywhere!,” it becomes very difficult to reverse that misunderstanding.

Now don’t get me wrong, it’s not entirely Young’s fault that other researchers put her criteria into practice, rather than hunting down data to either verify or challenge those criteria. She doesn’t have control over who uses her criteria. However, she has excellent recognition, and is in a position to revisit her criteria and research whether it actually reflects excessive gaming. With that data, she would then be able to modify her criteria so that it can actually start helping people - because right now it’s causing harm.

What about the process of research?

My own MA thesis was an interesting case. While the actual details are beyond the scope of a blog post, I do have plans to seek publication for a journal article that deals with sampling humans inside an MMO game. It was a very interesting process, but one that requires very unique considerations in its planning and execution. If you want more details on in-game sampling, I would greatly appreciate your comments or questions. It’s a journal article that I’d love to write, though I cover different aspects of my method briefly in areas of my master’s thesis.

Before I started in-game data collection, I had conversations with 4 respected professors, a dozen other grad students and one official IRB (human subjects committee) personnel. They all had wonderful and unique understandings of what was appropriate and not for using human beings in a study. The problem: none of them presented a deep understanding of games. As a result, some drastically misunderstood games, and I faced enormous pressure to do outrageous things, among them use Kimberly Young’s criteria for Internet Addiction Disorder. When it came to passing the scrutiny of the IRB for my thesis, I was allowed serious things. Not only was I given permission to show my subjects their relative level of addiction, but parental consent for minors was waived.

This is something that we have to be incredibly concerned with. Because games carry stereotypes that they, ‘aren’t serious,’ or, ‘are a huge cesspool for addicts,’ we as games researchers have a major responsibility to be not only deeply involved with watching the ethics of our own research, but we also have a responsibility to educate where appropriate. While planning and conducting my research I was very conscious of the ethics involved, and I tapped the experience of dozens of non-gamer experts. When I did finally have a conversation with a game aficionado and professor, however, I was presented with cogent critical feedback. He raised a number of very valid concerns, concerns that had been invisible up to that point. The point is this: people researching games, be they students or professors, more and more require the critical and honest feedback of other researchers who understand games.

There are ongoing mistakes, and most of them are honest. What we need is the will to correct them.

Neils Clark :: Feb.07.2007 :: Game Addiction, Research ::

Trackback URI | Subscribe to the comments through RSS Feed