There’s a definite need right now for a little bit of humility and stand up honesty when it comes to researching games. In the grand scheme of things, we haven’t really been playing MMO games, or even using the internet for all that long. Making mistakes in research is more than alright, it’s bound to happen. A bigger mistake is to fumble and then pretend like everything is hunky-dory. In particular, I want to talk about Kimberly Young, Immersyve Inc and Smith & Jones’ Game Addiction Center. What concerns me is that people with clear vested interests are being portrayed as neutral experts.
For years I’ve held up the hope that Young would aggressively revise her criteria for gaming addiction. Since introducing the idea of internet addiction in 1996, she’s really just emphasized three anecdotes: Internet affairs, unauthorized employee Internet use, and Internet-induced lapses in student productivity. Young has a vested interest in selling books, online therapy (@ 95$/hour it’s not the most expensive therapy ever, but there are certain therapy procedures that can’t happen over the internet). Most importantly, she sells herself as a credible expert. At best, she backs the claims up by citing market research. At worst, there are dozens of critical claims that she backs up by citing her own past research. While a number of experts have written this research off as schlocky and misleading, it’s still used to build studies at highly respected schools. Stanford most recently used it as a backbone for an expensive study that polled thousands over telephone. The more that her criteria is used as a panacea, the harder it is to stop and really analyze whether or not it’s accurate. You can read more about Young’s study and the Stanford surveys here.
Similarly, Keith Bakker of Smith & Jones addiction, the widely publicized Netherlands treatment center, joins Young in being one of the most quoted figures in addiction. Natrually, people want to know about what he has to say. I want to know what he has to say, but we have to remember that he has a vested interest in treating addiction under a very specific paradigm. In English, that means that he has a 12-step treatment program, and he wants to make a living by treating addicts that are checked into his program. I haven’t seen any huge research that he’s published, and there really isn’t a whole lot of research right now. Like Young and so many others, he’s probably winging it.
I really enjoyed the following quote on Smith & Jones from joystiq.com:
Smith & Jones, the Dutch addiction clinic that opened a gaming treatment program earlier this month, has been inundated with calls from pleading parents leading clinic director Keith Bakker to exclaim, “Computer game addiction is obviously an even greater problem than we imagined.” Whoa there buddy, let’s slow it down a bit. Is it possible for someone to actually be addicted to video games? And can you really claim that your anticipated call volume is directly proportional to the size of the perceived problem? How scientific.
At first, I was really excited that Drs. Richard Ryan and Scott Rigby of Immersyve had taken the tyme to write an article for Gamasutra. At first I was really getting into what they had to say. From their description there was a wonderful research methodology, and they were able to find factors which really get people to play. Relating to others, becoming competent, and feeling autonomous do seem to pull people into games, and I’m really looking forward to writing more about how those relate to our experience in different media. So at first, I was even down with Ryan and Rigby plugging the effectiveness of this model for helping game designers and playtesters.
But it got a little bit excessive.
…it goes right to the heart of what’s important about the overall player experience and to factors that are important aspects of critical and commercial success.
…the PENS model can strongly and significantly predict positive experiential and commercial outcomes
Gamasutra is a games industry trade publication, but I don’t think that a feature article there is the appropriate place for an 8-page sales pitch. What’s more, I had seen Dr. Richard Ryan quoted for two weeks in regards to this study. The CBC article which I discussed (thanks again to Jeff and Mike for linking me) the study was portrayed as showing promise at tossing MMO games into a favorable light,
“If you’re having a hard time convincing kids of all ages to pull themselves away from their video games, there’s a deep-rooted psychological reason, a study by U.S. researchers suggests.â€
The problem is that I had only ever seen Dr. Ryan quoted as a professor at the University of Rochester, and his study as being published in the journal of Motivation and Emotion. A few articles have mentioned Immersyve, but they never solidly connected this research to the company. They just noted new psychological draws to games. Do you see the problem here? You need something called disclosure when your research is proprietary. That means that you need to be make it obvious if you expect a commercial profit from this information. I really like the research work that was done by Immersyve, but they made a mistake. They started out describing research, and wound up turning it into a pitch for a product.
Mix the two and you lose credibility.
The more we learn about addiction, the more we’ll shift away from schlocky info. I hope. I was once told by a native Hawai’ian to, “follow the money.” Should we really have to worry about this in academic research? I mean, seriously. Right now we should be questioning the research, doing work to see if it’s accurate, and learning. Let’s do it.
Neils Clark :: Jan.22.2007 ::
Games ::
Absolutely great piece Niels. Thanks for taking the time to say it. I wish I had.
You know I just got a brochure in the mail about a conference for thrapists that’s being sponsored by Hazeldon, The Meadows, Betty Ford Center, and a few other “big names” in the “addiction treatment business.” Do you have any idea how much these places charge for “treatment”?
These conferences charge big money and while I think some of the presenters offer valuable information on treatment, others are questionable. One workshop in particular bothered me. It’s titled “THE INTERVENTION BUSINESS: THE PRACTICAL REALITIES OF MARKETING, SELLING, AND DELIVERING INTERVENTION SERVICES.”
Where there’s money to be made I automatically question the validity of the service being sold.
How much *do* they charge for treatment?
Neils,
I appreciate in your own writing the measured approach you are trying to take to the area of games and addiction. I think the area of gaming addiction is an important one that you have correctly identified is under-served theoretically and empirically. As with many areas of gaming, I believe society would be best served by committing itself to the process of objective science and discovery rather than premature or biased conclusion.
That said, I believe you fall into this trap yourself in the premature conclusions you draw about Immersyve, our intentions, and the quality of our work. Academics have their biases too — particularly about business. Specifically, you assume that because Immersyve has a for-profit component to our company, our motive is not to advance the field objectively, and that our model is merely a “proprietary†product. Its a common and perhaps understandable bias, but it is incorrect. Regrettably your mistake could have been easily avoided had you simply challenged us on our work being purely proprietary or immune from scientific scrutiny. Immersyve’s goal is to make its research and its measurement tools freely available at no cost to not-for-profit academic research (i.e. basic science research), allowing the scientific community to form its own opinion of our conclusions, build on them, and hopefully apply our model meaningfully to an understanding of the psychology of video games, even perhaps contributing to a better understanding of addiction. Expanding this understanding in society is a core component of our mission, and we are already beginning to collaborate with several health research projects who want to use our model in understanding the dynamics and effects of gaming. In addition, we are committed to publishing our own work scientifically, and we will be submitting work regularly through peer-reviewed journals. The affiliations of our team have always been openly disclosed by Immersyve (as in the Gamasutra article), and we certainly don’t have control over the nuances of language in articles written by others. As a case in point, we never published this work as a model for game addiction — yet that was often the headline that reporters (and subsequently you) chose! We are trying to address more fundamental motivational issues that we hope can inform many issues in gaming, addiction among them.
Do we also commercialize our expertise to the development community? Yes we do. We believe that we can help them make better games that are also more commercially successful. Respected scientists often consult to industry and revenue is often generated from solid science, and can even support it. While it’s certainly understandable to ask questions about the research from a company that has a profit component, its important to actually ask the questions to see whether they are also taking the steps we are to contribute meaningfully and openly to the science. The facts may pleasantly surprise you, and in this case, I sincerely hope they have. But if not, stay tuned for more scientific publications on this work and its application.
Best regards, and continued good luck in your own exploration of game addiction. I look forward to reading more in the future.
First of all, thank you for coming here and giving real critical feedback as well as praise. I earnestly appreciate both.
Secondly, I’m overall very impressed by you, the quality of your research and the philosophy that you seem to have laid out for Immersyve. You guys are creating new research, doing work to see if it’s accurate, and advancing our understanding. It was wrong for me to lump you in with people who have vested interests that are truly questionable. I’m sorry.
I think that there’s a great deal of value in the PENS model. The gamasutra article reinforced that, but prior to that I had played with your work alongside what I call media experience theory. There’s a lot of praise for your research in my posts, in terms of both methodology and validity. That’s why I also said that I was down with you, “plugging the effectiveness of this model for helping game designers and playtesters.†I’m all for an academic using their work to promote themselves (or their business). I do it whenever I get the chance. =P But at some point, a little plug becomes a big advertisement. There were times in Rethinking Carrots where I was confused. The word ‘commercial,’ (i.e. commercial success, commercial outcomes, etc) shows up a lot. I agree, the PENS model has commercial viability, but maintaining a commercial tone throughout – kind of cast this article as an advertisement from a business, rather than research from academics (who own a business).
Do you see what I’m saying? Certainly, this is a world apart from the other concerns that I raised in this post. Kimberly Young has had 10 years to refine a problematic criteria for different online addictions, causing all manner of problems for researchers and addicts. You guys are doing solid research.
Best of luck to you and Immersyve. I can’t wait to read more.
Is it too late to ask you for a job?
Hi Neils.
Thanks for your response and interest in our work, and I can certainly understand how the commercial language and focus of the Gamasutra piece might be somewhat unsatisfying to you, as an academic with a passion for the research and theory. Having just published the first scientific article in “Motivation and Emotion†in December, which really focused almost exclusively on the theory and method to the exclusion of commercial considerations, we thought it was important to put an article out to the industry that focused on the commercial side of the equation as too often academics are dismissed as not “getting it†when it comes to practical relevance. For pure researchers, we may have swung that pendulum too far outside their area of interest – and as the Gamasutra audience is quite diverse it’s difficult to appeal to everyone simultaneously. We’ll try to strike a better balance in the future.
It can be difficult to bridge the two worlds of science and industry and maintain credibility in each – but we think the value for both knowledge and the application of that knowledge will be served well if it can be accomplished. When I look at the “theories†of player experience and methodology that are discussed in the industry right now, I see a lot of smart, creative people generating interesting ideas – but where is the scientific method in formulating theory from these ideas, building objective models, testing hypotheses, and providing data supporting their internal and predictive validity? If the industry does not start to value this process more, we will simply have a growing list of untested ideas, and no matter how interesting they may be it will be difficult for them to contribute meaningfully to building this area as an applied science – whether this takes the form of commercial methodology in game development or societal/clinical understanding of games.
PENS is something we hope is an example of a different approach that has both practical value, but is also derived from solid methodology and theory. It’s just a first step into an area that is remarkably exciting to be in because there is so much work to do and more for all of us to discover. Overcoming the biases both industry and academia carry towards the other is just another layer to the challenge, and whenever confronted with them from either side of the aisle I try to see it as a blessing in disguise: A reminder to be mindful and diligent to both the honest discovery of knowledge and also making sure, wherever possible, it can be applied meaningfully to a commercial industry whose innovation and creativity is responsible for raising these kinds of questions in the first place.
Let’s stay in touch and continue to compare notes. Any chance you’ll be at the GDC in SF in March?
Scott, I couldn’t agree more with this - If the industry does not start to value this process more, we will simply have a growing list of untested ideas, and no matter how interesting they may be it will be difficult for them to contribute meaningfully to building this area as an applied science – whether this takes the form of commercial methodology in game development or societal/clinical understanding of games.
Too many brilliant researchers using one or two very bad methodologies — and especially in addiction, it gets frustrating seeing so many people who run in circles.
I’ll be at the GDC. Let’s compare notes. =P