So I was doing my daily search through gametab.com, as I am wont to do, and I came across this little gem:
http://www.1up.com/do/newsStory?cId=3156887
Dave Jaffe liveblogged during a Focus test for Calling All Cars. It’s funny to me because I’ve been to a lot of focus tests, and it’s interesting to see how people react to Jaffe’s impressions.
I mean from what I read, it sounds like a pretty normal focus test to me.
All sorts of things going wrong — check.
Frantically scrambling to get everything to work — check.
Players who are very bad at your game — check.
The strings of profanity (though I wouldn’t personally say them out loud, at least not where anyone can hear), are definitely things that go through your head while you’re watching one of these. So this account was a laugh a minute for me. I can relate.
For those of you who’ve never been to one, focus tests are so valuable precisely for all these things. They give you a chance to work out all the rough stuff before it gets to the consumer. And, while they’re definitely as frustrating as they sound, the good part about them is that at the end of it all you come away with a nice list of things that you can do to make your game even better (and certainly more accessible).
That doesn’t stop them from being frustrating as hell at the time, though 😉
Why are the focus test players so bad? Do they at least select people who play games? Are people really that stupid? What?
Oh, hey Dodger did you catch this: http://www.costik.com/weblog/2007/02/are-retailers-actually-screwed.html
It’s a little peek into the way that the game industry handles pricing, distribution, and marketing. Obviously game design is more fun, but I still found it interesting.
@1
In most cases, it’s not that they’re stupid, it’s just that they’re not as good as you are at your game. They don’t have the power to read your mind, either. Something that seems obvious to you, as a developer, is not necessarily obvious to your player. The focus test is, therefore, a reality check for the designer as to the accessability of his gameplay.
Of course, there are exceptions — like the people who don’t seem to grasp the basic concepts of playing a game… but those are like 1/20 or 1/30.
From what I understand, they select people based on answers they give to a questionaire. Usually it’s by what games they’ve said they own.
@2
I knew a bit of that, but that’s a very interesting take on the whole thing. They seem to be ignoring the outrageous profits from accessories, though — because retailers clean up on those. Like Best Buy — they sell an iPod and make a few dollars, if anything. But then they rob you blind on headphones, car chargers, wall chargers, fm transmitters, misc cables, software, etc…
Ah, so they do select likely people to actually play your game, but some total losers get picked up in your net. Still, you’re right that a lot of usefull stuff comes out of the focus testing, re: other people’s ability to reach the same conclusions that the developer did. Adventure games should be extra focus tested. 🙂
Yeah, there’s some filtering that goes on. I guess it depends on the size of the original pool. It’s not all that important when it comes to focus testing games for gameplay, though. I imagine demographic info comes in a lot handier when doing marketing focus tests, which I’ve never been a part of.
Oh man… when I was a tester I had to go through focus test tapes. Sometimes that was really painful! Some things that are extremely simple for a QA tester to pick up on can be nearly impossible for a focus tester. And I mean really easy things. Things that I did on the first try without thinking were sometimes impossible for a focus tester. People are all just different and we all play games and react to them differently.