Hundreds of thousands (estimation - no official number was given) of players participating in StarCraft 2 beta testing are generating plenty of gameplay statistics for Blizzard to fine tune the game before its official launch, but design director Dustin Browder is wise enough not to trust the numbers blindingly when making design decisions.
"The danger with a lot of this data is that you have to be very careful how you use it," he emphasized.
"With unit stats, I can tell you that, for example, in a Protoss versus Terran game, 12 percent of the time the Protoss build carriers. And when they build carriers, they win 70 percent of the time," Browder gave an example. "You could say, 'That must mean carriers are overpowered!'"
"That's not really true, though. It could just be that as you get towards the end of the game, if the Protoss have the extra resources to waste on a bunch of carriers, they're probably going to win anyway."
Without inspecting game replays and reviewing the design, it is impossible to determine which of the duo is the correct explanation, or if there is a third reason altogether, Browder noted before warning that "That stat alone actually tells you nothing. It's a very dangerous stat. If you listen to that stat, you can make all kinds of mistakes."
"The real challenge for us is to continue to sort the wheat from the chaff, to determine which stats are real, which stats are meaningful, and which ones we should be looking at to make a meaningful change," Browder concluded.