I’m posting below a section from the book Zombies Can Make You Think Better, How A Game Can Build Your Brain for those of you who want to know more about how to understand experimental and control groups in research studies. It explains why having a matched control group is the best way to test efficacy of interventions. Not all research studies do, but we used a matched control group in our efficacy study that demonstrated far transfer.
__________
Experimental and Control Versions of the Game
One of the challenges in research is coming up with a good active control condition. An active control group is desired so you can demonstrate that one intervention is better than another, not simply that one intervention is better than nothing. Also, without an active control group, the experimental group may get more attention from the experiment researchers and that attention may influence the results significantly, in which case, you may be proving that the attention matters, not the intervention. So, an active control group is preferred, but it is still tricky. For example, an active control condition in an efficacy study of Project Azriel could be doing crossword puzzles. Both the control and experimental groups would be participating in the intervention, and this is better than a control condition where subjects do nothing except take pre and post tests, but it is not ideal for multiple reasons. First, we want to be able to tell the efficacy of the intervention, in this case the cognitive training activities, in the context of a first-person shooter game. There is a risk that the subjects will guess their group assignment. If that happens, then those playing the game may guess they are the experimental group and, therefore, try harder than those who may guess they are in the control group who do not try as hard. Another problem is that if crosswords are the active control, you may have the problem of apples and oranges. If doing crosswords puzzles, despite popular belief, actually makes you less cognitively healthy, then the game’s impact would be over-presented. That’s why we were really excited about our experimental design. We were able to create a version of the game where the cognitive training activities never got harder. Both the experimental and control participants were required to play the game for the same amount of time. They had an identical experience. But the training activities for the experimental group adaptively became more challenging. This made for a good active control condition.
__________