A flexible gaming environment for reliably measuring cognitive control

Over the last few years, it has become accepted that reliable measurement of individual cognitive abilities requires participants to complete many more trials and/or to use tasks with larger effect sizes than are typical of existing cognitive batteries. This project develops a battery of cognitive control tests enabling efficient and reliable measurement of cognitive control abilities crucial for high performance under time pressure. The test battery is implemented in the Unity game engine, and accessible online using only a web browser with no installation. Gaming mechanics (e.g., variety, feedback, rewards, and a leader board) and an integrated story line maintain engagement over extended and demanding testing sessions. The battery implements most prominent measures of cognitive control including: 1) working memory (single and dual n-back tasks), 2) response inhibition (stop-signal task), 3) conflict tasks (Simon, Flanker and Stroop tasks), 4) multi-tasking, and 5) task switching. The different measures can be flexibly combined within a coherent “room-clearing” narrative, and self-contained tutorials enable easily deployed online testing. Novel versions of the conflict tasks were developed to increase effect sizes and reliability, and they were tested in an online experiment. We develop a rigorous methodology for quantifying the ability of the tests to produce reliable measurements of individual differences and report the results of applying it to data from the experiment. We conclude that these new conflict tasks produce much more reliable measurement than has previously been achieved.