Randomness, sample size, imagination and metacognition: making judgments about differences in data sets

Consider this:

An experimental treatment for mildly to moderately depressed patients is tested by taking two groups of 15 to the Caribbean. One group swims 4 hours per day for 4 weeks (the control group) and the other group swims for 4 hours per day for 4 weeks with dolphins (the treatment). Ten out of 15 of the dolphin group improve, whereas 3 out of 15 of the swimming only group improve (Rossman, 2008). What could you claim from this experiment and for what population? How certain would you be? Why? What types of things are you thinking about to inform your decision? What would you like to know more about? What are the big statistical ideas in this problem?

This was one of the problems that Grade 10 mathematics students were challenged to consider as a part of an action research project, where two teachers trialled a statistics and probability unit specially developed by the second author. A key intent of the five week unit was to help students develop conceptual and technical tools to be able to make informal inferences (statistical judgments) about whether claims based on one sample could be generalised to larger populations, and about whether differences in data sets were significant or whether the differences might have occurred by chance (a random event). Through using the TinkerPlots program (Konold & Miller, 2011) students could collect samples and create simulations that would enable them to develop visual and intuitive understandings alongside statistical interpretations.

The lead teacher was keen to expose her students to technical tools that might expand their notion of what mathematical inquiry is and liked the way that the unit combined ideas of probability and statistics, concerned that the normal way of teaching probability was around gaming contexts.