See the live dashboard for CensusAtSchool 2023-2024

Video: Randomisation Test [7:45]

Randomisation is always present in an experiment, so we need a type of test that helps us determine whether the effects we have seen in a randomised experiment are real or random.

(See also the Review Questions following the movie)

[Illustrated transcript (pdf)]

After you’ve watched this video, you should be able to answer these questions:

  • When we look at a plot of experimental data that compares two treatment groups, why is it not always just obvious which treatment is better? What question goes through our minds?
  • What is the basic idea governing when we will conclude that there is a real treatment difference?
  • Try to bring to mind the basic steps of the VIT visualisation of the randomisation test (and visualise the sorts of pictures that were being generated at each step).
  • What is the formal name for the tail proportions the visualisations produce?