-
Asked by Abena to Alex, Brian, Carolina, Courtney, Paula, Richard, Sara on 12 Mar 2018.
-
Richard Churches answered on 12 Mar 2018:
Actually, the large scale trials in England are only expensive because of the way they are organised. One of our teacher-led RCTs has 900 participants and has cost nothing more than teacher time. Also, the large trials in education are trying to standardise their protocols over six months with large samples. Teacher-led RCTs often model a laboratory approach with many being single lesson studies.
-
-
Paula Clarke answered on 12 Mar 2018:
RCTs don’t have to be expensive but when run by a research team the costs mount up. When we conduct trials we will typically employ research assistants, postdoctoral researchers, pay for buy out of teacher time and teaching assistant time, assessment costs, materials costs etc.
-
Brian Butterworth answered on 15 Mar 2018:
The answer is statistical. To get a reasonable effect size will depend on how much variance there is in the sample. In educational research there is usually a very large variance in the sample because learners will be very different for a host of cognitive, social and personal history reasons. This means you will need large samples to see anything significant. Bear in mind that there are lots of different ways of measuring effect sizes, but this rule applies to all of them.
Comments
Abena commented on :
Thanks for answering.
Richard commented on :
You can design trials that substantially reduce individual differences between pupils and which can then detect valid significant effects without a large sample. About 50% of the 140 teacher-led RCTs we have supported have been within-participant (repeated measures) – a form of design in which individual participants experience both the control and intervention. It is usual to ‘counterbalance’ (reverse for half the participants) the order in which things happen in this type of design, in order to balance out effects that might transfer from one condition to another (carryover or order effects). It is possible to counterbalance out the effects of different subject content, as well as the order in which participants experience conditions by double counterbalancing (see page 39 of Teacher-Led Research (Crown House)). Shorter treatment windows in teacher-led RCTs, and high levels of mundane realism (everyday-ness) also tighten up the design and reduce the effect of extraneous variables. For example, some of the neuroscience-informed teacher-led RCTs (the project we are working on with the Wellcome Trust) have taken place over 1 – 4 lessons and been within-participant. Another option, if you have to have different pupils in control and intervention, is case-matching prior to randomisation. Pupils are paired on important characteristics and then each member of the pair is randomly allocated to the control or interventions. In this way, there is a balance of differences between control or intervention.
Courtenay commented on :
It depends very much on who is delivering the intervention and things like dosage and definitely sample size. For most of the behaviours of interest, we expect small-medium effect sizes (in part due to huge amounts of variation within the treatment and control groups). To detect modest differences you do need to have large samples. It is also important to assess the skill of interest on a number of occasions, so that you can see how the new skill is maintained over time. Finally, some skills just take time to learn and improve. I’m interested in improving oral language skills and I just don’t think there are any quick fixes. Imagine how long it would take me or you to learn a foreign language – lots of instruction and lots of practice! and we have the requisite skills! A real challenge for researchers and practitioners in this field is to figure out how to offer sustained support in a way that is cost-effective – answers on a postcard please!