Question: Is there a generally-accepted set of guidelines for what evidence-based research in education is? I have seen papers without statistical analysis, with inadequate sample sizes, etc. Alternately, is there a blog or a twitter feed devoted to pointing out good or bad examples of education research?
Brian Butterworth answered on 7 Mar 2018:
I would say that the guidelines are the same as for any science, apart from the fact that education is not rocket science, it’s much more difficult than that. It’s difficult because education has a characteristic that make it different from natural sciences: education deals with conscious, reflective agents, not unconscious atoms or molecules. This means that what seems to work for Johnny, won’t work for Jimmy just because he sees that it works for Johnny. Also all learners come to formal education with a different set of cognitive capacities, inclinations, preferences, home and social environments, as well as distinct educational histories. This means, statistically, that there will be large variations in the sample under test. So, if you are testing method A against method B to see which is the better, you will need a large or very large sample to find a statistically significant difference between the two methods. Let’s suppose that method A leads to better learning outcomes than B, it is still very likely that there will be learners who do better with B than with A, and that neither method works for another subset.
The other problem is with controls: if A is carried out by one set teachers and B by another, or on one set of learners and B on another, then it will be difficult to tell whether it is the method or the teachers or the learners responsible for the significant differences. This is not an insoluble problem, but it does require sophisticated experimental designs.
However, if there are no statistics, saying A is better than B (for every learner?) lacks support.
The Royal Society set up a working party, on which I served, to look into the implications of cognitive neuroscience for education. They wrote “Education is about enhancing learning, and neuroscience is about understanding the mental processes involved in learning. This common ground suggests a future in which educational practice can be transformed by science, just as medical practice was transformed by science about a century ago.”
What we are now discovering that in medicine a method that works for most patients doesn’t work for a minority. This means personalising medicine: trying to match the method to characteristics of the patient to the method. Similarly, adapting education to the characteristics of the learner is the way forward. This means more sophisticated statistical and evidential methods.
Richard Churches answered on 15 Mar 2018:
There is a lot of debate about the phrase ‘evidence-based practice’ in education. In medicine and healthcare this is defined as more than the than the simplistic application of treatments based on research findings; it is about ‘integrating individual clinical expertise with the best external evidence’ (Sackett et al., 1996, p. 71). Practically, EBP is seen as involving a number of steps:
1. Diagnosis (assess the patient)
2. Create a clinical question to help
identify an appropriate treatment
3. Look at the research evidence,
critique it and select a treatment
4. Treatment, involving the
patient in the process
5. Evaluation of the effects of the
treatment (self-evaluation as a
In this way, each patient engagement becomes a form of research project for the clinician. Alongside this, clinical practice consists of guidelines that form ‘systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances’ (Field and Lohr, 1990, p. 38).
Field MJ and Lohr KN (eds) (1990) Clinical Practice Guidelines: Directions for a New Program. Washington:
Sackett, D.L., Rosenberg, W.M.C., Muir Gray, J.A., Brian Haynes, R. Scott Richardson, W. (1996) ‘Evidence-based medicine: what it is and what it isn’t’. British Medical Journal, 312, 71–72.
Is there any evidence related to learning in science museums? Any good advice how to make the visit more effective
What is the science of dealing with student behaviour for stealing? Do in school suspensions have positive outcomes?
I was just preparing a lesson, and deciding which exemplar of GCSE work to share with the students *first*. (The goal
The idea of an individual having a specific learning style has been discredited - but is mixing styles/approaches in a lesson also discredited?
Individual differences - in the book 'Battle Hymn of the Tiger Teachers', the staff describe a situation where they
Is there any evidence, specifically in educational settings, to say that a strong school culture (i.e. teachers all
Why are RCTs so expensive?
How common is it for researchers to replicate their own work?
The idea of learning styles is prevalent in some education circles and popular among the general public, despite lack
Is there any evidence related to learning in science museums? Any good advice how to make the visit more effective (2 Comments)
Do you have a set of data for learning curve? We do the finger maze activity with students but wondered if you have
We have become increasingly a visual society and multimedia learning promotes a combination of textual and visual
How can we help pupils with exam stress? How can we help them remember Science equations etc? (1 Comment)
My friend is about to move to an international school in South Korea who this year are employing a ‘positive psychology (1 Comment)
Is there any evidence related to learning in science museums? Any good advice how to make the visit more effective (2 comments)
How can we help pupils with exam stress? How can we help them remember Science equations etc? (1 comment)
Has mental health of pupils got worse, or is more reported? (1 comment)
My friend is about to move to an international school in South Korea who this year are employing a 'positive psychology (1 comment)
The 'redundancy effect' says that it is bad to read out the text of PowerPoint slides but the 'modality effect' says (2 comments)