Meeting Sixteen (15th January 2013): Randomised Controlled Trials in Educational Research

Happy New Year!

It’s been 6 months since our last meet­ing. Both @Alby and @AlomShaha have been unavail­able to run the club over the lat­ter half of 2012 and the autumn school term is tra­di­tion­ally a long and stress­ful term so, despite a few false starts, we’ve not been able to get things up and run­ning again. Until now.

The first meet­ing of 2013 (15th Jan­u­ary at 7:30pm) is inspired by the doc­u­men­tary on Radio 4 last week, where Ben Goldacre dis­cusses using evi­dence to inform pol­icy, con­cen­trat­ing on how suit­able ran­domised con­trolled tri­als (RCTs) might be when used in as evi­dence for social poli­cies in areas such as crime and edu­ca­tion. This pro­gramme can be down­loaded before Fri­day 11th Jan­u­ary from here http://www.bbc.co.uk/podcasts/series/r4choice. It’s not pos­si­ble for us to dis­trib­ute this pod­cast after this time as it con­tra­venes BBC pol­icy so down­load it as soon as possible.

There are some use­ful papers we could dis­cuss how­ever we suf­fer from the usual prob­lems  i.e.: a lack of open access to research papers. So for this meet­ing, we will look at the paper writ­ten for the Cab­i­net Office by Dr Ben Goldacre, Pro­fes­sor David Torg­er­son et al:

Test, Learn, Adapt: Devel­op­ing Pub­lic Pol­icy with Ran­domised Con­trolled Trials.

Haynes L., Ser­vice, O., Goldacre, B., Torg­er­son D. (2012). Test, Learn, Adapt: Devel­op­ing Pub­lic Pol­icy with Ran­domised Con­trolled Tri­als. Cab­i­net Office Behav­ioural
Insights Team
http://www.cabinetoffice.gov.uk/resource-library/test-learn-adapt-developing-public-policy-randomised-controlled-trials [down­load .pdf]

The paper doesn’t con­cen­trate exclu­sively on RCTs in edu­ca­tional research, but there are some exam­ples within that do sug­gest that RCTs should be used to form edu­ca­tion policy.

To com­ple­ment the paper here are some other open ref­er­ence mate­ri­als that are available:

• Intro­duc­tion to research: http://www.brad.ac.uk/acad/management/external/els/pdf/introductiontoresearch.pdf (from @Bio_Joe)
• A guide to RCTs in Edu­ca­tion Research : http://www.nfer.ac.uk/nfer/publications/RCT01/RCT01.pdf (from @Bio_Joe)
• A Sys­tem­atic Phon­ics Review: https://www.education.gov.uk/publications/eOrderingDownload/RB711.pdf
• A series of pub­li­ca­tions of in Edu­ca­tion research using RCTs by Pro­fes­sor Paul Con­nolly (@ProfConnolly, http://www.paulconnolly.net/) and the Cen­tre for Effec­tive Edu­ca­tion at Queen’s Uni­ver­sity Belfast in North­ern Ire­land: http://www.qub.ac.uk/research-centres/CentreforEffectiveEducation/PUBLICATIONS/
• The Edu­ca­tion Endow­ment Foun­da­tion men­tioned on Ben’s radio pro­gramme. http://educationendowmentfoundation.org.uk/projects/success-for-all
• Two Edu­ca­tional RCTs in Kenya : http://ipl.econ.duke.edu/bread/papers/0809conf/Dupas.pdf http://www.povertyactionlab.org/evaluation/impact-distributing-school-uniforms-childrens-education-kenya

Ques­tions to dis­cuss dur­ing the meet­ing are :

1. How well would the meth­ods described in the paper work in your school? Would you con­sider run­ning or tak­ing part in a ran­domised con­trol trial in your school? Why or why not?
2. What con­cerns do you have with the eth­i­cal con­sid­er­a­tions of using RCTs in edu­ca­tional research?
3. Do you agree with Ben’s asser­tion (in the doc­u­men­tary) that RCTs pro­vide bet­ter evi­dence (“the gold stan­dard”) than other meth­ods of research for social sci­ences.
4. Which edu­ca­tional inter­ven­tion would you like to see more evi­dence for, in the form of RCTs (or other methodology)?

The meet­ing is Tues­day, 15th Jan­u­ary 2013 at 7:30pm UTC, to be mod­er­ated by @A_Weatherall. Remem­ber to use the hash­tag #sciteachjc dur­ing the discussion.