#SciTeachJC: Discrete KS3 HSW

Grime, R (2012) A School’s Experience of the Discrete Teaching of Scientific Skills at Early Secondary Level, SSR 346 (.pdf)

Abstract: Students at age 11 or 12 took a course where scientific skills were taught discretely rather than in an integrated approach alongside scientific knowledge and understanding. There is evidence that this may be a more beneficial approach for developing scientific skills.

Discussion points

1. How would the experiences in this case study inform your personal classroom practice? Given the opportunity, would you recommend a similar approach in KS3 across your department?
2. With changes to controlled assessment at GCSE there is an assumption that many schools are teaching scientific skills in isolation, often with a strong emphasis on the context of the ISA/EMPA/etc. Does the paper support this approach? How could it best be managed for better learning as well as supporting students to achieve good results?
3. The paper shows how action research at a school level can provide evidence in support of changes. What lessons could you learn from this when considering policy changes in your department?
4. With the recent attention paid to the use of RCTs to inform education policy, how could the results of trials such as this one be used to inform the design of larger-scale studies? Would it be possible to avoid the outcome measure being used to judge schools rather than interventions?

With thanks to the ASE, who were happy to make the article open-access so we could discuss it. If you are interested in peer-led research — and not already a member — you may be interested in their membership options. These include subscriptions to School Science Review, the source of this article.

Metzler, Johannes & Woessmann, Ludger (2010) The Impact of Teacher Subject Knowledge on Student Achievement: Evidence from Within-Teacher Within-Student Variation IZA DP No. 4999 [PDF]

Abstract:

Teachers differ greatly in how much they teach their students, but little is known about which teacher attributes account for this. We estimate the causal effect of teacher subject knowledge on student achievement using within-teacher within-student variation, exploiting a unique Peruvian 6th-grade dataset that tested both students and their teachers in two subjects. We circumvent omitted-variable and selection biases using student and teacher fixed effects and observing teachers teaching both subjects in one-classroom-per-grade schools. After measurement-error correction, one standard deviation in subject-specific teacher achievement increases student achievement by about 10 percent of a standard deviation.

Discussion Points

The questions SciTeachJC will focus on for this chat are:

• “The results suggest that teacher subject knowledge should be clearly on the agenda of educational administrators and policy-makers”. If subject knowledge is demonstrably critical for teacher quality, should regular subject knowledge CPD be compulsory?
• What efforts have you made since starting teaching to improve your subject knowledge?
• Science teachers often teach subjects other than their own (as do many other teachers). Do you think you do a better job in your own specialism?
• “While there is clear evidence that teacher quality is a key determinant of student learning, little is known about which specific observable characteristics of teachers can account for this impact” — What characteristics do you think are most important in a teacher?

@AlomShaha will be moderating the discussion.

Abstract:

There has been a move in recent years towards the greater inclusion of social and ethical issues within science courses. This paper examines a new context-based course for 16–18 year-olds (Salters-Nuffield Advanced Biology) who are studying biology in England and Wales. The course is taught through contexts and has an emphasis on social issues and the development of ethical reasoning. Examination of a sample of reports written by students in 2005 as part of the course‟s summative assessment shows that utilitarian ethical reasoning is used widely and that the other ethical frameworks to which students are introduced in the course – rights and duties, autonomy and virtue ethics – are used substantially less often. In addition, students mostly argue anthropocentrically though many of them argue ecocentrically and/or biocentrically too.

In this paper I discuss the development, implementation and assessment of a way of teaching about social and ethical issues within a context-based course for 16–18 year-olds studying biology in England and Wales, focusing on the ethical frameworks used by a sample of the students on the course in some of their externally assessed examination material. The course in question (Salters-Nuffield Advanced Biology) was piloted from September 2002 and launched nationally in September 2005 as a result of approval from the national body charged with making such decisions (the Qualifications and Curriculum Authority). The course is taught through contexts and has a strong emphasis on social aspects of biology and the ethical analysis of biological issues.

Discussion Points

Science does not exist in a vacuum, we cannot teach it as a set of facts, figures, processes and ideas without brining in some context of where it fits in to the real world. Real-life applications are used in lessons to help explain what is being learned as well as to provide stimulus material as to the reason why we are learning about this. As scientific ideas become more involved in how we lead out everyday lives the area of ethics becomes more relevant. Is what science can do right?

Yeah, yeah, but your scientists were so preoccupied with whether or not they could that they didn’t stop to think if they should. — Dr. Ian Malcolm (Jurassic Park)

The ethical implications of science is part of the KeyStage4 Curriculum and A Level specifications, but do we teach it well in all aspects of science. The area of Bioethics is great for Biology topics but are there opportunities in Physics and Chemistry lessons. In Michael Riess’ paper he outlines the Ethical Framework that is used in the Salters-Nuffield Advanced Biology (SNAB) course which is now integrated into the Edexcel A Level post 2009.

To construct an ethical argument one of four possible areas could be used to justify your position

• Rights and Duties
• Maximising the amount of good in the world
• Making decisions for yourself
• Leading a virtuous life

Could this framework be extended to be used in all the science disciplines?

The questions SciTeachJC will focus on for this chat are:

• What topics outside of biology do you use to discuss ethics in science?
• Do you think the SNAB Ethical Framework is good enough to be used in all science/ethics lessons?
• How do you get students to think about other people’s opinions and viewpoints in class?

@Bio_Joe will be moderating the discussion.

Useful Links

Bioethics Education Project

Physics & Ethics Education Project

The Simpletons, a KS4 Science Upd8 activity introducing the ethical framework

#sciteachjc discussing RCTs in education as discussed in our Cabinet Office paper http://t.co/2mh508IR via @A_Weatherall

January 15, 2013 1900 via TweetDeckReplyRetweetFavorite

@bengoldacre

ben goldacre

Secondly, Alom doesn’t think people read to the bottom of blog posts , so I’ll post up the date of the next meeting at the top. The next meeting will be held 12th February and @teachingofsci will be your host. The topic is yet to be confirmed, however we’re aiming to do something that is able to have an immediate impact on your teaching the next day in the classroom.

A summary of the discussion on RCTs: (we had 222 tweets during the hour we were discussing)

@DrRacheal opened with the elephant in the room:

Well, I’ve got to say, as an “ex” scientist, it beggars belief why this system isn’t used already. #sciteachjc

January 15, 2013 1936 via webReplyRetweetFavorite

@DrRacheal

Dr Racheal

The responses to the first question: Would you con­sider run­ning or tak­ing part in a ran­domised con­trol trial in your school?

@alomshaha raised the concerns of a few people involved in the discussion:

@DrRacheal Some people think there might be ethical issues and also practical issues of *actually* carrying out RCTs? #SciTeachJC

January 15, 2013 1938 via webReplyRetweetFavorite

@alomshaha

Alom Shaha

@ryansecondarysc responded with a similar point to that made in the Radio 4 documentary:

@alomshaha @DrRacheal #sciteachjc ethical issues are overblown though surely. Less ethical to subject all students to a failing method?

January 15, 2013 1940 via TweetDeckReplyRetweetFavorite

@ryansecondarysc

B R

#SciTeachJC I think idea in paper, that we do all these different interventions *anyway*, mean it’s *more* ethical to find out which work

January 15, 2013 1941 via TwitterfallReplyRetweetFavorite

@teachingofsci

Ian

@oldandrewuk @ryansecondarysc @alomshaha @DrRacheal it’s the teaching practice that’s being experimented on not the students #sciteachjc

January 15, 2013 1948 via TweetDeckReplyRetweetFavorite

@Bio_Joe

Joe Wright

Both @teachingofsci and @miss_mcinerney suggested that RCTs might not be appropriate to run within school due to practicalities.

#SciTeachJC one of my thoughts was that it would work best between schools, rather than between classes.

January 15, 2013 1939 via TwitterfallReplyRetweetFavorite

@teachingofsci

Ian

@ryansecondarysc @alomshaha @DrRacheal #sciteachjc I agree that ethical issues are overblown; practical issues are more problematic.

January 15, 2013 1942 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

This was concurred by others in the discussion. The general feeling so far is that implementing an RCT within a school is problematic.
The idea raised by @bio_joe

Students are given different teachers based on the click of a timetable programme. within an institution can you see who’s good? #sciteachjc

January 15, 2013 1941 via TweetDeckReplyRetweetFavorite

@Bio_Joe

Joe Wright

@oldandrewuk @alomshaha @DrRacheal #SciTeachJC Exactly. Most people place kids in classes carefully based on behaviour dynamics.

January 15, 2013 1945 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

@miss_mcinerney @oldandrewuk @alomshaha And of course classes are set based on pupil performance, too. #sciteachjc

January 15, 2013 1946 via webReplyRetweetFavorite

@DrRacheal

Dr Racheal

What about between schools?

@teachingofsci #SciTeachJC But different schools often wildly different on many variables. This is more of a quasi-experiment than an RCT.

January 15, 2013 1946 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

There were some great ideas about the design of RCTs, mainly between @miss_mcinerney and @teachingofsci e.g.

@miss_mcinerney so recruit, pair schools that are as similar as possible, then randomize which receives A/B intervention? #SciTeachJC

January 15, 2013 1947 via TwitterfallReplyRetweetFavorite

@teachingofsci

Ian

@teachingofsci Why parallel? Why not ‘before’ and ‘after’? E.g. do 1 term w/out then 1 term with? Or compare to previous year? #sciteachjc

January 15, 2013 1948 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

I asked:

Could RCTs be done retrospectively? i.e. find settings that have implemented competing methods and study random students? #sciteachjc

January 15, 2013 1950 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

@A_Weatherall #sciteachjc that would be more of a case study than an RCT, as without pre-agreeing measurable outcomes, open to bias etc.

January 15, 2013 1952 via Twitter for iPadReplyRetweetFavorite

@ryansecondarysc

B R

@A_Weatherall #sciteachjc That’s quasi-experiment. Can be done, and very usefully, you just need to do stats a little differently to account

January 15, 2013 1955 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

@A_Weatherall + with big data from many schools: hierarchical stats to model effects of school/teacher separately, I think #SciTeachJC

January 15, 2013 1957 via TweetDeckReplyRetweetFavorite

@tom_hartley

Tom Hartley

We focused again on the ethics with question two What con­cerns do you have with the eth­i­cal con­sid­er­a­tions of using RCTs in edu­ca­tional research?

@oldandrewuk was concerned about whether teachers would be happy to take part in an RCT:

@ryansecondarysc @alomshaha @DrRacheal #sciteachjc There’s a difference between teachers trying what they think might work & a formal…

January 15, 2013 1952 via webReplyRetweetFavorite

@oldandrewuk

Andrew Old

@ryansecondarysc @alomshaha @DrRacheal …experiment organised by outsiders over-riding teacher’s professional judgement. #sciteachjc

January 15, 2013 1953 via webReplyRetweetFavorite

@oldandrewuk

Andrew Old

@SciTeachJC #sciteachjc My concern is that teacher’s would be made to persist with methods they see failing, in order to test them.

January 15, 2013 1957 via webReplyRetweetFavorite

@oldandrewuk

Andrew Old

#sciteachjc a teacher s professional judgement may be compromised and have a negative impact on student learning

January 15, 2013 1958 via Twitter for iPhoneReplyRetweetFavorite

@DFalc

DFalconer

@oldandrewuk @SciTeachJC in medical trials if the drug is doing really badly then they will stop the study, evalutaion is key #sciteachjc

January 15, 2013 1959 via TweetDeckReplyRetweetFavorite

@Bio_Joe

Joe Wright

RCTs are usually done in medical trials after they’ve been shown not to have a negative effect. Find out which is best #sciteachjc

January 15, 2013 2003 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

@miss_mcinerney made the point that

@A_Weatherall #sciteachjc There’s also evidence to suggest that if teachers don’t agree/believe in the intervention it’s less likely to work

January 15, 2013 2004 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

@miss_mcinerney nocebo by proxy :) #sciteachjc

January 15, 2013 2005 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

@alomshaha asked

@teachingofsci Are there any examples of good RCTs which have been done in education? @oldandrewuk #SciTeachJC

January 15, 2013 2005 via webReplyRetweetFavorite

@alomshaha

Alom Shaha

@alomshaha @teachingofsci @oldandrewuk Examples are work by Carole Torgerson, @ProfConnolly etc #sciteachjc

January 15, 2013 2007 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

#SciTeachJC a lot more is happening in the US it seems — see http://t.co/axzVqrBc for some discussion.

January 15, 2013 2010 via TwitterfallReplyRetweetFavorite

@teachingofsci

Ian

I would like to hear from a teacher that has been involved in an education RCT. What actually happened. #sciteachjc

January 15, 2013 2015 via TweetDeckReplyRetweetFavorite

@Bio_Joe

Joe Wright

Final question: Which educational intervention would you like to see more evidence for, in the form of RCTs (or other methodology)?

@alomshaha suggested his favourite topic:

@SciTeachJC Practical work in science ;) #SciTeachJC

January 15, 2013 2021 via webReplyRetweetFavorite

@alomshaha

Alom Shaha

Comparing SNAB with traditional approaches to #Alevel #Biology http://t.co/w7TAkBPT #SciTeachJC #ASEChat

January 15, 2013 2032 via TweetDeckReplyRetweetFavorite

@Bio_Joe

Joe Wright

but we descended (no thanks to me) into a discussion on ensuring that teachers were aware of any research that had been done and the lack of open availability to research.

#sciteachjc Research has to be made easily available to teachers and policy makers (SLT or gov). We must know why?

January 15, 2013 2022 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

Otherwise we are just going through the motions of implementation. http://t.co/eVOHwP2R #sciteachjc

January 15, 2013 2023 via TweetDeckReplyRetweetFavorite

@A_Weatherall

Alex Weatherall

#SciTeachJC *yes* — clearing house of ed research summaries, classroom ready, funded centrally for us all.

January 15, 2013 2024 via TwitterfallReplyRetweetFavorite

@teachingofsci

Ian

@teachingofsci #sciteachjc I agree. (In fact, I wrote something similar in a blog for @NatEdTrust http://t.co/hM1SMM94)

January 15, 2013 2027 via webReplyRetweetFavorite

@miss_mcinerney

Laura McInerney

Here is a selection of the links posted by @teachingofsci during the discussion

• Effect sizes: http://www.teacherstoolbox.co.uk/T_effect_sizes.html
• What works: http://ies.ed.gov/ncee/wwc/
• Discussion on RCTs in US: http://ies.ed.gov/ncee/pubs/20090049/section_2.asp
• Petty: http://www.geoffpetty.com/research.html
• doceo: http://www.learningandteaching.info/teaching/what_works.htm

There are loads of other valid points that were made, and @bio_joe, @oldandrewuk and @ryansecondarysc carried on the discussion afterwards. In all a good start to the year, and remember the next meeting on the 12th February. (Did you get to the bottom of the post?)

Here is a full Storify archive compiled by @eyebeams (thanks):
http://storify.com/eyebeams/rct-discussion-on-sciteachjc

It’s been 6 months since our last meeting. Both @Alby and @AlomShaha have been unavailable to run the club over the latter half of 2012 and the autumn school term is traditionally a long and stressful term so, despite a few false starts, we’ve not been able to get things up and running again. Until now.

The first meeting of 2013 (15th January at 7:30pm) is inspired by the documentary on Radio 4 last week, where Ben Goldacre discusses using evidence to inform policy, concentrating on how suitable randomised controlled trials (RCTs) might be when used in as evidence for social policies in areas such as crime and education. This programme can be downloaded before Friday 11th January from here http://www.bbc.co.uk/podcasts/series/r4choice. It’s not possible for us to distribute this podcast after this time as it contravenes BBC policy so download it as soon as possible.

There are some useful papers we could discuss however we suffer from the usual problems  i.e.: a lack of open access to research papers. So for this meeting, we will look at the paper written for the Cabinet Office by Dr Ben Goldacre, Professor David Torgerson et al:

Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials.

Haynes L., Service, O., Goldacre, B., Torgerson D. (2012). Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. Cabinet Office Behavioural
Insights Team
http://www.cabinetoffice.gov.uk/resource-library/test-learn-adapt-developing-public-policy-randomised-controlled-trials [download .pdf]

The paper doesn’t concentrate exclusively on RCTs in educational research, but there are some examples within that do suggest that RCTs should be used to form education policy.

To complement the paper here are some other open reference materials that are available:

• Introduction to research: http://www.brad.ac.uk/acad/management/external/els/pdf/introductiontoresearch.pdf (from @Bio_Joe)
• A guide to RCTs in Education Research : http://www.nfer.ac.uk/nfer/publications/RCT01/RCT01.pdf (from @Bio_Joe)
• A Systematic Phonics Review: https://www.education.gov.uk/publications/eOrderingDownload/RB711.pdf
• A series of publications of in Education research using RCTs by Professor Paul Connolly (@ProfConnolly, http://www.paulconnolly.net/) and the Centre for Effective Education at Queen’s University Belfast in Northern Ireland: http://www.qub.ac.uk/research-centres/CentreforEffectiveEducation/PUBLICATIONS/
• The Education Endowment Foundation mentioned on Ben’s radio programme. http://educationendowmentfoundation.org.uk/projects/success-for-all
• Two Educational RCTs in Kenya : http://ipl.econ.duke.edu/bread/papers/0809conf/Dupas.pdf http://www.povertyactionlab.org/evaluation/impact-distributing-school-uniforms-childrens-education-kenya

Questions to discuss during the meeting are :

1. How well would the methods described in the paper work in your school? Would you consider running or taking part in a randomised control trial in your school? Why or why not?
2. What concerns do you have with the ethical considerations of using RCTs in educational research?
3. Do you agree with Ben’s assertion (in the documentary) that RCTs provide better evidence (“the gold standard”) than other methods of research for social sciences.
4. Which educational intervention would you like to see more evidence for, in the form of RCTs (or other methodology)?

The meeting is Tuesday, 15th January 2013 at 7:30pm UTC, to be moderated by @A_Weatherall. Remember to use the hashtag #sciteachjc during the discussion.

Rather than an academic paper, these two articles discuss the effective use of literacy skills in science classrooms. Based on research and experience with young American students in the equivalent of Key Stage 2 and early Key Stage 3, the authors emphasize the need for much greater deliberate teaching of scientific vocabulary. Between the two short articles, suggestions are made for teaching both content knowledge and the ways in which we justify our knowledge, highly relevant for scientific method work.

Ross, D., Fisher, D., & Frey, N. (2009). The art of argumentation. Science and Children, 47(3), 28–31. Download .pdf

Grant, M., Fisher, D., & Frey, N. (2009). Science literacy is > strategies. Clearing House, 82(4), 183–186. Download .pdf

1. These papers are focused on younger students. How do the ideas apply to the age groups you teach?
2. Do you already provide writing frames or templates to promote discussion of scientific methods and the use of reasoned argument? If not, how would you adapt the examples given for distribution to students or as displays in your lab?
3. Do you use any specific texts or sources when providing wider reading opportunities to students? How do they respond to this? How do you assess their progress?
4. Do we take too much for granted in terms of students’ literacy development? How can this be addressed at a classroom, department and school level?

Tuesday 17th July at 7.30, to be moderated by @teachingofsci.

Abstract:

This article offers a review of the major literature about attitudes to science and its implications over the past 20 years. It argues that the continuing decline in numbers choosing to study science at the point of choice requires a research focus on students’ attitudes to science if the nature of the problem is to be understood and remediated. Starting from a consideration of what is meant by attitudes to science, it considers the problems inherent to their measurement, what is known about students’ attitudes towards science and the many factors of influence such as gender, teachers, curricula, cultural and other variables. The literature itself points to the crucial importance of gender and the quality of teaching. Given the importance of the latter we argue that there is a greater need for research to identify those aspects of science teaching that make school science engaging for pupils. In particular, a growing body of research on motivation offers important pointers to the kind of classroom environment and activities that might raise pupils’ interest in studying school science and a focus for future research.

This paper is still the most read and most cited paper from the IJSE and was in the top 10 most cited papers in Science Education Research according to Min‐Hsien Lee, Ying‐Tien Wu & Chin‐Chung Tsai, 2009 (http://dx.doi.org/10.1080/09500690802314876).

Key Conclusions:

• Students are alienated by school science a subject that has an increasing significance in personal and societal life.
• Research has identified this alienation as a problem but has little to say about a solution.
• Research suggests that good teaching and situational interest will improve student motivation
• Students desire opportunities for more practical work, extended investigation and discussion
• A better understanding of what science classroom activities that enhance student motivation is needed; the paper suggests that tasks need interest, importance and utility.

Discussion Points:

• This paper is influential but quite out of date now, how relevant are the findings and conclusions today?
• Is there anything missing from this paper that should be examined to understand student attitudes towards science?
• Are there any issues that we now have a better understanding of?
• Student interest in science lessons declines when they start secondary school, how can we as teachers stop that from happening?
• p1067 lists common aspects of teaching that are perceived to be effective, do you agree with that list? Is there anything you would add?
• “Science is a superb and dazzling hall, but one which may be reached only by passing though a long and ghastly kitchen” How can we as teachers make the kitchen less ghastly?

Thanks to Professor Jonathan Osborne for input on these discussion points. You may also wish to read this update paper (Attitudes Toward Science (Update) Osborne,Simon & Tytler) sent to me by Professor Osborne, the major element of it has been a focus on identity to try and explain the paradox of a society which is so dependent on STEM but which fails to engage many young people.

Abstract: Using a controlled experiment, we examine the role of nurture in explaining the stylized fact that women shy away from competition. Our subjects (students just under 15 years of age) attend publicly-funded single-sex and coeducational schools. We find robust differences between the competitive choices of girls from single-sex and coed schools. Moreover, girls from single-sex schools behave more like boys even when randomly assigned to mixed-sex experimental groups. Thus it is untrue that the average female avoids competitive behaviour more than the average male. This suggests that observed gender differences might reflect social learning rather than inherent gender traits.

This paper is rather long, but a great deal of it refers to the design of the experiment. Pages 22–37 contain the paper’s references and deal with experimental design and statistical results and are secondary to the paper’s conclusions. The key section of the paper is “Section V — Choosing to Compete”.

Key Conclusions:

• Single sex schoolgirls scored higher in the piece-rate round than coed schoolgirls. In the mandatory round there was no difference between single sex and coed schoolgirls.
• A girl who attends a single-sex school is 42 percentage points more likely to choose to enter the tournament than a girl from a coed school and “it would seem that a single-sex educational background has the potential to change the way women view tournaments.”
• A significant gender gap exists only between students from coed schools. Boys and girls from single-sex schools choose to compete equally frequently. Girls from coed schools at 71 percentage points less likely to enter the tournament than boys from coed schools but girls from single-sex schools enter as often as boys from coed schools: “there is no signi…cant difference in the probabilty of a single-sex girl and a coed boy chosing to enter the tournament.”
• A girl from either a single-sex or coed school randomly assigned to an all-female group is 16 percentage points more likely to choose to enter the tournament than a girl randomly assigned to a coed group, “suggesting that the environment in which a girl is placed affects whether or not she chooses to compete.”

Discussion Points:

• Should we encourage the adoption of single-sex groups for competitive work in coeducational schools?
• How can we encourage girls in coeducational schools to be more competitive? And should we encourage these girls to be more competitive.
• To what extent does the sex of their teacher impact upon girls’ decisions to compete or to take a subject at A Level?

Please suggest further discussion points in the comments below.