Five ways that the implementation of evidence-informed practice can go wrong

By Andy Tharby

It is early September, the first INSET of the school year. The school hall is a hive of expectancy, brimming with freshly rested minds and newly bronzed bodies. After a few moments, a member of the senior leadership team responsible for teaching and learning rises to the lectern. It takes a few more seconds for the hall to quieten to a hush. He clears his throat and begins:

“Our teaching and learning focus for the coming half-term is going to be retrieval practice. We expect teachers to implement this in every classroom and we would like every lesson in every subject to begin with a quiz based on prior learning. Thank you.”

He sits down again and the meeting swiftly switches to a discussion of the school’s new behaviour policy. Job done.

*

Although the example above is intended to be satirical, it is an unfortunate truth that poorly planned and insufficient implementation is rife in schools. It is one thing to know that a certain piece of evidence is likely to improve learning across a school or a department, but quite another to implement it in a way that will ensure that it has the intended impact. The EEF have produced a very useful guidance report entitled ‘Putting Evidence to Work: A School’s Guide to Implementation’. The following paragraph comes from its introduction:

Implementation is a key aspect of what schools do to improve, and yet it is a domain of school practice that rarely receives sufficient attention. In our collective haste to do better for pupils, new ideas are often introduced with too little consideration for how the changes will be managed and what steps are needed to maximise the chances of success. Too often the who, why, where, when, and how are overlooked, meaning implementation risks becoming an ‘add on’ task expected to be tackled on top of the day-to-day work. As a result, projects initiated with the best of intentions can fade away as schools struggle to manage these competing priorities. (p. 3)

The report begins by outlining two foundations for good implementation: treat implementation as a process, not an event; and create a leadership environment and school climate that is conducive to good implementation.

It then provides guidance on four vital stages of implementation:

Explore – define the problem you want to solve and identify appropriate programmes or practices to implement.

Prepare – create a clear implementation plan, judge the readiness of the school to deliver that plan, then prepare staff and resources.

Deliver – support staff, monitor progress, solve problems, and adapt strategies as the approach is used for the first time.

Sustain – plan for sustaining and scaling an intervention from the outset and continually acknowledge and nurture its use.

The remainder of this post will explore five mistakes that are often made when implementing a new teaching and learning strategy or evidence-informed approach. As an example, we will use the implementation of retrieval practice (currently in vogue) across a school; however, the five mistakes discussed below could apply equally to any other new school-based initiative:

1. The initiative solves the wrong problem. Retrieval practice may be simple and fashionable, but can it really solve the problems you have at your school? Might it solve problems in one area of the school but not another? If retention of information in the long-term memory is your issue (and it often is) then you should probably think about implementing retrieval practice, but if the issue concerns a general lack of student effort or perhaps problems students have with structuring and sustaining extended writing, you might be better off shifting your focus to these instead. It is wise to remember that a strategy like retrieval practice is a solution to a learning problem and that ‘lack of retrieval practice’ is not necessarily a problem you need to solve.

2. Insufficient or untimely training. Before implementing your change, you need to prepare thoroughly – which takes much planning and forethought. You should take the time to create a shared understanding among teachers and leaders of how and why retrieval practice can work effectively. This initial training should have a wide reach across your staff force. Ideally, you should aim for conceptual change: your teachers need to understand that retrieval practice works on the principle that using your memory improves your memory. To do this, they will require an overview of the available research evidence on retrieval practice, many practical examples of how it has been implemented in a variety of contexts and an understanding of the limitations of the evidence and associated practice. If training is not provided up-front, then you will find that teachers develop inflexible knowledge or misconceptions – for example, that retrieval practice is always synonymous with quizzing. Your teachers, therefore, may struggle to adapt their delivery to suit subject, topic and class differences.

3. Too much flexibility. As part of the training process, you will need to share the active ingredients of the strategy – in other words, the key elements that cause retrieval practice work. This will ensure that the practice in your school maintains fidelity with the original research findings. For example, you might choose the ideas that retrieval practice tasks should be low stakes, should be challenging but achievable and should centre on important concepts within the topic and the subject. If you give too much flexibility and do not stipulate the active ingredients – i.e. you create the conditions for what might uncharitably be dubbed a ‘free-for-all’ – it is likely that you will see one or more of these unintended consequences: teachers who do not implement the strategy at all; teachers who implement the strategy wrongly or poorly; or teachers who implement the strategy in place of other more effective approaches.

4. Too little flexibility. Schools and classrooms are incredibly complex places. Interventions will look very different in different contexts. Retrieval practice will not look the same in English and maths for example. It is also unlikely that a new implementation will be rolled out perfectly in the first instance; it will almost certainly require some adaption and careful reconsideration. If the implementation of retrieval practice becomes based on a series of hard non-negotiables, then it becomes difficult to shift the focus if or when this is required. It is important that room for flexibility and time for evaluation are built into any implementation; however, these should not be at the expense of the active ingredients, otherwise your evidence-informed intervention might mutate into something quite unexpected, perhaps even a Frankenstein’s monster of the original findings. It is also worthwhile pointing out that if you have chosen to become an evidence-informed school or practitioner, then you will have to accept that the evidence is sometimes contradictory and that you are willing to be proved wrong.

5. Switching to something new too quickly. It takes time for any new implementation to reach every corner of a school and have a genuine impact on student learning. Any new implementation will have its enthusiasts and early adopters but it will also have those who, for a number of reasons, do not accept the change in the first instance. It is important to monitor the impact of the implementation to celebrate and acknowledge success, and to provide further CPD and coaching opportunities where needed. Probably the worst thing you could do is to move on to a completely new focus next term. All new implementations require time, patience, honesty and regular renewal or rejuvenation if they are to fulfil their potential. By its nature, implementation is always a slow burner.

This entry was posted in General Teaching, Leadership and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s