Demand characteristics: Why remote learning may allow us to finally see what actually happens in lessons.

Determining the quality of teaching in a department is a vital and challenging part of every school leader’s job. This has had a recent twist with the move towards remote education and live lessons. 

Quality assurance is a difficult thing to get right, we often put too much emphasis on metrics and not enough emphasis on our own cognitive biases. I’ve delved into depth about these issues in Middle Leadership Mastery, so I won’t get on my soapbox right now. I will say that educational uber-genius Dylan Wiliam has talked about the difficulty in identifying good teachers by observation many times (try this from Durrington ResearchED loom). However, despite his calls for reason, the simple fact is that lesson observation is the primary tool that most schools use to determine the quality of teaching and in this time of remote learning that is not changing.

One of the things that makes lesson observations less reliable is the role of demand characteristics. Demand characteristics are when a person changes their behavior to meet what they think are the observer’s expectations. For the science lovers out there think of this as the psychological equivalent of Heisenberg’s Uncertainty Principle. This happens all the time in lesson observations. In my experience there are two types of demand characteristics I have noticed most frequently. I’ve decided to call them showponying and signposting. 

Showponying

Showponying occurs when a teacher tries to demonstrate their ability to fulfil all the school’s metrics in front of the observer. This leads to the lesson observation lacking authenticity as the choices the teacher makes are not the same as when they are not observed. It also makes feedback obsolete as you will be advising changes to strategies that are not normally used. In the early 2000s when performance management was linked to 3 high-stakes observations this was really common.

Signposting

Signposting is less of an issue. It’s the process whereby a teacher over narrates the activities of the lesson as a way to communicate to the observer what they are doing and why. Instead of saying “Now, class, we will complete questions 1-5,” it becomes “Now, class, we have just looked at two examples I have completed under the visualiser, then we did three partial examples, now I would like you to complete q1-5 which I have chosen based on your prior performance in last lesson.” It’s not the end of the world but it takes time and increases the complexity of the instructions, which is not beneficial to the students. This also often results in teachers continuing to talk throughout the task instead of letting the class work in silence.

Does remote learning offer us an opportunity to change the way we observe lessons?

I wonder if the ability to record live lessons, or even view pre-recorded lessons can really help remove these demand characteristics. For some time I have liked the idea of staff sharing videos of lessons routinely with line managers. This way they can get valid feedback on lessons that have been delivered without demand characteristics. It also solves the logistical issue of timetables: now I can observe classes that occur when I am teaching. Video lessons are also the best way to share practice. The videos, or more likely edited clips, can be shared with the entire department to illustrate good strategies I have observed and want to share. I know some schools have managed to find ways to regularly record their lessons and share them, by using expensive technology or having it built into GDPR risk assessments and parental permission slips, but so far I have been unable to make it work in my department. I am hoping that a small silver lining of this dreadful pandemic is a willingness to embrace recorded lessons as a tool for professional development and quality assurance.