Scientific teachers. Taste the words. What feelings do they evoke? Curiosity? Scepticism? Indifference? It probably varies depending on who you ask. But regardless of what you feel as a teacher, since 2010 there is a law in Sweden requiring teachers to work scientifically. Which seems reasonable, if the alternative is teachers working unscientifically. However, it may still be appropriate to start a methodology book on the subject with a little in-depth study of what science in education is actually good for.
But first a delimitation. This book is not about whether the knowledge content taught is scientifically based or not. The focus is on science as a verb. In other words, it concerns teachers who actively participate in educational science work on pedagogical and didactic issues, as consumers or as co-producers of research.
Why scientific teachers – two approaches
Science in education has two main approaches. One is about what ‘works’ in general, according to university researchers. This approach is usually referred to as the scientific basis, which means that teachers are expected to keep up with the pedagogical development of the profession and try to apply modern educational research findings in their teaching, mainly through professional development. Educational research is extensive and has made many advances in recent decades that can benefit students.[1] The more that teachers participate in various forms of science-based professional development, the better education we hope to see. Government initiatives are common, such as the Reading Lift and the Maths Lift.[2]
The second approach to scientific teachers involves teachers being involved in identifying what ‘works’ for them. This is often referred to as ‘tried and tested experience’, which means that teachers collectively and, at best, systematically, try to find ways of working that help them to improve student achievement in different situations. Some terms that capture this phenomenon are collegial learning, action research, school development and systematic quality work. Tried and tested teacher experience is supposed to lead to increasingly skilled teachers, which in turn means that students learn more, better and deeper. At least that is the hope.
Progress in other sectors of society can also serve as inspiration for educators. Scientific learning in the medical professions through clinical research has led to major advances in human health and longevity for over a hundred years. Doctors learn about what works by trying to help their patients in different ways based on various more or less well-founded hypotheses, and then carefully document their patients’ progress in medical records. It is now hoped that education via scientific basis and proven experience will be strengthened as strongly as in medicine, and that teachers will now be inspired by doctors’ clinical science.[3]
The two approaches are problematic extreme positions
Both approaches to teachers’ scholarship are problematic. There are few general truths in educational science.[4] Teachers can rarely get simple answers from external researchers on how to improve their teaching and their students’ learning. It is also methodologically difficult for teachers to exchange and trial each other’s unique personal experiences. Practices that work well for one teacher in one classroom may not necessarily work as well for another teacher in the next classroom. Unfortunately, there are few examples of successful attempts to work scientifically with teachers’ own unique experiences.[5]
These practical everyday problems for teachers have deep roots in philosophy of science. General objective truths are often sought through numbers and statistics. It can almost be described as an obsession among people to quantify their experiences.[6] PISA surveys and annual student and staff surveys are two telling examples of this numerical exercise. However, no matter how rigorously you go about it, you rarely succeed in producing figures that are practically relevant to teachers in their daily lives.
Instead, subjective truths are often sought through time-consuming methods such as interviews, logbooks and observations. This usually generates large amounts of text and thus excludes those teachers who do not have the time to devote to such data collection and analysis. Yet teachers working in this way rarely meet the minimum scientific standards of reliability and generalisability.[7] Researcher Donald Schön (1995, p. 28) summarises the dilemma for practitioners as follows:
The practitioner is confronted with a choice. Shall he remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?
The purpose of this book is to try to give teachers access to a new solution to this fundamental and almost timeless methodological problem. If the endeavour succeeds, teachers can be supported in combining rigorous science with relevant problem solving in everyday teaching. The ambition here is thus to describe a solution that is both rigorous and relevant at the same time.
A golden middle ground – critical realism
In this book, we will find a middle ground between the two problematic extreme positions. In between general and personal truths about what “works” in education, there are several promising methodological approaches that can help teachers in their daily work. These approaches will be described and explained in detail in this book. The middle ground is illustrated in Figure 1.1 below.
Figure 1.1 Critical realism is a promising middle ground between rigid general truths and vague personal experiences.
Rather than getting stuck in naïve and rigid objectivism (the search for general laws/truths) or, for that matter, vague and fuzzy subjectivism (that each individual has his or her own unique truth), this book is about how teachers can try to strike a good balance between the two, building on the strengths of both extreme positions. Philosophers of science have called this middle ground critical realism.[8] It is about being critical of the initial impressions of the things being studied, but still trying to find patterns.
Collection of teachers’ situationally dependent collective experience
In part two, we will go into more depth about what critical realism can mean for educators. But first, here is a brief practical introduction. Critical realism is about trying to see the hidden social mechanisms that govern people’s thoughts, feelings and behaviours, even when they are not fully aware of them. The focus is on what happens at a detailed level in the intricate and complex interactions between individuals in everyday life. The aim is to increase our understanding of the weak but important cause-and-effect relationships that exist in society. Despite the complexity of education, there are likely to be approaches and ideas that work better than others in different situations.
Critical realism focuses on precisely those situations in teachers’ daily lives where the relationship between cause and effect is most evident to them. In practical terms, this involves collecting and analysing data and reflective text from teachers about such situations. Data collection must therefore take place continuously in everyday life, precisely at those crucial moments when teachers see how a particular idea or method worked in practice for students. Such moments are often fleeting and emotional, either positively or negatively charged or both at the same time.
The middle ground of science is not without challenges. Regularities and cause-and-effect patterns are often weak in social contexts. Therefore, scientific teachers need to collect data from many different situations and from many fellow teachers. Otherwise, even strong patterns will not show up in the analysis. Numbers help the scientific teacher to look for patterns, and reflective text provides a picture of how and why these patterns emerge in different situations.
Education’s need for a scientific methodological middle ground
Why then might education need a scientific middle ground between extreme positions? It has mainly to do with how poorly the two established approaches to scientific teachers have worked so far. Educational researchers struggle to help teachers, and teachers struggle to help themselves through science. What unites the two approaches is their relatively low reputation, especially compared to other sectors of society, such as medical and natural science research.
The basic thesis of this book is that the problem with the two established approaches is a methodological problem. Established scientific methods such as surveys, interviews and observations do not work very well for scientific teachers. These methods are difficult and time consuming to apply. They rarely lead to interesting conclusions for the teachers themselves. The results are also too often uninteresting to a wider audience of colleagues. Perhaps it is too early to call the established methods dead ends for teacher scholarship, but it is not too early to consider alternative routes.
Teachers need access to a scientific method that works well in their daily lives without interfering with their important work. A method that can help them to scientifically develop their own colleagues and the profession as a whole, based on their unique situation in their own school. A method that delivers better education for students often enough. A method that is easy to use.
Choosing a scientific method – an impossible task for teachers?
It is no easy task for scientific teachers to choose a methodological path in the jungle of scientific methodology. Most of the time you end up with surveys, interviews or observations by convenience. But research methodology is more complicated than choosing between three methods of data collection. Figure 1.2 is called the ‘research onion’ and illustrates the complexity of designing a research study. If you also have a full-time job as a teacher, scientific methodology quickly becomes an almost impossible task. The only realistic way forward is for an external methodological researcher to make the choice for teachers. That is exactly what this book tries to provide – a set of methodological choices that work well for teachers.
In part two, we will describe in more detail the main theme of the book, a scientific method we have chosen to call designed action sampling and which this book is basically about. But already here we can see that the method is based on a number of carefully tested choices. These choices are shown in Figure 1.2 in bold italics.
Figure 1.2 The research onion shows the different methodological choices that need to be made in each research study. The methodological choices made in this book are shown in bold italics. Adapted from Saunders et al (2019).
Data collection – the missing link for scientific teachers?
At the heart of the research onion are data collection and data analysis. This is the core of scientific work and cannot be avoided if you want to make any claim to be scientific. Structured data collection is about documenting insights, which takes time. With the already heavy burden of documentation, it is therefore not surprising that many teachers feel sceptical about engaging in scientific work. Do we really have time for more documentation? Yes, we probably have to, if we want to realise the benefits of scientific methodology. But this need not be such a problem if we can find ways of collecting data that are simple and time-efficient, and thus work in teachers’ daily lives. This book describes one such way.
Data is always collected in order to be analysed, and a good analysis is only possible if the data collection has also been done well. The purpose of the analysis is to gain new insights into the mechanisms behind good teaching in different situations. This allows teachers to do more of what works well in each situation and less of what works poorly. Teachers who are aware of which mechanisms work well in which situations are more successful in their job.
Despite the fact that data collection is at the very core of teachers’ scholarship, it is the activity that works most poorly today in school scholarship. In a detailed review of various initiatives around Sweden, the school developer and investigator Lena M. Olsson (2018, pp. 78, 117, 119) bluntly states:
There are no signs that, within the framework of the models [for collaboration between researchers and practitioners], there is a more systematic compilation and analysis of teachers’ documented experiences of how changes in working methods affect students’ learning […] Systematic documentation of teachers’ experiences based on evaluation methods is often lacking in education and in skills development initiatives […] Without documentation as a basis, it is difficult for individual teachers to compare their observations from different occasions and see patterns in how teaching has changed.
Internationally, data collection is also one of the key gaps in educational development. Bryk et al. (2018, pp. 124-125) of the Carnegie Foundation for School Improvement in the United States state that educators need to learn from healthcare:
You cannot improve on a large scale what you cannot measure. […] For both teachers and students, time is a very limited resource. […] Measurements that will provide a basis for improvement work must be woven into the everyday pedagogical work. […] The lack of practical measurement methods is one of the most important differences between school and health care.
It is probably unrealistic to believe that we will have better teaching if we do not also engage in structured measurement of the effects of various initiatives. In order for such measurement to be woven into everyday teaching, each measurement session must be so simple that it takes around three minutes or less.[9] Later in the book we will show how this can be done.
Teachers as scientific leaders for their colleagues
For teacher-led science in education to work well, teachers need to take a great deal of responsibility in leading different aspects of the organisation’s scientific work.[10] Although managers have the formal and ultimate responsibility for an organisation’s pedagogical development leadership, in practice it is not appropriate or even possible for a manager to take on such responsibility alone.[11] Developmental leadership needs to be distributed among teachers and others who wish to exercise informal leadership or are expected to exercise formal leadership over their colleagues in the organisation.
Science also requires the participation of many, or even better, all teachers in an organisation. It is not good enough for a small group of teachers to do science in isolation. Because then patterns and connections become difficult to see, sceptical colleagues do not participate with their critical perspectives, comparisons are difficult to make and results are difficult to transfer to other teachers. Rather, successful educational development is characterised by unpretentious and boundary-crossing developmental cooperation, involving many different professional roles in the organisation and with some teachers exercising leadership over their colleagues within the framework of clearly defined forms of cooperation.[12]
Figure 1.3 shows a common distribution of roles in teacher-led educational science. Peer learning leader is a common title for those staff members in the school who lead other teachers’ scientific pedagogical development as well as have a leadership role in their own classroom.[13] Experts can be researchers at a university or other type of authority in a relevant area of development for the organisation.
Figure 1.3 Four key roles in teacher-led educational science.
Peer learning leader – a difficult role in dire need of a simple scientific methodology
Some synonyms for peer learning leaders are supervisor, team leader, development leader, subject leader, process leader and middle-ground leader.[14] In most cases, peer learning leaders also have the role of teacher at the same time. A special needs teacher or manager can also usefully take on the role of peer learning leader[15] .
There are many demands on the shoulders of peer learning leaders. They must lead and support the development work of many or all colleagues. In practice, this means creating clear structures, getting colleagues to actively participate in teacher-led research and providing them with feedback and encouragement after completing developmental actions. At the same time, they have to do their own job as teachers, often without any reduction in teaching time.
They must also ensure that the teacher-led research is scientific, sustainable and actively supported by managers. They should often also have contact with one or more experts, or be well informed about what different experts have concluded on a particular educational development issue.
Simplicity and clarity in scientific methodology is therefore not only desirable for peer learning leaders. It is rather an absolute necessity and a prerequisite for making teacher-led science work well in an educational institution such as a school, a university or a preschool. This book was written to support all those peer learning leaders who need a proven, simple and clear science methodology to lean on. But the book is also for managers who want to gain deeper insight into the methodological issues of teacher-led science. After all, it cannot be easy to be ultimately responsible for scientific work without also having some insight into scientific methodology.
Simplicity in science also requires a fairly high degree of standardisation. It is not compatible with simplicity and clarity for each teacher or team to be able to choose freely from the research onion in Figure 1.2. The method described in this book is therefore best used by all staff in the organisation. Those teachers who want to do more research can always use other methods in parallel.
[1] For an overview, see Håkansson and Sundberg (2012).
[2] For a recent review, see Kirsten (2020).
[3] For a paper on clinical research teachers, see Carlgren (2010).
[4] See Biesta (2007), Pring (2010) and Olson (2004).
[5] This is described in detail by Olsson (2018).
[6] Read more in Barrett (1979) and in Porter (1996). See also Bornemark (2018).
[7] Read more about challenges with research involving teachers in Carlgren (2010). For criticism of teachers’ scholarship, see Pring (2010, s. 136-140).
[8] A pioneer of critical realism is Roy Bhaskar. For an accessible overview, see Sayer (2010).
[9] According to Bryk et al. (2018, s. 130)who describe their experience in designing a practical measurement method.
[10] See e.g. Scherp (2013), Rönnerman et al. (2018) and Kroksmark (2019).
[11] Read more about this in Forssten Seiser (2019).
[12] See Jarl et al. (2017, s. 69-93).
[13] See Scherp (2013, s. 95).
[14] See Rönnerman et al. (2018, s. 23-29).
[15] Read more about their role in school development in Bengtsson and Kempe Olsson (2019, s. 183-187).
