Knowledge and the Curriculum - A Conversation with Prof. Dylan Wiliam

Welcome to the Future Learning Design Podcast.

Often the debates about how we should be teaching seem to be about instructional methods, but they're really about what we think schools should be doing. And so I hope that I can clarify the source of the disagreement so we can have more productive conversations. If you and I disagree about what education is for, we're never going to agree about methods because we don't even agree about what we're trying to achieve.

Hi and welcome to another episode of the Future Learning Design Podcast. My name is Tim Logan, your host, and this week is a huge pleasure to bring you a conversation with a personal hero of mine and someone who has made a huge contribution to education and teacher development. Dylan William is one of the world's foremost educational authorities. Over the years with partners Paul Black and Siobhan Leahy, Dylan has developed and helped to successfully implement classroom formative assessment in thousands of schools all over the world. Dylan is the Professor Emeritus of Educational Assessment at the UCL Institute of Education in London and the Executive Director of the Learning Sciences International Dylan Williams Centre. After a first degree in mathematics and physics, Dylan taught in urban public schools for 7 years, during which time he earned further degrees in mathematics and mathematics education. He served as Dean and Head of the School of Education at King's College London, Senior Research Director at the Educational Testing Service in Princeton, New Jersey, and Deputy Director of the Institute of Education in London. Dylan's most recent publication, Creating the Schools Our Children Need: Why What We're Doing Now Won't Help Much and What We Can Do Instead, analyzes the approaches that American schools have taken in order to improve student achievement and shows why they are unlikely to succeed, while at the same time providing clear advice about the steps that schools need to take to raise student achievement and close the achievement gaps. His other works include Inside the Black Box, as well as Embedding Formative Assessment: Leadership for Teacher Learning and the Embedding Formative Assessment Professional Development Pack.

Hi Dylan.

Hi there.

How are you?

Good, thanks. How are you?

Yeah, all good. Really happy to be able to talk to you, one of my personal heroes. I've spoken to John Hattie and Gert Biester. I don't know if you know Gert's work, but also heroes. So yeah, a huge pleasure to be able to speak to you today. Thank you so much for taking the time.

Yeah, it's good to be here.

Good.

The first thing I'd love to talk to you about is, as I said in our earlier conversation, a lot of the previous guests on the podcast have been, let's say, towards the more progressive end of the spectrum. And I had a discussion with Dan Willingham about if that even meant anything, you know, because clearly there are lots of different views within education, but there is a sense amongst a lot of the people I've spoken to that there are some challenges that we need to address in education, kind of on the meta level in terms of the purpose of education. So I'd obviously, I'd love to talk to you about your work, your research work, formative assessment, etc. But before we start that, just given that you gave your book the title The Schools That Our Children Need, or Creating the Schools Our Children Need, I wondered if you had any reflections on whether there was any larger questioning of the, the types of schools our children might need. There's something normative in that statement. I mean, what kinds of schools do you feel they need? Or perhaps we've, we've got it right and we should just crack on getting better at what we're doing. But just whether there was any reflection there about that shift of purpose, let's say.

The misleading thing about the title Creating the Schools Our Children Need is that I don't answer the question. But what I do give, I hope, is some tools for thinking about how individual schools and districts might go about answering that question. So I'm quite careful in my own work not to tell teachers what they should be doing. I think teachers have a lot of conflicting demands to balance, and therefore, you know, I'm not going to tell teachers what to do. What I'm going to hope to do is to give them some advice about once they've decided what to do, what they want their students to get better at, what are they going to do differently? And so I think the big idea in creating the schools our children need is that there are probably 4 broad categories of reasons to educate young people. The first, and I think most important, is personal empowerment. The second is passing on civilization from one generation to the next, cultural transmission. The third, at least in democracies, is Preparation for democratic citizenship. And the fourth is preparation for the world of work. And I think we need to do all those things. And often the debates about how we should be teaching seem to be about instructional methods, but they're really about what we think schools should be doing. And so I hope that I can clarify the source of the disagreement so we can have more productive conversations. If you and I disagree about what education is for, we're never going to agree about methods because we don't even agree about what we're trying to achieve. So that, that was the big idea of helping people come to a decision about what they think schools should be doing, and that might be different in different communities, different areas, at different times, in different countries. And the idea is people have some tools for thinking about these things and being explicit about the trade-offs. So there will be trade-offs. There's no perfect assessment system, there's no perfect educational system. What there is, is trade-offs, and. And you can either make those things explicitly and argue about them— yes, we think this is more important than that— or you can end up with them emerging as unintended consequences of decisions taken elsewhere.

Yeah.

And I think the former is better than the latter.

Yeah, absolutely. Do you feel that one particular— I mean, globally, obviously, as I recognize what you've said in contextual differences, whether it's a democracy or not, or a local community versus urban, rural, etc., there will be contextual differences, but Do you feel that there's been a dominance in a particular narrative about education? I mean, Chapter 1 of your book is quite a lot about the changing nature of work, and whether— I wonder whether that economic purpose has, you feel, become quite dominant in the debate about how education should be or improve.

I think the economic purpose has become dominant, but not in a very productive way.

Yeah.

So people are thinking about, you know, how can we give the students these skills? So for For example, recently we've had a narrative around the pandemic would have caused students to lose so many millions of earnings over their lifetimes. And I just think that those kinds of things are just speculative because you can actually argue, as Bryan Kaplan does, that education is largely a positional good. So if everybody has a bit less education because of the pandemic, that will do nothing at all to wages. I'm not saying that's right, I'm just saying I'm much less confident about these kinds of things than other people. So I do think it's important to prepare young people for the world of work because work is fulfilling, it helps you flourish and thrive in a modern society. The reason I focus on that more than the others is because it's the one where things are changing fastest.

Yeah.

So I don't think it's the most important, but it is the one where our existing ways of thinking about things, you know, the great things that have been thought and said, don't change that quickly. But the demands of the world of work are changing very rapidly. I think it's important to recognize that Very few people needed to be knowledge workers 50 years ago. Most of them didn't even need to read or to write in the course of a working day. They received their instructions from a supervisor, and that world is gone. The muscle work is disappearing and the brain work is increasing, and therefore I think it's appropriate to examine whether the systems we have in our schools are right for the kind of things we want, especially given that we have no idea what's coming. I remind people that You know, kindergarten children will still be working in the final quarter of the 21st century. We have no idea what's going to be needed. I think this probably also highlights a big issue of debate between progressives and traditionals. We do want to equip young people with skills, but if we alienate them in doing that, no matter how much we teach them, we'll fail them. Because young people need to be passionate and curious about learning for the rest of their lives, because most of what they need for their lives hasn't been discovered or invented yet. Yeah, so I think there's a trade-off there. We do need to increase standards of achievement, yes, we have to do it in a way that engages young people in learning how to learn and to being curious and open to new experiences rather than sending the message that school is not for me.

Yeah, yeah, yeah. I always admire how well you bridge that gap between the traditional and the progressives on Twitter and in general. I mean, you are universally loved, let's say that, not to fluff your ego, but, you know, I think I've got my own hypotheses about why that is, but what you've said there I think is absolutely right in the sense that it's not always that productive. The discussion, you know, the arguments within the education space are not always that productive, but also the way that we translate some of the emerging understanding of the way the economy changing, etc. The fact that, you know, we have these skills gaps, skills shortages, routinized jobs are going to be automated, etc. The translation work of what do we do with that information in a school context is not always particularly well done or kind of thoughtfully done. And I think you strike me as a pragmatist to some extent. You're, you know, you're trying to just apply the best that we can to be of use to young people right now.

Absolutely. And I think for me, the important point is to empower teachers to make smarter and better decisions in classrooms. And so I think what I try to do most of all is to give people tools for thinking with. You know, there aren't going to be universal solutions. So the important point is, what can I share with people about what I've learned, about what I'm finding out about, you know, various aspects of human flourishing that will help them make better decisions for their students in their local context? There aren't any universal solutions, I don't think.

No.

But there are some that come pretty close, and that's what I try to focus on.

Yeah, but I think just to, just to move slightly backwards to the question of purpose, I think that often that we, we put school and education to work to solve society's ills quite frequently, right? I mean, and the danger of that disagreement, as you said before, between those four different types of purpose, we, we get lost constantly in that debate and actually the thing that carries on is the status quo, even if the status quo isn't quite right. No, I'll take that back. It's never going to be quite right, but that's not what I mean. But the fact that there is these constant kind of navel-gazing debate, in the meantime, the system just keeps on running as it is. And so, and as you've said, there are clearly material or tangible benefits that are afforded by a better education right now, right? I mean, you lay that out, flourishing health-wise, individually, economically. But connected to that, I just wanted to say, how do you see that as being a particularly individual benefit accrued by an increased education, and how much of that is, and perhaps needs to be, more collective? Slightly rhetorical question, I guess, but I, I feel like there's an overemphasis of the individual benefits of an education, and I wonder how we can shift the conversation sideways to some of the more collective benefits?

Well, the difficulty with research in collective benefits is the research questions get very complex very quickly because you have to identify the groups and how they're constituted. But I think there's no doubt that individual benefits become collective benefits. But I think we also need to be careful about who benefits. And the work of an Italian economist, Vilfredo Pareto, is very useful here. Because people often focused on the average. Do we increase the average education? And what he pointed out is that you should be looking much more carefully at whether there's any changes you can make that benefit some people and don't harm anybody else. So for me, often people are talking about raising average levels of achievement, and that can often hurt the lowest achieving students. Or they talk about grouping students by ability, which certainly stretches the higher achieving students in many cases, but may actually lead to harms for lower achievers. And so for me, I think that the, the crucial thing is the lower end of the achievement range. I'm not that interested in, you know, helping a student get an A+ rather than an A. I'm very interested in helping a young person get to the level of skills they need to function effectively in society. And we can make that really quite concrete. So you can argue about the exact figure, but the Organization for Economic Cooperation and Development, the OECD, in its three-yearly testing program, PISA, identifies a threshold they call Level 2, and that's generally around 420 points on the PISA scale, where the international average is 500. And they reckon that 420 in maths and reading is what you need to function effectively in modern society. So for me, the focus needs to be on making sure that every single child leaves our schools ready to function effectively in society. And I think we may not get there, but if we start thinking about every student who leaves our schools without those abilities, think of those students as failures on our part, then I think we will begin to re-engineer our systems. So, so I think the collective issue is, is too difficult to grapple with. To be honest. But I think that focusing on the ones who are least successful in school and making sure that they leave— and, you know, as I said, this is what business people call a big, hairy, audacious goal. We may not get there, but by having as our basis, our goal, every single child leaving our schools ready to thrive in society, then we will re-engineer our schools in a very different way from the way we currently do, where we accept the wide range of achievement, the bell curve, as some people call it. We accept that as being normal. As Benjamin Bloom pointed out 50 years ago, the bell curve is what you get when education is unsuccessful, because the bell curve is what nature gives us. Students do vary in their ability to benefit from average quality teaching, and so if teaching is of average or poor quality, you get the bell curve. If teaching is of high quality, you get a much narrower range of achievement.

Yeah, it, it's still an ideological ambition though, right? I mean, personally I would agree with it as a laudable aim, but it, in a system that is quite fueled by the kind of accrual of educational capital and capital attracts capital economically and educationally, we end up with, I would say currently a situation where you have more attraction of educational capital and, and success accruing to the ones who already have it. Have the cultural capital, etc. Yeah, right.

And I think, but I think that's partly also perversely because of a refusal to recognize that some students find learning easier than others.

Yeah.

So Kathryn Paige Harden's book on the genetic lottery has come up for a lot of criticism because she's arguing there's a genetic component to educational achievement. And anybody looks at the evidence knows this. The fact is that some children are given brains even though they do nothing to deserve it. They get given brains that soak up school stuff. And other children get given brains that aren't quite so good at soaking up school stuff. And there's a genetic component to that. It's not meaningful to describe— to try to define how big that is, but the fact is that, you know, kids come to school with these different abilities to learn. And the question is, what do we do about it? And in the olden days, we accepted the distribution of achievement, that bell curve, as being somehow natural. But as David Hume, the Scottish philosopher, pointed out hundreds of years ago, you can't deduce an ought from an is. But the failure to recognize that some students do find learning more difficult than others, I think, has led to some rather suboptimal practice in schools, because a lot of schools assume that if students try hard, they can all learn equally well, and that is demonstrably not the case.

Yeah, but connected to that, I suppose it's a question of what they're learning, and I'd love to come on to some curriculum questions in a minute, but, but there is also the discussion around the fact that in terms of general intelligence or processing or, you know, whatever, that some people might be more efficient and faster and more effective than others. But given a shift in the what that we're learning in terms of tapping into some of the things you've mentioned already, curiosity, passion, you know, enthusiasm for a particular content of learning that isn't currently available in the standardized system, that's a narrative that is being discussed. You know, would that serve people more generally better at all, all ends of the spectrum economically, would that support better achievement because they're learning something fundamentally that they are really excited about, passionate about, it gets them out of bed in the morning because they're excited about going to school to learn that stuff?

I think there is little doubt, I mean I'm not saying no doubt, but there's probably broad agreement that the kinds of things we focus on in school, the traditional school subjects, particularly mathematics and science, to a lesser extent English, history and geography, they do seem to place a big magnifying glass on a very small aspect of human capability. And so we magnify those differences hugely. And a broader education that includes art and music and dance and drama might allow more students to be more successful in more different ways. So I think there's no doubt about that. My friend Guy Caxton talks about this. The fact is that, you know, if you have a broader curriculum, if you help people find the things they're really passionate about, then they're more likely to be focusing and turning up for school. So I think there's no doubt that ensuring a broad and balanced education reduces the focus on what Howard Gardner calls the logical-mathematical intelligence. There's lots of different intelligences that humans have. School places a very high value on a couple of them, and I think maybe decentering that would certainly make school broader. But I, but you know, I don't know whether that would be good or not. These are very difficult things to judge. There's the biggest trad versus prog debate is that progs think things will be better if their changes are enforced, and trads think that things will be worse if their changes are enforced. And nobody knows.

Nobody knows, right?

And, you know, this is Roger Scruton's argument that conservatives are really people who are much more valuable of the status quo, and radicals want to change the status quo. But we don't understand the kinds of unintended consequences we might get if these changes were made.

Yeah.

And so, you know, that's why we have to be quite careful, and we need to be thinking about what are the trade-offs, what would be better, what would be worse. So that's my major proposal for school reform, really, is for any proposed reform, what would be better if these things are implemented, and what would be worse? And if the person proposing the reform says nothing will be worse, then you shouldn't listen to them because they don't know what they're talking about. There'll always be trade-offs. My argument is they're better made explicitly.

Yeah, good one. I'd love to ask you about Roberto Baldino later, but we're thinking about pedagogy, but taking that question of what children are learning again, you kind of spanning this divide between progressives and traditions. You've been a real advocate for knowledge-rich curriculum, which I'd love to talk to you about because I think knowledge itself, just as a concept, gets tainted on either side. You know, it's a, it's a, it's a hot potato, a political football that gets chucked around. And I, I mean, I think it's fundamentally important, but I think we, we've perhaps also got, going back to your Howard Gardner point, got a quite a narrow view of what that is. So I'd love to talk to you about that a little bit in terms of, so firstly, why do you think a knowledge-rich curriculum is so important in general? And then I'd love to get into some of the specifics around that.

The biggest problem here is that when I say I want a knowledge-rich curriculum, people hear me saying I think children should memorize facts.

Yeah, yeah.

And so the real problem is that Paul Kirschner and his colleagues defined learning as a change in long-term memory, and people say it's more than memorization. Of course it is. To a psychologist, long-term memory is where you keep everything that you don't need to relearn how to do, like riding a bicycle. If you can ride a bicycle, it's in long-term memory. What I find really interesting is we ask people things like, "Do you know how to ride a bike?" We use the word knowledge for all kinds of things, and particularly in English, not so much in Latinate languages, but we also use the word know for people as well. Welsh and French actually have different verbs for knowing people and knowing facts. But so this word know and knowledge is misinterpreted, and so people automatically assume they're talking about memorization. So, so put that to one side. The reason I think a knowledge-rich curriculum is important is because, as the work of John Sweller has shown, what distinguishes novices from experts is primarily knowledge. So if we want to inculcate curiosity, then of course there's a range of things we can do to make students wonder about the world and things like that. But if we want them to be able to do certain things, and you may not want them to be able to solve quadratic equations, but if you do, then it— then the reason I'm advocating a knowledge-rich curriculum is you could have students exploring solutions. But what happens when you have those kinds of problem-based approaches to curriculum is they tend to create advantages for high working memory kids, and they're less successful for low working memory kids. So as the work of John Sweller has shown, there's a strange thing in learning called the expertise reversal effect. So he contrasts worked examples with solving problems, and for people who have a general level of expertise, then worked examples are less effective than solving problems. But for people with lower levels of expertise, then worked examples are a better way to build up those patterns in the brain that help them solve problems. And so what worries me about a lot of the stuff that's being advocated is that students just come in and do stuff and they're engaged and they're happy and, you know, teachers say the students loved it. Yeah, but did they learn anything? And I'm quite old-fashioned, I quite like the idea that students leave my classroom knowing stuff they didn't know when they arrived in the classroom. I'm in the change business. The question is what are those changes? Now I'm very happy that curiosity and resilience and all those other kinds of things should be an important priority, but it can't be all of it, and probably not even 20%— 20% probably an upper bound. The rest of the time we should be teaching students things to think with. So let's take science, perhaps the paradigm case. The whole idea that if you throw an object and there are no forces acting on it, it will never stop moving is a deeply unnatural idea. It's not what the world looks like, but this idea that objects remain in motion or at rest unless acted on by an externally impressed force is one of the most powerful things about dynamics. When I grab a cold door handle on a cold day, especially if it's metal, I feel that the cold is coming into my hand. But after a microsecond, my training as a physicist kicks in and I say, "Ah, no, it's the handle conducting heat away from my hand, and because the handle is metal, it conducts it better than if it was a wooden handle." And these ways of thinking aren't natural. It's taken hundreds of years for us to reach those kinds of insights. And I think that one of the things we do is to give the— our students these tools for thinking with, and that ends up as knowledge in their heads, these ways of thinking. And, you know, you could describe most of science education, for example, as model-based reasoning. You're reasoning using models. And how do students get these models? Well, they could discover them for themselves, and some students are quick enough to be able to do that for themselves, but other students aren't. So my concern is that if we have too much problem-based learning, then what we'll do is just expand the achievement gap by giving an advantage to those who have high working memories at the expense of those with less working memory.

Yeah, so I think there's obviously the— that's where the conversation often goes to, is straight to the pedagogy question and minimally guided instruction and, and all of that versus explicit instruction. I don't have the answer to this, but I've been thinking about this a lot in relation to, I mean, again, it goes back to your point about Howard Gardner, is that we prioritize a very particular type of knowing, which is propositional knowing, right? Like, as you've just talked about, all of those propositions are incredibly useful to understand the world and to interact with the world. But I wonder what your thoughts are on, do we miss other ways of knowing, to use that phrase, that are different to just a, a bunch of propositions, detached, context-free kind of propositions because I heard you say before, I think that, you know, teaching hasn't changed much in 2,000 years. But what has changed in 2,000 years is a very historically specific disconnection, you could say, from our kind of inhabiting the world, if you like, in terms of all of that stuff that came with the Scientific Revolution and Hobbes and computation and allowed us to kind of prioritize a particular type of knowledge that equates to effectiveness, success in the world. We then obviously, it then gets bound up with the economic system and therefore more of that stuff equals more success and financial and et cetera. But I, I just wonder what your thoughts are there.

I think there's a very interesting story here, which is the work of Lawrence Kohlberg on moral reasoning. So he developed some tasks that would explore children's moral reasoning. So the story is that a man called Heinz has a wife who's dying from a rare disease, and the pharmacist has a drug that will cure this disease, but he's asking for too much for it. And Heinz pleads, you know, here's the money I've got. The pharmacist says no. So that night, Heinz breaks into the pharmacy and steals the drug. And the question is, was Heinz right to do that?. And so what Kohlberg did was to look at the levels of re— of moral reasoning in students' responses. So at the basic level, Heinz shouldn't do that because he might get caught and put in prison. So that was a reward and punishment. And that's— he saw sophisticated, more and more sophisticated levels of reasoning about whether it was reasonable for the pharmacist to charge so much for the drug. All this work was done on boys, and when he tried applying it to girls to broaden his findings. He found some rather interesting resistances from the girls. They were saying things like, well, I'm sure they could have talked it out some more and found some reasonable compromise.

Okay.

They weren't willing to play his narrow logical game.

Right.

And Kohlberg, I think, concluded that this was girls suffering from an arrested form of moral development. One of his co-workers, Carol Gilligan, dissented from this. I'm sure she wrote a book— she wrote a book called In a Different 'Voice,' which is one of the founding texts of feminist psychology. And what she pointed out was that, that there are different ways of knowing, there are different ways of thinking. And there's a, there's a very nice book edited by Belenky et al. called 'Women's Ways of Knowing.' And what they're pointing out is that there are different ways of knowing. We've been aware of this for, for decades, and school still prioritizes one way, the narrow logical deductive way. And that has its strengths. But the important point I think we need to understand is the limitations of that kind of thinking. And it may be— this is a given, Michael Gibbon's point— it may be that we've solved most of the problems that we can solve from those narrow disciplinary perspectives. And what we may need in the future is more multidisciplinary work, people coming with different perspectives. Now, that I agree with. What I'm not sure about is how we do that. So my concern is that if we do too much cross-curriculum stuff in the schools, then we will end up with people who have no disciplinary-specific expertise in any subject. They'll just be a mess. So what I think we need to think through quite carefully is having people developing disciplinary strengths, but also the ability to work with people coming from different disciplinary strengths, so we get true multidisciplinary solutions rather than a kind of interdisciplinary mess.

Yeah, yeah, it's interesting because I mean, there's a lot of discussions around how old— to put it crudely— how old you have to be to then start doing multidisciplinary or interdisciplinary, transdisciplinary work, because you've got to build up this knowledge base of disciplinary, you know, facts and expertise in order to— but I also think that there's a broader point about completely different ways of knowing that are, we can of course connect across disciplines and, but there's, that's still at the level of proposition in some ways, right? Or talks about, I don't know, would you, would you agree? I think, I mean, I've never, I've never heard of that work, the feminist psychology work. I, I'd love to look into it, but different ways of knowing and being in the world, which are, which are just completely, they're, they're just, they're totally non-cognitive as far as anything can be. It's not, obviously everything's cognitive to some degree. But the idea of, I don't know, just relating with other people, the knowledge you get from place and, you know, being in a particular place.

So, I mean, I think the really interesting thing here is that we also have to be aware that psychologists do the kind of psychological work that gets published. There's a branch of economics called public choice theory, which is actually pointing out that everybody responds to incentives. So politicians respond to incentives and they do the kinds of things that will get them reelected. Psychologists respond to incentives, and they do the kinds of studies that will actually get them published in psychological journals. And that leads to a quite a narrow focus. So I did some work with some really top-line psychologists involving trying to work in schools with children learning from errors. And one of the psychologists in this project said she hated it. She couldn't wait to get back to her laboratory because she couldn't control any of the variables.

And she—

in a way, she was right to be concerned because the papers that came out of this project have been almost impossible to get published in mainstream psychological journals because they don't control the variables well enough. So I think we have to be aware that the knowledge we have about what works, in that crude sense, is driven by what research got done. And a lot of research didn't get done because it was too difficult. My favorite example of that is feedback research. So almost all of our findings on feedback are related to laboratory studies done by psychology professors on their own students. What we're beginning to discover is the most important thing about feedback is what people do with the feedback, and that depends on the relationship between the giver and the receiver of feedback, much of which has never been researched because it's just too messy. So this again, where, where psychological research, where cognitive science research speaks authoritatively, we should listen. But we need to be aware that most of what we want to do, psychology hasn't got anything to say, and therefore we have to work to rely on our judgment and maybe some models that we found useful elsewhere. Yeah, but that's where I think we go too far, is when we apply solutions in situations for which those solutions were not really designed.

Yeah, the other thing I've heard you talk about is tacit knowledge versus explicit knowledge, and I think quite powerfully the idea of tacit knowledge in terms of teacher development, but I also wonder what's the role of tacit knowledge in a classroom with young people? Maybe then do they not, I don't know, do they not have enough yet because they haven't been explicitly instructed sufficiently to— obviously it depends what you're talking about, but I wonder if that conversation about tacit knowledge is relevant to young people just as it is to professionals in an organization?

Of course.

So I mean, the tacit-explicit distinction— what's really important there is whether we can actually express our own understanding in words.

Yeah.

And so, for example, there's some nice work by Sam Weinberg recently about training students to spot fake news stories, and my guess is that the knowledge that those students developed as a result of that intervention being able to spot that looks like a fake news story. That could well be tacit. Yeah, they might not be able to say how they know it's a fake story, but they've been through lots of— they've looked at lots of fake stories, now they've got a kind of sense, an intuition about is this a fake news story or not. And of course they could then tell you stories about that, but we also know from quite a lot of psychological research that people often confabulate. So when people are asked, why did you do that, they often give reasons that were completely unrelated to the reason they actually did that thing. So I think we have to be a bit cautious, but I think there's the work of Nonaka and Takeuchi on organizational knowledge, I think is relevant here. So I think there is a role for tacit knowledge, but I think we should also be seeking to come up with descriptions of tacit knowledge. So sometimes students may gain tacit knowledge and sometimes it's— there's no point in going any further. So once you can ride a bicycle, there is no point in trying to make that tacit knowledge explicit. Yeah, it doesn't benefit. There's no— you're not going to figure it out. Nobody's figured out how people ride bicycles. It's, it's very complicated. Just leave it. But sometimes taking somebody's tacit knowledge and then trying to get them to explain why they're doing it, particularly at the whole class level— what was there about this article that made you think it was biased? And different students might pick on different things, and then that might produce some explicit knowledge for the whole class. And then they could then try applying that explicit knowledge to something else, like a different set of newspaper articles, to see if those were likely to be fake news or not. So I think that what Noma Kanta Guchi called the knowledge creating cycle of moving from learning by doing, through discussion, through sharing experience, just trying to make the tacit explicit where it makes sense to do so.

Yeah.

But also being aware that sometimes The best you can do it, in fact the only thing you can do, is to recognize that this expertise will always be tacit and it's not— it may not be possible or it may not be cost-effective in terms of time to try to make that explicit.

Yeah, yeah, but then I suppose the assessment system, the summative assessment system, prioritizes very clearly explicit knowledge. It has to by definition.

Well, no, I mean it needn't. We could actually have more practical assessments.

Sure. Yeah.

We, you know, we could actually have practical assessments in a whole range of subjects, but we tend not to do so because they tend to be less reliable because the factors intrude and they tend to be more expensive. Yeah. But I think the important thing is recognize this is a trade-off. And so the extreme example here is America, the United States, which actually spends something like $150,000 educating each child on average across the US.. And yet because they evaluate student achievement with multiple-choice tests, they compromise that huge amount of money. They screw up the way they spend $150,000 for the sake of saving a few hundred dollars on the cost of assessment. So I think we have to be very careful that in fact our, you know, assessments create incentives People respond to incentives, and therefore you may need to actually spend a bit more on the assessment in order to get more productive things happening in classrooms. Yeah, I mean, this is what Lauren Resnick calls tests worth teaching to. So let's have, let's have high-stakes tests because I don't think there's an alternative apart from kind of a kind of patriarchy of alumni going to elite institutions. Let's have high-stakes tests, but let's have them being of such good quality that we're happy if teachers focus on teaching students to achieve on the test. And that would require oral assessments in languages. We'd actually have a spoken language test, maybe even English. We'd have conversational tests. We would actually have an assessment system that assessed all the things we thought were important.

Yeah.

If you don't do that, then when it's— when the stakes are high, people will focus on the things that get assessed.

Yeah, absolutely. Maybe that's a perfect segue just briefly into Roberto Baldino's work because I've never heard anybody else talk about it other than you in relation to— what does he call it? SAM?

Solidarity Assimilation Methodology.

There we go. Yeah. And just that point about incentives of assessment. I think that's such an interesting example of putting into practice something which tries to develop a different set of incentives through the mechanism of assessment. Maybe you could just talk briefly about that. I think that's fascinating.

I've heard him talk about this, but I haven't managed to track it down in any of his writing. He's written quite a lot about solidarity assimilation methodology.

Yeah.

Let's take the standard psychological research from the Johnson brothers and Robert Slavin about effective cooperative learning. Now, when people talk about cooperative learning, they often talk about students working together, and it's, you know, I think broadly it's a good idea that children learn to work together. They then also talk about children learning together, and that's fine, but too often the focus is on the quality of the product rather than the individual learning. Now, to the employer who works with the business team, they don't care who came up with a solution, whether it was one person or all of them. If the best solution is the best solution, it's the best solution. Some people focus on cooperative learning by focusing on the average learning in the group, But I think that most people research cooperative learning focus on everybody learning, and that's why people who advocate cooperative learning say that the big goal here should be individual accountability and group goals, so that every individual member is responsible for putting forward their best learning efforts and those efforts are visible to everybody else. So one way to do this is with an example of this solidarity assimilation methodology. The idea is that we have students in a class working as a team, so they're set to work on Chapter 9 of the maths textbook and they work on this chapter and when they think they're done, each child, each student in the group is tested on the content of that chapter and each member of the group is awarded the score of the lowest-scoring member of the group.

That's right, right.

Yeah, whose identity is never revealed. Now people think this is appalling, especially in individualistic cultures.

Exactly, yeah.

But it's not actually a collectivist approach. It is an individually based approach that maximizes learning for each individual.

Yeah.

And what we're saying is you are responsible for the other members of your group. The aim is that everybody learns this. No one is done until everyone is done. And so There's a trade-off here. You know, anybody objecting to this is— you're absolutely right, the score that student got may not reflect what they actually learned because they actually may have learned far more. But that mechanism creates very strong incentives for every member of the group to worry about the learning of every other member of the group.

Yeah.

Here's the really interesting thing: in football teams, in the orchestra, everybody knows that the whole team's performance, the whole orchestra's performance, is going to be harmed by one person not doing what they should be doing, and therefore you get that cooperation. What is interesting is we shy away from that in the assessment of academic subjects, and I think that, that's the important observation for me, is that if we want to assess students for high-stakes purposes like can a student go to university, maybe individual assessment is relevant, but when you're thinking about how to maximize learning, particularly how to maximize learning for each member of the group, each individual, then I think something like that solidarity assimilation methodology may be a useful way of thinking about this.

Yeah, it's fascinating, isn't it? I mean, and I don't think he did it— as I understand it, I don't think he did it either-or. I mean, I think he was also allowing them to go to do summative individual summative assessments, and both of them were aggregated or pulled together into the credit score for their transcript. So I think it wasn't either-or, but I agree, it's such an interesting way to incentivize effort and group responsibility. I think it's fascinating.

Yeah, I'd go further. I would say that, you know, when you do the solidarity simulation methodology, you shouldn't use that as a contribution to the final score, but you might want to report it separately. So it's— so you're not reporting on mathematics achievement here, you're reporting on ability to support others in a group. So we actually recognize that school achievement is multidimensional, and I think a lot of the problems we have in school achievement are that we want to describe mathematics performance as a single variable rather than saying, well, here's how well somebody worked with others, and here's how good they were on solving routine problems, and here's how good they were at mathematical investigation.

Yeah.

And we add them all together, and we could argue that it'd be more useful, particularly for parents, to have these things reported separately.

Yeah, more useful and more resilient potentially, but less efficient.

Well, you can always—

it's the trade-off.

You can always combine those. Yeah, but the point is, at the moment the testing agencies decide how those variables are combined. Yeah. And I'm saying let's let the user define them. So if you want to take the average across those 6 profile scores, then knock yourself out. But what I'm saying is that's your decision. Yeah. You chose to weight those things evenly or place more weight on this. My worry is that, of course, the old-fashioned ones would probably be given greater weight and the others might well be ignored. So I see an argument for combining them as well. But I still think certainly at the formative level, you know, before the, the final assessments, that more multi-dimensional information would help both students and parents and maybe even teachers make smarter decisions about what to do next.

Yeah, and maybe universities when they're, you know, giving them places on their courses. But just so then, just I'm conscious of time, but one of your, your really interesting talks that I heard you give at ResearchEd was in 2017, was about teaching being research-based, a research-based profession. And obviously you, you are fantastic at kind of always backing up your, your assertions with some relevant research. And so I think you live that way of being, but at the same time It was quite a provocative talk because you were saying it will never— teaching will never be a research-based profession. And I wondered, have you— has that changed in your views, changed at all in, in that respect since then, since 2017? And also, you know, whether it has or it hasn't, what, what is it that you think is so difficult about creating a research basis for what works, to coin a phrase, in teaching?

The important thing about that talk is I was very careful to give a definition of what it would mean— yes— for teaching to be a research-based profession. And the definition I gave was that for the majority of decisions the teachers took during the course of a day, they would have high-quality research evidence that would indicate that a certain course of action was more likely than others to be successful.

Yeah.

And so you can think of somebody in medical diagnosis, you know, you can think of a cardiologist or an X-ray radiographer. And the fact is that most of the decisions they've, they've got, they've got pretty good evidence about this procedure versus that procedure in these situations. Not perfect, but what I'm arguing is that I think it is never going to be the case that there'll be enough research taking into account the differences between different contexts and different content and different personalities. There will never be research that would actually give teachers clear guidance about what to do next. And I'm absolutely convinced. It's almost a kind of mathematical point. You know, there's the famous three-body problem in mathematics, that if you have one ball hitting another ball, you pretty much know what's going to happen. But if there's three bodies involved, then tiny differences in the velocity or the trajectory of one makes the results completely unpredictable. And so, you know, we know this from chaos theory in mathematics. Even the simplest systems can exhibit chaotic behavior. And so what I'm suggesting is that it'll never be possible to get that kind of advice, a kind of clear research-based advice, this is the best thing to do. That's why I tend to talk about best bets. So the research can actually stop you from wasting time on things that are highly unlikely to be successful, like learning styles. Now unlike some of my colleagues, I don't say that learning styles don't exist. I think they may exist, you know, there have been 75 different categorizations of learning styles that I know about, and you know, none of them have been shown to have any utility. But I'm not saying that somebody might not come along in the future and say this one actually works. But right now it is highly unlikely, given the amount of effort that other people have put into this, that you on your own would find a way of using learning styles productively in your own teaching, and therefore it's probably not a good use of your time. On the other hand, being more responsive to your students' needs, a curriculum that systematically builds capabilities in students, those have been shown over quite a large range of studies to be more successful. It doesn't guarantee that you know how to do it, but these are likely to be more effective ways to spend your time. And I think the time thing is really important. I've come to the conclusion that opportunity cost is the single most important concept in educational improvement. People often say, well, what works? And that's not the important point, because even if it works, if the effect is small or it's very hard to get that effect, then it's not relevant. Every hour that teachers spend doing one thing is an hour they don't have to spend on something else. So for example, we now know that teachers with more subject knowledge are slightly more effective, but that's not irrelevant. That's not at all relevant, because the question is, how do you get teachers more subject knowledge?. And it turns out that spending time on teacher subject knowledge doesn't actually produce much gain in student achievement. So my question for every school leader is this: let's say you have 25 hours of teacher professional development time per year. That's a reasonable international average. What use of that 25 hours for your teachers is going to have the biggest benefit for your students? I think that's the only question we should be thinking about. What use of the time we've got. Don't wish for better people, don't wish for more time, there isn't any. Think about what use of the time we've got is going to have the biggest benefit for our students. And then also bearing in mind Pareto's point, you know, even if it's successful for some students, is it successful for all students? Yeah, looking at those trade-offs.

Yeah, and the other, the way I know Pareto is that 80/20, you know, 20% of what you do will—

That's what he's best known for.

Yeah.

So he's known for pointing out that 80% of Italy's wealth was owned by 20% of the population, and people associate him with the 80/20 rule. But his most important insight was that if there's something you can do, if you can redistribute the resources you've got to make at least one person better off without making anybody else worse off, that's a technical decision. If you have to make other people worse off to make some people better off, then that's a very— that's a much more political decision. Amartya Sen has pointed out that even if it's a technical decision, it might not be the best technical decision because you might benefit some people more than others. But I think that general idea of, are there better ways of spending the time we've got before we start thinking about asking for more teachers or more time or more money? Yeah. And often there isn't. Often you are Pareto optimal. The only way to improve this system is to spend more money on it. But in the meantime, let's make sure that the time that teachers have is being spent in the way that has the greatest benefit for their students. And that can also be long-term versus short-term. So I'm not saying— we know that actually teachers who are effective in the short term are often less effective in the longer term, because every teacher builds on the foundations laid by her predecessor. So we have to think about the long-term, short-term thing. But in general, what use of teachers' time benefits students most is the best question to ask.

Yeah, and I mean, as you've kind of alluded to and explicitly said in relation to the context, the entanglement of the context, the messiness of the classroom, the reality. There are— that's also why we need to, in whatever way, invest in the teacher professionalism, because ultimately they are the ones that best understand that context to be able to make hopefully good judgments on the fly as the context evolves.

Absolutely. And that's why I think if you want an improving education system, there should be two non-negotiables. One is that if you're a teacher in this system, I don't care how good you are, you're going to get better, even if you're already the best teacher in the state. So yes, you're going to get better, and you'll be getting better at something that benefits students, which is where the research comes in. I'm not saying you have to follow that research study or this research study, but I'm saying that you should have some prima facie evidence that what you're spending your time on is likely to be of benefit to your students. And that's the end of the top-down bit. You will be getting better at something, and it'll be something that benefits students. And after that, leaders should step back and just say, now teachers, you've decided what you want to do, how can I help you? How can I support you in doing what you think you need to be doing to make your practice better?

Absolutely. And if you're gonna— your best bet on one thing, then I mean, I can't leave you after an hour and not talk about formative assessment. So, you know, your best bet's got to be formative assessment, right? After 25 years in the business.

It has, but it's, it's, it's for a kind of strange set of reasons. So the 5 questions I ask are: does this solve a problem we have? So if you've got good subject knowledge, then increasing subject knowledge isn't going to help. Number 2: how much extra achievement will we get in months per year? And this is where I part company with John Hattie. Effect sizes in education are meaningless. We don't know what they mean. So when a new educational initiative comes across, I want to know how many extra months of learning do you get? And people say, well, it's very difficult. Yeah, but it's the only thing that matters because any improvement in education manifests itself as an improvement in the rate at which students learn. Otherwise, it doesn't improve education. So the fact you find it hard to estimate that is your problem, not the teachers'. School leaders can't make smart decisions about where to invest their time unless they know how much improvement we're likely to get, and then also how much it will cost. In money, sure, but more importantly, as I've said, in terms of teacher time. The fourth question is, can we implement it here? So class size reduction does improve student achievement, albeit at great cost, if you have a plentiful supply of additional teachers. But if you don't, then it's a disaster. So, you know, things will work in one context, not another context, and the local leaders are usually better able to judge that than politicians. In education offices. The last question is, do we know what to do? So right now we know that both curriculum and formative assessment are key drivers, and some quite good randomized controlled trials in the United States have shown that just changing textbooks can improve student learning by 25%. Some textbooks are more effective than others, but of course that finding is probably not robust to changes in curriculum from one state to another. We now know that some curricula are more effective, but we aren't able to judge in advance which curricula are going to be most effective. What we do know now is that encouraging teachers to spend something like 1% of their time supporting each other on developing their use of classroom formative assessment improves student learning by 25%. So it's not the only thing that we could do. It's not even the most cost-effective thing we can do because good curriculum is really cheap. Bad textbooks cost the same as good textbooks. The reason I, I think that formative assessment needs to be more of a priority right now than curriculum is because we know what to do. We know how to support teachers in improving their classroom formative assessment. And so while we're figuring out how to make curriculum better, we should be focusing on helping teachers become more responsive to their students' needs. What one teacher described as making the students' voices louder and making the teachers' hearing better. It's as simple as that.

I love it, absolutely. And yeah, no, and I mean, that randomized control trial, the Education Endowment Foundation one, I mean, that's some quite impressive— for the 1% of teachers' time that I've heard you talk about it, I mean, that's an incredible incredible outcome in terms of enhanced learning achievement for a very efficient use of teachers' time, but also empowering them to be in charge of those teacher learning communities rather than, as you've also said, bringing external, you know, supposed experts in. You know, it's about empowering the people in, in that context to lead that process.

And, you know, obviously the work is flattering because it's an evaluation of a product my partner Siobhan Lee and I developed, but I think what's really important about that study is how it cuts across so much of the received wisdom. People say that professional development has to be subject-specific. It wasn't. It has to be led by external facilitators. It wasn't. It has to involve, you know, extended face-to-face contact on the curriculum that you're dealing with. It wasn't. And so I think it's a useful kind of existence proof of structures that help teachers get in charge of their own professional learning. And support each other in getting better. And, you know, the evidence is that it scales because it was 140 high schools, half getting money, half getting the resources. And so I think that that is why right now my advice is, if you're serious about improving student outcomes, you need to be supporting teachers in making greater use of classroom formative assessment. Right now we know of nothing with a greater cost-benefit than that. Maybe the story will change in 5 years' time, But right now it's a pretty good bet.

Maybe it will be learning styles one day, but I doubt it.

Could be, could be.

No, good. Sorry, I'm very conscious of time. You've probably got another meeting to go to right now, but thank you, Dylan. I mean, an absolute pleasure to talk to you. I can talk to you all day about these things. There's so much in there. I mean, the contribution you've made to the profession and to empowering teachers is, I mean, it's hard to describe. So thank you so much for taking the time today and for your general work. Wonderful.

Thank you. It's been fun.

Brilliant. Thanks, Dylan.

Bye.

Bye-bye. We hope you've enjoyed this episode.

Please feel free to continue the dialogues with our guests, with us on our blog or on social media, or within your own networks.