Assessments and Innovation in Education: A Barrier and Opportunity

Welcome to the future of education. And now here's your host, Michael Horn. Welcome, welcome, welcome to the future of education. I'm Michael Horn and I am delighted by today's conversation that we have for you. Because in my mind, assessment is single handedly both one of the significant bare barriers that is holding the education system back, but it's also one of the biggest opportunities for innovation that can propel it forward to the learner centered education system that our students need, our society needs and so forth. And I would just say as a bit of prelude before I introduce my guest on today's episode, the problem with today's assessments in brief, are that they really are a reification, if you will, of today's time based learning system. In today's system, right, we offer learning experiences. We test and assess students make progress to the next unit, the next subject, the next grade level, whatever it might be. And only afterwards do they get the results with very little ability to do anything with that feedback. Or maybe it's a formative assessment that they get in the context of their learning, but it's not a something that actually really helps with their transparency around their learning. It's often not a high quality assessment of any sort. And so they don't really get credit for those sorts of learning experiences in any meaningful way. And what's interesting is as a result, assessment often becomes synonymous with an autopsy in our education system. It's just a basically a view of what the student did, but not an opportunity to improve their learning. Which frankly is a shame because in a mastery based learning model we would offer learning experiences to students. We'd still do testing and assessment, but that testing and assessment would essentially be almost real time or interactive to inform what a student did next, and only when they truly demonstrated mastery would they move on. It's a very different model for assessments and it requires a very different model of thinking about valid validity and reliability and so forth. And in that model, assessments could be shorter, they could be more frequent, and they could be both for and of learning. So break this trade off between that formative and summative by performing both. Our guest today, Arthur Van der Veen, the CEO of New Meridian, knows a heck of a lot more about all of this than do I. And he's in my mind helping create these innovations in assessment to really propel the system forward toward this much more learner centered vision that I have outlined. Arthur, first, welcome to the future of education. It is great to see you. I'm excited for you to talk to us Teach us and correct everything. I just messed up about the future of assessments. But thanks so much for being here.

Michael. It's great to see you and a real pleasure to be here with you.

So let's just dive right in. Like tell us a little bit about your own background and what is New Meridian.

So I've been 25 years working to improve K12 education in the US both in the for profit, nonprofit, public sector, private sector. And a few things have animated me over those 25 years. I developed my expertise with assessment when I was at the College Board and when I led assessments for the New York City Department of Education where I also had the opportunity to help launch and run the Innovation Zone under then Chancellor Joel Klein. An interesting intersection because the IZONE was all about how can we develop and support schools in developing more student centered personalized learning models, doing that work while also running all of the assessment programs for New York City Department of Education. And so you just kind of hit the nexus of kind of my career passions about how can we make assessments better support innovative, new, more personalized student centric learning models.

So it's a fascinating background and again to my mind it's like maybe the most important topic in education to think about how we actually move meaningfully forward. I'm curious just to like start with a little bit more history, which is no Child Left behind ushers in the minds of the public at least these high stakes assessments arguably, right. They'd been going on for a decade at least before no Child Left behind in the States and so forth. Then there's the Common Core, there's Smarter Balance park, all these sort of innovations on top of those summative assessments. And then people start to say, well, gee, maybe we want a little bit more innovation in this space. But they, they kind of restrict it to grade level assessments and things like that. I'm just sort of curious, where in your mind is the state of state assessments at the moment as you look at the landscape.

Starting from the end and working backwards? Perhaps we've all just experienced perhaps the biggest disruptive shock to our education systems in the pandemic over the last two, two and a half years, which I think put in question many of the assumptions around the NCLB era accountability models. We as a country just had a crash course in hybrid and blended learning where the locus of education instruction moved from kind of school based standardized central education instruction into our dining rooms and our dining room tables. And that has been, I think will have a lasting disruption. Technology enabled more personalized learning Models have been anchored into not only our schools but but our parents and our family's expectations. So I think it's opened up space for real innovation. NCLB did tremendous things. It's part of the original 1965 ESEA, which was commitment to civil rights. We need to know that every school is serving all of our students equitably and fairly. NCLB helped us move along that path tremendously. But there is room for improvement, to be sure, in our state level assessment and accountability models. And they are a constraint on innovation as we try to get to more student centered learning and assessment supporting those models.

So I'd love you to just dive in and in a moment I want to get into what you all are doing at New Meridian to break that down. But just stay with us for a beat on that, which is how are the current assessments still holding us back from these personalized models that you've advocated and tried to push at the Innovation zone and every, you know, elsewhere in your career? How are the current assessments holding us back from that specifically in your mind?

I think it's the power of the accountability regimen that we operate under. And so we currently have a single end of year snapshot of student learning which reduces a student essentially to a single score. And that's not in the spirit of kind of student growth and learning. And yet our states, our districts, our schools, they are so constrained in order to meet the kind of technical requirements to ensure that those scores are valid and reliable, comparable, all the things they need to be. But wouldn't it be great if we could develop a model that is more instructionally valuable, more closely aligned to what's taught in the classroom, deliver more data and information to teachers and students and families along the way, and then in aggregate try to build that technically valid, summative score from those multiple assessments that provide a more holistic view of student learning throughout the year. That's the vision that we're working on here, enumerating.

So let's go right there, which is Numeridian is trying to build these valid, reliable pictures of student learning. It sounds like much more frequent throughout the year. What's the actual work look like, what's the product look like and what, you know, what sort of, who do you work with? Is that at the state level, district level and so forth?

Yeah, we are, we're piloting this, what we call instructionally aligned assessment system right now in Montana and Louisiana. We both states won each one a competitive grant for state assessments Grant, CGSA grant to help Fund that work and philanthropy has been also helping us fund the design and development of this early pilot. The challenge, if you can imagine, of aligning assessments to what's taught throughout the year, from a state's perspective, they can't impose a single curriculum on districts. There's too strong a commitment in this country to local control. And so it's always been the challenge of how do you align to what's been taught when you're trying to get that summative score. That's why we do end of year assessments, because at least we know based on the standards, students should have mastered this by the end of the year. Once you start to press into what's taught, when then you start to get challenges. So the way we've tackled that is we take grade level standards and from an instructional perspective and a curriculum perspective, how can we create mini assessments, a collection of maybe 18 to 20 mini assessments, micro assessments that can be flexibly selected, sequenced, stacked and administered throughout the year. So irrespective of what curriculum you're using, maybe it's a high quality instructional materials, a green rated curriculum, or maybe it's something that's put together locally at the district, you can create a configuration of assessments throughout the year that cover the standards and support that end of year summit of school.

So let me just make sure I understand. So basically you have a body or almost a library of of assessment modules, if you will, and based on the curriculum and what a student is working through. And I guess if I've chosen to go a certain content approach, another district or state has gone a different content curriculum approach, we can pull off in modular fashion, almost different assessment or packages of assessments, it sounds like that have been vetted, they're valid, they're reliable, and deploy them alongside the learning so that we can get that instant feedback, it sounds like, about how the students are learning. Am I starting to get that right?

Absolutely. And in Louisiana and in Montana, we're working with selected districts for this first year who are using different curricula because we want to show that each district can select and sequence according to their instructional plan. And we want to do the research and development then to validate the cumulative score and see that it serves for those summative purposes.

So that's super interesting because now I guess that's the other question I have, which is what are the role of the traditional end of year tests in this version? Is it something that they're sampling against to just say, hey, did the score that New Meridian put together for the Student predict accurately and in line with and so we don't need this end of year summative assessment. Are you expecting maybe more precision? What's the benchmark and the role of those old assessments, if you will, as you move into this new model.

For a transitional phase, we will use them as a benchmark. It's part of the validity research. We want to see how we categorize a student in terms of mastery of grade level standards versus how they do on the end of year state summative. And we'll do those analyses to see how reliably both approaches determine whether that level of mastery. Ultimately, the ideal is that we don't need that end of year summative because we've proven out that this approach both provides that real time instructional value back to the teacher and student and provides that comparable reliable measure of grade level mastery by the end of the year. And there's some other advantages of doing multiple measures throughout the year. Of course, as you started to describe earlier in a mastery based learning model, you want the feedback. You need the feedback because it informs how you correct and adjust where you go next, what kind of precursor standards or understandings the student doesn't have. You want that real time information in a mastery model to move the student along. So that's the value of doing this throughout the course of the year. And then building that cumulative measure as well will dramatically reduce overall testing time. And that's a real goal for this whole design is we want to have a single coherent system of assessments that both support instruction throughout the year and provide that summative score.

My head's going a number of different directions because there's some exciting features here. I'm just sort of curious in terms of the day to day work of Numeridian, are you all building the actual assessment items and checking them for bias and all those things to make sure that they're valid and reliable? Are you taking assessment items from others and what do these assessment items look like? One of the criticisms obviously of no Child Left Behind ERA state assessments was that they tended to be quote unquote, low level multiple choice, not performance based. So there's a bunch of questions in there, but I'm just sort of curious what the work is and what the output of those assessment items looks like.

We, we're designing and developing the items ourselves entirely. So we've been numeridian founded in 2016 and since 2017 we've been supporting the former park states with their assessment design and development for their state level Summatives. One of the reasons I founded New Meridian because early when I was in New York City, it was close to the design of the PARCC assessment. It really did push forward, advance the field in terms of the quality of assessments, focusing on critical thinking, problem solving, deep engagement with content and strong emphasis on writing. And it's because I believe the PARCC assessment was such high quality, I wanted to start New Meridian to ensure that that legacy continued. So we bring that history, that commitment to deep engagement with content and more authentic tasks to all the work that we do. There are some new challenges here. When you start to try to give more timely information through the course of the year, you do need machine scorable items. The field there too has advanced dramatically over the last eight to 10 years in terms of technology enhanced items that create a more authentic or deeply engaging task will provide an immediate score. And then in our design we'll still have open ended tasks that require scoring. We'll do those at certain periods within the year because we think it's important not only in writing, but also in mathematical modeling, in showing your reason in solving open ended problems quantitatively. Those are important performance tasks and we will be using AI scoring to support the quick turnaround of those scores as well.

Super interesting. It sounds like you're going to address what your former boss, Joel Klein always complained about, which was he. He used to always tell the story that it was, you know, December 1st and he liked to joke that he probably went to you actually in, in the story and said, hey Arthur, how are we doing this year? And you'd say, well, when the assessments come out next year, December, I'll tell you how we were doing at this point last year. And it always frustrated the heck out of him.

And he's not alone. We've got to get scores closer to the point of instruction if they're going to have any real value. And politically, I think families, state education agency leaders, chiefs, they recognize that. And so I think there's more momentum now to get more utility out of state level assessments than there ever has been in the past.

And to two other buckets of questions that I'm curious about because you alluded to the fact that how you can design these assessments for different curriculum or content choices to sort of yield to the local control desires of districts. Obviously Louisiana has said we're going to be super centralized in terms of the content, as I understand it, and make sure the assessments are aligned around that. As a result of that, it sounds like you're trying to transcend the church challenge that interim assessments have played in. In. In this world of accountability, which is to say interim assessments that are relatively content agnostic have propped up to try to predict how a student is going to do on the end of your summative assessments and give the school a lot more information. But instructionally not super helpful because they haven't been content aligned. It sounds like you're trying to disintermediate or get rid of that system as well, if I'm understanding correctly. Is that an accurate portrayal?

Yeah. Not super helpful is, I'd say an understatement. I would say actually quite problematic in terms of the impacts that interims have on curriculum and instruction. They primarily support a growth measure and prediction to the end of year state assessment. The way they do that is they essentially measure the same end of year blueprint three times a year, even at the beginning of the year. And that creates a rather artificial measurement of growth because they're measuring stuff the student hasn't seen yet. They haven't had the opportunity to learn that content and that's frustrating for the student. The other impact that interims have on instruction is too often it tempts teachers to focus and drill on the skills that are tested by the interim assessment, which pulls them out of a coherent instructional plan. We want rather to let the dog wag the tail by creating these modular micro assessments that can be flexibly aligned to what's being taught. All high quality, quality curricula are aligned to standards. You can look at the standards in instructionally coherent clusters and you can align to 80 to 90% of a district scope and sequence or an adopted curriculum. If you have enough of these micro assessments to support that kind of flexible alignment. That's the approach we're taking.

It's incredibly exciting, frankly. I mean it. And doing away to your point with the deviation from coherent scopes and sequence of curriculum for low level drill and kill, that doesn't actually have much value in frankly boosting learning or test scores ultimately, as I read it, would would be a welcome departure. I'm curious. You're not just doing this in the, you know, ELA and math areas. You're also doing innovative things with science assessment as I understand it. Can you tell us a little bit about that work?

Yeah, we have. Two years ago we launched the Science Exchange driven by the fact generally the. It's. It's ridiculous how much money states spend on customized end of year assessments that are all essentially measuring the same standards. Yet every state creates their own custom assessment underlying Our model in all subjects is states should be able to share assessment items and realize those efficiencies by licensing content and using that in their state. That's especially true of science assessments for states that are aligned to the next generation science standards. Really deep set of standards, multidimensional cross cutting and to create high quality NGSS science assessments is very expensive. So some states are farther ahead than others. We wanted to set up a mechanism by which states could share those NGSS aligned content. We handle the licensing, the liability, the risk, all of that to help, to let states help states. And so we've been using that approach. We have a handful of states who are doing that now, some contributing, some licensing, some doing both because they can refresh their science assessments using other states. High quality and we do a quality review and give them feedback on that. We're also now looking at high quality interim or a more testlet space system for science that has its own challenges. But we think the same things apply in science as they do in math and ela. If we can get more instructionally valuable information back to the classroom, we're going to help learning outcomes.

It's incredibly exciting. Last question as we sort of wrap up here, which is you suggested a problem with the growth scores, if you will, that the current state of interim assessments give. Something I had never thought about, frankly. Uh, I had always thought about the problem with some of those growth scores is that they're norm referenced. And so, you know, if you're in the fifth percentile of learners, it's sort of comparing you against the fifth percentile learners. And if you, you know, grew, quote, unquote, a grade level, well, that's actually compared to the others at the 5%. So it's sort of building in lowered expectations. Huge problems there in my mind. But I think the growth question is still an interesting one in the sense of, gee, if I'm really working at a second grade math level, even though I'm in the fourth grade, I actually would love to know, am I starting to master those standards over time and am I growing to a point where I'll be on grade level so I can keep up with. Particularly in math, it's a cumulative subject. I'm curious how you think about that in the system because I know right now the federal requirements say you can only assess on grade level, but it seems to me that what you've built could be much more adaptable to whatever each individual student is in fact learning and be assessing that to give a much more robust picture of what they have mastered and what they have not yet mastered, perhaps. So maybe it's not a growth score, but a more transparent reflection of what each learner knows and can do. Am I reading that right or what's wrong with that picture?

No, I think, I think you're reading it right. There's a lot in what you just said. We are, we're especially on the math side. We're designing around the learning progressions developed by student achievement partners which extend across grades. And there is a, I would say strong emphasis coming out of the pandemic to focus on learning acceleration rather than remediation. And that means focus on grade level content, identify where students are struggling and look at the precursor standards from prior grade or even before. And how can you build those knowledge and skills to help them be successful with the grade level content? That's strongly the model emerging out of the pandemic to accelerate the recovery. And so since we're building on those cross grade progressions, we can support the identification of those prior foundational skills. But we want to keep educators focused in the on grade curriculum. We think that's essential. Too often students get identified below grade level and, and they get locked into the cycle of remediation and they never catch up. So we've been designing for a model focuses on grade level, helps identify kind of precursor skills when needed and supports educators with those.

Super interesting. Arthur. Final thoughts as we wrap up here on where this future could go with assessments and what sorts of learning models we might see in the future as a result.

I do think we're walking through a door where the movement from standardized instruction and assessment is giving way to more student centered competency based learning models. The more we can have micro assessments that are supporting giving feedback to what students are learning on their own pace, the more we can move to that student driven kind of student centered learning model. And we right now are designing for the district and school level to configure. But imagine we see the next iteration of this where the teacher and then the student can say I'm ready for that assessment now, I'm ready to demonstrate my mastery of that competency. Let me take it. And then build and level up through the course of the year on a true student centered competency based model. And if we can do that and meet federal accountability requirements from that system, then I think we've really found the holy ground.

Now that gets exciting. Arthur Van der Veen of New Meridian, thank you for joining me on the future of education and for the work you are doing. I appreciate it.

Thank you, Michael.

And we'll be back. Next time on the Future of Education.