Going Beyond the Numbers with Anne Mickey at Sands Anderson

Welcome everyone. We're so glad for you to join us today. We are just going to get started here in a moment while we let some more folks in from the waiting room. Hope everyone had a chance to grab lunch. Maybe you're sitting down with lunch for this. Let's see here. Alright, give it about 20 more seconds, and then I'm gonna get started. All right. Wow, I love seeing all the participants here this afternoon. Okay, I'm gonna go ahead. Hello and welcome to Parallel's first Virginia Learn-a-Latte. Thank you for taking the time to join us. We are excited to be here today with Ann Mickey from Sands Anderson. My name is Crystal Weber, Director of District Partnerships at Parallel, and I will be your host. Thank you, Crystal. I will share a brief parallel overview in a moment, but let's be honest, I know why you're here. Me too, to see Anne. So, Anne, thank you for joining us. I know many of you know Anne, and from what I gathered at the recent VCASE conference, she is a loved and trusted resource for Virginia administrators, and we are honored to have you with us today and look forward to continuing this partnership and our common goal of supporting special education staff and students in Virginia. So Let's do a quick icebreaker as folks continue to join. I also may have a little hidden agenda of wanting to plan a vacation in Virginia, so help me out. And for any of you who know me or maybe have had a happy hour with me, you may know that I love to travel domestically, internationally, and honestly, sometimes in my own city, I'll just go explore for the day. So last week I visited the Locals Food Hub. It's a mini market in downtown Louisville, Kentucky, where I live. It also does things like wood-fired pizzas. It sells local goods. I was in search of honey sticks. What I found was a Louisville kid who started beekeeping at age 11 as a mental health outlet and to cope with stress. He wrote a book about it, and he now runs a local beekeeping business with his mom and his uncle, and they sell the honey sticks at the market. So I was ecstatic to find this out, and I was very excited in telling the folks who worked there what I do. And I just had this moment of feeling like knowing I was in the right place and feeling really proud about the work I do to bring resources to special education teams like yourself and students. And that it matters, and that what you do really matters even more. So I wanted to share that. But next VKS in October, come to the booth for your honey sticks. I will have them. All right, where are we going? Have we finished the poll? Ah, the island. Okay, I knew that was gonna happen because then I was gonna have to pronounce it. So if we have any phonetics fans, you can drop the pronunciation into the chat. But yes, I would love to see the ponies on the island. All right, but before we turn this over to Anne, I want you to think about the coming school year, and as you prepare to welcome your special education team back, which challenges are already on your mind? So you can drop answers in the comments if you feel strongly. The dreaded burnout, right? If caseloads are starting high, they're only going in one direction. True story. Covering gaps. Maybe sometimes it's easier for you to step in because you're a former school psych or an SLP or a teacher, or you're someone on the team who's joined us today who's picking up, you know, that extra load, and now you're burnt out and everyone's burnt out, and it's really tough. Consistency. This is where I wave my wouldn't-it-be-nice wand, but wouldn't it be nice if you could balance caseloads for your SLPs, bring in part-time support to prevent assessment bottleneck, or provide caseload management instructional coaching for those new or provisionally licensed teachers? I'm just going to let data speak for itself down there, but these are just some of the scenarios that Parallel can help you with next school year. And how do we do this? We work with districts. We partner in these 4 areas: specialized instruction, speech-language pathology, psychoeducational assessments, and behavior and mental health counseling. I normally won't read a slide to you, but in case anyone looked away, I just want everyone to hear what it is that we are doing to partner with districts. And we are 100% virtual, and that is 100% on purpose. We intentionally built our practice around the research done by experts in the field. One of them is our Chief Clinical Officer, Dr. Jordan Wright. And the research is on the effectiveness of teletherapy, and our platform and service model were designed with best practice in mind. So everything we do is research-based and legally defensible. And at this point, many of you have likely utilized some kind of virtual support. Rest assured, I can read your minds. I know what you're thinking. But really, I'm here today to tell you how we truly do things different at Parallel. And if we could go just jump to the next slide. Thank you, Anne. So starting with, we only hire top providers for our network and work incredibly hard to support those providers. We want to ensure they are the best fit for your district. So if you meet with me, if we've had a consult, you will hear me ask, what does a great SLP or school psych or teacher look like for you? What qualities do you need? We also offer them clinical management, mentorship, and systems that allow them to prioritize you and your students. Because of that, we're able to retain those providers, 98% of them, year over year. And as you know, retention is easy. To that end, I'm going to speak quickly to our partnership model. This is my favorite thing to talk about. So again, if you've ever been in a consult with me, you've heard me talk about Linda, your dedicated customer success manager. When in doubt, call Linda. It's my motto. It's what I do when I'm in doubt, and that's something you'll hear me say as well. She's a former administrator, very proactive, very empathetic to what happens in the district. She's experienced in virtual implementation and support. And really wonderful to work with. We also have clinical managers that are service line specific who are exceptional, and they make sure that all things clinical run smoothly. This is, I would say, the key differentiator for providing services with fidelity is having a dedicated and relevant support team. A lot of companies in our space are scaling back on customer and clinical management while we are scaling up, and this is our foundation. So, okay, differentiator number 2, a proprietary platform built for therapy and assessments. Ours is called Pathway. We built it on the research of Dr. Jordan, who is the leading researcher in K-12 for virtual service and assessment delivery. It's called Pathway. I think I said that it's FERPA compliant. It allows us to provide services more efficiently, to continuously track progress and maintain visibility on outcomes throughout the year. And as Anne might agree, and as we're going to talk about some today, good data is key in supporting day-to-day decisions that improve your outcomes. So speaking of Dr. Jordan and outcomes, one thing that we are really proud of is having the data to back up what we say. So when virtual services are delivered the right way, students can absolutely meet and even exceed their goals. Our white paper looked at outcomes from the 24-25 school year, and the results were really encouraging. Across all of our service lines, 98% of students met or exceeded their IEP goals. So you can drop a line in the chat, or you can sign up at the end for a consult with me, or just email me, and I will send you the outcomes white paper along with our hardest-to-reach Students white paper, which just was published recently. All right, last but not least, we are your Virginia team. I already told you a bit about me, but one thing I will add is that I've been serving in the special education sector, specifically in teletherapy, since 2013. So I've seen the evolution. I know what works and what to look out for, and I am here as a resource for you. Please use me. Some of you will recognize Tamar's name. She is my partner in outreach and in making sure you have Parallel top of mind when staffing becomes a challenge. And as I said before, Linda is simply the best in the business. So one more plug, one more hurrah for Parallels partnership model. It's truly the reason that I'm here and that I enjoy my, my work. Okay, so last slide before Anne. If you want to meet with me, please use this code. I would love to speak with you in the next few weeks to come up with a strategy plan for your district. I always tell folks, meet with me when you don't need me. It's the best time to evaluate where we might fit in your program. So go ahead and scan the QR code. This is good way also to make sure we get those white papers over to you. And with that, I'll hand this over to Anne. Quick note, if you have any questions throughout the webinar, there is a Q&A at the end, so you can drop those in the Q&A chat. All right, Anne, thank you again for being here with us.

Yeah, thank you, Crystal, and thank you to Parallel for having me this afternoon. For those of you who don't know me, my name is Anne McKee. I am an attorney with the Education Law Group at Sands Anderson. I've been practicing in special education law for more than 10 years. Very privileged to have this job. I think very highly of all of the educators who are supporting students with disabilities across the state, and so it's a real privilege and honor to get to work with you every day and to get to have conversations like this, right? Because I know oftentimes when you're interacting with me, maybe it's because a conflict has arisen, but I love doing stuff like this, right? Proactive thinking about how we can put practices and other tools in place to ensure that we are not only fulfilling our legal obligations, but doing things that are going to help, you know, secure and really develop those relationships with the families that we are dealing with every day. And the topic that I'm going to be talking about, or the title of my presentation today, is Going Beyond the Numbers: Telling a Student's Story of Progress Through Meaningful IEP Goal Data Collection and Analysis. So you might be thinking to yourself, Anne, how often in your particular role are you collecting data? I'm not collecting data in my day-to-day practice, right? But I am reviewing IEP goal progress data on a daily basis. You know, when I get involved with a case, you know, maybe there's a dispute about whether a student has made appropriate progress over the course of the year, or relatedly, is their IEP appropriately addressing their student's needs that may be impeding progress in some way? And so in reviewing a lot of student data, again, on a daily basis across a variety of wide variety of contexts, you know, there's 2 main problems that tend to come up. And the first was alluded to, I think, in an earlier slide, but sometimes it's just we're struggling to have that underlying data, right? We might have an IEP progress report, we might have a report card, we might have SOL data or MAP testing data, some standardized assessments, but maybe there, there's a little bit of a gap in terms of some of that underlying raw data that ties directly to the IEP goal. I will say that's getting better. I do see progress in that area across the state with the many clients that I work with, but I will say even in divisions where maybe I have a case where there has been robust data collection, another issue I'm sometimes seeing though is there's a real heavy emphasis on the quantitative aspect of data collection that might be given a stack of sheets that will give me all these percentages on how a student accurately or didn't accurately answer a particular question, but I think the problem sometimes with focusing on that quantitative piece is it doesn't necessarily give me the entire picture of whether a student has made progress or not, because there's a lot of qualitative things that we need to take into account when we're actually assessing whether a student has made progress over time. And I will tell you that I think this issue does tend to drive some conflict at times, that, you know, parents might be given— parents or families might be given data about how their student has been doing, but it's not given with any sort of context that, you know, we're giving these percentages, we're giving these standardized scores, but families aren't really told how to interpret it, or maybe they're not given sort of this overall context of how this shows ultimately at the end of the day how their child has progressed over the course of the year through the specialized instruction that you all are providing. You know, at the end of the day, I think parents just really want to hear the story of progress, right? They want to hear the journey that their child has been on through all the hard work that your staff has been doing. They want to understand how we get from point A to point B over the course of the year through the IEP programming that you all have developed. And as a school board attorney, I want to be able to do the same thing. So if I'm going into a due process hearing, or if I have— there's some sort of dispute, I want to be able to show a judge or a hearing officer or a family, right, when we're trying to resolve this, hopefully without having to go to litigation, how the child has gone from what I often describe as point A to point B. And to do that, at point A, we need to know where the child started, what their baseline data and status quo is in a particular area of need. And then I want to be able to show not just data about how they did, but how through the programming that you all are implementing, we got to point B, right? How they've made measurable progress, where they've— we've moved the needle in some way, right, to get them further along in the curriculum or in an area of need that's impeding their progress in some way. That's That's the hope. We want to tell the story again from point A to point B and not get stuck in the statistics of the data, which is something that I feel like we're struggling with, with staff sometimes. So how do we accomplish that? Well, I think there's sort of 3 elements to this story of progress, if you will, and these are the things we're going to break down in more detail today. The first, I think, is drafting effective IEP goals, right? Establishing the story we wanna tell a year from now about where a child hopefully has made progress. Then of course, once we've hopefully crafted a, what I will deem later as a SMART IEP goal, that we are collecting the data towards that goal and not just focusing on quantitative measures, making sure we're going beyond the numbers, if you will, to provide that complete picture of a student's progress. Once we've done all this hard work of delivering specialized instruction, collecting the data on progress, we want to then tell the story, right? And hopefully that story is going to match what we anticipated a year ago with the goal, but we're going to talk a little bit about how do we craft IEP progress reports in a way, again, that's going to be meaningful for families to understand how their child has gone from point A to point B over the course of the IEP life cycle or whatever reporting period we may be in. So, let's get started. Although I will say before I jump in, if you do have questions, we're going to try to have a Q&A at the end of today's session. I've been told that the best place to put those questions are in the Q&A function. And I think that's at the top of your bar. If you don't see it right away, I've been told you can check the little thing that has the 3 dots more, and there should be a Q&A button there. So, again, encourage you to ask questions. which I will try to get to at the end of today's presentation. Okay, let's start with drafting IEP goals. So before we get into best practices, as a reminder, under the IDEA, every IEP has to include a statement of measurable annual goals that's both academic and functional that are designed to meet the needs that result from the child's disability, And that maybe you're impeding that child's ability to make progress, whether we're talking about in the general education curriculum or in an alternate curriculum, again, based on that student's unique circumstances. So what does that mean in practice? Well, a student's annual IEP goals, to me, should reflect what the IEP team predicts a student is going to be able to achieve in a year, hopefully given specially designed instruction and support from your staff. But under my storytelling framework, the goals are the story in my mind that the team hopes to be able to tell about the student's progress in a year's time, again with the support that's been identified within their IEP. I think if goals are individually tailored to a student's specific needs and follow what we're going to talk about in a second, that SMART framework, your team will be in the best position, I think, to tell the story of a student's progress clearly, and effectively throughout the course of the year. So let's talk about what I mean by that SMART framework. I'm sure this is something many of you have heard about, right? The idea of SMART goals. But I wanted to kind of tailor it to the conversation that we're having today. And I'm going to break these down in more detail. But when I say SMART, what does that stand for? Well, S stands for specific, that we think that we ultimately should craft goals that are targeting a discrete skill rather than bundling several new skills together. Should be M, measurable, describing an observable, countable action with clear mastery criteria. A is for achievable, you know, it's appropriately ambitious given the student's circumstances, not just a guess. R is for relevant, targeting needs that result from the disability and support access to the general curriculum. And T is for time-bound, that there should be a clear timeframe for the goal, and as we will talk about, clear mastery criteria that has clear trial counts and other factors that help give us that full picture of progress. But again, let's walk through those each in more detail. So S again is for specific. Goals should tackle one skill at a time whenever possible. And why? Well, if you bundle too many skills into one goal, it can make it more challenging to collect data on the goal and more challenging to explain goal progress, I think, to families. So let's compare those 2 goals that I have on the screen here. You know, a goal that says by February 2024, Johnny will write a 5-sentence paragraph, turn in his essay on time, and show proficiency with grammar and spelling at 80% accuracy. That's bundling 3 skills into 1 goal, and not all of those skills clearly build off of one another. So for example, drafting a 5-sentence essay is really measuring Johnny's written expression skills versus turning in the essay on time is really tackling executive functioning skills. Both of which might be impacting Johnny and need to be worked on. But when you're tying those kind of different skills together, that's obviously going to make it a little bit more complicated to give sort of a complete mastery picture. Also, if Johnny does then master one of the 3 skills targeted in the initial goal, that's progress, of course, but it ultimately again is going to inhibit your team from saying that Johnny's fully mastered that goal because you've tied his success with that skill to the success of the other skills, which just makes that mastery criterion evaluating a little more complicated. I think a better version of the goal would isolate progress monitoring to a more discrete skill, such as by February 2024, given a writing prompt, Johnny will write a 5-sentence paragraph with a topic sentence, 3 supporting sentences, and a conclusion at 80% accuracy at 4 out of 5 opportunities. That's discrete, right? If we're targeting more so the written expression aspect of it, there's still, there's still multiple parts to it, But again, it's highlighting that written expression portion so that we can make sure that we can give families a more accurate understanding of how Johnny's written expression goals are hopefully moving from point A to point B over the course of the year and not tying that into his mastery of a separate skill regarding executive functioning. Next, M is for measurable. Again, a goal is measurable when it's describing an observable, countable action, usually. So we want to be really careful when we're drafting goals to watch out for language that's maybe overly vague, like student will improve reading comprehension. To me, that's not measurable what improvement means. We need to target it and we need to root it in something more concrete, or student will demonstrate, quote, appropriate behavior. The fix again is to kind of, kind of find those vague verbs and instead replace them with specific observable actions. You know, so for example, improving reading comprehension, we might want to instead say correctly answer literal and inferential comprehension questions with 80% accuracy in 3 out of 4 trials on grade-level passages. So again, to develop these SMART goals, we want to root out those vague verbs, finding ways to have more objectionable, countable actions that's going to make it clear when has, you know, the student in question actually mastered this goal. Achievable is for A, and that's talking about that goals need to be appropriately ambitious for the student. You know, the Supreme Court addressed this in this Endrew F. v. Douglas County School District case that I know you all have heard about, that held that an IEP needs to be appropriately ambitious given the student's unique circumstances and should offer a student the opportunity to meet challenging objectives. In practice, that means that your team needs to look at baseline data, the nature of a student's disability, and other relevant factors to determine what level of progress is realistically ambitious for that student over the coming year. We're ultimately wanting to avoid a scenario where we're making a goal too hard, right, for a student so that we're, you know, not setting ourselves up to really not be able to show sufficient progress over the course of the year. So really making these goals individualized to the students so it is achievable, but again, appropriately ambitious. We want them to meet challenging objectives. R stands for relevant, and it kind of ties in with achievable, right, that we want goals that are really unique to that student and are addressing the needs that are impacting their ability to make progress in the general curriculum. You know, the team identifies those needs by again reviewing baseline data about the student, both in academic and functional areas. You know, we have to always remind teams it's not just a matter of academics. There can be functional needs, areas like behavior, social skills, that also might be impeding access to the curriculum and to being able to make progress that we might need to target through a goal. And again, data that might inform that are teacher observations, prior goal progress data, standardized assessments, That's not an exhaustive list. There's lots of things we can review to kind of garner what is really impacting the student. What do we need to target? I know there's often a focus on developing IEPs that are standards-based, right? And while grade-level standards, I think, can be an appropriate place to start conversations around what we draft IEP goals about, to me, we always have to make sure that goals remain individualized to a student, because for some students, Achieving a grade-level standard may not be possible within a particular year based on their unique circumstances and just where they fall at that particular time. So again, we need to remind staff that while that can be a starting point, we ultimately need to make sure that every goal is uniquely tailored to that particular child's needs and the unique circumstances that again might be impacting progress in some way. T, time-bound and clear trial count. So mastery criteria for every goal you have should be unambiguous to anyone reading the IEP. For those of you who have done trainings with me before, you know I often talk about the substitute teacher test, that we should be writing every IEP in a manner that a substitute teacher who is not— wasn't involved in the development of the IEP can pick up the document and say, okay, I know exactly what I need to do, and in this context, I know exactly what the student needs to accomplish to be able to master this goal. I often— I would say 80% accuracy tends to be the most common criteria that I see drafted for IEP goals, and that can be a good starting point, but standing alone, it can often raise more questions than it answers. You know, clear mastery criteria should ultimately specify the type of task that the student is accomplishing, the measurement tool that's going to be used, whether we're talking about like a rubric, a probe, a tally, work samples, what the threshold is again for mastery, and the consistency of that mastery. And also how much staff support might be needed or how much staff support is appropriate to be able to say that the student again has made progress on that particular skills. All of those factors, you know, to me provide a more complete picture of what a student has learned or gained through the support of their IEP versus saying again, a percentage of accuracy is just to me one piece of an overall picture. So if I put this into practice, I've given you here a goal that says, given a grade-level text, student will improve reading fluency comprehension, and written response skills as measured by teacher observation, achieving 80% accuracy by the end of the IEP year. I see goals like this all the time, right? But hopefully from the discussion we just had, you can immediately point out some things that ideally could be improved about this. I think the first thing that hopefully should stand out is the word improve, right? What does that mean that we're going to improve reading fluency? And even though there's an 80% accuracy measure at the end, I don't think that's going to give us a clear way of telling a story of how a student has moved from point A to point B. So some possible revisions to this might be that we focus, for example, on reading fluency rather than bundling it with comprehension and written expression skills. And by targeting sort of a particular word-per-minute fluency level, we are rooting it right in something that's countable that we can then measure across time. We also talk about 95% accuracy on a particular number of trials, right, or across a consecutive number of opportunities. But another version is focusing on comprehension, right? After reading a grade-level passage, student will correctly answer 4 out of 5 literal and inferential comprehension questions on 3 consecutive opportunities. So we— when I do a more comprehensive IEP training, we could give lots more examples to this, right? But again, wanting to help our staff understand that by drafting SMART IEP goals, keeping in mind all those factors, we're hopefully going to set ourselves up really well over the course of the IEP life cycle to collect clearer data, and then again, hopefully ultimately be able to tell families a really clear picture how their child has moved from point A to point B over the course of the year. So let's talk about progress and data collection. So as a reminder, under the IDEA and our Virginia special education regulations, our IEPs are required to state how we're going to measure progress across the life cycle of the IEP. Now, to be clear, neither the IDEA nor our Virginia regulations expressly mandate a specific method for IEP goal data collection, right? That's up to the IEP team to decide what's going to be appropriate based on a particular student's needs. But regardless of what is picked, what you ultimately pick does need to be explicitly laid out in the IEP itself. This is a box that's given, I think, as part of a sample IEP form that VDOE has published in the past. It gives you a non-exhaustive list of things that the IEP team might decide are appropriate to use for data collection purposes, and I think this is helpful to look at not only just to spark ideas, but again, it emphasizes that we might be using both quantitative and qualitative measures to be tracking data on goals, but also that teams should not feel contrived to a specific list, that there's meant to be some creativity here so that we are pulling out the most effective data collection tool possible based on that child's needs and based on whatever the goal says. But again, we wanna make sure that if we list something here, we absolutely should have artifacts, uh, in the student's file or in our notes showing we've done this. You know, so I'll see teams often check 4 different things on this list, right? And I think sometimes there's a misconception, especially amongst new teachers, that that means that it might be one of those 4 things that we are using for data collection. I think the spirit of this is that if you are checking something, that means you intend to use that data collection method for IEP progress monitoring. So we want to make sure to reinforce with staff, if you've checked something here, we should expect to see that particular method, you know, if it's documented somewhere that we can pull this out when we're doing the IEP progress reports at the end of the reporting period or at the end of the year. My advice to teams to kind of help ourselves on the front end of this process is to have a specific conversation about what data collection will look like when you're actually drafting IEP goals. I find that doesn't happen very often, at least in the IEP team meetings that I participate in. There's, you know, a heavy discussion on the IEP goal itself, making sure the wording is really precise, and then there will be a really brief conversation, if conversation happens at all, about what's checked in the box underneath the goal. But I really would encourage your staff to have a more explicit conversation about what data collection is going to look like in implementation at the same time that you're drafting the goal. Just to make sure everybody's on the same page. But I think it's also going to help prevent the possibility that you're picking a data collection method that doesn't actually make sense with the goal at issue. I will say, going back to this slide, checklists is one of the ones that I see where this can come up most often. That checklist in theory sounds great when you're developing a goal, but when teachers actually get into the implementation side of things, they find the checklist actually didn't make sense for that goal after all. And so having that conversation at the time that the goal is being drafted again can kind of hopefully help avoid that particular problem, make sure that the family's expectations are clear about what they should see in terms of data, and then avoids us having to come back as an IEP team, because of course the question is, well, and what if we find ourselves in that situation? I would say you want to revise the IEP, right? Because you don't want to be in a situation where there's a concern that the IEP wasn't implemented as required, Because we didn't use a particular data collection method tool. So if you're finding that that tool just doesn't make sense, we might need to revise the IEP in some way, either by changing the data collection method or changing the goal itself. Either way, just to make sure that we're continuing to follow the IEP as required and having meaningful data collected for that particular student. So I want to go through an example to kind of illustrate what can happen when we focus a lot on the quantitative side of data collection. Because again, I started the presentation talking about how I am seeing a lot of improvement across the state with data collection, but what I'm sometimes seeing is that there's this heavy emphasis again on quantitative measures, and we lose sight sometimes of the fact that we also might need to be tracking additional qualitative things to be able to give that full picture of a student. So let's look at this example together. So This is an example where the goal is Ann will label uppercase and lowercase letters with 100% accuracy on 4 out of 5 opportunities by August 2026, with data collected via staff observation. To take data on this goal, a special education teacher developed the table that you see on the screen where she tracked how many uppercase and lowercase letters she observed the student correctly identified during a data collection session. She even put a column on the table to include notes and information About prompting. You know, I would say at first blush, I think this data collection method seems fairly robust, right? We're getting some good quantitative data, and there's even a section that's provided to kind of give some level of qualitative information. But I think based on this data sheet alone, I think it's possible that different people can look at this and come to different conclusions about what it says about the student's progress on this particular skill. And I really think someone could probably look at this data and say, that it supports emerging, insufficient, or sufficient progress. So here are some of the issues that I see with this. You know, I think the first question is, was Anne labeling the same letters correctly in each trial, or was it hit or miss? You know, for example, on April 1st, did Anne identify the same uppercase letters that she identified on April 8th, but then added 2 additional letters, you know, showing that progress? And if that is what happened, what do we think contributed to that progress? You know, was it the substantial redirection that's noted in the 3rd column, or was it something else? Also, I think a question that is posed from this is, how were the letters presented to Anne? You know, were they presented all at once? Were they presented in groups? And if so, again, how might that have ultimately impacted how Anne performed? Also, because my performance on this goal, if I'm Anne, is tied to both my performance of uppercase and lowercase letters, because I had such different progress on those 2 categories, you know, what am I going to say then about overall progress since the success of one is tied to sort of the success of others? So again, I give this as an example of something that may look robust at first, but I think once you dive into it, and this is what I often have to do as the attorney, is I'm given something like this and then trying to figure out how to interpret interpret it, I realize there's a lot of questions that I don't know the answer to from this data collection sheet. So my advice for a teacher maybe whose data collection table this is would ultimately be that I think we either need to modify the goal in some way and/or modify the data collection tool that is being used to monitor it. I think with respect to the goal, we may want to make it more discrete to focus on just upper or lowercase letters Or break that goal down into benchmarks so we can bifurcate our data collection a little bit differently. But I think also we need to give this staff member maybe some clear direction on how data collection trials should be run so we can develop a more complete picture of how Anne is performing on the goal and some of the factors that may impact her performance in some way. So let's look at sort of a revised version of the goal, or the data collection tool and goal, I should say. So the new goal says, given a field of 5, Anne will label uppercase letters with 100% accuracy on 4 out of 5 opportunities by August 19th, 2026. So what I like about that goal is now we've targeted a more discrete scale, and maybe this is a benchmark, right, in terms of letter identification, and maybe lowercase letters is a different benchmark here. But at least for this benchmark, we have a clear discrete scale. We're just going to focus on uppercase letters. We're identifying how those letters should be Introduced, right? We're going to do 5 at a time. And we've identified the number of trials that we're going to do. So 4 out of 5 trials, and we're looking for 100% accuracy, which I will say, as a general matter, I try to avoid that because I think as humans, we all know that perfection is very difficult. But in this particular sense, we'll go with that. And I think this table is going to do a better job of giving us sort of a clearer picture of what happened, right? So I have 5 clear trials here. So I know I'm going to have data that aligns with the goal mastery criteria. And then I've laid out the letters more clearly, and then circle the letter if correct, X the letter if incorrect. I'm giving clear directions to whomever is going to use this data sheet on how it's intended to be utilized. And hopefully it might turn out something like this, right? That if I got the letter correct, you're supposed to circle it. If I got it wrong, I'm supposed to put an X over top of it. And immediately I can glean a lot more than what I was able to glean from the table we looked at earlier, that I know that Anne is doing a really good job with the letter A, and generally speaking with the letter B, but it's the letters C and E where there's some more variability. That's going to help me know where I need to target more explicit instruction over the next quarter, right? So again, this is a small tweak with this goal, but it's helping to give more context And more observation, right, to how Anne was doing, to give the teachers more direction again on instruction, giving families more of a clear picture of where their child may still be struggling and what's preventing them overall from mastering the goal that we've identified. Now, of course, there's ways we can further improve this, right, that I just wasn't able to fit on the screen here, right? There's more things about how much staff involvement may have been at play when they were working on this goal. Or other observations about how the student was performing that day that may be relevant. Also, we could be inserting some information about maybe what tactics or tools or instruction seem to be meaningful for Anne to further her progress on some of these things, right? The letter D she seemed to improve on over the course of these different trials, what may have contributed to that? So definitely more that you can add to that, but wanted to give this as an example of making sure we're really triaging our data collection tools to make sure it's giving us the most meaningful and impactful actual data possible. Let's look at a different example together. So let's say we have a goal where the student is trying to reduce verbal aggression towards peers when the student gets frustrated. Um, if you'd only track frequency of this goal— did the behavior happen, yes or no, in a particular context— you're going to miss a lot, right? You'd miss information about maybe coping strategies or tools that help the student avoid engaging in that behavior if we do see a reduction over time. Or we might miss out on whether the student just instead substituted an alternate behavior when they became frustrated. So maybe it wasn't verbal aggression anymore, but they engaged in something else that maybe we don't want the student to be engaging in. And then also, what level of staff support was needed to help the student maybe reduce verbal aggression in that moment? Or was there a lack of staff support because we're trying to build independence, but the student just doesn't seem to be making progress without that staff intervention. All of that's really important again for providing a full context and picture of student progress. So I've given here again just a chart that could be utilized for something like that. Again, not saying this is a perfect chart, there's ways it can be improved, but what I do like about it is it's giving, you know, different trials in which we were observing student data. It's giving the setting in which that was happening, which that can often be a really important data piece that's not tracked if explicitly by staff, but maybe we're seeing behaviors in a setting, for example, where the student— it's really a non-preferred subject, English for example, or math class. Maybe if the student struggles in that subject, we tend to see increasing behaviors there, and that's a, you know, that should be a cue to us of maybe we have to provide more intervention in that setting. So tracking that can be really helpful. What was the trigger for the frustration? Again, that can give us meaningful insight onto how to help further support the student. There is a yes/no here, but again, we go on to talk about what replacement behavior was used or what coping strategy was used and the level of support. I think for that level of support column you see there, I'd want to have some sort of guide of what is a 3 versus a 2 versus a 1, so a staff member knows if they hadn't had any involvement in developing this beforehand, but just didn't have room on the screen again to put all of that. And then again, some qualitative information about some further observations. So. Again, this isn't necessarily perfect, but I wanted to give it as an example of trying to pull out all these contextual data points, both quantitative and qualitative, can just be super important to give that complete picture to families. Okay, I want to give a quick warning of something I've seen a lot this year, which is I've seen a lot of staff using things like Google Survey or other types of like electronic submissions to staff members, particularly at the high school level where a student might have 6 or 7 different teachers, and we're trying to take data across, across a variety of settings. So case manager, to do data collection in part, will send out one of these surveys to staff to elicit information about how the student is doing on an IEP goal and to get information for the progress report. You know, I think the pro of doing something like that is obviously it's an efficient way to collect data. We can sometimes get some good quantitative information from that, and it is documenting our efforts, right, to collect data on the goal, which is always positive. But I think the issue becomes that it does sometimes place too much emphasis on the quantitative side. I'll see, you know, a lot of yes/no questions in these surveys that don't get the contextual information that we sometimes need. But it also can be prone to a lot of errors, things like not all staff complete the survey, and so it skews the results in some way, or the questions are vaguely worded and staff interpret it differently. So I just wanted to put out this warning that I'm seeing increasing use of this, but we have to warn our staff there can be pitfalls to this. So we need to be careful and really analyze before we use this. Is this going to give us the most robust data and accurate data to be helpful for IEP progress monitoring purposes? So I gave here an example that I won't go through, but just again, avoiding just yes/no questions, because that's oftentimes not going to give us the full context. I'm going to skip through this to get to our IEP progress reports because I want to make sure that we have time to answer questions here at the end. So as a reminder, the IDEA, again, sort of establishing the legal standard, the IDEA requires that the IEP state when periodic progress reports will be provided. For example, that might be through quarterly reports, it might be reports that are provided every 4 and a half weeks, so halfway through the quarter, but at minimum we have to make sure that we are providing progress reports at the same cadence that we are providing reports to families of students without disabilities during the school year, so that's usually at the frequency at which we are providing report cards. Some school divisions also do interim reports. We just have to make sure that we're consistent with the IEP and consistent with our schedule for non-disabled students and their families. To me, IEP progress reports are really the payoff for everything we've covered up to this point, right? A good IEP progress report should clearly explain, citing to the data that we've collected, how the child has progressed towards or mastered the annual IEP goals during that particular reporting period, maybe identify any factors that are preventing mastery at that point, and then hopefully outlining the specially designed instruction or supports that help the student to make the progress that they made. And I say outline, I mean, this is your time, right, to really celebrate the efforts of staff and what the IEP has done to help the student. I would say most progress reports I read, you know, can be very literal of just this is the frequency of accuracy, you know, tied to that goal. But I think a progress report can do a lot more than that, right? It can not only talk about that progress data, but again, putting it in context of, and we saw this progress through the work that staff were doing with the child through the IEP. And to me, with all of the hard work that staff do, this is the type of progress report I really don't want to see or don't prefer to see, right? The goal— the progress report that basically just says sufficient progress, sometimes insufficient progress, sometimes mastery, but there's no narrative given, right, that really puts the story behind the progress that we saw. And I think, again, it probably goes without saying, but for a family that receives something like this, right, they don't really know then what take away from this, that some families say, great, my child's made progress, but I don't really know what that means. Why haven't they mastered the school yet? What might be preventing that from happening? And maybe how does this compare to last quarter, for example? There's a lot of questions that this might elicit, and we can head that off, right, by trying to provide a more robust narrative. So I'm giving here a different example of an IEP progress report that I'd ultimately love to see for every student, you know, where we have the IEP goal And this progress report is not only providing that quantitative information. So at the top, this is a goal about a student reading a grade-level fiction passage, summarizing the main idea and key details, and with so much accuracy in particular trials. So this progress report gives that quantitative data, right? Out of the 5 trials, how did the student perform each time? But instead of just giving that quantitative information, this case manager goes on to give a narrative. That sort of outlines why we saw the data that we did, and maybe some of the things that helped the student to make progress and some of the things that maybe impeded progress. So for example, this particular narrative, for example, talks about how the student was observed to rush through the passage prior to attempting to answer some of the questions. That's really important context for families to know, right? Because that suggests that maybe this is an underrepresentation of what their true abilities are and that they actually would make more progress if they didn't rush through. Also gives us the cue that maybe we need to work with the student on perseverance or attention to task or building in breaks to avoid that from happening. So again, there could also be things to improve about this, but I provide this as an example of just not focusing on the quantitative if we can at all, trying to provide some of that qualitative context so families can see the story of progress and know how to partner with the school team to help the student continue to make progress going forward. I think something that we need to also be mindful about in talking with staff about this issue is that we really need to be focused on describing progress in language that families will understand. In our Virginia regulations, we actually have a requirement for the present level information that is supposed to be described in objective, measurable terms, and that when we are reporting on test scores, we have to provide those in a manner— or describe them, excuse me— in a manner that's self-explanatory or provide any explanation That may be needed to interpret the results of that. While that regulation is not specific to progress reports, I really think we should be applying sort of the same standard to progress reports. That again, a family member, a parent who maybe is a non-educator, should be able to pick up a child's IEP progress report and be able to understand generally what it means, right? Be able to interpret the data that's provided, and to the extent that something's not self-explanatory, making sure we're giving them appropriate. Context so that they're not reading it confused. Um, you know, I think the area I see this with the most tends to be when we give a lot of statistics from standardized assessments. So I see this all the time, particularly in present levels, but sometimes I'll see this in progress reports too, where maybe we have a reading-related goal or a student is struggling with reading and the teacher gives, you know, 7, 8, 9 different standardized assessment data points, all of which is a positive, right, from the standpoint of we're trying to evaluate progress based on standardized measures, which is great. But in this particular example, you might have a parent that doesn't know what MAP stands for, right? I know because I read these all the time, that's Measures of Academic Progress. But, um, or, you know, these other acronyms— what is Lexia? What is FastBridge? Families might not know what this means, or they might not know that WPM stands for words per minute. We want to spell out acronyms as a starting point, right, so the parents know what this means and what they're reading. But again, I think there's still a lot more we could provide context to with this. What does it mean to get a 204 on the MAP? Where does that place the student in context with their peers, both at that, at their grade level, or maybe at their age level? Where did they come from with this? Is this showing progress on some level? Same with FastBridge, right? What does 38 words per minute mean? And what grade level was the passage that they read to get that particular score? So I give this as an example of, you know, this is a, this is great that we're taking standardized assessment data, we have some good quantitative measures for families to see, but if we're not giving it appropriate context or providing it in a way that they will understand, it may fall flat, right? And that's where conflict can sometimes arise. So we want to try to avoid that. So this is, you know, really isolating one particular data point, but I give it as an example of some of the things that can be helpful to pull out about it, right? So if I'm talking about a MAP score, you know, instead of just telling me what my score is, this is trying to give some context around it. So not only do you see on this slide how the student has performed over time, so I see fall, winter, and spring scores, and that the student score has slowly gone up over time, but I also can put this 236 in context, right? That it's telling me sort of what percentile that score is and how it compares to division and state averages. For those of you familiar with the MAP score, you know that the MAP test gives a lot of these statistics, so we could pull this out of that report. But sometimes families might need even further interpretation of that. And one thing it may not tell you, for example, is how Jordan was the day that he performed this particular assessment, right? Again, if rushing is something that happened, or the student just wasn't feeling well that day, or maybe didn't seem overly motivated that day, that could impact these scores, right? And potentially suggest the score is an underrepresentation of Jordan's ability in this area. So all of this context again is going to, I think, help families be able to see that, that progress story and be able to work with staff to collaborate to support Jordan going forward. A question I often get is, well, what do we do if we do feel we need to say that a student has not made expected progress towards a goal? Because that's going to happen, right? And sometimes that's going to be because of factors that are outside of our control, you know, that the student was absent for a significant period of that reporting period. Or they were ill, so maybe they were still coming to school, but their illness clearly impacted their ability to put forth full effort, not because they didn't care, which is they weren't feeling well, right? And that naturally can impact how we're performing. Maybe at the end of the day, we realized the goal was overly ambitious, right? Or there was a staffing change midway through the quarter, and this is a student who really values those relationships and performs better with staff that they're familiar with, and maybe that change just impacted things a bit. During that transition period. I would say if you find yourself in this situation where we feel the need to say insufficient progress, we should be doing whatever we can to give context to that information by outlining the factors that we think may have contributed to this. And even then beyond that, going through what do we intend to do next quarter, maybe to change the tide on this so that hopefully next quarter we can say that we're now making sufficient progress again. So, I would say, at baseline, telling staff, don't ever just put insufficient progress without giving some sort of narrative. And if you are then going to give the narrative, hopefully giving appropriate context for why we think we are where we are, and what we're going to do going forward to, again, change the tide on this progress story. So, to wrap up before we ask questions, kind of bring this back together, at the end of the day, the message I'm hoping to have you convey to your staff is that Obviously, IEP goal data collection and progress monitoring is critically important, but we have to make sure that when we're thinking about our efforts to collect data and to monitor progress, that we're not getting so fixated on quantitative data. While quantitative data is really important, it's important that we also focus on the qualitative measures that are going to impact how we look at the quantitative data and ultimately impact how we tell the overall story of a student's progress over the course of the year. At the end of the day, we want to be able to show families the story of how their child has moved from point A to point B, and I think to effectively do that, you have to have both these quantitative and qualitative pieces to give that complete picture. And so, you know, by doing that, the goal is hopefully again reassuring families that their child is in fact making progress within our program, that we're implementing the IEP as required, that this program continues to be something that we should work on with the student over the course of the year. And then for the staff, you know, it's helping them with instructional planning and thinking about again how we can continue to tweak things or do things to support that student with hopefully then getting them to the finish line of being able to make progress in the general education curriculum. So I really hope this presentation was helpful today. You know, I encourage you to utilize some of the graphics if, if that's helpful to you to illustrate some of these things And again, if I can answer any questions either at the end of this presentation, which I think Crystal will lead us through, or offline, please don't hesitate to reach out and I would love to talk with you. But with that, I will hand it over to Crystal to see if we have any questions.

Thank you, Anne. Greatly appreciate that. And we do have a couple of questions here that I will start with. I just want to remind everyone too, if you go down to the bottom or top, wherever your controls are, You might be able to see the Q&A box. Yes, it is being recorded and the link will go out to you after. If you don't see that, as Anne mentioned earlier, there's a little box that says more with a circle and 3 dots. You can click there and that should pop up your Q&A if you would like to drop a question in. But I'll go ahead and start with what we have. And again, Anne, we're so appreciative. I know that the folks here were really looking forward to listening to you. I really, I feel like I learned some stuff. So that's always great to be able to collaborate in that way, and I, and I again really appreciate you.

My pleasure.

Let's see. Okay, so for school teams that are already stretched thin, how do you balance the ideal of engaging in substantive progress monitoring with the practical reality of staff capacity and time?

Yeah, that's the million-dollar question, right? And that's always the feedback I get when I talk about this topic is like, Ann, I completely agree with you that we want staff to not just focus on this quantitative piece, we want it to be more robust. I just don't know how my staff are going to find the time. And so I'm not going to try to pretend that there's an easy fix to that solution, but what I would say is something I had mentioned earlier, which is at the IEP table, when you're setting aside this time to talk about the IEP, actually take some time to talk about progress monitoring and data collection so that everybody's on the same page leaving that meeting, what generally this will look like. And to the extent you can even build in some time to talk about like what you're envisioning the data collection tool to look like, I think that's also going to be helpful so that on the back end we don't realize that we have a problem with something that we then have to have another meeting, right, and take up more staff time. So I think planning on the front end is really helpful, and maybe if even if that's not happening in the IEP meeting, doing things like having maybe especially our newer teachers show their data collection tools to a mentor, right, to say on the front end And does this look like this is going to get me what I need at the end of the progress reporting period so that it's going to be meaningful? Because I think if you have a really good data collection tool in front of you, it's going to go a lot faster and it's going to make your progress reporting a lot quicker too. So I know that that's sort of an ideal and again doesn't fully address the operational capacity issues that we often have, but my best advice is the planning on the front end is really going to pay off and should hopefully prevent the need to then have further meetings and questions and concerns from families when they're not getting sort of that full contextual information that they're looking for.

Amazing, thank you. Um, here's one right here. What are your thoughts on attaching goal data collection tools and templates to the IEP?

Great question, and I know that's something a lot of my clients, uh, grapple with because in the spirit of transparency, that can be a really helpful thing, right? And it if the data collection is clear, it can be a really impactful way to further illustrate the narrative that you have in your IEP progress reports. The only caution that I give to that is sometimes if it's not super self-explanatory on the front end, sending that data collection can cause parents to get confused. So I think if you want to have some sort of data review with families, sometimes it can be better to do that in the context of a meeting rather than just attaching it to the IEP progress report and sort of of sending it home without having that opportunity for sort of a contextual conversation. But again, if you do that, offer a meeting at least, because we again want parents to make sure that they have the opportunity to ask questions so that they're not misinterpreting the data or not putting it in appropriate context. So I don't have anything against it, certainly not required, but if the family wants to have a meeting, that can then be a great way to sort of explain what the data is to avoid there being any sort of confusion about it. But just as a random aside, if you are sending it home, please make sure staff are scanning it or keeping a copy of it. There's so many times where staff are like, oh yeah, I just sent it home, but then they don't keep the copy for themselves, and we want to make sure we do that. So anything that's sent home, please make sure that staff are maintaining it on your end so that we can pull it up later if questions arise.

Amazing, thank you. Here's another one. Anne, how do you feel about Progress timelines. Sorry, wondering how Anne feels about how progress timelines jive with the regulations. We currently provide IEP progress with interims and report cards, but that sometimes seems overwhelming and leads to incomplete data pictures. Does providing IEP goal progress with report cards alone meet our legal obligation, even if all children are receiving interim reports halfway through each report card term?

So if your division is doing interim progress reports for all students to include students without disabilities, then we should be providing IEP progress reports consistent with that. That's what the regulation says, is we have to be doing it as often as we're providing reports for students without disabilities. But let's say your division wasn't doing that. You know, I'm in IEP meetings a lot where families will ask for it more frequently than every 9 weeks or whatever the reporting report card period is. And I think staff have some understandable concerns sometimes about that, like I'll see sometimes requests for like, every 2 weeks I want data. I think it's appropriate sometimes to push back and say, is 2 weeks actually going to give us a meaningful snapshot of progress? Is that enough time for us to really see the needle being moved for a particular student? And so it may be appropriate to push back to say, I completely understand you wanting to have transparency around the data, and I know you're eager to know how your child is doing, so it's a well-meaning request, but from an educator standpoint, from an expert standpoint, I really think every 6 weeks or 9 weeks is really what's gonna give us a more appropriate gap between data points to have meaningful conversations about, are we moving the needle at the right pace, or do we need to change something? So just know that, yes, we sometimes will get requests from families to have fairly frequent progress reporting, but that might not always be appropriate based on the unique needs of the student, their goal, And, you know, and what's, what's reasonable to infer between data points.

Thank you. And we have 3 minutes left. And if you don't mind, I'm just going to that last slide. I'll throw one more question out so that you can answer this. But real quick, just a reminder, I would love to meet with you. Please utilize me as a resource as you consider resources and things like data tracking and progress monitoring, which is a big part of what our platform can support. But as you're doing that, I will also just ask one more question because it seems to be just very relevant right now. More and more staff— and we have 2 minutes in— more and more staff, more and more of my staff are turning to artificial intelligence to support IEP drafting and progress monitoring. Is this something you encourage or see problems with?

I mean, I think it's both. That I think that I'm using artificial intelligence a lot in my personal practice too, to develop, you know, help develop webinars and things like that. So it's a, it's a wonderful tool, but the bottom line advice I would give is it should never be the endpoint. It should only be a starting point where it might help to jog an idea, right? Or to help make something more measurable. You know, if you're struggling with how to word a goal more specifically, it can be a great way to give you suggestions on how to get rid of those vague verbs we talked about earlier and to have something that's more measurable, objective, countable. So I think it can be a helpful tool, but again, should only be a starting point. And then, of course, the legal advice I will always give is should not be a place where we're inserting personally identifiable student information unless you're utilizing a program or proprietary platform that one of your providers might be using that has those protections in place, right? So good, it can be a good starting point. I don't, I don't stand here saying never use it, but I think there are some things we have to be cautious about both for student privacy reasons but also to make sure that things are robust and effective, and that should just then be a starting point for that conversation.

Amazing. Thank you. It seems like the question of the day or of the times. So thank you for that, Anne. Again, we appreciate you. I, I know how much I got out of this. I'm sure that the folks here who do this day in and day out did as well. Also, just to everyone listening, I want to say that I appreciate you. We appreciate you. We know that your job is not easy and that you do a lot to support our students, your students, students in the community, and that it's not easy. So Please have a wonderful day, and we're going to end right on the hour. Thank you.

Thanks, everybody.