When and When Not to Lean on Tele-Assessments and Tele-Therapy for Your District: SY 23-24 Planning

Hello everyone and welcome, welcome! Thank you for joining us at Parallel Learning on this very busy time of year, which I like to refer to as Maycember. It's the second coming of December, but just in the middle of the year. I'm Taylor Beely. I'm a school psychologist and I'm also the Director of School Psychology and Mental Health Services here at Parallel. Parallel is committed to building the foundation of of different thinkers and learners to thrive. And I'm here to introduce Dr. Jordan Wright. Before I do that, please be advised that you can enter in any questions that you might have using the Q&A feature at the bottom of your screen, and not the chat, if you don't mind. Thank you for sending us all of your questions and thoughts as we get to hear Dr. Jordan speak today. Dr. Jordan is our Chief Clinical Officer. He's also the founding director at NYU Center for Counseling and Community Well-Being, and he has conducted a body of research specifically around teleassessment with children and adolescents. He's authored the only book on the topic, Essentials of Psychological Teleassessment. He's also the author of the Handbook of Psychological Assessment, the most widely used text on assessment in the field. And today we're going to talk a little bit more about when to lean into teleassessment and when not to lean into teleassessment as you're thinking and planning for the coming school year. So, Dr. Jordan, thank you for joining us. Welcome to everyone else on board.

Thank you very much, Taylor, and we're going to invite Taylor back a little bit later for the Q&A. So thank you, thank you, thank you for joining, and thank you to Parallel for inviting me to give this presentation. Here's a little bit of an agenda. We're going to talk about the research behind tele-assessment and the research behind tele-services, support services for kids, specifically those who are different learners and different thinkers, as Taylor said, and how it can be used in schools. And we are going to talk a little bit about when it shouldn't be. There, there, it is not perfect, and the, the research is far from perfect, and so we're going to talk about when to rely on it and when not to. And then I'm gonna, I'm gonna put my clairvoyant hat on and talk about the future of telework the way that I see it. And then there will be a little bit of a Q&A at the end, the last 15 or so minutes. Hopefully we will have time to answer your questions. So as Taylor said, please do enter your questions in the Q&A. She's gonna be monitoring that and posing those questions to me at the end, and hopefully we can get to As many as possible. So this is our agenda for this meeting, and I am excited. Thank you for joining us. So we're going to start with the teleassessment research. Teleassessment is the provision of— we're going to talk today about psychoeducational assessments through virtual means, right? This is very different. We're typically giving tests, IQ tests and academic achievement tests, in person, in the office with a kiddo. And so this is a shift. This is a shift in thinking, certainly one that was necessary over the past few years. The lockdown helped us, I think, put some eyes on what can be done and what can't be done. But I will say the research predated the pandemic. There was research on this. I'm going to present a lot of my own research, And some of the studies that I conducted, and they started well before COVID-19 hit. So we kind of got lucky in that vein. So we're going to start with the fact that there is direct, specific evidence that there are certain tests we can give through teleassessment means without compromising their reliability and validity. This is fantastic. This means that we have studied very, very directly, very specifically, a few tests that are very, very widely used in psychoeducational evaluations. And we have evidence that they are okay, that moving them online does not introduce so much error. It doesn't change the process so much as to render the results uninterpretable or unusable. These are just a few of my published studies. This is just for the nerds out there who want to go and look them up. I will talk a little bit more specifically about each one, but these are my studies that came out 2018, 2020, and they are summarized in, as Taylor said, the book The Essentials of Psychological Teleassessment, which Susie Rayford and I published in 2021. This has a review of all of the literature to date around all tele-assessment. What can be done? What can't be done? We go specifically through a bunch of tests and discuss what evidence is there, what evidence is not there, what can be done ethically, what cannot be done ethically. We tried to be as methodical as possible when going through this. I encourage you to pick up a copy of this. The disclaimer is, if you buy it, I will make a few pennies each time you buy this book. That's my disclaimer. All right, so let me talk specifically about some of these studies. And I will preface this by saying that equivalence studies— these are equivalence studies. We're looking at, is it equivalent to give, in this case, the WISC-V? This is a very widely used IQ test. one of the most widely used IQ tests out there for kids, maybe the most. Is it equivalent to give it virtually as compared to giving it in the traditional in-person way? And doing this research is more complicated than it sounds. You can't just give a kid the test twice. You can't just give it to them in person and then give it to them again in tele-assessment because they will have learned some stuff. So you expect somebody, you expect a kid to do better the second time they take a test, like this no matter what, because they will have had exposure to it. They will have learned what the task is. They will have practiced a little bit. This is called practice effect. So you have to do some methodological gymnastics to make this viable to do this research. It makes this research a little bit more complicated, a little bit harder to explain, and a little bit harder— we have to really go into the statistics, which I am not going to go into. I will have my email at the end of the presentation. And if you want to talk it through, please feel free to reach out to me. But what I found with the WISC is that every subtest on the WISC can be administered virtually without altering the scores significantly, without that change in method producing any error, producing any change in the way that kids are interacting with the test, all except for one. There's one subtest— I won't belabor the point, it's a supplementary subtest that I never use anyway, very few people use it. Um, and so it's, it's not super important. Um, what's most important with the WISC-V study, um, is that I compared every single subtest, every single task that a kid took in, um, tele-assessment versus traditional assessment. We compared— I compared those one-to-one. I also compared, uh, the index scores, the cluster scores, full-scale IQ because we wanted to make sure— I wanted to make sure that it's not just the tasks themselves, but the higher-order scores do not change significantly. And we are lucky because we found that actually it doesn't change the IQs that emerge from this test. That means we can use the test in a tele-assessment context and use the norms, use the comparison sample from the traditional context without introducing a lot of error. This is fantastic. This is really helpful for us. It was very helpful for us during the pandemic, but it is continuing to be helpful for us in thinking about how we can accomplish virtual assessments. Similarly, the Woodcock-Johnson-IV Cognitive Tests, another very, very widely used set of tests for cognitive assessment, for IQ. This study showed that everything was equivalent. There were no alterations of any scores. Again, at the task level as well as the aggregate or scores that are put together for a full-scale general intellectual ability score or a fluid reasoning score, all of these, it showed No significant variation. There was a third study that I'm not going to go into using the RIAS. RIAS? RIAS? I don't know how it's pronounced. The Reynolds Intellectual Ability Scales or assessment scales. And it similarly showed mostly unaffected scores by moving it online, though there were some scores that were affected. And that's going to be really important if you use the RIAS, if you use tasks similar to the RIAS. It's going to be really important to know which tasks cannot be given because they were shown to be not equivalent. And then comes the academic testing. And here again, we get very, very lucky. Woodcock-Johnson Tests of Achievement, which are a very widely used set of tests of academic achievement, reading, writing, math, and such, showed no alteration. That's good, right? If we're testing math, if we're testing math fluency, how automatic math is in the basics of math, how quickly, easily, automatically you can solve those problems, we would hope that it wouldn't be affected whether you're doing it with me sitting there in person or me on a screen. We would hope that if I'm giving you word problems, it shouldn't make a huge difference if the word problem is presented on a screen or if it's presented on an easel in person. And what we found is that actually it wasn't significantly affected, and that's really, really useful for us. That means what we're testing is mostly the academic achievement, the actual academic skills that a kid has. We're not being affected significantly by the change in method, the change in mode. We're not really testing how savvy you are at the computer or interacting through virtual means. We're not testing that as much. We're testing the academic skills and that's what we're aiming for. So these are 4 tests. I didn't put the RIASE on here, but these are 4 tests that I've studied specifically that have direct specific evidence that we are okay to use them. We're okay to use these types of tests to evaluate a kid's intellectual ability, cognitive skills, current cognitive abilities, academic skills. We're okay using these tasks. We have direct evidence that this is okay. There's also more indirect evidence, and the Essentials of Psychological Tele-Assessment Susie Rayford and I developed a methodology for understanding what has support and what doesn't. And there are things that don't have support that I'm going to talk a little bit about when we talk about the limitations. But beginning with cognitive tests, processing, executive functioning, speech-language tests, most of the tasks that are involved in these kinds of tests not all of them, but most of them require the same inputs, mediation, and outputs as those that I have studied, that I have direct evidence. What that means is the input is the stimulus, what I would present to a kid. That's the input. The kid has to take that in, and how they get that information is the input. Mediation is what they do with it in their brain, how they work with it. And then the output is how they communicate their response to me. Are they pointing to something? Are they selecting multiple choice? Are they saying something aloud? So as an example, there are tasks of working memory— very, very common tasks of working memory require the reordering of numbers. So I give you a bunch of numbers and I have you I say the numbers aloud to you and you reorder them in some way, backwards order, forwards order, whatever. And then you say it aloud in the reordered way. This is a task that requires you to like focus, pay attention, take the information in, stick with it, work with it in your brain, and then produce an outcome. There is a task just like this on some of the tests that I studied. That means that other tasks that have the same inputs, mediation, and outputs, we have indirect evidence that those should not be significantly affected by the change in mode, right? This is where we would expect the change in mode to affect the testing, to affect the kid, either in the way that it is taken in by the kid, the input, What they have to do in the moment, or how they communicate it out. So if there's another task that requires an auditory input— me, the psychologist, saying it out loud so that the kid hears it— the mediation is them working with that information in their brain to create some, some solution, and the output is saying it out loud. They have to say it out loud so that I hear it. another test that has those same inputs, mediation, and outputs, we have pretty strong support saying this should be okay because we've tested multiple tests with these inputs, mediation, and outputs, and they're okay. And the majority of cognitive, cognitive processing, underlying processing, auditory processing, executive functioning, speech, language, the majority of these tasks, these tests, these measures have very, very similar or identical inputs, mediation, and outputs as those that have direct and specific evidence already. Those have been evaluated directly. That's true for academic tests as well. The good news is there are limited ways we can evaluate academic ability, right? If it's a reading comprehension task, then you're going to be reading something. You, kiddo, is— you're going to be reading something, and I'm going to be asking you questions about it. So that's presenting some sort of visual stimulus for you to read. You reading it. The mediation is reading it. Then me asking a question aloud or presenting a question on the screen. Some are done slightly differently. And then you have to answer the question either by pointing, selecting a choice, Saying the answer out loud. All of these academic tests are fairly similar in terms of their inputs, mediation, and outputs. They might be slightly different in the types of questions they ask or what they're pulling for, but typically most academic achievement tests have very, very similar structures. They're about reading, they're about writing, they're about math. If we have a math fluency test or a writing fluency test, it's going to require you to write as quickly as you can or answer math problems as quickly as you can using that pencil and paper, which we can do remotely, right? I can ship you a pencil and paper. I can have a school aide or a paraprofessional hand it to you while you take that test. I can watch you do it. I can give the instructions. They're all fairly similar in their structure. So it's not that we have to use the Woodcock-Johnson Test of Achievement. We can use other academic tests as well, and we have good evidence. And again, the Essentials of Psychological Teleassessment book, we went through very, very methodically and anally, one by one, a whole bunch of tests to look for where there was evidence and where there was not evidence. And again, there will be some where there is not evidence. So a note about tele-assessment. Most of what I just presented was about testing, but most assessment methods are already fully equipped for the tele-assessment procedure. We know this. If we think about this, you're— you, you know this, part of a comprehensive psychoeducational evaluation, psychoeducational assessment includes record reviews. I don't need to be in person. I mean, if you, if your district or your school has paper copies of everything, maybe you'll need to scan them. Maybe you'll need to find a slightly different way to do this. But reading records is just fine on the computer. There is no need to be in person for any reason to read academic records. Interviews. Interviews have some of the longest history of empirical support, of research support. This has been done over and over for decades. Research on are Are we getting the same quality of information in interviews if we do it through teleassessment procedures, through telehealth, through an online platform, on a telephone? The research has supported this over and over and over. The quality of data that we get even through telephone, even if you're not doing it face-to-face on a video conference, is just as good as in-person data. This is a robust part of the research. Behavioral observations. This is a big question that people have, right? Behavioral observations in the psychoeducational assessment context means 2 things. One is classroom observations. Are we, you know, some, some schools like classroom observations. I'm not a huge fan of classroom observations. The research shows that they don't add actually that much value to a full comprehensive assessment, because we're going to be talking to teachers anyway. We're going to be getting feedback. We're going to be sending surveys to teachers, to other school personnel. So these classroom observations are not usually— they don't add a lot of value, but some places really like them, and some school psychologists really like them. And actually, The tele-assessment method of doing classroom observations is even less intrusive than a stranger sitting in the back of the room, right? There's something in psychology called the Hawthorne effect. The Hawthorne effect is a finding in the, in the psychology literature that says that people behave differently when they know they're being watched. They behave better, worse, differently. It's just different when they know they're being watched. So having a stranger in the room in a classroom watching your every move, a kid could behave differently than their normal everyday. If that person is somebody that they know, it's a school aide, it's a— in my daughter, my daughter's in middle school and they have a lot of trainee teachers. And if it's a trainee teacher that they're familiar with already, just sort of holding an iPad unobtrusively in the back of the room, kind of, it's pointed at that kid to do a behavioral observation, but it's actually less intrusive. than if I, a stranger, came and sat in the back of the room with a clipboard and my glasses and, you know, I took notes whenever a kid did something. So it is possible to do these classroom observations. The other thing that behavioral observations means in a psychoeducational assessment is an actual interaction with the kid. When the kid is taking tests, when the kid is doing those IQ tests, those academic achievement tests, Do we see the behaviors of what that kid is doing? Are we seeing them click their pen a lot, which I tend to do? Are we seeing them get up out of their seat? This does change a little bit in teleassessment. I've written a little bit about this, and what we found is— and some colleagues and I have done some work in this area— what we found is that Behavioral observations are a little bit different. So we may miss a tapping foot, right? Because if you're watching me right now, you're seeing me from the shoulders up. You're not going to see if I'm tapping my foot. However, it forces us to focus a lot more on the face. So it forces us to focus and not get distracted by a tapping foot. So I will see if a kid is going like this and chewing their own lip. Or, or kind of having a very subtle behavioral tic or, or some other behavior that shows, oh, they might be a little bit anxious. Or so we are getting slightly different behavioral observations, not better, not worse, just different. So we are still able to do these behavioral observations, classroom observations. School and teacher input again doesn't change. Teacher surveys, school surveys. If we send a survey to a teacher in a, in a very traditional psychoeducational assessment, most often now we're actually sending a link via email and they fill it out on the computer. That happens exactly the same in, in a tele-assessment. So this is ready-made for tele-assessment. Other data may be input from parents, maybe input from tutors, but maybe other stuff. Pretty much everything is, is ready-made for online gathering of those other data. Survey data— if I'm surveying a kid, if I'm surveying a family, we're doing this typically online anyway. Very few schools are using real pencil and paper things and then having the psychologist go and enter that into the computer. More and more research is coming out that if we have them fill out surveys directly on the computer, we reduce error a lot. There is user error when kids or parents or teachers fill stuff out by hand, and then we have to manually enter that into the computer. That is a step that is taken out when we administer things directly on the computer, and more and more measures are only having that as an option anyway. And finally, the test data. Finally, we get to the actual tele-testing. So tele-assessment is all of this put together, and we know this, right? We know that if you're doing an IEP process, if you're looking at a 504, if you're looking at what supports a kid needs, you're never going to like hinge a decision on one data point that comes from one test. one time, one day with one kid, right? You're never gonna make that decision. We use all the data. And our decisions are really based on all of these data together and where they overlap and where they intersect. So you can see 1, 2, 3, 4, 5, 6 out of these 7 things we're already kind of doing for the most part through telehealth, through online procedures. Maybe 5 of the 7, if you don't count the behavioral observations. We're already doing it. It's already fully equipped to be doing it through tele-assessment procedures. Again, for the nerds out there, I published with some colleagues last year a paper on evidence-based psychological assessment. We're calling it EBCPA. This is in a flagship journal, the Professional Psychology Research and Practice. It's one of the American Psychological Association's big journals, and we did a deep dive on what is evidence-based assessment, because we wanted to know, are we— if we're doing teleassessment, if we're doing other stuff, if I'm switching up the way that I'm administering something, am I doing it in an evidence-based way? If I'm not considering diversity, Am I doing it in an evidence-based way? And we wanted to know really what does the research show is evidence-based. And the take-home messages, of course, we have to consider context and culture, right? We have to understand test scores within a whole context of a kid's life, within the whole context of their cultural background, their experience of privilege, their experience of oppression. And like I said before, it is all about Multi-method evaluation, multi-method assessment procedures. It's where the overlap is of all of our different data sources. That should be our decision-making point. Shouldn't be driven by any one data point. It should be reinforced and revalidated over and over by different data points. All right, I promised the limitations, the limitations of teleassessment. The first one is kids under 7. Kids under 7, I said that I would talk about the RIASE study, and it's not just that one. There are other studies of even in-person work when you use iPads, when you use tablets or laptops. Kids under 7 actually interact differently with technology. In a way that affects the data. So we really shouldn't be doing the tele-testing with kids under 7, not the tele-assessment. Remember, all those other things can be done, but we may need to think about, okay, let's do the testing part, the IQ, the academic achievement, the cognitive processing, all that stuff. Let's do that in person in the traditional way for kids under 7, but we can still then do some interviews with parents, with Teachers, we can do observations, we can do all that stuff through tele-assessment procedures and put it all together in that evidence-based way that I talked about in the article that I presented. So under 7s, problematic in terms of the data that we get. Then there are just some tests that aren't going to be possible because, one, they might not be logistically possible, or two, they may not have been studied yet, their input, mediation, and output. I'll give 2 quick examples. One, there is a task, it's an executive functioning task, where it's kind of a, it's a big sheet of paper with a connect the dots. And the dots are big dots with numbers and letters. And the task, it's executive functioning, right? It's, can you control your impulses? The task is to alternate numbers and letters, right? Our impulses say go from 1 to 2 to 3 to 4, or A to B to C to D. So to control yourself and switch your strategies, you have to go 1 to A to 2 to B to 3 to C. This is a very, very traditional executive functioning task test. But the input— that sounds doable, right? You just put the piece of paper in front of them. However, it is timed and mistakes have to be corrected in the moment. That's part of the input. So the input is a piece of paper with the connect the dots. The instructions connect them alternating, and the input is correcting you in the moment. And we can't do that easily and effectively at a distance. So this is just not going to be perfectly possible yet. The other one, you know, there are other tests that require like bubbles or toys of certain sorts that we just— we're not at a point where we can really replicate actual bubbles online in a way that Interacts— that kids can interact with in the exact same way as they interact with bubbles in real life. So there are some tasks, some tests that are not going to be able to be used, and you need your school psychologist to know exactly which ones have that support and which ones cannot be done. All right, tell us services. I'm going to talk about support services and the research behind them. This is a little easier because the research is much deeper and stronger and longer lasting. So teleservices are things like teletherapy, telespeech, language therapy, remote specialized instruction, remote tutoring, remote remediation, things like Burton-Gillingham reading remediation or these sorts of things. They have a long history of use and validation. This is, this is actually the easier sell today. Think about speech and language therapy. The overwhelming majority of interaction in speech-language therapy, the overwhelming majority is about speaking, right? It's about the kids speaking aloud. It's about listening and speaking. Yes, there are worksheets. Yes, there are things about reading aloud. There are things, um, the, the overwhelming majority doesn't require a lot of like writing, writing, writing, or speeded writing, or these sorts of things. Um, so speech-language therapy has a long history of very effective use and support for being done remotely. It's being done widely, and kids are improving, which is all we really care about at the end of the day, right? Cognitive behavioral therapy, CBT, and other types of supportive psychotherapy also has a really strong history of research support. One of my favorite research studies looks at a very specific type of therapy called PCIT and actually found that teletherapy services using this model were actually stronger than in-person services. And it had a lot to do with the setting. It had a lot to do with if we're worried about a kid's behavior at home or a kid's behavior at school, let's do the therapy with them at their home. Let's do the therapy with them at their school. If we're worried about anxiety in school, rather than coming to my beautiful office that is safe and quiet and comfortable and loving and supportive, and it's like walking into a big hug, like, let's put them— let's keep them in their context and do the work there. So we can do it remotely, and it's got a lot of support for therapy. Instruction, educational instruction. Math instruction, reading instruction, writing instruction, these sorts of things. You all know online tutoring companies, they work. There is, again, a long history of support for this. And we're going to talk in a— I'm going to talk in a few minutes about how we are not going to really, in the future, be able to disaggregate education and technology at all. I put this in here, executive function coaching, only because If I were in charge of the world, if I were in charge of a school or a district, this is the area that I would flood the school with. I would, if I, especially elementary and middle school, I would flood them. I know SEL is very, very sexy and I'm a big fan of SEL and self-regulation and these sorts of things, but executive function coaching I see more and more as one of the most fundamental skills for kids to access the curriculum, for them to access learning. If they can plan, organize, get themselves in a state where they can really focus, this, I think, would be where I would throw a lot of resources personally. And there's good strong support for doing this in a tele-remote context. All right, how is this used in schools? First and foremost, the good side. It's efficient, right? You log on, there's not a lot of setup, especially in the testing where I have a lot of materials. Usually it's all right, right there built in. It's easy. The efficiency frees up resources for the school. Frankly, it just frees up your own resources because it's kind of on demand. It improves access. It's been shown to improve access as long as there is reliable internet access and the devices needed, then especially like rural schools where maybe a school psychologist isn't within an hour or 2 of, of that kiddo, or virtual schools, or very densely populated schools where school psychologists and other support providers are very overwhelmed. We know that this provides quicker access to these assessments, to these services, and it reduces burnout. Tends to reduce burnout by allowing more variation in the day. We know, for example, school psychologists are so overwhelmed with evaluations, with assessments, with IEPs, that they often don't get to do a lot of the counseling. They don't get to do some of the stuff that they really want to be doing. And they— this allows them to free up some time to be able to vary their workload and think about what they're doing in a more planful and varied and maybe more engaging way. And it does tend to reduce burnout. These are the good things. There is some burgeoning research that in addition to efficiency, it can actually even improve quality of the services. So the next generation of research in this area is not just is it equivalent or as good as, but can it even be better? Can it deepen learning? Can it deepen speech? Can it deepen language? Can it deepen these skills that kids need? And especially with online metrics, with the advent of AI and all these things and being able to capture data, we can actually get more real outcomes in real time to reinform what it is that we're doing. So this is actually going to contribute to very, very high-quality services as well as just efficient services. It does come with some drawbacks. There are some costs to schools. The first one is it takes coordination, right? The, the school, the in-person staff at the school, you need someone there to coordinate the schedule, to help coordinate the scheduling, to find that quiet room, the private room. If a kid is doing counseling in school and they're doing it remotely with a remote counselor, then they need that quiet room. They need those headphones. They need, depending on their age, they need someone to log in for them. There is some coordination needed. That doesn't have to be a school psychologist, though. It doesn't have to be the sort of higher-paid support personnel. We can get creative about who is helping coordinate. Onsite help. We do sometimes need hands-on help logging in. Sometimes with tele-assessment, we might need someone to hand the record forms or the response booklets or a pencil without an eraser or something like that. We may need a helper onsite to kind of help the remote psychologist doing the work or the remote SLP doing the assessment. And collaboration, and this is a big one. Schools need to be willing to collaborate with remote personnel. The flip side of that is schools need to find those remote personnel, those companies, those organizations who know how to collaborate well with schools. This is not easy. It's not intuitive always how best to collaborate between these two very, very differently working entities. The in-person school, we're used to doing things one way. The remote supports or the remote assessments are doing things slightly different. So that collaboration, that communication is really, really key.

All right.

Now my last, I promised my last agenda item is about the future of telework. I'm going to put my psychic hat on. When it comes to education instruction, when it comes to— I know my daughter here in middle school in New York City uses quite a few apps. She gets sent home with math homework that's on one app and then a math evaluation each week on a different app. And education is getting more and more intertwined with technology. If you've been to a conference recently in education, in the future of education, in special ed, in whatever. If you've been to a conference, I guarantee you've seen a lot of presentations on how we can leverage technology to improve education. This is especially true for kids with learning and thinking differences, right? Those who have some roadblocks with more traditional education, we need to get more creative. And technology allows us some latitude to get more creative in how we are providing education to these kids who are ADHD or different thinkers or dyslexic, these sort of different learners. This isn't going anywhere. So technology in education is here to stay. We just have to embrace it and help it get better.

Right?

We can't just sort of blindly say, okay, well, let's just try whatever's out there. We need to do some work. We need to do some research and make sure that we're actually improving education through technology. Tele-support services. Students, if the lockdown did anything, it was proof of concept. Kids are really getting much more comfortable in interacting through virtual means, through especially video conferencing means like this. Kids are very comfortable. So doing speech therapy, doing counseling, doing executive function coaching, doing educational remediation or supports through tele-means is not only viable, but again, the research is showing it can, it can place itself In a way that deepens and improves outcomes and reinforms itself with those outcomes that we can collect in real time. And this is a little insider information. Don't tell anyone I told you. I work with a lot of the test developers and the testing companies, and they are actively developing the next generation of tests. You saw the WISC-V. That means it's the 5th one. It's actually more than that. There were other iterations. The Woodcock-Johnson before. These tests get updated. And the next generation of tests, a lot of them are being developed fully online. They're being developed— the test developers know this is the future of assessment, of psychoeducational evaluation, of psychological assessment. This is the future of it. So we need to probably get rid of some of our more antiquated ways of evaluating cognitive functioning, academic functioning, and think about more sophisticated online adaptive ways of doing it. I am on a task force right now with the American Psychological Association thinking about what are the biggest issues facing the assessment world, and one of them is our antiquated methods are just too easy to find online, right? You can, you can find the items, you can practice, A lot of kids do practice to appear gifted on these tests. They just, they practice the types of tasks that they're going to have to do because we are antiquated in our methods. If we develop these tests online, we can get smarter about adapting them and making them different and harder to fake or cheat or practice. So this is actively being developed right now. This, this won't be a decade away. This is a year away. With a lot of the tests that are being developed right now, they are being developed online. That's my, that's my psychic hat. And now, I promised I put my, my email address up here. Please feel free if you, if you have questions, about any of this, if you have concerns, thoughts, if you just want to nerd out with me about it, please feel free to email me, jordan@parallellearning.com. And I see Taylor is back with us. Taylor is fantastic. And I am excited to do a little bit of Q&A. And I am going to leave this up for one more second. And then I'm going to take it down so that our faces are nice and big on your screens.

Thank you, Dr. Jordan. Thank you for sharing your predictions with us too. It's a lot to think about. Very exciting to think about how the landscape is going to change. We do still have some questions that came in. The first question is if you could talk a little bit more about how the WISC is conducted virtually, specifically the block design subtest, and what does that look like in a virtual administration?

Great question. Someone out there absolutely knows what they're talking about with this. I didn't know who would be in the audience, but block design is a task that has physical blocks and you have to A kid has to manipulate them to match a stimulus, a picture. And this is really hard in the tele-assessment environment. In actual reality, it would be really easy to develop something like this online, but that's not the way it's developed currently. It's developed with actual physical blocks. The WISC-Online does not use block design. It doesn't use the physical blocks. Again, we got a little bit lucky way before the pandemic. The test developers, the major one is actually my co-author on the tele-assessment book. The, the, the— one of the adaptations that have been made is called block design multiple choice. And of course, it's going to get something slightly different. It's going to get at a slightly different task. But block design is meant to measure visual spatial ability. And we can do that through what's called block design multiple choice. It's just a stimulus with multiple choice questions that get really complicated. And it kind of looks like block design. And they did research on it and they normed it differently. So you don't, you don't use the exact same norms. You don't, you can't use the norms that you would use with block design, but you use the norms that you would with block design multiple choice. And that gives us as strong a Visual Spatial Index, which is what that loads onto. So you have to know how to do it. Again, a tele-assessment is not as easy as being good at assessment and then just kind of throwing it up online. You've got to train in how to adapt these measures to the online environment. You have to know exactly how it works. And for that one, that's, that's a kind of a little preview of how That particular one works.

Thank you, that's great. Um, staying on the topic of assessment, what advice would you give to a school team who's thinking about dipping the toe in the waters of doing tele-assessment to help kind of relieve the testing caseload for their team and allow them to focus on some other needs in their school? What would be kind of some considerations you would be thinking about? To see if maybe teleassessment would be right for your school and your students?

Yeah, so there are a couple of considerations. One is why you're doing it, and Taylor, you just— the question sort of told us that it's to relieve some of the on-site support staff, and that's a really good reason, right? We can engage with some sort of teleassessment, some company that does it, And, and relieve the onsite staff to be able to vary their work. So think about why you're doing it and do it for the right reasons. Not, not— there are probably plenty of bad reasons to do it. To dip your toe in the water, I strongly recommend finding a company that does it and does it well, that has sort of a, a proven track record or a provider who has done it that you could contract with. that knows exactly what they're doing. Now, this is easier said than done. There are a lot of providers out there who think they're doing good teleassessment and may not be doing fully research-supported teleassessment. There are certainly companies out there that are cutting a lot of corners, and there are others that are not. And I, you know, I happen to work with Parallel, which I think is a fantastic company because I work there, and fantastic people work there like Taylor. But we're not, you know, we're not the only company, but we take very seriously doing it in alignment with the research support, and that's super important. It can't just be a school psychologist saying they kind of know how to do it. Like, I could probably do that. can help you through the process, can really collaborate with you and walk you through dipping your toe in. There are things I guarantee you're not thinking about. There are problems that will arise that I guarantee you're not thinking about. I'll give you an example. A kid could absolutely take a laptop and just— I was going to close my laptop— and just close it and walk out of the room, and a tele-assessment professional wouldn't have a lot to be able to do there. So The evidence base says we need to know where that support person is. We need to know exactly who to call, how to call them in the moment for that extra onsite support with a para, with a school aide, with a trainee teacher, whoever it is that's our point person for that moment, for that assessment has to be there. We have to know some of these potential pitfalls so that we know we're doing it well. Well, and then give it a shot. Give it a shot. We know that it can be done validly. If you think it can be helpful for reducing the workload of your school psychologists, your SLPs, your OTs, whoever it is, if you know that it's going to reduce their burnout, their overwhelm, give it a shot.

Yeah, absolutely. Talking about the research around more of the therapy side of tele- Telework, teletherapy, teleassessment. Tell us a little bit about adolescents specifically in terms of how successful has teletherapy been for working with middle school and secondary students?

Oh, that's the easiest question ever. I thought you were gonna say— I thought you were gonna ask a really hard one and say, tell me about like ADHD elementary school kids who are all over the place. That'd be a harder question. Adolescents, are, you know, they are what we call digital natives, right? They were born into FaceTime. They were born into iPads. They were born into these things that I wasn't because I'm old. So the research has shown that is a huge benefit for kids and adolescents. The research has really hoarded. This kind of— there's also something really interesting in the literature. There's something a little bit safer. So when I am— I'm sitting here, I am looking at Taylor. If Taylor were my therapist right now, if I were in the room with her, I would be staring at her and she'd be staring at me. When we're on, you know, a teleplatform like this, I can kind of— I can look at the cam. I'm looking at the camera right now. I'm not looking at Taylor. There's something a little safer about this environment for kids. It's not as vulnerable and dysregulating as sometimes in-person services can be, which frankly, when you're in a school, when you're doing school-based services, it's not the type of counseling where we want to break you down.

Right.

It's not psychoanalysis where we want to do 18 years of breaking you down to your core emotional states and that kind of stuff. No, it's usually skills-based. It's usually, we want to help give you some coping mechanisms. We want to work you through improving. We have maybe IEP goals or other therapeutic goals. And so we want it to be a more safer and more comfortable space. And we have— the research has shown that this can be a much more comfortable I can definitely attest to that.

Last kind of question we have here is just a desire on behalf of the audience to kind of— what resources, other research, if they wanted to dive into the research, or if they're looking for resources for teleassessment and teletherapy, what direction would you point them to?

Yeah, so there are a couple resources, and again, I'm going to Hawk my own stuff for a second. The— for teleassessment, the Essentials of Psychological Teleassessment book is the only one. That's the only game out there. Normally, I will say this, normally I would never have written it as a book because it's research and we're reviewing the research and that's always changing, except that we knew this wasn't going to change anytime soon. Because of the pandemic, it's hard to do that kind of research, that equivalence. research because we don't have the typical. It was hard for the past 3 years to collect data that's the standard in-person practice because you had PPE or you had nervousness or you had anxiety around, you know, or 6 feet distance, which I never have 6 feet of distance when I'm testing a kid in person. That just doesn't happen. So the tele-assessment book, if you're interested in that, for a much freer Review of this, I just published a white paper. If you go to parallellearning.com, the white paper is right there. There are some resources on tele-assessment. I've published a white paper on executive functioning and executive function coaching and where the evidence is around that, and those you can download for free. And those summarize the literature around it. You know, if you, if you Google it, you're probably going to find some really good stuff out there and some really bad stuff. But there, there are good resources. The American Psychological Association, I've worked with them to put out guidance on teleassessment. They have a really good set of guidance on telehealth in general and teleservices. NASP does as well, the National Association of School Psychologists. has a fantastic set of resources and guidelines for teleservices. I've summarized those in the white paper as well, but you can go straight to the source and go to NASP. I would recommend those as more sort of legitimate scholarly resources rather than, you know, some random person's blog or psych post somewhere on the outskirts of the internet. That's just me. Thank you.

That's a huge help. I know, especially the clinicians among us, we always like to dive into the research first and make up our own minds. So I appreciate that. As we wrap up today, I'd like to thank everyone for joining us again. We do have a gift card giveaway, and the winner is Lisa McDaniel. Lisa, you are the fortunate winner of our gift card. So keep a lookout for that. And again, just don't, don't hesitate to contact Jordan by email. Be sure to reach out to us at Parallel Learning if there's anything that we can do to support your learnings and your school communities, and we just thank you for taking the time. Bye, everybody.