Workshop: Designing Engaging Pre-College Research-Based Learning Programs

Good morning. Hi there. My name is Rory Fulger and I am the Pre College Education Programs Manager at Wolfram Research. My kind of general background is that. So I have a bachelor's degree in Computer science and a master's degree in Education and Technology. And I've been running education programs at wolfram since around 2019. And today what we're going to do is discuss the kind of ecosystem of learning and development opportunities for middle school, high school and college students at Wolfram and beyond. The kind of overview of the programs and the potential pathways that students can take. We're going to discuss what they learn, how they learn it, and the reasoning behind our program design. So we're going to get into some pedagogy and some pretty exciting stuff around constructionism and constructivism and project based learning and research based learning and active learning or our big buzzwords for the day. So we're going to start out with a bit of an overview of, of our programs. So while the Pre College Education Programs team runs a bunch of programs online in the US and internationally, we're going to be focusing on kind of our big three today. So those are the Wolfram Middle School Summer Camp, the Wolfram High School Summer Research Program, and the Wolfram Emerging Leaders Program. So we're going to talk through what each of those three programs really involves. We're going to do a kind of overview look at the student pathways, the learning objectives, the pedagogy, and the reasoning behind our program design. So we're going to start out talking about the middle school Summer camp. So our middle School Summer Camp we Sell was founded in 2020 to try and address our enrollment gender ratio at our high school program. So the middle school Summer camp is just for girls and for gender non conforming students. And it's a five day virtual workshop for about 25 kids aged aged about 10 to 14. And the kids need to be really excited by STEM, but we don't ask for any previous program experience. And, and during the middle school summer camp, we kind of grapple with the basics of computational thinking and coding in Wolfram language. So we use a lot of code to do cool things like we create a bunch of kind of 2D and 3D art, we write music, we explore machine learning, we create interactive content, we do a lot of really active learning. It's all online. So we do everything in kind of. We have lots of small breakout groups where we have a very high teacher to student ratio. So it's about, about three or four students per per teacher. That we have. So it's a very, it's a very tight program and the kids learn a bunch of computational thinking stuff and it's very exciting. Our really big program, our flagship program is our high school summer research program where we have around 60 high schoolers from all around the world. And they come to Boston and they engage in a really intense learning experience for three weeks. And this is, this is the program that I'm, I'm really going to be talking about today. So we, for, for this program, just to give you a bit of a background, our staff team is staffed by kind of directors, mentors and TAs. So our mentors are subject experts. Around half of the mentor team are Wolfram employees, and the other half are industry experts, graduate students, and occasionally exceptional undergraduate students. And each mentor is assigned three to five students for whom they're like, really academically responsible. Our teaching assistants are younger Wolfram language experts who are graduates from our, from our programs. So we feed them in and they're responsible for organizing social activities and providing academic support to students, most commonly with coding issues. So students spend the first few days of this program learning Wolfram language and computational thinking in active learning classes and in workshops. And then the kind of, the big thing of the program is that students spend. The first students work with myself and with Stephen Wolfram to choose a project tailored to their interests, their strengths and their goals. And then throughout the program, the students work with an expert mentor to develop their project. And at the end they publish a computational essay on Wolfram community to display their work. These projects are incredibly diverse. Going from, you know, mapping the mapping hand movements between ASL finger signs and go. Going from there kind of to creating video games from. We had a student this year who created Pac man levels from city street maps. So instead of, you know, making, making a maze or whatever, he used street maps to create the spaces for his little Pac man to move around all the way to, you know, some pretty intense kind of Wolfram physics project projects. And, you know, we have a huge diversity of projects. Alongside their projects, students attend a variety of academic and non academic activities which include Q&As and small group discussions with, with Stephen Wolfram interactive special topics talks on topics like cryptography, astrophysics, food data, computational farming, software development, natural language processing, the Wolfram physics project, analyzing fungi for virus development, all kinds of, well, not for virus development, I guess, but for immunization development. Very cool. They have workshops on things like video game development, esoteric languages, niche math, debugging and refactoring. Skills, design, thinking, whether. So they, they do these workshops where they're very involved and they do a lot of coding every day there's two or three project presentations, kind of like a conference type vibe from mentors and teaching assistants where they present their own Wolfram language projects so that the students get a huge amount of exposure to the different types of things that can be done. We have minor tracks, we have, these are creative tracks where students choose to focus on something very non technical. We usually do creative writing, art, music or theater and they get to pick one and they spend five evenings out of the three weeks kind of focusing on this creative activity with the goal of creating an entry for kind of end of end, end of program talent show. And then we have social activities that range from field trips to MIT to chess tournaments to basketball, to product design and entrepreneurship workshops, to debate stuff to Dungeons and Dragons and murder mysteries and watching cartoons and there's a lot of fun. So the kind of big learning objectives for, for the summer research program are learning computational thinking, learning to code in Wolfram language and learning a huge amount of their of content for their project. So they choose their, their project content and they, they learn a lot about that. Our kind of top tier program is the Wolfram Emerging Leaders program which was founded in 2019 to help students extend their experiences from our summer programs. So Wolf Emerging Leaders program whelp takes the top 50% of the high school alumni for our WELP high school program and a small selection of undergraduates from previous years of, of our, of our summer program or from the Wolfram summer school. So these programs are similar but slightly different. So the, well, high school students are sorted into small groups based on their shared interests, are assigned a subject matter mentor and then as a group over the course of a school semester, part time, they define, propose and then do a project and then they publish their computational essays at Christmas. So they have created a, again a huge diversity of projects that are more intense, more, more, more involved than their school, than the, than their summer projects and the world. Undergrad students do an independent research project, so instead of in a group they do, they do an independent one over the course of a full academic year, also under the guidance of a subject matter expert. And kind of over the course of the program, students attend a bunch of workshops which have ranged from learn to use GitHub to writing resumes and LinkedIn profiles, from 3D video game development to creating tools to hack daily life and from multi variable calculus to LLMs and AI. And all this kind of stuff. So our kind of big, big goals for this are that they kind of further develop their computational thinking and their coding skills, they develop a deep understanding the content that they're working with, and they also learn how to manage a research project, how to organize their time, how to collaborate with a team and improve their programming and subject skills. And then beyond just attending these programs, our students are also involved in them. So successful alumni are invited to become teaching assistants and mentors. So select WHELP students are invited to join our teacher training program, which, which happens during that school year. And being a TA or a mentor at the middle school summer camp or at the high school summer research program is their capstone project for that qualification. So alongside doing their WELT project, students learn how to helpfully critique other people's work, how to teach concepts, how to run social activities, how to become strong leaders, and it's pretty great. So students who successfully complete the teacher training program at WHELP U at the undergraduate level are invited to become mentors at the high school summer research program, where they have their small group of students to mentor. And successful high school students are invited to become teaching assistants at that program or sometimes mentors at the, at the middle school program, which requires slightly less subject involvement. And then people who are successful summer TAs or mentors are invited to become mentors at welp to invited to become our subject matter experts for younger students. So we have a large number of students who have cycled through the system over the last five years, including a bunch of people who've attended the high school program in 2019, 2020, and are now in the kind of your mid to upper undergraduate years who were mentors of the summer program this summer. And they did fantastically well, in some cases better than our kind of, you know, industry professional people because they've had some, some real teaching training. So just as a kind of, just to show you kind of how your journey might go, if you were involved in a program, you might attend the middle school camp as a student, then you might attend the high school research program as a student and then attend the WHELP high school program that school year. And then you might attend the summer program as a teaching assistant, or you might attend the and, or you might attend the middle school camp as a mentor, and then you would probably attend the WELP high school version again and then attend the summer program again as a teaching assistant. Usually you would do that twice. Then you would attend WHELP U as a student once you become an undergrad and Then you might attend the high school program as a mentor. So get that bit of a promotion going on there. And, and then you might attend Whelp as a mentor. You might do, do well kind of throughout that, that school year as a mentor for younger students. And then you can kind of keep, keep repeating that as, as many times as you want. I've I've often, I've often told people when they're asking me when they're trying to send their kid to the middle school camp and they say, you know, well what, what can we do after this? I'm like, well, you give me your kid, give me your kid now. I can entertain them for the Next, the next 10 years. Which is a pretty cool thing to be able to say. And then kind of beyond that, they become interns at Wolfram. They can join other Wolfram programs. Stephen Wolfram runs a coding club where he does lots of live coding with the kids. They can join the Wolfram Institute as junior researchers to do physics projects things and metamath things. They can join the Wolfram Student Ambassador program which is a, where we support them to do research projects, to run events at their schools and universities, to run hackathons, to run, you know, coding clubs, to teach to, you know, be involved in the community. All kinds, all kinds of things. They, they get very involved, which is very cool. So before we get into our kind of pedagogy stuff, I just want to talk about who our students are. So we don't, we don't really select for academic performance or school grades. So our application process includes several short essays, a coding problem set and an interview. So while we don't select for school grades at all, we don't care about that. Most of our students are gift and talented or have been accelerated in school. They're all very, very smart. And around 20% of our students have some form of learning disability. So usually adhd, autism spectrum disorder, dyslexia, stuff like that. So students who are gifted and talented and also have a learning disability are referred to as twice exceptional. So they're exceptional on, on both ends of their, their scale. And that's often associated with kind of asynchronous development. So being really good at something and not something else and also with, with lower confidence and lower, lower self esteem because that can be, it can be a confusing way to exist in our, in our traditional school system. So, and we'll get more into that. But students have to be demonstrably passionate and adept in any STEM related or STEM adjacent field. But because we acknowledge the kind of opportunity gap in STEM for a variety of reasons, be that gender or geographic or racial or economic or anything. There's lots of reasons why some people might not have as much opportunity for exposure to stem. We offer a, we don't require students to be experienced coders in any way. So instead we offer a kind of two day pre program workshop for students who haven't taken computer science classes before where they can, where they're able to kind of take a shallower on ramp to learning Wolfram language, where we teach them some, some more intro to CS skills there. And out of our 60 students who attend the, the high school summer research program, we usually have around 15 who, who also attend the workshop who've never taken a CS class before. And it's amazing to, to watch them, to watch them grow through that very, very quickly. It's, it's very cool. A really great thing about running a summer program instead of running a school is that our students for the most part are really genuinely choosing to be there. They volunteered to spend their summer learning new things. And this means that they're really intrinsically motivated. So this kind of intrinsic motivation primes them to learn skills and content more easily as opposed to being, as opposed to their motivation being dependent on a grade or a reward or a punishment. So we, we don't want them to depend on external factors for their, for their motivation. So due to the kind of really wide range of skills and experiences and interests and abilities, it's really important that our programs be personalized and rigorous and supportive. So we really focus. The high school summer research program is focused on this project that they create. And by allowing students to choose the direction for the bulk of their work during the program, we're able to offer learning support to a much more diverse set of interests and abilities than if we tried to passively teach content. Right. There is no way that we could teach active like actively teach the, the variety of content that these students learn to do their projects. You know, we couldn't have a three week program that focused on biology, but also on physics and also on math and also on digital humanities. You know, it's just, it's just not, not possible to, to pack all of it in. So instead of doing that, the kids get to pick their project, they get to really focus on their content that they're motivated on, that they're motivated by. And, and we step back slightly from that and help them to learn the content rather than teaching it to them. Which brings us to our research Informed pedagogy. So, and we, before we move on to that, had question, question in the chat asking how the giftedness assessment is done, is there any formal testing and if the disabilities kind of require any, any medical records. So we, we don't, we don't get any, we don't ask for any information from their schools or for assessing their, their giftedness. We, we experience their, their giftedness in, in meaningful ways rather than needing to assess it. So we, in their application, we have them write essays and do an interview and complete a coding problem set in wolfram language, which 99% of them have never touched before. So we get a kind of uninterrupted baseline for that. And that means that it's pretty easy for us to assess whether a student is going to be able to cope in our pretty intense environment. And for disabilities, we don't ask for any doctor's notes or anything. Parents have the ability to tell us any disabilities that their kids had and request any accommodations that their kid might need. So that's how we get that information. And then we make a lot of effort to make sure that our students with disabilities are supported in various different ways. Our most common, like I said, most common disabilities that we have are adhd, Autism Spectrum disorder and dyslexia. And we have various ways that we work with those kids slightly differently to, to our neurotypical students, which we can talk more about if we're, if we want to towards the end. But yeah, great question. Thank you. So now that, now that we, now that we have an understanding of the kind of range of programs that we offer, let's, let's talk about the, the pedagogy, talk about the, how we design this curriculum, how we, how everything works, why it works. So, yeah, so we're going to be focusing mostly on our, the high school summer research program, which is, you know, as we said, our kind of flagship program. We have, we have 60 kids who learn Wolfram language and do a project that is designed, the project is, is designed by myself and Stephen in collaboration with each individual student. And at the end they publish their computational essay on Wolfram community and they have written a whole essay, they do a whole research project which is very cool. So at wslp, the Wolfram Summer Research Project program, students benefit from a meticulously designed curriculum which offers support and freedom on kind of both ends of this, of this spectrum. So it's a very learner centric program rooted in social constructivism, which we'll get into in a moment and it's designed to meet a bunch of kind of hidden curriculum goals. So while our exposed curriculum goals are exposed learning goals are learning to code, learning computational thinking, learning research project skills. We have a bunch of hidden curriculum skills that we're working on. We want students to grow in confidence. We want them to develop a growth mindset. So thinking about struggling and failing in a growth oriented way. So instead of, ah, I failed to do this thing, I struggled with this thing, that means that I am a failure and I will never be able to get better. What we want them to have instead is a growth mindset where they struggle with something and they think, fantastic. This is a real opportunity for me to grow. Doing things that I'm good at is not a great learning experience. I want to do things that I'm bad at so that I can get better at them. And so we try to really encourage, encourage that. We want them to kind of overcome any fear of failure that they have. We want them to embrace struggling with hard content and skills. We usually find that the vast majority of our students are terrified of, of failing or struggling in, in any meaningful way. They've, they've kind of usually been been taught in, in school that failing means like failing is, is the end of, of something, right, rather than the beginning. So they failed, so they got a D, they got an F. They aren't allowed to take the next class. They're get in trouble with their, their parents and their teachers. And so they've, they've really been. And, and they're also, you know, everybody is, is rewarded for getting the answer right first time. And the longer it takes you to find the answer, the less smart you are. But what we want to encourage is the longer it takes you to find an answer, that means that you've chosen a, a hard problem and that's fantastic. And that means that you're going to learn infinitely more skills than if you had picked an easy problem. We also want to strengthen their self efficacy and self concept. So the self efficacy is how a person views their own ability to perform tasks or to meet goals. And self concept is how a person conceptualizes their self, their relationships to others, to others, and their own academic strengths or abilities. So somebody with, with low self efficacy or concept might think, you know, oh, I'm bad at this, I'm, I'm bad at making friends, I'm bad at math, all of this kind of stuff. And we want to strengthen these, these things to where they think that, you know, I'm good at learning from my experiences. I'm good at finding out, finding out information. I'm good at asking for help. We want all this kind of stuff. So our kind of key pedagogical concepts for these programs are this little list here. And we're going to go through these kind of one by one. So social constructivism is. Everything that we do in our, in our education programs is rooted in theories of social constructivism. And social constructivism is a big, big learning theory that posits that learning happens when a student constructs new knowledge in their own mind. And that happens when they interact and collaborate with other people. So, for example, when you sit down with a child and you read with them, you read to them, you're pointing out words, you are helping them to construct in their mind language models and storytelling. So they are learning from that interaction. They're learning from, they're not learning to read from the content of the book. They're learning to read from constructing these models in their mind, guided and aided and abetted by the social interactions that they're having with the people around them. This is really in contrast to this more kind of traditional filling with knowledge model where we take fully formed knowledge and try and shove it into a kid's mind. And, and what we, what we, what we're actually trying to do is helping them to construct their own knowledge rather than taking fully formed knowledge that is not easily assimilated into their model. So this means that there's a big emphasis on collaboration, on working through problems, on figuring stuff out instead of just being told, we want them to work with other people. We want them to find information for themselves. We want them to find information from disparate resources and put it all together and kind of to steal some terms from machine learning to kind of generatively build their model from different types of information. The social constructivism. There are three, three kind of big things within social constructivism that I want to talk about. The zone of proximal development, the more knowledgeable other, and scaffolding. So these, these three big, big concepts are very important. So I'm actually going to do the zone of proximal development first is this kind of zone between what you can do and what you can't do. Specifically, the zone between what you can do and you can't do is what you can do, but with help, right? So if you can read the word cat, you can probably read the word tree with a little bit of help, but you can't read the word anti disestablishmentarianism. You just, it's just beyond your capability. If you can add two numbers together, you can probably add three numbers together with some help, or you can add two bigger numbers together with some help. But you can't do linear algebra. That's just outside of your, your zone of proximal development. So the goal for the curriculum designer or for the teacher is to find for each student the next step that they can take with assistance. So we do this in projects by finding a project that's at the upper end of their zpd, their zone approximal development and helping them break it down into steps that they can take with the guidance of their mentor or their ta. Which leads us to the more knowledgeable other. So the more knowledgeable other, the MKO is a, is a construct that can be a person, a book, a computer, a course, whatever. But the MKO's task is to bridge the gap between the students current skill level and the skill level that's needed to accomplish the task that they're trying to do. So you know, back in this social constructivism model we're building here, interaction and social development to build skills to build content knowledge. That is very cool. We kind of have four levels of, of more knowledgeable others during the program, kind of starting at their, their peer group. All the kids are in groups of. We group relevant projects together so their peers are available to collaborate, to offer assistance. We There is rarely a time where students aren't helping each other actively, which is very cool. Then we have teaching assistants who as we said, are graduates from our programs, successful graduates from our programs. So they are kind of near peer, not quite peers, but just above the group, which has many benefits that we don't need to go into. But these TAs are experts in Wolfram language and they get to help the students with individual tasks. Then we have our mentors. So each mentor has a group of three or four students with projects that are relevant to their area of expertise. So mentors are subject area experts and they're, they're great Wolfram language coders. And then we have the director team who are experts in, in project development, in research, in Wolfram language and in their own content area, although that's, that's less relevant because they don't necessarily do content mentoring. And the concept of the more knowledgeable other is that they are assisting with a task, but they're not just telling them the answer. So we need to be really careful with how much help we give students. So if they come to us and ask us a question and we tell them the Answer. I view that as a failure of, of our staff team. So what we do instead is that we help them find resources. We ask probing questions to try and figure out what the kind of question under the question that they're asking is. And we help them find resources to solve it by themselves or to solve it with somebody else. We is a firm rule in our staff training that we never tell a student the answer to a question. Academic, an academic question, we'll tell them where the bathroom is. But yeah, the kind of next, next big, big thing is scaffolding. So scaffolding is about building curricula around the slow withdrawal of support. So students learn from interaction with the teacher and from their peers and being given adequate support for a new skill. They also learn from not being given help. So what we want to do is set them up with tasks that are at the right spot in their zone of proximal development, that they need less and less help as we move through our curriculum. So we use scaffolding most during our Intro to Wolfram Language curriculum in the first few days, where we start out explicitly teaching a task and, and then students solve problems relevant to that task and, and relevant to the tasks that they learned earlier. And each task becomes increasingly complex. And then they move on to doing explorations, workshop things where they're given a larger task with fewer instructions. So let's just look at a couple of examples of that. So early on in the curriculum, we might talk for 10 minutes about how to define a function in Wolfram Language. And then we might say, okay, now that you've, now that you've been talked at for a couple of minutes on how to define a function. Now define a function that takes an image and blurs it. So we give them an explicit instruction and they go away for a few minutes. And then either by themselves or in a pair or in a group, depending on how they prefer to work, they solve this, this mini problem. They do this, this exercise, and then later, you know, a few, a few hours later, usually they'll be given an exploration. And explorations are things like this where they are told, build a classifier that takes an image and decides whether it looks closer to Harry Hermione or Ron from Harry Potter. So just create a classifier that looks at new images and guesses which category they belong to. And these are all the instructions that they're given. So use the classify function to sort images. This is how you import images, create a classifier, get images in different ways, build a mini interface and just do it. And they're given a kind of extended period of time to do that. We have about 15 of these, these kind of mini projects that they can, they can choose which ones they work through so they can practice different skills that they're more interested in. And so given this task with fewer instructions and then there's peers and TAs and mentors walking around to have questions asked of them. So this kind of scaffold of slow, slow withdrawal of support makes it, makes this a social constructed environment. The kind of next big thing is active learning. So during our class time, I really minimize teacher speaking time to no more than 10 minutes. I prefer to never speak for more than 10 minutes. And students are actively engaged while I'm talking. You know, they'll tell me what to write next in my code. They shout out answers, they ask questions. They are highly engaged throughout that, that experience. And then after each 5 to 10 minute teaching block, students work on a small set of problems that are designed to allow them to practice that skill that they were just taught in combination with skills that they were taught in previous little blocks of teaching. So it all builds, you know, this thing builds on this thing, builds on this thing. And it's all highly active where they are doing things, they're learning by, by doing, which is fantastic. And that is during our class sessions, during the project time, we get even more, we get even more intense with our kind of independent stuff. So many of you, I assume, have heard of project based learning. Research based learning is a slightly less common thing because it's slightly more difficult to manage. But we do research based learning designed around a project. So that's the, that's the vibe we're going for. So the computational essay, if you haven't seen a computational essay before, these are interactive kind of essay slash research papers where you have text and code inputs and outputs that are often interactive that people can use and they are, you know, publishable, shareable, all this kind of stuff. And students, as part of this summer program write a computational essay about their project where they explore the, the content, they explore the code that they've, they've written, this kind of stuff. So they do a project over that course of the summer and we, in the project selection process, we work very hard to ensure that this project is based on their interests, on their current skills, is within the, is, is at the high end of their zone of proximal development and that they, so, so that they will be capable of doing it, but it'll be really hard. We, we want projects to be Very difficult for them to do. And we usually will have a kind of minimum viable product. Like if they accomplish this goal, then great, they've, they've accomplished something meaningful. And then the project, which is the, the goal and then a kind of an extension. Like what happens if they, if they finish this project, then what should they do? We usually have some extension ideas for each project. And this takes, this takes a long time. We usually spend about around an hour choosing a project for each individual student, including. We spend a lot of time before the program coming up with potential projects, many of which don't end up happening because the kids have different interests to the ones that we predicted. So we spend a lot of time coming up with a project and matching this project. And the students are involved in that selection process. We don't just hand them a project and send them away. They hang out with us for a while to talk about what they're interested in. And then once we've kind of allocated them a project, we, we talk to them in detail about, you know, what they think, if they, if they want to change anything, all this kind of stuff. And then they're matched with a mentor who becomes their, their mentor through. Throughout the program, mentors have groups of three to five students, most often three. And all of those students are kind of have relevant projects to each other. So they get a nice little academic peer group to work with. So that's what we do. Let's talk about the kind of, the theory behind that. So a really, really big theory behind project and research based learning is authentic problem solving. So students, we need students to engage in real world problems. And so project based learning constructs a, what's called a driven paradigm to students where the, the problem drives them to the core concept of particular subjects. So instead of being like we, you are going to learn linguistics, go learn linguistics. Instead of doing that, which is pretty pointless, like you can't just learn linguistics. That doesn't mean anything. Instead of doing that, you have a project where you say, okay, you are going to create a interface that, that gives the, now I've forgotten the word. That gives the, the kind of ups and downs, the inflections of language for lines of poetry that comes up with the, you know, how the, how the poetry scales across the line and what the inflections are. And you're going to do that for iambic pentameter, you know, Shakespeare vibe poetry. And you're going to create a little interface that shows people where the inflections go. And in order to do that you're going to have to learn a bunch of linguistics and you're going to have to learn a bunch of computer science things and probably some history. And so the, the project acts as the, the driver towards the, these core concepts of a particular subject and gives it motivation that gives, makes it meaningful. Instead of, you know, reading a book about iambic pentameter, which sounds soul crushing to most people, you are figuring it out. So you're constructing this knowledge. You are working on something worthwhile. You're motivated by the, by the project to learn new stuff. Next important thing is autonomy. Students are responsible for what the project is, how they solve problems within it, how they write it up, how they communicate what they've learned to others. And teachers and the mentors and the TAs are acting kind of like coaches. So they're guiding students as they develop questions and problems. They're helping students to find and read through and sort and evaluate information. They're giving students the opportunity to draw their own conclusions. They're providing the time and the opportunity for students to, to communicate their results. They'll usually go through several iterations of their computational essay to, you know, and how, how they're going to communicate these results to the world. Students having autonomy over their, over their research project is really important because this means that they are learning, they're learning things about themselves. They're learning, you know, their own management skills. They're learning how to, you know, how, how to think critically about information that they're being given. You know, we, we don't want them to just be given an, an article or a blog post or a book that says the answer to their question and then they write it down in a different way, like a book report. We want them to be very active and autonomous in what their project ends up being, how it ends up working, what they want to display as their, their results at the end. So they are not only responsible for doing the project, they're also responsible for choosing what it means to do this project. You know, does, does this project mean lots of UX design or does it mean lots of written research, lots of reading, or does it mean lots of experimentation, computational experimentation, where you run sets of code a bunch of times and you're simulating something, you're collecting results and data like what, what does doing this project mean? So they are very autonomous during that, which is fantastic. We have a kind of the skill development, we're focusing on content and research skills. So we want them to identify and clarify issues, challenges, questions, puzzles, all this kind of stuff. We want them to find and process information. So we don't want to give them content, we want them to find it. We want to support them in researching and finding their information. We want them to think critically and creatively. Usually these projects have never been done in meaningful ways before. They're usually very unique projects. So we, we need them to be thinking critically and creatively. We want to apply their knowledge and ideas to draw conclusions that are often unique and, and fascinating. And we want them to learn how to communicate results. So there's a bunch of kind of skill development learning objectives surrounding doing research which many of these kids end up going further in the kind of academics which is fantastic. And our kind of final thing is that we want them to be active researchers, right? We want them to work through the entire research process in a really self regulated manner guided by their own research questions. And we want to take on a facilitating role role. So we want to be that, that more knowledgeable other facilitating them but not telling them what to do. We want, you know, we want them to push the, the limits of their, of their zone approximal development further and further and further. We want to facilitate this development of a kind of researcher's mindset, the ability to objectively examine data or a situation and find enjoyment in solving problems. We want for them to gain in, in what we call effective motivational gains which means higher tolerance for obstacles, desire to learn, increased disposition towards working with ambiguity, higher research self efficacy. You know, wanting them to think that they are good researchers, that they are, that they have skills. And so by, by doing active research projects they kind of in, instead of, you know, being involved in someone else's research project like they might be, you know, as, as an undergraduate they might join a lab and you know, be under, you know, six layers of, of research administration where they, they do a task towards some, some professors research project which of course can be a very valuable experience. But instead of doing that, we want them to be really active researchers involved in this project with, they are the, they are the, the leader which is very important. And then after the summer in the emerging leaders program where they choose their own project and they collaborate in a research group doing a project together that introduces even more skills that will be helpful in doing research in the future if they choose to do that, but also helpful if they have nothing to do with academia ever again where they get to learn all kinds of independent independent thinking skills. The big question though is, you know, does, does this work? Is it, is it valuable? Hopefully so my, my kind of evidence for it working are that kids keep coming back. We have dozens of returning students and many of our alumni return as teaching assistants or mentors or in our further education programs. 98% of WHELP alumni last year said that it was their most influential educational experience. So I was, I was pretty proud of that statistic. Every single student since I started running the program in 2019 has published a computational essay. And I think what's most important is it is just amazing to see the impact of these programs on the students skills and their confidence level. So we had a couple of stories. A kid who was terrified at the beginning to even run a line of code in case it was incorrect and he just couldn't, he would type code and, and he couldn't, he couldn't run it. He was like, I, I can't, I can't do shift, return and execute this line of code because what if it is wrong? And I'm like, we're going to get over that, I promise. And by the end I, I heard him shout from the other room and he goes, that's so cool. And I walked in, I walked in and he had found a bug in his, in his large piece of code that was screwing the whole thing up. And he'd spent the whole morning trying to figure out where this bug was by individually running the micro pieces of code. And he found this, this, this bug and he fixed it. And he was just, just screaming in, in how delighted that he was for to have done this, which was fantastic. You know, I have a kid this year who asked her mentor literally every 45 to 60 seconds what to do next. You know, she, she wanted micro tasks so that, so that she could know what to do. So we came up with a plan for getting stuck. You know, what, what do we do when we're stuck? What do we do when we don't know what to do? And so we, we came up with a plan with her and by the end she was working independently and asking for help when she needed it. She had, you know, resources, she had her checklist, things to go through before she, you know, asks a grown up for help. Young. We look in the documentation, we Google it, we sit with it for a second and try and figure it out. And you know, she, she knew what to do. It was fantastic. I have a few young adults this, this year who attended the program when they were teenagers and work incredibly diligently to help their, their students now that they are mentors to explore, help their students to explore their topics and to have fun. And it is remarkable watching how, how much they've, they've grown since they were teenagers. Most importantly, just in general, we have all of these kids who are now able to keep failing and struggling and then picking themselves back up and keeping going with the, with the stuff that they, that they're doing. They're able to just to really grow within their, their content and their skills. And I think that this kind of program where we have them be active learners, active researchers, instead of trying to, instead of focusing on us being good teachers, we focus on them being good learners. I think that's really helpful in developing these kind of skill sets. I've, I've talked slightly longer than I wanted to talk, so before I open up for questions. If you or your kid is in middle school or high school or undergrad, then they should attend one of our programs. It's really fantastic. If you are a Wolfram language expert or a subject expert and you're interested in being a mentor or in being a guest speaker. So if you're not into Wolfram language particularly, but you are really knowledgeable about something, then we have a guest speaker program and they come and give, give talks. But you should email camp adminolfram.com to tell us that you're interested in becoming a mentor or doing a guest talk. If you are interested in running a program in your, to teach computational thinking or Wolfram language with our curricula, email adventures-staffolfram.com and we'll help you kind of get set up. If you're interested in discussing the pedagogy and design decisions in more detail, you can chat with me or you can email me@roryfwolfram.com and I will, I will get back to you and we can have a, we can have a chat. Okay, so in our last, in our last two minutes, I want to open up to questions. I have one question. Um, in the chat. Yeah. Gift gifted students have. So, so I did my, my master's thesis on gifted and twice exceptional students and they, yes, often receive a bunch of approval and told that they're amazing and fantastic when they do things that they find easy just because those things are not easy to other people. So they get lots of, you know, wow, you're so amazing, you're so smart and that's great. But we being, being exceptionally praised for things that you are finding easy doesn't, doesn't help you to grow. You know, we, yeah. So we, we want to make sure that we are helping them not give up. Okay, so scrolling through this, Yes, there are lots of other kind of theories and pedagogies that we do incorporate. Though the white box, black box thing, we, we do do pairing students with professors in academia. We, we definitely do do. In the Wolf Emerging Leaders program, we, we do facilitate, um, them working with, with professors. Um. Yeah. Yeah. Okay. Um, right. If, if you have any other questions, feel free to email me rory fwolfram.com or get in touch in any of these other ways if you're interested in those things. I hope that this was educational, informative, and I am looking forward to connecting with anybody in the future. Thank you very. Much.