Social Engagement in Online Learning

Welcome to my talk, Social engagement and online learning. So my name is Clinton Bradford. I work with a company, Silva, based out of Switzerland, which is using Wolfram Technologies to build a next generation online learning platform. So I have a quick video just showcasing some of the things that we've been able to do using these Wolfram demonstrations, these Wolfram language assessments with our editor in order to create more dynamic questions and more dynamic answers than most platforms are able to. So we can have question cards, we can write in assets in Wolfram language, we can even pull in data from Wolfram Alpha to do the computations, to do the answers. But this isn't what I'm planning on talking about today. The new thing that we're going to do is we're going to going to do social engagement. So in many in person classrooms, when teaching a lot of material, it's really valuable to talk to other other people, to work in groups and to do group based assessments. And that's somewhere where online tools have really lagged behind. So here we've got, well, here we've got in some embedded Wolfram interactables being graded given dynamic feedback using Wolfram language. Anyway, All right, so we've got interaction, collaboration and dynamic social content where people can compete with each other, where people can interact, where people can be ranked, see how other people are doing and have their content shown to other people at scale. So being able to take large courses, online courses and asynchronous courses and run these experiences is something pretty new on Silva and something we're able to do with our platform and with Wolfram language. So here's an example of an in classroom run of this in an asynchronous class being run in Switzerland where individuals are participating on their laptop and they're being shown a dynamic leaderboard of who's what, groups, which teams are leading in the game. And if we want to see what it looks like from the player's perspective, well, we have this individual ant. So this is an example of a game theory exercise. It's really common to do in Classroom Activity 4. We're going to pair up people, we're going to have them run through the prisoner's dilemma with each other. So a couple rounds of that classic problem which is described here, where you can choose with another player whether to cooperate or whether to betray. You make these choices in secret. And if you choose to cooperate, both of you, in the classic example, cooperate against a state doing a criminal investigation. But you could also choose to frame this question in other ways. You can choose to cooperate and each get a one year sentence or you can choose to tavel on the other person and they'll take the entire fall. They'll get five years and you'll get zero. But if you both tell on each other, then you'll both get the harsher sentence of four years. So this is a classic green theory problem because it reveals that the best solution for everyone combined that cooperate solution can still be worse in each individual decision. But just saying that students doesn't work. So we're going to illustrate it by having them engage in this game in several rounds with each other. So we're going to start out by setting up this game room. We're going to have a bunch of students join. So you can see we've got our main student here and we can have four different students in this example who are going to join the game. Then they're going to be paired up and then they can start going through the exercise individually. So we have our two students here in this game. You can see the student and their opponent. And we can see this generated interface built within our editor, which I'll talk about in a bit for running a game. So they're going to choose to, this player is going to choose to cooperate at first we're going to see that they were betrayed. So they got five years and then we're going to run through another round. So now they're working with that information. Okay, the other person chose to betray me on that first round and so maybe they'll choose to betray on the second round and we'll see oh, betrayal in both. So we can run these experiences and as we saw, we can have live updating data showing what kinds of dynamics people are, what kind of choices people are making. We can do analysis after the fact. We can run this in sort of an AB testing system to check. Okay, how does the wording of the question, if we frame it as positive or negative points, does that make betrayal more likely? We can use all sorts of and variance analysis by running these games at scale. And we can do that by building out in the editor this drag and drop editor, these individualized experiences. So out of these grid table graph components we can link them together using Wolfram language and using front end Javascripts and just quick WYSIWYG styled front end editor. We can create these games, these interactive experiences, group people into opponents or teams and really create these dynamic experiences for education. All right, but then this is only a live example of what the kind of thing that we can do with this. We also want to run these game theoretic experiments in other ways so we can do asynchronous competitions as well. So for example, a classic kind of code question that you might have seen in a programming course is to take a game like this, write a little, a little artificial intelligence. And so we can use our coding system, we can use our Wolfram based assessment system to create a assessment or to create a quiz for the user to fill in. All right, to write a program themselves in Wolfram language in a restricted version of the language, so whitelisted based on safe functions for user input in an online assessment. We can have them write their own little script and we can have them post it. So for example, we could have them do a multiple round prisoners dilemma simulation and we could run through their code, passing them these arrays of past values and have them write a little program like function rounds Betray. There the program lets you, Which would be the example of the Betray solution, which chooses Betray every time. And we can have them automatically analyze these by processing it as Wolfram language code, by doing some code inspection to get results. So for example, a classic solution to this problem is the copycat solution where each round you do something the same thing as the opponent did the previous round. So we can write that out as a, the participant could write that out in Wolfram language as this function rounds and then last at last of the rounds, which says, you know, choose my opponent's answer in the last round. And we can then process this in our editor. So we can say, all right, we can run it through Wolfram language function. We can say, all right, your submission, we can check whether it throws an error. We could inspect it to see that it, that it's a function, what variables it accesses, we can check its return type, all of that standard code analysis stuff. We can also do, you know, custom code analysis. So we could do a five round matchup versus a particular strategy, say, and we can see how many points they get using these functions to aggregate it. We could also do a five round matchup versus say the steadfast trader. We can do these, these experiences and we can show the student the responses using our platform. And so now we've collected from these students these answers and we've successfully, we can successfully validate, okay, this is a solution to the problem. The next big thing that we can do is we can take these and build out of them a unified solution in one Wolfram language which takes all of these student dancers, collects them together and then runs A little competition on them. So we can take this prisoner's dilemma and make it into a prisoner's dilemma tournament using different evaluations to evaluate it. So here we can gather, for example, all of the student dancers and we can run them against each other. So we can create in app in the website, we can create this custom report that we can use to gather up the student answers as functions as Wolfram language functions objects, and we can process them together to create a unified game experience or a unified sort of result. We can create this, we can rank it so we can sort these lists by their total sentence. We can do any processing you want to do in Wolfram language on this and then turn that back into individual feedback for the students, take that back and show it to the students. So, So we can do this using a couple different models for the creation of the games. So the games themselves have a structure with rounds and they've got a structure with dynamic content in those rounds where information is piped between the various cards and the various questions. And we can model that information in terms of a graph based structure for constructing these. So as each person goes through the assessment, we want them to be able to move at their own pace, as we saw in the video. And to do that we need to know whether there are available, whether the information they need is available in the next card. We can do that by, and we can do that by using a graph based model for information. So each user action can generate sort of a node on a graph. So it can generate, so choosing betray, or in this case, in this example of a positive prisoner's dilemma situation, choosing to split the potential or to attempt to steal it from the other player. Each user input can generate a, each user input can generate a event that we can listen to and that we can make decisions based on and we can create a graph of these decisions. So, for example, we can say, okay, their answer. We need an answer from both players in order to evaluate the first question. So we can say, okay, we've processed the answer from both participants, now we can evaluate the first question. So now the results are available. And these computed results that are available we can then pass to the advance to round two so that we can say, all right, they've got round one results, so now they're allowed to advance to round two. And then from round two they can give answer to. We can create this tree based or this directed acyclic graph based model for the question that allows us to offer the instructor a pretty, a pretty wide range of Freedom in constructing derivative content and constructing content that. Constructing content that works with a complicated flow while restricting the user's ability to advance. And we can forbid the instructor from consuming content that depends on itself using this model. So we could, for example, prevent them from trying to use a variable from round two in answer one. Then the next thing I'd like to show is another. There's another game run in this, another classic game theory game, the contribution game. So the way that this, the way that this game works is each round there will be a individual amount of money, in this case in the framing. So we're going to give each player, say, $100 to start, and they can choose how much of that money to put into a common pot. The money in the common pot will be doubled and then distributed evenly among the remaining players. And by running this for several rounds, we can get that same kind of game theoretic cooperation versus betrayal information out of these by. You can get that same sort of cooperation versus betrayal information out of these by seeing who chooses to contribute nothing while receiving an equal share or who chooses to contribute the most. And again, we get that same sort of trade off between players being able to choose. So here we have a slider component where each player is choosing how many of that those first hundred francs to contribute. Here we have the second player contributing, the third player contributing roughly even amounts for the first game. We can see they've all received results and they're all able to share seeds for a second round. Now they're contributing from a larger pool. We can see these live updating results. Oh well, this person contributed this much. And we can see who's benefiting and who's losing round to round. And we're able to do a live leaderboard for this for the whole class or for an individual group. We're able to do live analysis of how many people are contributing and what, as well as the delayed analysis for these complicated. For these composed game experiences. So here we can see in the last round, various players are choosing to contribute relatively little and they end up with a fairly even result. Sorry, in the fourth round. Now here in the fifth round, our second player is contributing only a little bit, our third player is contributing only a little bit. They're holding on to more as they proceed. And we can see that represented in live information for the instructor. We can see a complete aggregate score. So we also have a system whereby you can run individual polls where you can take in information from the students this way and provide live analysis on that information into Various graphs or aggregates. So we can take individuals, have them participate, have them type questions, submit information. So we're creating these online experiences in the hopes of demonstrating a sort of new way to process the online education experience to make it less isolated, to make it more direct feedback and to hopefully provide a lot of the benefits that people were getting from in classroom activities to students who are perhaps doing the course in around the world in their homes, unable to produce. That was the content I've prepared. Thank you for coming to the talk. Have any questions? Yes. Yeah. So we have a lot of interactive components using Wolfram language. I'm going to go back to this and pick out some. So if I go here. I can just. I could pull out some examples on the site in a bit. But here's a example question which is using the Wolfram knowledge base to, to ask a geography question, to ask people to pinpoint a city on a map. And this is done as a Wolfram interact running through a Wolfram running through a Wolfram server sort of passing in, you know, as, as these interacts do. This is for what. It's not an embedded notebook, I believe. It's not a feature I made. It was made by our Wolfram specialist, but it's a, it's an embedded dynamic. And then we have, you know, these other dynamics like slide, you know, classic slider dynamics as we've seen and more of these point selection dynamics. We have a lot of this interactive content done using this Dynamics. But for the live games we're mainly focusing on content made with the interactive editor that's a little easier to control the flow of and the live information on than the dynamics. Yeah, We have a Wolfram EPC and a Wolfram language. A W. Wolfram language server. Yeah, application server, I think we have. Yeah, yeah, I believe so. So for the, for the interactive games, currently we're not using Wolfram language in the. That front end sort of moment to moment computation. The Wolfram language is being used in the analysis portion so that the. Yeah, So we have, so we have the data that's being generated from these games that you can. Or that we're working on creating these analytics and feedback for. And using this we can create, we can parse that information through Wolfram language to allow us to leverage like the statistical analysis and the data processing tools Wolfram provides to get those insights about comparing the different runs of the different game with different wording with a positive or negative connotation. We're using Wolfram language to do that processing mainly and that's its role in the, that's its role in the, in the live games, not in the actual front end. Right now. We are planning on sort of moving towards a more general model for computation for that and for some other things. So in a lot of the areas of our application, we have Wolfram language as the, you know, it's the first class language for our app. It's the one that you're going to be presented with when writing custom analytics or it's the standard notation for entering a lot of the answers. But we are planning on moving more things to being able to be done in a variety of languages and as part of that, moving the game computations to be done in Wolfram language and smaller compiled snippets. But that's a future feature. Thank you for coming and seeing what we've been working on. I am Clinton Bradford. If you've got any more questions, you can reach out to the contacts at Silva AC or to me by email at clintonilva ac. And I'd love to talk about math education, about STEM outreach, and about Silva. It.