Teaching Numerical Methods with IPython Notebooks 3 | SciPy 2014 | Aron Ahmadia, David Ketcheson

So as I mentioned before we started the break, we're going to talk a little bit about Software Carpentry's approach to teaching with the IPython Notebook and actually teaching hands-on labs in general. So first, a little bit about what Software Carpentry is. And I'm actually not going to take you to the main landing page of Software Carpentry. I'm actually going to take you to the operations, which is the second tab here. And this includes a checklist of how a software carpentry workshop is run. And it turns out that there are always many pieces in making a hands-on laboratory work. It's not like a lecture where you can just show up with your computer and plug into a presentation system and go. And it turns out that even that simple operation with an instructor and a lecture and a laptop is a very challenging thing. As anybody who's come with maybe a Mac 5 years ago or a Linux laptop, just plugging into a display can be a half-hour ordeal.

Right.

So now, instead of just working with a single display, here as an instructor, the person is going to come in. They're probably going to be teaching with another instructor. And there are going to be as many as 20 or 30 or sometimes 40 or more learners that are all bringing in their laptop. And then we're going to synchronize on our course material. And we're all going to execute through this regardless of what you brought in at the beginning of the day, whether you brought in a Windows machine or a Mac, FreeBSD, Linux. Whatever it is, whatever thing you have, we're going to make it work. And we're all going to get along. And we're going to be executing code together. So if you asked me if you could do this 5 years ago, I would have told you you were crazy.

Yeah.

'Cause actually, almost nobody was doing this 5 years ago. It was, it was really, really hard. And virtual machines were around, and people would sometimes distribute a virtual machine, and then everybody would figure out how to get that virtual machine going, and it was still painful. Now, there are a million technologies, and there are technologies that are coming in from the web development side. So some of you have heard of tools like Docker and Vagrant, and these are actually very common tools for system administration, and they're coming in and they're creeping in into web development. I would encourage you to go see— Dav Clark, I believe, is going to give a presentation on what they're doing at University of Berkeley— University of California, Berkeley— later this week. But on our side, from Software Carpentry, we don't believe in virtualization. We don't believe in virtual machines. There are a couple of instructors who have used them, but the overwhelming opinion is that you should bring students something that they can use for their research, because Software Carpentry is there to help students students get better at their research. It's not about learning material, though we sometimes do that. We sometimes will teach a specific set of research material or course material. But it's actually about enabling you to do computing better for your research. So at the end of the day, we want to give you a tool that helps you become more effective. So this is a little bit different from a classroom, right? A classroom is deliberately a more controlled environment. And because of that controlled environment, sometimes you can go further in a specific direction. Software Carpentry tries to be a broad impact. And so part of that is whatever students install on their computers, they can take away with them. So that's one of the first ways that Software Carpentry differs from a typical classroom, is that we actually expect students to work on their own laptops in the platform that they'll be using later on. Yes? Can you tell people where that page is?

Ah, yeah.

From the front page, it's the second link. It's the Operations link.

That's the Bitly page.

Yeah, the Bitly page, teach_ipython.

teach_ipython, go to this, and then click on operations.

Yeah.

This is also— it's the operations link for boot camps in the Software Carpentry main page. OK. So because of this complexity, we don't just have a single instructor who is responsible for everything. First of all, because Software Carpentry is a little bit like a traveling show. Frequently the instructors are coming from somewhere else. There will be a host who's responsible for managing logistics on the ground for those instructors sort of before they arrive and during the boot camp day. So coincidentally, David hosted me for a Software Carpentry workshop in Saudi Arabia last year. So he acted as the host. He took care of all the initial details, and then I showed up and taught.

Yeah.

For 2 days. The instructor is, of course, responsible for the material. Usually there's a tag team of instructors. Sometimes there's more than 2. The reason that you have more than one person in the classroom at any given time should be obvious based on how things have gone today. Right? So somebody gets up in front of the room and they start talking. And of course, they have no idea what's going on in front of the 40 screens in front of them. So the second instructor is walking around. They're looking for people with sticky notes stuck on their laptop. And they're answering troubleshooting questions. And this is essential. In fact, it's— we usually have more than just 2 instructors. We have a small team of helpers. And these are locally recruited volunteers. Maybe they're graduate students. Maybe they're people who are just excited about Python or computing in general. Sometimes they're just people from the community. They're not necessarily even students at the university. So a sort of famous example for me is in January when we were teaching a Software Carpentry workshop at MIT, Jessica McKellar, who's the, I believe, current president of the Python Software Foundation and is a local Cambridge resident, is super excited, of course, about teaching Python at MIT, which is where she's an alumni, came and helped out. And she mostly just stuck around for the introductory section that she was available to come, but that was OK because we could just sort of have somebody float in, help out, and then come out. Again, not necessarily— yes, Eric? Oh, go finish your thought. OK, I was going to say, this isn't necessarily something that easily applies in the classroom. But the takeaway here is that if you are going to be doing a hands-on lab and you would like to move quickly and cover a lot of material, and you have people moving from a lot of different laptops or operating systems or maybe necessarily software packages, you will want to have people in the classroom who can help them. And they may be other students. It may be graduate students or TAs, but it's different from your common lecture model where there's only one person in the room who is sort of responsible for giving out content. Yeah, go ahead, Eric. Eric is a Software Carpentry instructor, by the way. I can't emphasize enough the importance of the helpers, and I've taught several Software Carpentry workshops where the helpers were kind of an afterthought. It was like, oh, we need to get some helpers, and maybe they would show up on the day The day of, maybe not, and I wouldn't know who any of them are or anything like that. I like— I've found that like the best workshops have been ones where we've gotten at least 2 or 3 helpers well in advance and who are part of the planning process all along so that they know what's going on too and can be like fully involved in teaching and helping people set up. It's for me like those are the ones that go the best, because otherwise I'm trying to teach and run around and fix 40 laptops at the same time. It's impossible.

Yeah.

So I've definitely done Software Carpentry without having any helpers, and it was much, much more painful. Even having a second instructor makes a big difference. So that was the operations guide. So that's some of the— almost the infrastructure that goes in. So there's other advice in there, and it's good general advice about How a classroom should be laid out. If you're going to teach something like this as an informal course, you have to make sure that you have a good wireless internet connection. Yeah. But even things like power access, making sure that the room is accessible, all of these things. And of course, the sticky notes. So what are the sticky notes for, and how are they supposed to be used? I kind of alluded to this earlier. Usually we hand out 2 colors of sticky notes. It used to be that you could actually get green and red. Now it's like a neon yellow-green and a neon pink, but they sort of mean the same thing. And it's a great way to find out where your audience is. And this is actually independent of having computers, but it really closely ties in with the computers. And so I'll say something like, put your sticky notes up if you're following along with me right now and paying attention and not on Facebook. And I mean it. Go ahead and put your sticky notes up. And I'll actually know how many people are actually on Facebook, and how many people are checking email, and how many people are with the lecture. And then the people who are on Facebook will see that everybody else is actually doing something. And then they'll realize that they need to maybe pay attention, because maybe something's going on that they're going to miss. But it's a good way for 2 reasons. So the first thing is the sticky note requires you to do something. You cannot be passive. When I ask you to put a sticky note up. So for example, if I say, hey, if you have a problem, put a sticky note up, that actually isn't sometimes enough. Because sometimes people will sit in front of a problem and try to fix it themselves or be stuck and not want to raise their hands. So the other way to do it, of course, is say, put your green sticky note up if you've gotten this far.

Right.

And that's easy when there's a green sticky note for I'm doing OK and a red sticky note when I'm in trouble. It's hard when you're trying to dual-purpose the sticky notes, which we did today, and as you can see, is with mixed results. OK, you can take your sticky notes down. Some OCD factor in me is freaking out at the sight of all of these flags. OK, it's like having 70 issues on the issue tracker that have been unaddressed. Some more general advice. Now I'm on the first link. Some of this is directly applicable. Some of it isn't. Making time to— this one is, making time to assume that there will be installation problems when you are running software is so true. And the ways to mitigate this are by having a very regimented process and only following that process and not breaking away from it. So if you go to a Software Carpentry workshop, the instructors will be using an installation procedure that they have used several times before. And if you are lucky, you will also have instructors who have debugged problems on all 3 operating systems. And so some of this is experience, and some of it is just doing it the same way each time. Interacting with the audience, pausing, asking questions, making them write code on their own computer— all of these things are part of sort of the mantra. This is actually— would sort of fail the Software Carpentry sniff test, because the amount of code that we've actually had you run and force you to write on your own has been very relatively little. A typical Software Carpentry workshop would have a lot more hands-on. But this course is not necessarily about the hands-on side of it. It's about how to create the hands-on atmosphere. So, you know, David's idea, which was really awesome, was that we would make you write a course lecture. In this. And if we had a day, we would have definitely done it. Because we had this constrained amount of time, we just wanted to expose you to some of the things that were available. But if you want to extend this another 4 hours, I'll sit here and evaluate your course material. And David will too. I'm volunteering him.

OK.

So this last little piece before David takes over for styling is probably the most useful thing that you're going to find here. And it's the troubleshooting link. I'm gonna come back to it in a sec. The troubleshooting link here, which is a Wikipedia page compiled by the Software Carpentry instructors of all of the common things that you will hit if you were trying to install Bash, Python, or Git on a computer. Right? All of the common problems. They're documented here and solutions are here. So, and if you follow the basic Software Carpentry installation instructions, which are not here. I'll put a link up to them. They're in GitHub. I'm not even gonna be able to recite the link off the top of my head, but there's a set of common default instructions we use for R, Python, and the other tools that are sort of commonly used in a Software Carpentry workshop. So this is handy, and it's a wiki, so you can also edit yourself if you come up on something new that isn't covered here. Okay, sorry, there was a question back here and I stepped past it.

The room arrangement.

So what is your room setup? Do you work in lecture room setup like we're here?

Or do you do more of the hybrid? The ideal room setup is actually something that looks a lot like this. This is almost a perfect room. Here is what I would change besides the fact that you've got this nice set of stadium-style seating where everybody gets to plug into their laptop and is relatively close. I would improve the wireless. I would only have one of these screens up, probably. Or I would push them out wider and I would have a chalkboard or a whiteboard, because it's really important to be able to write to show examples. But this is a great room, actually. You don't often get to teach in good teaching environments. It's— I mean, it's dark. There's no glass windows overlooking a beach, but we can't all teach software carpentry workshops in Hawaii. Yeah?

So just another take from our experience.

Yeah, if you're teaching, if you're traveling for a hands-on, right, or, you know, you don't know anything about the room that you're gonna be in, make sure that there's gonna be a sufficient power, you know, spot so people can work. Okay, I'm gonna turn this over to David, and we can talk more about software carpentry after the panel discussion.

Thanks, Arden. Let me just go back to the— Oh, I guess here. So I just want to mention I left a link here in the examples section to a full course. So I said that most of my experience teaching with IPython Notebooks is with this semester-length course on numerical methods, but I also taught a couple of months ago for the first time a 1-day course. It was basically 7 hours To both some university students and also some industrial engineers from Audi, actually, a course on hyperbolic— solving hyperbolic PDEs. So you can see here there are just 6 notebooks. And the Lesson 0 here is similar to what I would give at the beginning of my university course, although it's even more abbreviated because we just took 1 hour to teach them Python. And these are just links to the notebooks on nbviewer, or if you click here then you get the GitHub repository with all of them. And then I also wanted to advertise— well, of course everyone will be at Lorena's talk tomorrow, but you know it's very much in the same vein as this tutorial, and I'm also giving a talk tomorrow on Teaching with IPython Notebooks. Actually, I submitted the talk before I realized— before I knew I was going to do a tutorial. But also there's a proceedings paper that I wrote that goes with that, that covers many things that we haven't had time to talk about here that are more sort of philosophical. All right, so let's go to styling notebooks. from the notebook server here. It should be the last one unless you've created others, unless you have an untitled notebook like I do there. Okay, so adding style to the notebook. I should say before I go through this that the CSS file that I'm going to show you and that I've used in these notebooks is basically, you know, composed of stolen bits and pieces from all over the web, everybody else's IPython CSS files, especially Lorena Barba's Arrow Python CSS. Okay, so you can make your notebooks look a little bit different from everyone else's, which is kind of nice, and maybe make them more useful also by modifying how they look and also modifying a little bit the way they behave using CSS. The appearance of the notebook. So the notebook is composed of HTML, and, you know, the modern way to change how HTML pages look is using Cascading Style Sheets, or CSS. And I am not a CSS expert by any means. I've struggled through using CSS several times in my life, but— Okay, I may have to— redirect questions, but feel free to ask. So the way we've been applying— oh yeah, so there's one way to apply a certain style to all notebooks that you open on your computer, right? And that's by putting a file called custom.css in your IPython profile directory. I'm not going to show you how to do that. If you Google it, you can find it, or, you know, probably a lot of people at this conference can tell you how to do it. You just put one file there, and then every notebook that you open on that machine looks that way. It's styled that way. You can also, you can have multiple IPython profiles on your machine, and when you launch the notebook you can tell it, I want to use this profile, so the notebooks will look that way. But the problem that I usually want to solve is not to make the notebooks look a certain way on my machine, but to make them look a certain way on my students' machines, right? I want, or on whoever is using them. It's nice to, just like publishing a book, right? You want it to be formatted a certain way. If you If you're publishing a notebook, you say, I want it to look like this, right? Maybe just for stylistic reasons, maybe for, you know, important practical reasons. So the way to do that right now is to have a box like this one of code that imports HTML from IPython's display module, right, the HTML command, and then opens whatever file you want and injects that as HTML into the page, and that's what this is doing. So if you run this box, then as you know, you get the reformatting of the whole page.

So yeah. It should actually be ipython.display. Core display will work, but core is kind of an implementation detail.

The official API is kind of high-level.

All right, thanks.

Sorry, we just caught that.

OK, good to know. Now I have to go edit all my notebooks. Okay, so I have faith that one day soon we'll be able to inject this kind of stuff without having a box of code that you have to run, but maybe that's impossible because of security. Anyway, we'll see. There's a link here to a proposal by Aaron and I to make this less intrusive.

Yeah?

Can the CSS be a remote file?

I believe so. It could— I could point to a URL there. Yeah, it doesn't matter.

I mean, you'll have to fetch it, but you could use urllib and fetch it.

OK. Oh, I can't just— yeah, OK.

Not necessarily the HTML, but just like CSS href equals—

Yes. If you inject the href—

href equals a single HTML tag with the URL.

OK, right. Yeah, yeah, OK. So maybe that's a better way to do it, in fact, right?

Because you can tweak it.

In one copy forever. Yeah. I don't have to have a—

Even in the GitHub repository, it's always— you can tweak it.

Yeah.

OK.

So that might be a more useful practice. After the talk, we'll implement that. Yep. Sounds good. OK. So let's— I want to just walk you through this CSS file, not because this is the greatest CSS file ever, but because it's one that I understand. All right? So I'm going to use %cat here. This is the shell command cat. To list the contents of the file. This actually isn't that great. There's no syntax highlighting. It's really long. Oh, it should have opened a scrollable— I wonder why it's not. I thought it would open a little scrollable window because this is really long. But I can get a kind of nicer view of it using %load. So what that does is create a new cell where it's actually put all that content into here. And okay, it's actually syntax highlighting it as if it were Python code. But fortunately, CSS and Python are similar enough that it's kind of useful. All right. So, but what I'm going to do— oh, I can actually run this code now, right? Oh, here's what I need to do. Aaron taught me this earlier. If I put %%html there and I run it, then it's just like having loaded it from a file. I'm doing exactly the same thing, right? Oh, I see. And so in fact, I could go and change different things. So I could say, I don't know, I want my H1, instead of Philosopher, I want it to be Arial. What? Comic Sans. Comic Sans. All right.

But you have to have that available, though.

Yeah, let's see. Hmm, it didn't find that, so it's just using whatever default it has. But you can see that it did change the look of the notebook.

Okay.

Right, okay. I'm not going to— okay, so now I want to walk you through each piece of the CSS file just in case, well, just so you understand at least one example of a CSS file. And at the bottom of this notebook there are links to a few other people's IPython notebook CSS files. All right, so the first part of it is actually pure HTML. It's kind of hard to read here because, because it's wrapped, these long lines, but all it's doing is loading a bunch of Google Fonts, right? So these are all links to font.googleapis.com because I'm using a bunch of Google Fonts. And so in fact, if you run these notebooks without an internet connection, then you'll just get whatever the default fonts are instead because it needs to go get them from Google. But the nice thing about Google Fonts is that you can fetch them anytime you have an internet connection. And they have a lot of nice fonts.

All right.

And then— so that part is just HTML. And then there's this style tag that says, OK, now we're doing CSS. We're done. This is not HTML. This is CSS. And each CSS— I don't know what the right term is— but section of the CSS file has sort of a label here which says, What I'm— the styling I'm about to tell you about applies to this part of your document. Okay? And then it has a bunch of attributes, one or more attributes to apply to that part of the document. Right? So what this does is apply the Computer Modern font face. Actually, I'm not sure to which part of the document this one does. Okay, let's go on to the next one because I know what this does. So the notebook panel is this region in the middle, right, from here to here.

Okay.

And here I'm setting— oh, actually no. Yeah, so here I'm setting actually the background outside of that, right? This #888, that's gray, right? That's hexadecimal for gray. And then Aaron, do you know what this color does? Why I have background and color here? I guess I could— I can try changing one of them real fast to just see.

I will freely admit that my CSS looks roughly like a monkey at a keyboard.

OK. I'm not actually sure what that second one does. OK, sometimes you set these things and you forget.

Color changes just the text color.

Yeah, this one I— OK. Oh, oh.

If the background was some particular color, you would want to change the text to a different color that has good contrast with the background.

OK, so I think if I change this, it will change that gray out on the edges. Yes, so I've just changed it to black by changing— the background there. Okay, this isn't so important. Let's move on to the more essential parts of the style file.

I think that's the color inside of whatever your element is.

Right. This background tag is— oh, and this must be for text that's out there, but there's no text out there, so probably this line is superfluous. Okay, this says that cells should be 800 pixels wide. Okay, this is just because if you A lot of people will be on their laptop with a really wide screen and they'll maximize the browser, and then if you have long paragraphs of text they get really hard to read. Or if you open it on nbviewer, right, it'll be this really wide thing and when you get to the end of one line it's really hard to tell which line is the next one. So I've limited it here to 800 pixels per cell. So that's the distance from here to here. And then the notebook itself, right? I've told it, okay, you can only be 1,000 pixels wide for similar reasons. And this margin: auto makes sure that it's centered. This says that the notebook background should be white. And this padding: left just makes sure that, that this cell isn't flush against the edge there, because that's kind of ugly. That's just aesthetic. Feel free to ask questions as we go along. For me, Markdown lists by default are a little bit too compressed. This adds a little bit more space between bullet points in Markdown lists. Okay. Now we get to really interesting things. So this one, which I stole out of somebody's CSS file— I can't remember whose— I never would have figured this out by myself, but This is how you refer to a running cell in the notebook. And this says put a 3-pixel-wide solid almost black line border around running cells. And that's that nice really visual indicator that this cell hasn't finished executing rather than just the asterisk. And I would love to see this actually become default in the notebook, Fernando. He's not listening.

Don't go there. All right.

Customizable.

Yeah, yeah.

I think it would be great to have a more noticeable indication that this cell hasn't finished running.

You could use CSS3 animations to make it kind of pulsate while it's running.

Yeah. I don't want to have seizures. I just want to know. Anyway. Animated GIF.

We've agonized over these things.

For hours and hours.

The fear is making something that flashes, that flashes too much and becomes distracting.

Yeah.

We're always open to continuing to fine-tune this. Our approach with a lot of these things is actually to be ultra-minimalistic and potentially insufficient rather than to overdo it. And so we have a really very plain, very vanilla base layer and let people Yeah.

So, so this isn't.

Making this kind of easier to package this aspect of it, which actually is somewhat gets technical because we're talking about control profiles is managing other things that you want you want to separate those. So it really is about theming. There's been some discussion. We can talk about that at the sprints. I mean, remember, we're going to have two days of sprints about Python at the end of the conference. We're happy to. And part of the point of coming here is for us to get feedback. We're not claiming that we have all the answers. What we do try to do is be fairly slow moving so that the base layer is solid.

Yeah, that's good. OK, I won't criticize that. OK, so if you run this cell, right, it keeps that black border for 5 seconds.

Another important point, we plan on refactoring all of our CSS.

Okay, that makes me sad, but all right. Yeah, it will be.

It will break everything with the goal of making it easier in the future. This is what you get for being early adopters of customizing something that is poorly designed for customization.

All right, well, I think it's great that we can customize it, so I'm really happy about that.

Consider yourself a martyr for the cause.

Yeah. Now it's kind of fun playing around with the styling. So what does this one do? This is what puts this nice, you know, tan box around each code cell and also its output to kind of keep them together, because I find that sometimes it's hard to tell where the output from the previous code cell ends and where something new begins, especially if you have output from a cell that may be text and and then you have more text below it, right? So if I run this, right, I can see clearly this text came from that code, right? And now here's some other stuff that's unrelated to that code, or at least not, not a result of it, right?

What font would do that?

What typeface would do that? This text and this text, I believe, are the same typeface because here my text is code. All right, if you're— so There will always be a situation where— by the way, if you double-click on one of these cells, you can see the way I'm rendering them is triple backtick CSS. So this says apply syntax highlighting to this as if it's CSS code, which is what it is. So I think these are actually the same font or very similar.

It's not a corner case.

Yeah.

But maybe it's a corner case. It's unusual. Okay. And then this just says all text cells I want to render with this font, this particular Google font, and I want a little more spacing for them, and I want the font a little bigger. Okay. And also I want text paragraphs to be even narrower than my code cells. Right. Okay. We're almost to the end. This is— the rest of this The next several sections are just which fonts I want to use for my headings, okay? And how big I want them to be, and what color I want them to be, and so on. So this is the H1. So this is what's setting the style for this title here, right? Before I changed it here. Yeah.

Green.

Right. So this happens to be green, that color of green that's there. This is the font that you've seen in all those titles. Okay, and then H2 and H3. And by the way, H1, H2, and H3 are things that you can get in Markdown by using, you know, 1, 2, or 3 pound signs, or you can get them by setting the cell type. H4 through H6, you can't get them by putting— I don't think— actually, maybe— yeah, you can. Never mind. I forgot.

You should always use headers.

Check this out.

Why? And that's because you get an HTML anchor element to the headers, and it provides more structure to the notebook. So the Markdown for the rest of the— any tool that's parsing it or looking at it will only just kind of see Markdown. Do a little more work to figure out that you intend to put a header there. If you make a header cell, everything knows that it's a header cell, and it's much easier to work with and keep track of in the future. And the HTML anchor thing is just huge. You can actually link to subpages or subsections of your notebook just by clicking on that link and right-clicking. You'll actually— clicking on a heading and right-clicking will give you a link to that section of the notebook, both in nbviewer and in the library.

So if I'm writing a quick notebook of notes for myself with some code that I'm playing around with, I may just do it this way. But if I'm going to distribute it to other people, then I would use header cells.

There may also be circumstances where you want to have a typographical difference, but you don't necessarily want it to appear as a header or something like that.

Yeah.

I mean, we run into that problem with indexing LaTeX headings all the time. The same thing, you know, you end up with a scientific thing, It's got a 14-page table of contents and 4 pages of actual text.

Okay, and really important, if you want to change the font for code, here's how to do it, right? So the notebook uses CodeMirror to render the code. So you just put .codemirror and then tell it what font to use, and you can adjust the size and things like that. Of course, you should always use a monospaced font for code. Right, so that each character takes up the same width. Okay, and then there's this closing style tag to tell it we're at the end of the CSS. You know, everything else is HTML. Okay, and then here are just links to CSS files or discussions of CSS for IPython notebooks on the web.

Any—

yep? Can you go back to the— What is recommended by linking 2 notebooks? Being in one notebook and say, maybe you should go and look at a section of another notebook. Is that what it is?

Yeah, you can use— it's the same way that you would navigate a webpage that somebody is using. You would use relative links if it's something within the page, like something within the site. And if it's something on another site, you would use an absolute link, like the full URL. That's it.

There's no special And you just use that to paste anything?

In the Markdown anchor, you just put in square brackets whatever you want to say, and in the parentheses, just copy and paste the filename. Even if it has spaces in the filename, it works fine. You just put the filename to the notebook or the directories. You can have pound and the anchor marker if you want to link into it. And one thing that's worth noting is that if you have a notebook called index, index.ipynb, that one will show up first at the top of the directory listing. That way, if you have 20 notebooks on all kinds of things, that one is always at the top and it's an easy entry point for your readers to start. That's how we're organizing our own documentation now, is in each directory there's an index notebook and that one contains the links. And therefore you can— because it's another notebook, you can explain whatever else and you can have other So for tutorials, it's a good way of packaging. The index has the intro material, setup, whatever, and links to all the sections.

How long is it before GitHub is going to automatically start rendering and finding these files? There's a camera. Oh, okay.

Okay. So this will work, for instance, right? If I'm just within the materials for a single course and I want to link to another notebook, this is what I would do. Right? And then if I click this, right, it opens that notebook. By the way, I just realized these 4 links here to HyperPython, Aeropython— somehow the actual links didn't get put in. I'll put them in later so that that's— so those are there. But if you Google HyperPython or Aeropython, you can find the CSS files for those. Yeah, any other questions? And if you go to the— I don't know if you have any— in HyperPython I have links between some of the notebooks. I have the first one. Yeah, okay. All right, so now we are going to open it up for sort of a panel discussion. We have lots of experts in the room besides Just Aaron and I. Questions about the notebook, about teaching, about any of the other topics that we've covered today, everything's fair game, or something you wish we would have covered that we didn't. How do we want to do this, Aaron? Do we want to have people sit at the front or just kind of redirect questions based on content?

I don't know.

Well, so we only have one mic, so there'd just be kind of like a shooting line.

Does anybody want to volunteer to be on the panel?

We're going to talk about teaching, yeah.

Just let anyone ask questions about teaching with IPython notebooks. So we are submitting questions.

Yes, that's right. Okay, I think we'll start with questions from the audience and then I'll give each of our panelists, our standing panelists, a chance to answer and then we'll just keep going until everybody's tired and they can't stand anymore. How's that sound?

Okay.

Our questions are from the audience.

Yeah, so start thinking about your questions, of course.

Yeah?

So it was mentioned earlier about grading and comments, and I'm sort of stuck on that. In particular, how to keep students from turning in something that they just found online or they got from a friend. Can you speak a little bit more about what that metadata is in the archive?

Okay, and I'm going to repeat the question so that they get saved in the video. The question is regarding turning in IPython notebooks and when students are handing in, what are the sort of technologies available for making sure that they're turning in something novel or their own work as opposed to something that they found on the internet? And then there were some questions about whether there was anything in the metadata of the notebook itself that might help. And I think there are some interesting answers there. So it's a little bit of a technology question as opposed to a policy question.

I will not answer the technology question, but the pedagogy questions. Have you seen the YouTube video by Eric Mazur on something like assessment is the silent killer of education?

Yeah, and I mean, I tend to not— I teach college students, so my feeling is if they're going to cheat, they're just wasting their time and money, and that's not really my problem. So I mean, I don't want to really police it, but on the other hand, I'm starting to think that if they do a really good job, I'm going to get 20 identical-looking notebooks.

Right, so everybody has a different style to how they want to treat How do you do this in the classroom? I talk to every student in every class because I have a small class. I have 20 students. And I go desk by desk and I speak to them at every class, in every class. So I am aiming for mastery in the sense that I will work with the students until they have completed the assignment. And so in that sense, in a way, they all deserve an A. And so what do I do? In addition to that, I make them present. I give them projects where everybody's going to do something different. But to— we are thinking with some colleagues about ideas to do some assessment that is more automated. Not assessment in the sense of getting a grade, but formative assessment so students know in the as they are progressing through material that they are achieving some result. And what would be nice is to transfer the— what we do in development, in software development, to education in the sense of developing a testing framework where if students have been able to complete a certain assignment, Their code needs to pass the unit test. So you could have— if we could co-opt the testing philosophy and framework from software development to this education, I think that would be really, really great. That requires a lot of creative thinking on the part of the instructors on how to create that material. That would be nice. As to— Within the notebooks, what I would like is for the notebooks to be able to have some sort of facility where you can write some code that somehow calls home, in the sense queries from a server and a particular answer when the students are executing some part of code to check that the answer is correct. And that would be a nice way for the students to get that satisfying green checkmark next to some answer. All of this, I believe, is just ideas at the moment because this is happening too fast.

I'm going to come in with some actual comments about what's possible today and what you could do with a little bit of effort or a medium amount of effort or a lot of effort. So first of all, the idea of homework solutions that are automatically graded or automatically checked and tested exist within computer science curriculums. There are several universities that have been doing it for a while. The first case or the first one that I'm aware of was out of Carnegie Mellon. It was a systems programming class where you actually logged into a machine and your homework was a sort of an interactive thing where you solved the problem and then that problem was graded for you on that machine. That's not the sort of thing that we think about, right, SSHing into a machine anymore, but people do think about going to web pages. So the same way that a machine could automatically grade for you, your local IPython notebook session probably couldn't, but a remote system like SageMath Cloud is sort of this perfect environment for— the kernel is already executing remotely, and so it can be something a little bit more than an IPython kernel. And we didn't talk about how there are multiple kernels. But there are. So what we've all been looking at is the IPython standard sort of kernel console, right? But you can run Julia, you can run R, you can run other programming languages, you can run MATLAB from an IPython notebook. It's actually happy to use any of these. But what I'm talking about actually is an IPython kernel that has been augmented just a little bit so that if it sees, for example, a cell magic That says homework. Instead of saving that locally, of course, because this is running on a cloud, this kernel could actually make a connection or save something in a database or do something else that's actually a little bit above and beyond. And then you can talk about things like automatic mini-grading. To come back a little bit about tracking whether an IPython notebook is a copy or different, there's some very first principle things you can do. The first is that every IPython notebook When it is saved is actually signed now. So, IPython 2.1, every notebook is signed when it's saved, and that signature is not like, this was created by Aaron Abadieh, unfortunately. That signature is a hash that is unique to the computer that created it. So, the answer, of course, is if you get 2 notebooks with the exact same hash, that's a copy. Unquestionably a copy. There was no other way that that was created except Somebody copied the notebook from one computer to another. It was not created on 2 separate notebooks. Okay, sorry, yes. Somebody could open it, save it, and then they would completely go around that. So this is not a model. However, if you are really interested in keeping track of how people are progressing and working, I would really suggest that you use Git.

Mm-hmm.

Because A, you're teaching people how to use a modern version control system that has some pain on its own just learning it, but the— Git model is basically every time you save or checkpoint your code, that's sort of signed by the person who wrote it, and that is like, this is my name, this is my email address, I say that this is my stuff. And you are constructing a much more elaborate lie when you turn in a homework assignment that has a Git commit that has some set of files on it, because A, if it's something that took a little while to develop, that Git history is harder to prove.

That's an excellent, excellent point.

Thank you.

Okay, sorry, that was some technology things completely aside from the policy.

We've got—

should we keep going? Just one reinforcement. I've had my students create a GitHub account in a classroom, and I told them that, you know, they have to just post the link to that GitHub address that holds the notebook, and I'll go in and see the commits. And sometimes I go in and see, you know, you haven't committed anything for 10 days. What's happening? So GitHub provides a very nice way to check on students and how they're progressing.

One more thing. So I just want to make a point about the kind of exercises and what the answers to the exercises are. So in that multigrade notebook, for instance, there was a point where They were supposed to code up Jacobi's method. They do that in class. That's not something that they're going to get graded on. They just need to do it so that we can move to the next part. And some of them will struggle with it. They'll talk to the person next to them and figure out how to do it. The instructor will come and help them. But that's just something that they need to do. Everybody's going to get that done, and they're not going to get an A or a B or a C on it. And then there are exercises at the end. And the exercises are written so that the answer is never code, and the answer is never the plot produced by your code. It is an explanation of what is happening that shows that you understand some concept behind it. So that's one way that I get around this. Of course, you can copy the explanation from the guy next to you, right? Still. But anytime you're turning— anytime— yeah. You really— I tell them, I should not need to read your code. Now if something bizarre— if they say something bizarre in their answer, then maybe having their code is useful because I can see, oh, you did this wrong, right? And that's why you think that this is what's going on. But I try to emphasize to them that code is how you communicate with a computer. Code is not how you communicate with the professor.

Yeah, that's an excellent point. And sometimes textbooks will say something like, reproduce this Poincaré section. Well, okay, they know it has to look exactly like this, so I get all those. And you're absolutely right that there's no science in that. So the exercise needs to be rewritten, not reproduce this plot, but then asking them to infer something from a new plot that they can't just check against the one they know.

And especially, like, try, you know, changing this parameter. How does the Poincaré section change? why, you know, that kind of thing, right? Doing something different than what's in the textbook. Okay, thank you.

I'm going to move us to the next question.

I actually had a comment. So I used to work for the math department, and we have an online testing system there. And one of the things that, just as a thought, if you're going to be doing pedagogical notebooks, is that one of the features is that students could get partway through a problem, they could submit it, And there's a database of known common errors that are made. And if one of those errors matches, then they get a hint back and says, look at this, try again. And so they get 3 attempts on these, which are effectively just to make sure that the students are working on the problems. And their time working is recorded by the web server, but not necessarily the results of what they've actually done.

Yeah, so that's worth repeating for posterity. The idea is that there is a web server where students can submit solutions and the web server is actually aware of common mistakes that students make. And so it will identify some of these common mistakes when they're submitted and it will kindly, I suppose, reject the answer and also point the user to some hints that are actually specific to that student's particular mistake. Okay, so before this question wanders, do we have a new question?

Yes. So I do teaching at SUNY Albany and we use team-based learning.

Okay.

Essentially, you divide the class in groups of 5, 6, and it's a flipped classroom approach. And I'm curious if you have applied the notebook in that type of environment.

Okay, so is there— the question is, do either of the panelists have an An example of a notebook that is taught in a team-based flipped classroom approach, teams of 5 or 6?

I used my AeroPython notebooks in a flipped classroom, but there wasn't specific teams. It wasn't a team-based learning approach. So what we did is I— well, basically there were no lectures in class, and the students had to read. This was a flipped classroom with no videos, so not Flipped classroom does not equate online videos. It's one of the things I want to point out. Although I did previously teach a CFD class with some videos that contained the lecture material. And the whole class period was used on active learning in various forms. We did pair programming, all sorts of— Basically, the students were coding in class and interacting with each other and answering questions and doing peer instruction and things like that. So the class was all active and the content was read by the students within the notebooks. So the notebooks contained written content, the equations, the derivations, sometimes supporting video, and the students had to do the work of studying the theory and working out the class derivations in their own time. Sorry, the equations, the mathematical derivations in their own time.

So do you actually make them work in teams of 2?

Yes, very many times, yes.

Thank you. I've also just had them work in teams of 2. I think it's really hard for more than 2 people to get around one computer, but I've thought that with syncing on SageMath Cloud, for instance, or if we get it natively in the notebook eventually, right, where you could have 5 or 6 students on their own laptops sitting around a table but all editing the same notebook. Maybe it would be total mayhem, but maybe it would be really fun to have them work that way.

Eventually on Wednesday. All right.

Should we record that? Yes, a follow-up. There's a follow-up question.

When you create the teams, did you make some effort to create the teams so there is one person who knows a little bit more than the other, or you let them form the teams by themselves?

Never let the students form the teams by themselves because they choose really bad partners. They choose their friend and their colleague and not the person who's going to help them. The next best thing is random. I use random. There is some way, there are some tools to, if you have a large class, to pair students up But I don't use any of that. I used a free app called Attendance that I downloaded on my phone. You can create a CVS file of all your students and then upload it. And then you have some little 2 clicks to create groups of 2, groups of 3, groups of 4 for randomize. Or you can write your own code for that, I suppose.

That's good.

Is this a new question?

Yes.

OK, great.

Basically the question is, how do you think about teaching the basic ideas behind Git in a largish classroom with quite a wide variance in technical abilities?

OK.

So the question is, how do you teach the basic ideas What are the basics behind Git in a large classroom with varying degrees of technical ability? And for what parts of Git should they focus on? You can boil Git down to create a repository and make commits and push those commits to GitHub in probably 15 to 20 minutes.

I do it in 5 with the graphical app.

I was thinking of the graphical app, actually, but 5 minutes I think is too short of an introduction to— Click, click, click, click. Yeah. Okay. But I guess we should—

Unless you hate that.

No. We should say that that way there are dragons, right? Because you are On a very, very small piece of the map with Git, and if you step off in any direction, there's an infinite amount of confusion that you can cause. So you just have to be careful to keep students in a very small fenced area with how they use Git. And so you don't introduce things like forking, you don't introduce things like pull requests or issues or merges or branches or anything. You just have them use the API to push. That's sufficient for anybody who is just using Git in, say, an undergraduate context. They're not actually using it for their own research or software. If I was teaching to graduate students or if I was teaching to people who were using Git for research, I would invite a software carpentry instructor to come teach for about 3 hours, and you can get a very solid introduction to Git in about half a day. It gets you all the way through things like branches and pull requests and issues, how to use them on GitHub in addition to the core model of Git, which has less of some of those web things, but helps you get out of corners.

Let me jump in. My constituency, of course, is not computer computer science students. It's engineering students, mechanical engineering. So my experience with this was— my philosophy, first of all, is to never give them a tool until they need them. So little by little, when they need them, they feel the need, they want to learn some more. And if they get in trouble, they get out of trouble, then they will want to stay on task until they solve something because they need it. So I start them with the GUI. Like any app, they download the app and they're up and running in literally no time. And what I observed with these students, mechanical engineering students, is that over time, some of them started realizing the value of this. And they were telling me, you know what? I've moved all the homeworks of my other classes now to GitHub because I have them all in one place and I can't lose them and so on. And they were really excited. The guy next door, he says, you did that? Why did you do that? And they start sort of sharing the same way that we do with every single tool that we have on our phones or anything. Somebody tells you how they're using it, and they like that, and they find it useful. And then you come in and say, okay, so there's a tutorial here that you need to go and read because it's going to tell you how to get out of trouble. Now that you have everything there, not just the homework for this class, you better learn this part. And you start pointing them in the right direction. But my philosophy is that they have to have a reason for it. And it has to be sort of just in time and gradually get them interested in it. Another one there.

Yeah.

We are actually right at 5 o'clock. There's less things outside, you know, there's no snacks or other exciting things. They clear all the food away at the break. I think the video cameraman probably has to go. So I think we're going to officially wrap up here and sort of dismiss the class. Obviously, we're still around. We don't teleport up to our spaceship and fly away. We're still here.

I'm available.

After these courses. So we'd be happy to meet with you individually or in groups. So thank you all. You were a really attentive audience. It was really fun to see all this excitement around both the IPython Notebook and teaching and education and getting students to that next level. I know that for some of you it doesn't feel like you're advanced or leading the pack, but you very much are. by being here at this conference and sort of taking part in this activity. I mean, you're going to go back to your institutions and you are going to be the leaders now and teach all of this material to everybody and, you know, blaze new trails with this. So thanks again for coming, and it was a real pleasure.

Thank you.