WEBVTT 00:00:01.645 --> 00:00:04.000 The following content is provided under a Creative 00:00:04.000 --> 00:00:05.540 Commons license. 00:00:05.540 --> 00:00:07.840 Your support will help MIT OpenCourseWare 00:00:07.840 --> 00:00:12.230 continue to offer high quality educational resources for free. 00:00:12.230 --> 00:00:14.830 To make a donation or view additional materials 00:00:14.830 --> 00:00:18.790 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:18.790 --> 00:00:19.750 at OCW.MIT.edu. 00:00:22.840 --> 00:00:25.060 PROFESSOR: So I am going to start this talk 00:00:25.060 --> 00:00:27.340 by explaining the title. 00:00:27.340 --> 00:00:29.620 So we'll start with active learning. 00:00:29.620 --> 00:00:31.420 What is active learning? 00:00:31.420 --> 00:00:34.180 In my mind, it's this pretty broad term. 00:00:34.180 --> 00:00:37.000 It's a form of learning that attempts to involve students 00:00:37.000 --> 00:00:39.916 more directly in the learning process. 00:00:39.916 --> 00:00:41.290 And there are all sorts of things 00:00:41.290 --> 00:00:43.780 that fall under the umbrella of active learning. 00:00:43.780 --> 00:00:45.790 So things like having students engage in group 00:00:45.790 --> 00:00:49.540 work, in class-wide discussions, in debates, 00:00:49.540 --> 00:00:52.690 acting out certain concepts, group problem solving 00:00:52.690 --> 00:00:54.580 sessions, all of these things. 00:00:54.580 --> 00:00:56.200 And so in all of those activities, 00:00:56.200 --> 00:00:58.180 students are engaged. 00:00:58.180 --> 00:01:00.610 They're not passively listening to a lecture, 00:01:00.610 --> 00:01:03.050 they're actively participating. 00:01:03.050 --> 00:01:07.060 And there is some evidence that this style of learning 00:01:07.060 --> 00:01:09.350 is good for a lot of things. 00:01:09.350 --> 00:01:13.630 There is evidence to support the effectiveness of student 00:01:13.630 --> 00:01:17.740 engagement in exam scores, failure rates, how 00:01:17.740 --> 00:01:21.220 well students remember content, student attitudes, study 00:01:21.220 --> 00:01:22.430 habits. 00:01:22.430 --> 00:01:24.970 And there's also evidence that active learning 00:01:24.970 --> 00:01:28.090 has a disproportionate benefit for minorities, students 00:01:28.090 --> 00:01:31.330 from disadvantaged backgrounds, female students, 00:01:31.330 --> 00:01:33.430 and male dominated fields. 00:01:33.430 --> 00:01:36.970 So there's evidence that active learning creates this more 00:01:36.970 --> 00:01:39.230 inclusive environment. 00:01:39.230 --> 00:01:43.660 So it's this wide, wide variety of techniques reported 00:01:43.660 --> 00:01:45.390 to have a lot of benefits. 00:01:45.390 --> 00:01:47.230 So that's active learning. 00:01:47.230 --> 00:01:49.240 Now, what is 6.033? 00:01:49.240 --> 00:01:52.820 So 6.033 is computer systems engineering. 00:01:52.820 --> 00:01:55.510 It's a required class for all computer science 00:01:55.510 --> 00:01:59.870 majors, which means as a result, at MIT it's quite large. 00:01:59.870 --> 00:02:02.620 So like Dipa said, we have about 400 students. 00:02:02.620 --> 00:02:03.850 It runs once a year. 00:02:03.850 --> 00:02:07.180 So every spring we get 400 new students. 00:02:07.180 --> 00:02:09.850 Most of them are juniors and seniors. 00:02:09.850 --> 00:02:14.050 And in some ways, 033 is a typical MIT class. 00:02:14.050 --> 00:02:15.800 We have lectures twice a week. 00:02:15.800 --> 00:02:17.860 We have recitations twice a week. 00:02:17.860 --> 00:02:23.290 But in most other ways, 033 is not a typical MIT class. 00:02:23.290 --> 00:02:25.840 For one, it's a CIM class. 00:02:25.840 --> 00:02:29.470 So it's a Communications Intensive in the Major class. 00:02:29.470 --> 00:02:31.630 That means that most of our assignments 00:02:31.630 --> 00:02:35.110 are based around writing or oral presentations. 00:02:35.110 --> 00:02:37.840 It's not a class where students are implementing or building 00:02:37.840 --> 00:02:39.490 a lot of things. 00:02:39.490 --> 00:02:43.000 We focus heavily on reading and on writing. 00:02:43.000 --> 00:02:46.690 In particular, we have tutorial sections each week, 00:02:46.690 --> 00:02:49.810 dedicated to the communication material, 00:02:49.810 --> 00:02:53.020 taught by dedicated communication staff. 00:02:53.020 --> 00:02:57.160 The major assignment in 033 is a design project. 00:02:57.160 --> 00:02:58.690 It takes most of the semester. 00:02:58.690 --> 00:03:02.380 The students completed in teams, usually teams of three. 00:03:02.380 --> 00:03:05.470 And it's comprised of a few written assignments 00:03:05.470 --> 00:03:07.450 and oral presentations. 00:03:07.450 --> 00:03:11.500 One of our big focuses for that project is getting students 00:03:11.500 --> 00:03:14.770 to recognize the trade-offs they need to make when they're 00:03:14.770 --> 00:03:17.140 designing a system, and getting them 00:03:17.140 --> 00:03:18.730 to defend those trade-offs. 00:03:18.730 --> 00:03:20.140 So this is not just a class about 00:03:20.140 --> 00:03:23.410 how computer systems work, but how we design them, 00:03:23.410 --> 00:03:25.930 how we decide what choices to make when 00:03:25.930 --> 00:03:27.970 we're building those systems. 00:03:27.970 --> 00:03:30.280 So the last thing that makes 033 pretty unique, 00:03:30.280 --> 00:03:32.800 and is what I'm largely going to talk about here, 00:03:32.800 --> 00:03:35.770 is that our recitation sections are pretty intense. 00:03:35.770 --> 00:03:39.250 They are taught by faculty, senior research staff. 00:03:39.250 --> 00:03:41.980 We have about 25 students per section. 00:03:41.980 --> 00:03:44.380 So you are right to think that, oh, my gosh, they have 00:03:44.380 --> 00:03:46.720 like 18 sections per semester. 00:03:46.720 --> 00:03:47.540 We do. 00:03:47.540 --> 00:03:48.970 It's a lot. 00:03:48.970 --> 00:03:52.690 Each recitation focuses around a technical paper 00:03:52.690 --> 00:03:55.240 that the students have read beforehand. 00:03:55.240 --> 00:03:57.190 And the goal for those discussion sections, 00:03:57.190 --> 00:04:01.030 those recitations, is for them to be largely discussion based. 00:04:01.030 --> 00:04:03.550 And in fact, that has always been the goal of 033. 00:04:03.550 --> 00:04:08.470 033 has been around for almost 50 years at this point. 00:04:08.470 --> 00:04:10.480 It's been around since before I was born. 00:04:10.480 --> 00:04:13.270 And the goal has always been for these recitations 00:04:13.270 --> 00:04:15.100 to be discussion based. 00:04:15.100 --> 00:04:17.649 During the recitations, we're talking about, 00:04:17.649 --> 00:04:20.709 we talk about the system that the paper describes-- 00:04:20.709 --> 00:04:21.920 does it work? 00:04:21.920 --> 00:04:23.920 What were the trade-offs that the authors made? 00:04:23.920 --> 00:04:25.160 Do we like this system? 00:04:25.160 --> 00:04:27.900 How would you design this system? 00:04:27.900 --> 00:04:30.840 All right, so that is 033. 00:04:30.840 --> 00:04:34.800 Now, why put active learning into 033? 00:04:34.800 --> 00:04:38.370 And you might think that it's just because, well, there's 00:04:38.370 --> 00:04:41.910 evidence that active learning can be a good idea, 00:04:41.910 --> 00:04:44.160 so why not try it? 00:04:44.160 --> 00:04:45.250 That's fair. 00:04:45.250 --> 00:04:49.260 We were actually trying to solve a pretty specific problem 00:04:49.260 --> 00:04:50.960 in the class though. 00:04:50.960 --> 00:04:54.180 So like I said, ideally the recitations 00:04:54.180 --> 00:04:56.670 are based heavily around discussion. 00:04:56.670 --> 00:04:59.100 And again, that has always been the goal. 00:04:59.100 --> 00:05:02.910 Because this is a class about system design, not just about 00:05:02.910 --> 00:05:06.460 how these systems work, one way that we teach 00:05:06.460 --> 00:05:09.480 that is by having the students analyze and critique 00:05:09.480 --> 00:05:11.280 existing systems before they go out 00:05:11.280 --> 00:05:12.910 and try to design their own. 00:05:12.910 --> 00:05:15.570 And so, what are the goals of the designers? 00:05:15.570 --> 00:05:17.640 Does the system meet those goals? 00:05:17.640 --> 00:05:19.290 Are those goals important? 00:05:19.290 --> 00:05:22.119 If you were redesigning the system, how would you do it? 00:05:22.119 --> 00:05:23.160 What would your goals be? 00:05:23.160 --> 00:05:24.784 What trade-offs would you have to make? 00:05:24.784 --> 00:05:26.130 Things like that. 00:05:26.130 --> 00:05:28.350 A large part of that instruction happens 00:05:28.350 --> 00:05:30.910 in the recitation, where we're having these discussions. 00:05:30.910 --> 00:05:32.430 And they go and apply those skills 00:05:32.430 --> 00:05:34.350 to their design projects. 00:05:34.350 --> 00:05:38.070 So to get that to happen, it's important that the students are 00:05:38.070 --> 00:05:40.290 engaged in recitation, that they're 00:05:40.290 --> 00:05:42.810 reading the paper beforehand, they're asking questions, 00:05:42.810 --> 00:05:45.018 they're answering questions, they're coming prepared, 00:05:45.018 --> 00:05:46.082 things like that. 00:05:46.082 --> 00:05:48.690 And for a long time, that was not happening for us. 00:05:48.690 --> 00:05:51.000 We had a lot of trouble with this. 00:05:51.000 --> 00:05:55.230 We found that students weren't willing to volunteer answers. 00:05:55.230 --> 00:05:58.540 When we'd ask them a question, nobody would raise their hand. 00:05:58.540 --> 00:06:00.960 We found that some of them weren't even reading 00:06:00.960 --> 00:06:02.370 the paper beforehand. 00:06:02.370 --> 00:06:05.280 So they're just kind of coming in confused. 00:06:05.280 --> 00:06:07.410 As a result of all of this, our instructors 00:06:07.410 --> 00:06:10.140 would end up spending the first 15, 20, 25 00:06:10.140 --> 00:06:13.710 minutes of the recitation lecturing about the paper, 00:06:13.710 --> 00:06:17.070 because so many of the students didn't come prepared. 00:06:17.070 --> 00:06:19.680 And then that just made things worse, as you can imagine. 00:06:19.680 --> 00:06:22.381 Students know that they're going to come into recitation, 00:06:22.381 --> 00:06:23.880 that they're going to get a lecture, 00:06:23.880 --> 00:06:26.490 so why bother reading the paper, if your instructor is just 00:06:26.490 --> 00:06:27.615 going to explain it to you. 00:06:27.615 --> 00:06:30.330 And so it was just kind of making things worse. 00:06:30.330 --> 00:06:33.570 We weren't getting to teach the material that we really 00:06:33.570 --> 00:06:35.700 needed to teach, because we were spending time 00:06:35.700 --> 00:06:37.470 rehashing these papers. 00:06:37.470 --> 00:06:39.350 The students weren't learning how 00:06:39.350 --> 00:06:41.100 to read these papers, how to analyze them, 00:06:41.100 --> 00:06:42.360 how to critique them. 00:06:42.360 --> 00:06:44.590 And honestly, it wasn't much fun. 00:06:44.590 --> 00:06:46.960 It wasn't all that much fun to teach this way. 00:06:46.960 --> 00:06:50.820 Occasionally, we could get a discussion going in a class. 00:06:50.820 --> 00:06:54.210 But we would then encounter another problem, which 00:06:54.210 --> 00:06:56.480 is pretty common, where we'd have a discussion, 00:06:56.480 --> 00:06:58.680 but it would be dominated by two or three students 00:06:58.680 --> 00:07:01.260 in the room, the same two or three students. 00:07:01.260 --> 00:07:03.540 And then other students are tuning out, 00:07:03.540 --> 00:07:07.020 or even becoming sort of visibly uncomfortable, 00:07:07.020 --> 00:07:10.960 not wanting to volunteer these answers. 00:07:10.960 --> 00:07:12.330 So that's where we were. 00:07:12.330 --> 00:07:13.350 And we had this problem. 00:07:13.350 --> 00:07:17.379 We really wanted the recitations to be this really fun place, 00:07:17.379 --> 00:07:19.170 where the students were coming, and they're 00:07:19.170 --> 00:07:21.450 wrestling with all of these new ideas 00:07:21.450 --> 00:07:23.340 and proposing these new things. 00:07:23.340 --> 00:07:24.690 But we didn't have that. 00:07:24.690 --> 00:07:27.660 We had this place where students aren't coming prepared, 00:07:27.660 --> 00:07:30.360 or if they're prepared, if we can get a discussion going, 00:07:30.360 --> 00:07:32.250 it's not the kind of discussion we want. 00:07:32.250 --> 00:07:35.550 It's sort of dominated by one or two people. 00:07:35.550 --> 00:07:38.880 So after some thinking, I thought 00:07:38.880 --> 00:07:40.560 active learning might be a good approach 00:07:40.560 --> 00:07:43.032 to solve both of these problems. 00:07:43.032 --> 00:07:44.490 So, as I mentioned earlier, there's 00:07:44.490 --> 00:07:45.990 some evidence that active learning 00:07:45.990 --> 00:07:48.990 can be good for creating an inclusive environment 00:07:48.990 --> 00:07:50.130 in a classroom. 00:07:50.130 --> 00:07:52.380 So for instance, one common technique, 00:07:52.380 --> 00:07:54.300 which I will talk a bit about later, 00:07:54.300 --> 00:07:56.370 is you pose a question to the class, 00:07:56.370 --> 00:07:59.060 but put everybody in small groups, 00:07:59.060 --> 00:08:01.200 have the students discuss in small groups, 00:08:01.200 --> 00:08:03.420 and then kind of bring the classroom back together 00:08:03.420 --> 00:08:05.640 for a wide discussion. 00:08:05.640 --> 00:08:07.200 Students are in their groups, they 00:08:07.200 --> 00:08:10.711 get a chance to kind of vet their ideas with their peers. 00:08:10.711 --> 00:08:12.960 It's sort of hard if you're in a group of two or three 00:08:12.960 --> 00:08:14.090 to not participate. 00:08:14.090 --> 00:08:16.354 You sort of have to say something. 00:08:16.354 --> 00:08:18.270 So they'll get to vet their ideas with people, 00:08:18.270 --> 00:08:19.490 they'll get some reassurance. 00:08:19.490 --> 00:08:23.790 And in theory, they are then more willing to bring that idea 00:08:23.790 --> 00:08:26.050 back to the whole class. 00:08:26.050 --> 00:08:30.360 That type of activity can also give you a relatively easy way 00:08:30.360 --> 00:08:33.480 to make sure one person doesn't dominate the discussion, 00:08:33.480 --> 00:08:36.210 by saying, OK, I split you guys up into five groups, 00:08:36.210 --> 00:08:39.143 and let's hear from each group now. 00:08:39.143 --> 00:08:42.330 Moreover, I thought if we did a lot of active learning, 00:08:42.330 --> 00:08:44.960 it might keep the students engaged for longer. 00:08:44.960 --> 00:08:46.572 It would help us avoid lecturing. 00:08:46.572 --> 00:08:48.030 It would help us set this precedent 00:08:48.030 --> 00:08:50.430 that you come to recitation having read the paper, 00:08:50.430 --> 00:08:52.415 we're not going to lecture for you. 00:08:52.415 --> 00:08:54.040 So that's kind of where we're starting. 00:08:54.040 --> 00:08:55.740 That's the baseline for this talk. 00:08:55.740 --> 00:08:59.610 Now my goal for all of you is that this talk is, I hope, 00:08:59.610 --> 00:09:01.290 exceedingly practical. 00:09:01.290 --> 00:09:03.090 I'm going to talk about some specific ways 00:09:03.090 --> 00:09:05.370 that I trained my staff to do this, 00:09:05.370 --> 00:09:07.020 the types of active learning activities 00:09:07.020 --> 00:09:08.850 we do, some lessons we've learned, 00:09:08.850 --> 00:09:10.960 how we've evaluated our success. 00:09:10.960 --> 00:09:13.590 I hope you'll be able to apply some of these ideas 00:09:13.590 --> 00:09:15.750 to your own classes if you want. 00:09:15.750 --> 00:09:18.570 I do want to start, though, with a disclaimer. 00:09:18.570 --> 00:09:21.540 This is really a case study of how active learning has worked 00:09:21.540 --> 00:09:23.320 in 033. 00:09:23.320 --> 00:09:26.310 I'm not here to make claims that active learning is 00:09:26.310 --> 00:09:29.470 a solution to all of your teaching problems. 00:09:29.470 --> 00:09:30.530 I think it's very good. 00:09:30.530 --> 00:09:32.940 It's probably not the solution to everything. 00:09:32.940 --> 00:09:33.790 Nor am I even-- 00:09:33.790 --> 00:09:35.590 I'm not even here to really evangelize it 00:09:35.590 --> 00:09:39.061 over any other particular pedagogical schemes. 00:09:39.061 --> 00:09:41.560 I think it works well in 033, I think we've done some really 00:09:41.560 --> 00:09:42.642 cool things. 00:09:42.642 --> 00:09:44.350 I hope you're excited to hear about them. 00:09:44.350 --> 00:09:48.610 So active learning. 00:09:48.610 --> 00:09:50.590 As you've probably picked up at this point, 00:09:50.590 --> 00:09:53.620 what I'm going to talk about, we're infusing the recitations 00:09:53.620 --> 00:09:55.192 in 033 with active learning. 00:09:55.192 --> 00:09:56.650 And so that means that really it is 00:09:56.650 --> 00:09:59.560 my recitation instructors who are on the ground doing 00:09:59.560 --> 00:10:00.610 a lot of this. 00:10:00.610 --> 00:10:03.730 And I will admit, when we were starting out, 00:10:03.730 --> 00:10:05.590 I sort of thought-- 00:10:05.590 --> 00:10:09.550 I had this amazing idea to do active learning recitations. 00:10:09.550 --> 00:10:11.410 And I was like, well, I am done. 00:10:11.410 --> 00:10:13.030 I have solved the problem. 00:10:13.030 --> 00:10:15.160 I just told my instructors to implement it. 00:10:15.160 --> 00:10:18.100 It was like, guys, active learning. 00:10:18.100 --> 00:10:20.530 It didn't work. 00:10:20.530 --> 00:10:25.150 I was perplexed, because I told my instructors correctly 00:10:25.150 --> 00:10:26.980 that these types of activities would 00:10:26.980 --> 00:10:28.780 help increase participation. 00:10:28.780 --> 00:10:31.120 My instructors were the ones complaining about the lack 00:10:31.120 --> 00:10:32.110 of participation. 00:10:32.110 --> 00:10:34.580 And so I just thought come on, what's the problem. 00:10:34.580 --> 00:10:37.600 Well it turns out you cannot tell your instructors to do 00:10:37.600 --> 00:10:40.000 a thing that they've never done before and just have them 00:10:40.000 --> 00:10:41.470 magically do it. 00:10:41.470 --> 00:10:43.420 In particular, you can't tell your instructors 00:10:43.420 --> 00:10:45.940 to fundamentally change the way they teach 00:10:45.940 --> 00:10:48.040 and magically have that happen. 00:10:48.040 --> 00:10:49.720 I would say it's difficult enough 00:10:49.720 --> 00:10:52.150 for us to change the way we teach, 00:10:52.150 --> 00:10:55.900 much less to get other people to change the way they teach. 00:10:55.900 --> 00:11:00.430 So the first part of my talk is changing 00:11:00.430 --> 00:11:03.130 the way other people teach in four easy steps. 00:11:03.130 --> 00:11:06.550 So step one is get everyone on board. 00:11:06.550 --> 00:11:10.110 So this is the first thing that I did, last year in 033. 00:11:10.110 --> 00:11:13.450 I will keep saying last semester, but last year. 00:11:13.450 --> 00:11:16.540 I presented active learning as our primary goal 00:11:16.540 --> 00:11:18.430 for the entire semester. 00:11:18.430 --> 00:11:22.720 So we did not wait until the course got settled down 00:11:22.720 --> 00:11:23.840 to start doing it. 00:11:23.840 --> 00:11:26.230 In fact, our very first staff meeting, 00:11:26.230 --> 00:11:29.350 which happens before the semester has even begun, 00:11:29.350 --> 00:11:31.090 was about active learning. 00:11:31.090 --> 00:11:32.750 We had everyone on the staff there. 00:11:32.750 --> 00:11:35.500 So me, all of my recitation instructors, 00:11:35.500 --> 00:11:38.140 all of my TAs, all of the communication instructors. 00:11:38.140 --> 00:11:42.400 This is about 30 some people here. 00:11:42.400 --> 00:11:44.380 And we talked about active learning. 00:11:44.380 --> 00:11:47.380 We talked about why it's a good idea, the evidence that it's 00:11:47.380 --> 00:11:50.440 a good idea, evidence that it is better than lecturing. 00:11:50.440 --> 00:11:53.830 We talked about why running the recitations in this way would 00:11:53.830 --> 00:11:57.190 support the other learning objectives in 033, 00:11:57.190 --> 00:12:00.490 in particular the objective of getting students to design 00:12:00.490 --> 00:12:03.704 their own systems and evaluate their own tradeoffs. 00:12:03.704 --> 00:12:05.620 I explained to my instructors that there would 00:12:05.620 --> 00:12:08.320 be extensive support for this. 00:12:08.320 --> 00:12:11.140 And that we'd be checking in throughout the semester 00:12:11.140 --> 00:12:13.240 to make sure this was really working for them 00:12:13.240 --> 00:12:15.720 and working for the students. 00:12:15.720 --> 00:12:20.980 Now I did expect some pushback on this in this meeting. 00:12:20.980 --> 00:12:25.210 Partly, I am asking a lot of my staff here. 00:12:25.210 --> 00:12:27.340 And also, I am a lecturer in charge 00:12:27.340 --> 00:12:30.350 of a lot of faculty, who have been here for a very long time. 00:12:30.350 --> 00:12:32.740 So I was not quite sure how this would go. 00:12:32.740 --> 00:12:34.660 I do not know if my staff knew about this. 00:12:34.660 --> 00:12:37.240 I actually went so far in preparing for this staff 00:12:37.240 --> 00:12:40.570 meeting to not only write out all of my objectives 00:12:40.570 --> 00:12:42.610 and why active learning was a good idea, 00:12:42.610 --> 00:12:46.780 but to write my own rebuttal to myself and then rebut that. 00:12:46.780 --> 00:12:50.350 So I sort of had, OK, if I didn't want to do this, 00:12:50.350 --> 00:12:51.935 I'd be asking-- well, I don't think 00:12:51.935 --> 00:12:53.560 active learning is a good idea, I don't 00:12:53.560 --> 00:12:54.726 think this is going to work. 00:12:54.726 --> 00:12:57.400 And I had prepared answers for all of these questions. 00:12:57.400 --> 00:13:00.160 I looked at the notes for that meeting to prepare for this. 00:13:00.160 --> 00:13:03.370 And it really went on for quite a while. 00:13:03.370 --> 00:13:06.820 I did not actually get that much pushback here, 00:13:06.820 --> 00:13:08.390 because my staff is amazing. 00:13:08.390 --> 00:13:12.790 But also, I talked to a few of my instructors ahead of time 00:13:12.790 --> 00:13:15.280 about this, people who had been with the class 00:13:15.280 --> 00:13:19.150 for a while, who I knew would be maybe a little bit more game 00:13:19.150 --> 00:13:20.620 for this idea. 00:13:20.620 --> 00:13:23.370 I kind of got them on board first. 00:13:23.370 --> 00:13:26.630 I also appealed to everyone's scientific nature, 00:13:26.630 --> 00:13:28.930 explaining that this was somewhat of a research 00:13:28.930 --> 00:13:31.570 project, which was true. 00:13:31.570 --> 00:13:33.800 I did want to know would this work in 033. 00:13:33.800 --> 00:13:35.590 I was heavily biased towards the fact 00:13:35.590 --> 00:13:37.090 that I thought it was going to work. 00:13:37.090 --> 00:13:40.530 But we had methods to evaluate this throughout the semester. 00:13:40.530 --> 00:13:43.150 And I told them if this goes horribly, 00:13:43.150 --> 00:13:44.600 we won't keep doing it. 00:13:44.600 --> 00:13:46.530 I'm not going to burden you with this. 00:13:46.530 --> 00:13:50.080 And I enthusiastically mandated that this is what we 00:13:50.080 --> 00:13:51.850 were trained for this semester. 00:13:51.850 --> 00:13:54.100 In previous semesters, I had suggested 00:13:54.100 --> 00:13:57.230 some of these ideas, encouraged my staff to try it. 00:13:57.230 --> 00:13:59.570 And that really wasn't enough. 00:13:59.570 --> 00:14:02.160 And so on the one hand, get everyone on board. 00:14:02.160 --> 00:14:04.480 It was one staff meeting. 00:14:04.480 --> 00:14:06.130 I will say, it was really tempting 00:14:06.130 --> 00:14:09.370 for me to not have that staff meeting so soon. 00:14:09.370 --> 00:14:11.500 033 is such a big class. 00:14:11.500 --> 00:14:15.112 It has so many people involved. 00:14:15.112 --> 00:14:16.570 It feels like we're just scrambling 00:14:16.570 --> 00:14:19.330 to get the logistics worked out in the first week or two. 00:14:19.330 --> 00:14:22.520 Even for us assigning our students to sections 00:14:22.520 --> 00:14:24.890 is this whole thing. 00:14:24.890 --> 00:14:28.490 But doing this really set the tone for the whole semester. 00:14:28.490 --> 00:14:31.759 I was kind of surprised at how helpful this was. 00:14:31.759 --> 00:14:33.800 Everybody knew the active learning was important. 00:14:33.800 --> 00:14:36.350 Even the people who weren't the recitation instructors, 00:14:36.350 --> 00:14:39.560 all the TAs, all the communications instructors, 00:14:39.560 --> 00:14:41.090 they knew that it wasn't optional. 00:14:41.090 --> 00:14:43.460 It was their primary goal of the semester. 00:14:43.460 --> 00:14:45.260 Any other changes that we made in the class 00:14:45.260 --> 00:14:48.770 were largely going to be in support of this goal. 00:14:48.770 --> 00:14:52.010 So step one, get everybody on board with your mission. 00:14:52.010 --> 00:14:56.170 Step two is do a lot of work, do a lot of planning. 00:14:56.170 --> 00:14:58.440 So step one was our first staff meeting. 00:14:58.440 --> 00:15:01.592 Step two, what's happening in our second staff meeting. 00:15:01.592 --> 00:15:03.050 So one of the things that we did is 00:15:03.050 --> 00:15:06.920 we talked about a pretty well-known list of something 00:15:06.920 --> 00:15:10.850 like 278 different active learning activities. 00:15:10.850 --> 00:15:14.060 I knew it was a lot to ask really 00:15:14.060 --> 00:15:18.090 anyone to read through all 278 active learning activities. 00:15:18.090 --> 00:15:19.610 And so I annotated this list. 00:15:19.610 --> 00:15:21.770 And we kind of discussed the highlights 00:15:21.770 --> 00:15:24.080 in the staff meeting. 00:15:24.080 --> 00:15:27.110 I am going to go into detail in the second part of this class 00:15:27.110 --> 00:15:30.440 about some of the specific activities we do in 033, 00:15:30.440 --> 00:15:33.140 but I'll give you some examples from this list just 00:15:33.140 --> 00:15:34.577 for concreteness. 00:15:34.577 --> 00:15:37.160 So one of them, which I really like to do in lecture actually, 00:15:37.160 --> 00:15:37.866 is-- 00:15:37.866 --> 00:15:38.990 they all have funny names-- 00:15:38.990 --> 00:15:41.000 OK, so fingers on chest. 00:15:41.000 --> 00:15:44.537 So this is having students vote on a multiple choice question, 00:15:44.537 --> 00:15:46.120 but using like fingers on their chest. 00:15:46.120 --> 00:15:48.110 So option one, option two, option three. 00:15:48.110 --> 00:15:50.179 And that way, the students can't see what 00:15:50.179 --> 00:15:51.470 other people are voting really. 00:15:51.470 --> 00:15:54.110 They're not quite like, oh, everybody else 00:15:54.110 --> 00:15:55.140 is voting for one. 00:15:55.140 --> 00:15:57.701 They can't really see, so they tend to be more truthful. 00:15:57.701 --> 00:15:59.450 A lot of times in lecture, I ask them just 00:15:59.450 --> 00:16:00.824 do they feel good about something 00:16:00.824 --> 00:16:02.570 or bad about something. 00:16:02.570 --> 00:16:06.470 Another one, a fairly common one-- think, pair, share. 00:16:06.470 --> 00:16:08.660 So posing a question to the class, 00:16:08.660 --> 00:16:10.910 asking students to think about it, 00:16:10.910 --> 00:16:14.210 then to share their ideas with a partner. 00:16:14.210 --> 00:16:17.630 And then maybe go and address the larger class. 00:16:17.630 --> 00:16:19.310 And then a third one, which I like just 00:16:19.310 --> 00:16:20.870 because it has a nice name-- 00:16:20.870 --> 00:16:23.140 the name for it is human Tableau. 00:16:23.140 --> 00:16:24.980 And so this is having groups create 00:16:24.980 --> 00:16:27.410 living scenes which relate to the classroom 00:16:27.410 --> 00:16:29.170 concepts or discussion. 00:16:29.170 --> 00:16:31.060 So that one's a little bit more out there. 00:16:31.060 --> 00:16:33.170 But these are the types of activities 00:16:33.170 --> 00:16:34.702 we're thinking about. 00:16:34.702 --> 00:16:36.410 And of course, we went through this list. 00:16:36.410 --> 00:16:38.118 We looked at a whole bunch of activities. 00:16:38.118 --> 00:16:41.680 We're not going to end up using all of them in 033. 00:16:41.680 --> 00:16:44.570 But going through those list, it was a really good exercise 00:16:44.570 --> 00:16:45.740 for all of us. 00:16:45.740 --> 00:16:47.450 It gave everyone a much better sense 00:16:47.450 --> 00:16:51.870 of what is active learning or what can active learning be. 00:16:51.870 --> 00:16:54.650 But even kind of armed with that list, 00:16:54.650 --> 00:16:57.470 it's still a lot to tell your instructors, 00:16:57.470 --> 00:17:02.910 here's your 278 activities, now go, now do active learning. 00:17:02.910 --> 00:17:05.780 So in addition to this, I pre-planned 00:17:05.780 --> 00:17:07.760 a lot of active learning techniques 00:17:07.760 --> 00:17:09.950 into each recitation. 00:17:09.950 --> 00:17:11.990 So normally, before active learning, 00:17:11.990 --> 00:17:14.300 the way that I would plan recitations 00:17:14.300 --> 00:17:16.819 was strictly just for technical content. 00:17:16.819 --> 00:17:18.710 I would give my instructors a handful 00:17:18.710 --> 00:17:21.440 of major technical issues they needed to hit on. 00:17:21.440 --> 00:17:24.710 It helps keep our recitations somewhat uniform 00:17:24.710 --> 00:17:26.960 so that we can be fair on exams. 00:17:26.960 --> 00:17:28.850 We do test the technical material 00:17:28.850 --> 00:17:30.570 on exams in this class. 00:17:30.570 --> 00:17:32.150 Beyond that though, my instructors 00:17:32.150 --> 00:17:35.030 have a lot of leeway to teach these topics however 00:17:35.030 --> 00:17:37.880 they want, a lot of leeway on how they teach in general. 00:17:37.880 --> 00:17:39.620 I had never before tried to steer them 00:17:39.620 --> 00:17:41.820 in a particular direction. 00:17:41.820 --> 00:17:44.380 But now, in addition to that technical content, 00:17:44.380 --> 00:17:47.990 what I would do is I would put two or three active learning 00:17:47.990 --> 00:17:51.470 activities or techniques into each recitation. 00:17:51.470 --> 00:17:53.930 So things like pointing out places 00:17:53.930 --> 00:17:56.030 where they could have students break into groups 00:17:56.030 --> 00:17:58.700 and think about a particular question, pointing out 00:17:58.700 --> 00:18:01.820 places where they could hold a debate in the class. 00:18:01.820 --> 00:18:04.400 And again, I'm going to talk about these specific activities 00:18:04.400 --> 00:18:06.965 we do in the second part of this talk. 00:18:06.965 --> 00:18:09.230 But planning activities in this way 00:18:09.230 --> 00:18:10.910 meant that my instructors has had 00:18:10.910 --> 00:18:13.130 a lot of ideas of how they can make 00:18:13.130 --> 00:18:15.020 this work in their sections. 00:18:15.020 --> 00:18:18.665 Giving them multiple ideas and multiple types of ideas really 00:18:18.665 --> 00:18:20.540 meant that they could pick the ones that they 00:18:20.540 --> 00:18:22.040 were the most comfortable with. 00:18:22.040 --> 00:18:24.290 So instead of saying, hey, here's this really, really 00:18:24.290 --> 00:18:27.290 out there idea, everybody go try it, some of them 00:18:27.290 --> 00:18:29.570 might try some of the more extreme ones, 00:18:29.570 --> 00:18:31.670 but everybody could kind of find something 00:18:31.670 --> 00:18:33.620 that they were comfortable with. 00:18:33.620 --> 00:18:37.090 So get everybody on board, plan things, 00:18:37.090 --> 00:18:39.920 And the next thing that I did was support my staff 00:18:39.920 --> 00:18:41.550 as individuals. 00:18:41.550 --> 00:18:43.310 So I didn't plan these activities 00:18:43.310 --> 00:18:44.960 and set the staff free. 00:18:44.960 --> 00:18:48.220 I actually observed them throughout the semester. 00:18:48.220 --> 00:18:49.970 And I made sure to stress that it 00:18:49.970 --> 00:18:51.560 wasn't any sort of evaluation. 00:18:51.560 --> 00:18:55.220 I was not there to say, oh, boy, Karen, 00:18:55.220 --> 00:18:57.694 this is really not working out. 00:18:57.694 --> 00:18:59.360 I was just there to see what was working 00:18:59.360 --> 00:19:01.430 and what wasn't, so that we could all 00:19:01.430 --> 00:19:03.710 iterate on these things. 00:19:03.710 --> 00:19:06.350 In practice, this was great. 00:19:06.350 --> 00:19:09.620 I, of course, found way more good things 00:19:09.620 --> 00:19:11.610 in doing this than anything problematic. 00:19:11.610 --> 00:19:13.190 Most of my feedback to my instructors 00:19:13.190 --> 00:19:15.170 was, hey, you did this really awesome thing, 00:19:15.170 --> 00:19:16.753 you should do more of that, and can we 00:19:16.753 --> 00:19:18.270 share it with everybody else. 00:19:18.270 --> 00:19:20.435 And so, I really, in the end kind 00:19:20.435 --> 00:19:22.560 of thought of myself more as a cheerleader for them 00:19:22.560 --> 00:19:25.000 and what they were doing, than someone who was coming in 00:19:25.000 --> 00:19:27.310 are really critiquing anything. 00:19:27.310 --> 00:19:30.190 I also got a lot of great ideas from them, 00:19:30.190 --> 00:19:34.812 both in how to present certain technical content and twists 00:19:34.812 --> 00:19:36.520 on some of the active learning techniques 00:19:36.520 --> 00:19:38.320 that I had suggested. 00:19:38.320 --> 00:19:42.160 So related to that, I'm observing them as individuals, 00:19:42.160 --> 00:19:45.160 but we would also talk a lot as a group 00:19:45.160 --> 00:19:46.870 about how this was going. 00:19:46.870 --> 00:19:49.210 Every staff meeting we'd take time 00:19:49.210 --> 00:19:52.600 to talk about what went well the previous week. 00:19:52.600 --> 00:19:55.150 Our instructors would share active learning ideas 00:19:55.150 --> 00:19:57.920 that they tried, report on how they went. 00:19:57.920 --> 00:19:59.420 This was really nice. 00:19:59.420 --> 00:20:02.410 It got some of my more hesitant instructors kind 00:20:02.410 --> 00:20:04.930 of on board with this, to say like, oh, well, 00:20:04.930 --> 00:20:07.210 that person tried it and it seemed like it went really 00:20:07.210 --> 00:20:07.710 well. 00:20:07.710 --> 00:20:09.640 Maybe I'll try that next time. 00:20:09.640 --> 00:20:11.950 And of course, the thing that turned out 00:20:11.950 --> 00:20:15.490 to be so great about this, is that all of my instructors 00:20:15.490 --> 00:20:18.700 have these amazing ideas on how to teach certain things. 00:20:18.700 --> 00:20:20.710 But to them, it doesn't seem special, 00:20:20.710 --> 00:20:24.130 like this is just always the way I've taught operating systems. 00:20:24.130 --> 00:20:26.410 And so, now we have this space where we're getting 00:20:26.410 --> 00:20:28.030 to share all of these ideas. 00:20:28.030 --> 00:20:30.430 A lot of them relate to how our active learning has gone. 00:20:30.430 --> 00:20:33.070 But sometimes it's just, hey, I taught this paper 00:20:33.070 --> 00:20:35.120 this really cool way. 00:20:35.120 --> 00:20:38.320 It has also-- the fact that we've now kind of fostered 00:20:38.320 --> 00:20:41.290 this space for discussion at every staff meeting, 00:20:41.290 --> 00:20:43.780 everybody is pretty willing to bring up 00:20:43.780 --> 00:20:46.850 things that aren't going as well, to maybe come in and say, 00:20:46.850 --> 00:20:49.030 hey, I tried this active learning thing 00:20:49.030 --> 00:20:50.994 and it did not work at all. 00:20:50.994 --> 00:20:51.910 What do you all think? 00:20:51.910 --> 00:20:54.330 Is there something I could have done differently? 00:20:54.330 --> 00:20:57.360 Is this just not something that will work for our class? 00:20:57.360 --> 00:21:00.040 And so that has been really, really great, 00:21:00.040 --> 00:21:04.330 having this space for my staff to kind of reflect on this 00:21:04.330 --> 00:21:07.450 and not feel judged in any way. 00:21:07.450 --> 00:21:12.250 So this is the first part of getting active learning to work 00:21:12.250 --> 00:21:16.550 in 033, which was a lot of staff training. 00:21:16.550 --> 00:21:18.830 But now the second part-- 00:21:18.830 --> 00:21:20.770 what do we actually do? 00:21:20.770 --> 00:21:23.040 What are these activities? 00:21:23.040 --> 00:21:27.160 I would like to give you some specific examples from 033, 00:21:27.160 --> 00:21:31.240 but I will also try to give you some general feedback as well. 00:21:31.240 --> 00:21:34.420 I realize you probably won't be able to take the 033 activities 00:21:34.420 --> 00:21:38.620 exactly into your classes, so let me talk about some 00:21:38.620 --> 00:21:40.550 of the common things that we do. 00:21:40.550 --> 00:21:43.030 So these are things that we've seen work really well 00:21:43.030 --> 00:21:44.710 in our classes. 00:21:44.710 --> 00:21:48.220 So one of the most common types of active learning that we do 00:21:48.220 --> 00:21:51.070 is one I have already referenced a lot, which 00:21:51.070 --> 00:21:55.810 is to put students in small groups, have them do something, 00:21:55.810 --> 00:21:59.030 and then bring them back for a class-wide discussion. 00:21:59.030 --> 00:22:01.600 So let me give you some really specific examples. 00:22:01.600 --> 00:22:05.110 We have a recitation where we talk about content distribution 00:22:05.110 --> 00:22:06.100 networks. 00:22:06.100 --> 00:22:09.130 The paper they read describes various goals 00:22:09.130 --> 00:22:12.680 of those networks, so performance, reliability, 00:22:12.680 --> 00:22:14.920 scalability, things like this. 00:22:14.920 --> 00:22:16.420 And what we'll do in that recitation 00:22:16.420 --> 00:22:19.300 is we'll divide the class up into groups. 00:22:19.300 --> 00:22:21.310 And we'll have each group think about what 00:22:21.310 --> 00:22:24.310 aspects of the system support a particular goal. 00:22:24.310 --> 00:22:28.120 So you think about performance, you think about reliability, 00:22:28.120 --> 00:22:30.880 you think about scalability, et cetera. 00:22:30.880 --> 00:22:33.430 And then we'll bring the class back to a discussion 00:22:33.430 --> 00:22:35.500 once they've thought about that for a while. 00:22:35.500 --> 00:22:38.920 Another example, when we talk about naming in the Unix file 00:22:38.920 --> 00:22:42.370 system, we start by asking them, what things have names? 00:22:42.370 --> 00:22:43.810 What things in Unix have names? 00:22:43.810 --> 00:22:45.310 And again, we'll put them in groups. 00:22:45.310 --> 00:22:48.370 We'll try to get every group to come up with as many things as 00:22:48.370 --> 00:22:49.510 possible. 00:22:49.510 --> 00:22:51.490 Because when we bring them back together, 00:22:51.490 --> 00:22:53.870 we'll try to get a different thing from every group. 00:22:53.870 --> 00:22:55.670 So you can't all just say files, you 00:22:55.670 --> 00:22:58.660 have to say like files, and directories, and processes. 00:22:58.660 --> 00:23:01.440 And there's a lot of things in Unix that have names. 00:23:01.440 --> 00:23:04.560 I'm not here to make you Unix experts. 00:23:04.560 --> 00:23:07.690 And so, in both of those examples, like I said, 00:23:07.690 --> 00:23:10.570 we'll bring the class back together for a discussion. 00:23:10.570 --> 00:23:14.020 We'll try to have each group say something. 00:23:14.020 --> 00:23:17.860 And as I said before, we've seen very clearly, talking 00:23:17.860 --> 00:23:19.990 in these small groups before hand, 00:23:19.990 --> 00:23:24.430 really makes the shyer students a lot more confident. 00:23:24.430 --> 00:23:27.610 And asking each group to contribute to the discussion 00:23:27.610 --> 00:23:30.520 means that we don't end up with something that is dominated 00:23:30.520 --> 00:23:33.160 by one or two groups. 00:23:33.160 --> 00:23:36.130 Often, we'll even be a little bit more relaxed about it. 00:23:36.130 --> 00:23:40.150 We will just ask a big question, some sort of thought question, 00:23:40.150 --> 00:23:43.265 not just a tell me this fact from the paper. 00:23:43.265 --> 00:23:45.050 We'll break them up in the small groups, 00:23:45.050 --> 00:23:47.050 have them think about it, and come back together 00:23:47.050 --> 00:23:49.570 for a discussion. 00:23:49.570 --> 00:23:51.820 It's nice, it breaks up the recitation. 00:23:51.820 --> 00:23:55.240 It gets them kind of talking and doing things. 00:23:55.240 --> 00:23:59.500 We occasionally will have our TAs teach in this class. 00:23:59.500 --> 00:24:02.510 And we encourage them to use these techniques as well. 00:24:02.510 --> 00:24:04.600 And one of them asked me a really good question 00:24:04.600 --> 00:24:06.430 that I realized I had never thought about. 00:24:06.430 --> 00:24:08.800 She said, what do I do when they're in groups? 00:24:08.800 --> 00:24:12.070 Like, do I just stand there and stare at them? 00:24:12.070 --> 00:24:14.770 Probably not, that's probably not good. 00:24:14.770 --> 00:24:17.970 So to tell you what we do while they're in groups, 00:24:17.970 --> 00:24:20.446 sometimes our instructors might actually check in 00:24:20.446 --> 00:24:21.820 with each group, to say, oh, hey, 00:24:21.820 --> 00:24:23.260 what are you guys thinking about? 00:24:23.260 --> 00:24:25.900 Sometimes, you're kind of lightly monitoring, 00:24:25.900 --> 00:24:28.210 you're walking around. 00:24:28.210 --> 00:24:30.730 I know when I cover recitations, sometimes 00:24:30.730 --> 00:24:33.327 I use it as time to prep for the thing that is coming next. 00:24:33.327 --> 00:24:35.410 So if I need to draw something on the board that's 00:24:35.410 --> 00:24:37.550 going to take a minute, I'll put you all in groups 00:24:37.550 --> 00:24:39.880 so you can discuss while I draw that. 00:24:39.880 --> 00:24:43.330 So this works really well as a technique 00:24:43.330 --> 00:24:45.450 to get students engaged and talking, 00:24:45.450 --> 00:24:48.010 and honestly, it's also nice from a classroom management 00:24:48.010 --> 00:24:48.709 perspective. 00:24:48.709 --> 00:24:51.250 It breaks things up, it gives you a little break from talking 00:24:51.250 --> 00:24:53.390 and you can drink some water. 00:24:53.390 --> 00:24:55.546 So that is kind of our starting point. 00:24:55.546 --> 00:24:57.670 This is one thing where I would say we do something 00:24:57.670 --> 00:25:01.870 like this in almost every single recitation, every Tuesday 00:25:01.870 --> 00:25:03.250 and Thursday. 00:25:03.250 --> 00:25:06.220 The second one is one that I really like, 00:25:06.220 --> 00:25:08.960 which is to have the students debate. 00:25:08.960 --> 00:25:12.160 It's a little bit harder to work into a recitation. 00:25:12.160 --> 00:25:15.010 You need a question that can be debated. 00:25:15.010 --> 00:25:18.280 For us, we have one recitation where the students 00:25:18.280 --> 00:25:20.230 read two short papers. 00:25:20.230 --> 00:25:22.799 And the papers actually come to opposing conclusions. 00:25:22.799 --> 00:25:24.590 And so it's an interesting lesson for them, 00:25:24.590 --> 00:25:26.410 well, how could this happen in science? 00:25:26.410 --> 00:25:30.890 How could data say two different things? 00:25:30.890 --> 00:25:35.640 It is mind-blowing for them, honestly, it's pretty wild. 00:25:35.640 --> 00:25:38.790 And so we'll split them into two teams. 00:25:38.790 --> 00:25:42.280 And we'll give them each a paper, have them debate. 00:25:42.280 --> 00:25:45.130 The students usually really like this. 00:25:45.130 --> 00:25:48.290 They love to argue, so they're very excited to do this. 00:25:48.290 --> 00:25:51.580 It does involve a bit more monitoring. 00:25:51.580 --> 00:25:54.700 You want to make sure that one team, or one person, 00:25:54.700 --> 00:25:56.470 or one team doesn't dominate. 00:25:56.470 --> 00:26:00.040 To combat that, we'll always have the teams converse 00:26:00.040 --> 00:26:02.350 beforehand, kind of get your argument together 00:26:02.350 --> 00:26:05.290 so you're ready to prepare it for the class. 00:26:05.290 --> 00:26:07.600 I personally think something like that could also 00:26:07.600 --> 00:26:09.140 work in small groups. 00:26:09.140 --> 00:26:12.820 Maybe you have groups of four, teams of two, debating. 00:26:12.820 --> 00:26:16.060 I will say, when I've taught this recitation, of these two 00:26:16.060 --> 00:26:19.550 papers, there's like a clear winner in my mind. 00:26:19.550 --> 00:26:21.880 The last time I taught this, the opposing team 00:26:21.880 --> 00:26:24.400 made some very salient points. 00:26:24.400 --> 00:26:27.730 So they really do well with this. 00:26:27.730 --> 00:26:30.370 So this one is one that I think is really fun, 00:26:30.370 --> 00:26:32.920 just a little bit more difficult to work in 00:26:32.920 --> 00:26:35.290 because of what it requires. 00:26:35.290 --> 00:26:37.240 A third one, something that we've actually 00:26:37.240 --> 00:26:41.830 started doing more of this year, is having them draw pictures. 00:26:41.830 --> 00:26:45.850 So we'll ask them, in groups, to draw pictures 00:26:45.850 --> 00:26:48.210 on the board of whatever system we're talking about, 00:26:48.210 --> 00:26:50.790 or maybe some component of that system. 00:26:50.790 --> 00:26:53.350 And what we end up with is a wide variety 00:26:53.350 --> 00:26:56.700 of drawings of varying levels of artistic skill. 00:26:56.700 --> 00:26:59.290 And we can come together and talk about, 00:26:59.290 --> 00:27:01.180 what is each drawing showing? 00:27:01.180 --> 00:27:03.850 What are the common things that we're seeing 00:27:03.850 --> 00:27:05.380 in all of these depictions? 00:27:05.380 --> 00:27:08.270 What level of abstraction is each drawing capturing? 00:27:08.270 --> 00:27:12.490 That's something that comes up a lot for us in 033. 00:27:12.490 --> 00:27:15.280 I like this for a few reasons. 00:27:15.280 --> 00:27:19.420 For 033 specifically, part of our communication curriculum 00:27:19.420 --> 00:27:23.620 involves teaching the students how to design and draw figures. 00:27:23.620 --> 00:27:27.610 So this, for us, this is just a great way for them to practice. 00:27:27.610 --> 00:27:30.970 It also forces them to figure out exactly what the system is 00:27:30.970 --> 00:27:32.000 doing. 00:27:32.000 --> 00:27:33.880 And so we can expose some areas where 00:27:33.880 --> 00:27:35.320 they might have missed something, 00:27:35.320 --> 00:27:36.580 which always happens. 00:27:36.580 --> 00:27:39.520 We expect them to read the papers before they come in. 00:27:39.520 --> 00:27:41.500 They should be able to draw something. 00:27:41.500 --> 00:27:45.570 It might not be perfect, and that gives us a great space 00:27:45.570 --> 00:27:48.540 to explore that, like why was this confusing? 00:27:48.540 --> 00:27:51.400 What is it about the paper that made this confusing? 00:27:51.400 --> 00:27:52.900 It's also just fun. 00:27:52.900 --> 00:27:55.030 So the students get a kick out of it. 00:27:55.030 --> 00:27:59.535 Now the last activity that I want to talk about, this is, 00:27:59.535 --> 00:28:00.910 I'm going to say that this is one 00:28:00.910 --> 00:28:03.010 of our more extreme activities. 00:28:03.010 --> 00:28:05.600 We actually do it pretty frequently. 00:28:05.600 --> 00:28:07.300 We will have students act things out. 00:28:07.300 --> 00:28:09.790 Also, the hardest to make an icon for, 00:28:09.790 --> 00:28:12.250 I just didn't know what to do there. 00:28:12.250 --> 00:28:14.880 They're acting, you can tell. 00:28:14.880 --> 00:28:19.030 Now, I tell people about this a lot, and whenever I say it, 00:28:19.030 --> 00:28:21.670 like, oh, we act things out in 033 all the time, 00:28:21.670 --> 00:28:23.920 they are for sure intrigued. 00:28:23.920 --> 00:28:27.050 They are not sure exactly what that would entail. 00:28:27.050 --> 00:28:30.470 So I was going to have you all do this activity, 00:28:30.470 --> 00:28:34.060 but we're half going to do this activity. 00:28:34.060 --> 00:28:36.220 So I kind of want to go through one of our examples 00:28:36.220 --> 00:28:37.640 actually in detail. 00:28:37.640 --> 00:28:41.302 We have them act out a system called MapReduce. 00:28:41.302 --> 00:28:42.760 But for this to make any sense, you 00:28:42.760 --> 00:28:45.970 need to understand a little bit about what MapReduce does. 00:28:45.970 --> 00:28:48.340 So you're going to become-- 00:28:48.340 --> 00:28:50.280 I don't want to say MapReduce experts, 00:28:50.280 --> 00:28:53.230 MapReduce novices for the next five minutes. 00:28:53.230 --> 00:28:57.090 All right, so, brief aside of technical material. 00:28:57.090 --> 00:28:59.200 MapReduce is this system for performing 00:28:59.200 --> 00:29:01.330 parallel computation. 00:29:01.330 --> 00:29:04.000 And so here's the canonical example. 00:29:04.000 --> 00:29:06.970 We have a large piece of text. 00:29:06.970 --> 00:29:10.240 The Library of Congress has a lot of books in it. 00:29:10.240 --> 00:29:12.520 These books, we're going to digitize this, 00:29:12.520 --> 00:29:14.920 and so the data from these books is spread out 00:29:14.920 --> 00:29:17.590 on all different machines. 00:29:17.590 --> 00:29:19.060 So we have a lot of computers. 00:29:19.060 --> 00:29:21.160 Let's say, you all can be computers. 00:29:21.160 --> 00:29:24.910 So each one of you has a book from the Library of Congress. 00:29:24.910 --> 00:29:27.700 And here's the problem that we want to solve, 00:29:27.700 --> 00:29:29.410 the canonical MapReduce problem. 00:29:29.410 --> 00:29:32.170 We want to count the number of occurrences 00:29:32.170 --> 00:29:34.952 of each word in this entire corpus. 00:29:34.952 --> 00:29:36.910 So I want to know, how many times does the word 00:29:36.910 --> 00:29:38.240 apple appear? 00:29:38.240 --> 00:29:40.480 How many times does the word ball appear? 00:29:40.480 --> 00:29:42.310 How many times does the word the appear? 00:29:42.310 --> 00:29:43.180 OK? 00:29:43.180 --> 00:29:45.119 It's the canonical problem. 00:29:45.119 --> 00:29:47.410 You all don't look like you actually want to know that. 00:29:47.410 --> 00:29:49.930 You do, you want to know how many times the word 00:29:49.930 --> 00:29:52.850 apple occurs in the Library of Congress. 00:29:52.850 --> 00:29:57.400 So you can imagine doing that with one computer, where 00:29:57.400 --> 00:30:00.370 that computer just reads every book in sequence 00:30:00.370 --> 00:30:02.760 and keeps track of these counts. 00:30:02.760 --> 00:30:04.880 And that would be very slow. 00:30:04.880 --> 00:30:06.880 So instead, what we can do is we can have 00:30:06.880 --> 00:30:09.250 multiple computers participate. 00:30:09.250 --> 00:30:12.280 You all have a portion of the Library of Congress. 00:30:12.280 --> 00:30:14.180 You all can read your data. 00:30:14.180 --> 00:30:17.090 You can make accounts for your pieces of data. 00:30:17.090 --> 00:30:19.630 This is the map phase. 00:30:19.630 --> 00:30:23.980 And then we need some way to kind of combine those results. 00:30:23.980 --> 00:30:26.830 So if Dipa finds that the word apple occurred five times 00:30:26.830 --> 00:30:29.150 in her text, if Jesse found that it occurred six times, 00:30:29.150 --> 00:30:30.820 I need some way to add those up. 00:30:30.820 --> 00:30:32.900 That would be the reduce phase. 00:30:32.900 --> 00:30:34.960 And so part of the insight of this system 00:30:34.960 --> 00:30:37.930 is that there are a lot of common problems in computer 00:30:37.930 --> 00:30:40.576 science that can be described this way. 00:30:40.576 --> 00:30:42.200 This is a system that came from Google. 00:30:42.200 --> 00:30:44.350 So Google is doing this on your data all the time. 00:30:44.350 --> 00:30:46.840 They are mapping and reducing everything about you. 00:30:46.840 --> 00:30:49.580 So let's see how that would actually work, 00:30:49.580 --> 00:30:52.320 how this system would work. 00:30:52.320 --> 00:30:55.264 Jessie, come play your role in this thing. 00:30:55.264 --> 00:30:56.680 JESSIE: Do I have to wear the hat? 00:30:56.680 --> 00:31:00.300 PROFESSOR: You do have to wear the hat, it's active learning. 00:31:00.300 --> 00:31:02.200 I've got to write master on it. 00:31:04.840 --> 00:31:07.231 JESSIE: I'm ready. 00:31:07.231 --> 00:31:08.980 PROFESSOR: I think a lot of things in life 00:31:08.980 --> 00:31:10.307 should be more fun. 00:31:10.307 --> 00:31:12.640 So there's really no reason for Jessie to wear this hat. 00:31:12.640 --> 00:31:15.730 But there's also not a reason for her not to wear this hat. 00:31:15.730 --> 00:31:16.230 So-- 00:31:16.230 --> 00:31:18.172 JESSIE: There might be a reason. 00:31:18.172 --> 00:31:22.044 AUDIENCE: This is about who this is fun for. 00:31:22.044 --> 00:31:24.779 JESSIE: Fortunately, I can't see you, so now I feel comfortable. 00:31:24.779 --> 00:31:27.070 PROFESSOR: Well, you can tilt the hat back a little bit 00:31:27.070 --> 00:31:27.760 maybe. 00:31:27.760 --> 00:31:30.820 So in MapReduce, there is the one machine 00:31:30.820 --> 00:31:34.480 that acts as the coordination for this, the master machine, 00:31:34.480 --> 00:31:36.520 as Jessie is so-- 00:31:36.520 --> 00:31:38.960 doing such a great job acting for us. 00:31:38.960 --> 00:31:40.920 And so what the master's job would be-- 00:31:40.920 --> 00:31:42.430 you don't have-- you're just-- 00:31:42.430 --> 00:31:43.900 we just want to visualize you. 00:31:43.900 --> 00:31:45.790 You're just a prop, yes. 00:31:45.790 --> 00:31:47.980 So it would start by assigning jobs to workers. 00:31:47.980 --> 00:31:51.400 Now you all, I didn't make hats for all of you, there wasn't-- 00:31:51.400 --> 00:31:54.610 I know, I'm very sorry, but you're all workers. 00:31:54.610 --> 00:31:56.354 Only the master gets a hat. 00:31:56.354 --> 00:31:57.770 All right, so you are all workers. 00:31:57.770 --> 00:32:01.660 So one of the first things Master Jessie would do, 00:32:01.660 --> 00:32:05.470 would be to assign map jobs to workers. 00:32:05.470 --> 00:32:07.810 And she would tell you things like exactly what data 00:32:07.810 --> 00:32:08.740 to operate on. 00:32:08.740 --> 00:32:12.830 So imagine a scenario where you, a worker machine, 00:32:12.830 --> 00:32:15.664 has data from a ton of different corpuses on it. 00:32:15.664 --> 00:32:17.080 So Jessie actually has to say, OK, 00:32:17.080 --> 00:32:20.267 worry about the Library of Congress part of your data. 00:32:20.267 --> 00:32:21.850 And you would begin operating on that. 00:32:21.850 --> 00:32:24.245 You'd count your words. 00:32:24.245 --> 00:32:27.100 And you'd write out some results to your disk, 00:32:27.100 --> 00:32:29.359 that you also have as a machine in this. 00:32:29.359 --> 00:32:31.900 And then when you were done, you would report back to Jessie. 00:32:31.900 --> 00:32:35.626 And you'd tell her some various meta information, kind of like, 00:32:35.626 --> 00:32:37.000 these are the words that I found. 00:32:37.000 --> 00:32:38.840 You don't have to send her all of your results, 00:32:38.840 --> 00:32:41.464 because the Library of Congress is big, you have a lot of data. 00:32:41.464 --> 00:32:45.280 But you might tell Jessie, hey, I found apple. 00:32:45.280 --> 00:32:48.340 Now, once all of you are finished doing your map jobs, 00:32:48.340 --> 00:32:51.090 Jessie has all of this kind of meta information, 00:32:51.090 --> 00:32:53.882 she can start assigning the reduce jobs. 00:32:53.882 --> 00:32:55.840 And the reduce jobs are a little bit different. 00:32:55.840 --> 00:32:59.205 So Dipa if it was your job to reduce, say, the word apple, 00:32:59.205 --> 00:33:00.580 you need to know a couple things. 00:33:00.580 --> 00:33:02.705 You need to know, well, which one of these machines 00:33:02.705 --> 00:33:04.360 actually found the word apple. 00:33:04.360 --> 00:33:05.830 It's not necessarily they're going 00:33:05.830 --> 00:33:07.051 to be in every piece of data. 00:33:07.051 --> 00:33:08.800 And Jessie would be able to tell you that. 00:33:08.800 --> 00:33:11.650 She'd basically say, hey, Dipa contact these workers, 00:33:11.650 --> 00:33:15.190 they have the data you need, you can add them together, 00:33:15.190 --> 00:33:16.140 in this example. 00:33:16.140 --> 00:33:19.570 MapReduce functions can be much more complex than this. 00:33:19.570 --> 00:33:21.360 I thought let's start slow. 00:33:21.360 --> 00:33:24.940 All right, so the reduce workers would go, you'd reduce, 00:33:24.940 --> 00:33:26.980 you'd report results back to Jessie, 00:33:26.980 --> 00:33:30.359 and then we'd have our full counts. 00:33:30.359 --> 00:33:32.650 I'm trying to decide how long I want you to stand here, 00:33:32.650 --> 00:33:34.900 and the answer is for the rest of the talk, 00:33:34.900 --> 00:33:37.540 but you can almost sit down. 00:33:37.540 --> 00:33:39.290 All right, so this is the basic-- 00:33:39.290 --> 00:33:40.300 JESSIE: I thought I'd have more work to do. 00:33:40.300 --> 00:33:42.008 PROFESSOR: I thought that would be a lot. 00:33:42.008 --> 00:33:44.260 I didn't prep you for that. 00:33:44.260 --> 00:33:45.190 Don't go yet, though. 00:33:45.190 --> 00:33:47.140 JESSIE: I'm just moving over. 00:33:47.140 --> 00:33:49.370 PROFESSOR: So this is the basic idea of MapReduce. 00:33:49.370 --> 00:33:51.380 This is the type of thing we'd actually expect 00:33:51.380 --> 00:33:52.850 the students to know coming in. 00:33:52.850 --> 00:33:55.820 They've read the paper, and they should understand this sort 00:33:55.820 --> 00:33:57.680 of level of coordination. 00:33:57.680 --> 00:34:00.680 But there are a lot of details that we left out. 00:34:00.680 --> 00:34:03.440 For instance, I didn't ask you all, hey, what 00:34:03.440 --> 00:34:04.880 happens if a machine fails? 00:34:04.880 --> 00:34:07.460 Worker Dipa crashes, what do we do? 00:34:07.460 --> 00:34:08.659 AUDIENCE: [INAUDIBLE] 00:34:08.659 --> 00:34:10.310 PROFESSOR: All the time, right? 00:34:10.310 --> 00:34:12.560 Is that data that she was storing, is it 00:34:12.560 --> 00:34:14.120 anywhere else in the system? 00:34:14.120 --> 00:34:15.110 How do we know? 00:34:15.110 --> 00:34:17.600 What if she had half finished the job? 00:34:17.600 --> 00:34:18.659 What do we do? 00:34:18.659 --> 00:34:21.110 Or even, what if there was no crash, but like let's say 00:34:21.110 --> 00:34:25.900 the worker Janet is just running really slow, like she's going, 00:34:25.900 --> 00:34:30.320 she's working hard, but everyone else is finished 00:34:30.320 --> 00:34:32.480 and we're just waiting on Janet. 00:34:32.480 --> 00:34:34.399 And these are those types of things. 00:34:34.399 --> 00:34:36.440 Those are the interesting things about MapReduce. 00:34:36.440 --> 00:34:39.080 That's the point where there are tradeoffs for the students 00:34:39.080 --> 00:34:39.706 to think about. 00:34:39.706 --> 00:34:41.455 So it's not just important to think about, 00:34:41.455 --> 00:34:44.090 oh gosh, what does happen if a machine crashes or is slow, 00:34:44.090 --> 00:34:46.440 but why does it happen that way. 00:34:46.440 --> 00:34:52.040 So to do this, we actually have them act out this system. 00:34:52.040 --> 00:34:53.929 And from the way that I've described it here, 00:34:53.929 --> 00:34:56.330 you might think, well, I have one student be the master 00:34:56.330 --> 00:34:58.100 and everybody else is a worker, that's 00:34:58.100 --> 00:35:00.740 kind of what I was setting you all up for. 00:35:00.740 --> 00:35:03.200 This is not actually how we do it. 00:35:03.200 --> 00:35:06.650 Normally-- for one, it doesn't work with the size 00:35:06.650 --> 00:35:07.860 of our recitations. 00:35:07.860 --> 00:35:11.600 Even here, this is a lot of workers for one student 00:35:11.600 --> 00:35:12.820 to be in charge of. 00:35:12.820 --> 00:35:16.010 And also, if Jessie was maybe more of a shy student, 00:35:16.010 --> 00:35:20.000 she's like this is the worst day of my entire life. 00:35:20.000 --> 00:35:23.090 Katrina has made me wear a hat, and I have to stand-- 00:35:23.090 --> 00:35:26.190 I am in charge of everything, it's not good. 00:35:26.190 --> 00:35:27.140 It's not good. 00:35:27.140 --> 00:35:31.072 So in practice, here's how we do this activity. 00:35:31.072 --> 00:35:32.280 You can sit down if you want. 00:35:32.280 --> 00:35:33.310 You don't have to be here anymore. 00:35:33.310 --> 00:35:34.685 JESSIE: Should I keep the hat on? 00:35:34.685 --> 00:35:36.560 PROFESSOR: I think, definitely, yes. 00:35:36.560 --> 00:35:39.020 I think absolutely. 00:35:39.020 --> 00:35:41.990 So we do start by assigning students roles. 00:35:41.990 --> 00:35:44.100 But we will usually have them in teams. 00:35:44.100 --> 00:35:46.220 So there will be a master team. 00:35:46.220 --> 00:35:49.670 Each worker will be a team of maybe two or three students. 00:35:49.670 --> 00:35:52.950 This is good for a lot of reasons. 00:35:52.950 --> 00:35:56.120 Again, for the reason of getting the shyer students 00:35:56.120 --> 00:35:57.840 to be a little bit more comfortable, 00:35:57.840 --> 00:36:01.010 no one student is in charge of something. 00:36:01.010 --> 00:36:04.220 From a logistical standpoint, again, I 00:36:04.220 --> 00:36:06.980 glazed over many a detail in MapReduce, 00:36:06.980 --> 00:36:09.920 but you can imagine that the master has a lot to do, 00:36:09.920 --> 00:36:11.480 a lot to keep track of. 00:36:11.480 --> 00:36:14.870 And so we will often, we'll have our master team. 00:36:14.870 --> 00:36:17.940 And we'll actually give each student in that team a task. 00:36:17.940 --> 00:36:22.130 So one might run the clock, timing in the system 00:36:22.130 --> 00:36:22.890 is important. 00:36:22.890 --> 00:36:24.920 We'll see an example of that in a second. 00:36:24.920 --> 00:36:28.250 One keeps track of what map tasks to complete. 00:36:28.250 --> 00:36:31.400 One assigns the tasks, things like that. 00:36:31.400 --> 00:36:33.920 We have found-- we've actually done this activity 00:36:33.920 --> 00:36:35.270 a number of times. 00:36:35.270 --> 00:36:36.900 It gets better every time. 00:36:36.900 --> 00:36:39.710 One of the things we learned the most quickly 00:36:39.710 --> 00:36:41.240 is that the master-- we had to give 00:36:41.240 --> 00:36:43.240 the students who were doing the master extremely 00:36:43.240 --> 00:36:44.510 detailed instructions. 00:36:44.510 --> 00:36:47.020 If we just said do the master's job, 00:36:47.020 --> 00:36:48.920 they were not ready for that. 00:36:48.920 --> 00:36:51.230 In some sections, in fact, some of my instructors 00:36:51.230 --> 00:36:54.290 prefer to do this where the instructor just is the master, 00:36:54.290 --> 00:36:57.170 and so they can be kind of in charge of coordination. 00:36:57.170 --> 00:37:00.200 Workers in this example get a little bit of an easier job. 00:37:00.200 --> 00:37:02.810 Again, we're putting them in teams of two to three. 00:37:02.810 --> 00:37:06.050 Sometimes we might give them a sheet with words on it, 00:37:06.050 --> 00:37:08.810 and actually ask them to count the words. 00:37:08.810 --> 00:37:12.050 Sometimes we'll give them a sheet that kind of just 00:37:12.050 --> 00:37:15.290 says here's the words in your data and here are the counts. 00:37:15.290 --> 00:37:18.560 That might seem like, oh, gosh what's the point, 00:37:18.560 --> 00:37:20.930 they're not actually doing the worker job. 00:37:20.930 --> 00:37:24.650 Counting the words in the text is not the interesting part 00:37:24.650 --> 00:37:25.740 of this system. 00:37:25.740 --> 00:37:27.500 So sometimes we'll just skip that. 00:37:27.500 --> 00:37:30.860 It also, again, lets us do some very precise things 00:37:30.860 --> 00:37:32.300 with timing. 00:37:32.300 --> 00:37:34.220 So once these roles are assigned, 00:37:34.220 --> 00:37:36.410 we're going to run the protocol. 00:37:36.410 --> 00:37:39.260 So the person keeping track of the master clock 00:37:39.260 --> 00:37:42.920 will advance it, one click, two clicks, et cetera. 00:37:42.920 --> 00:37:45.680 Various map jobs will complete. 00:37:45.680 --> 00:37:47.930 Most of the time, when we give them a paper that says, 00:37:47.930 --> 00:37:50.390 here's your words and your counts, we'll also say, 00:37:50.390 --> 00:37:53.610 hey, it takes three clock cycles for you to complete. 00:37:53.610 --> 00:37:56.300 And so once the clock has ticked three times, 00:37:56.300 --> 00:37:59.300 they'll report back to the master team. 00:37:59.300 --> 00:38:02.120 Because of how we set up those map tasks, some of them 00:38:02.120 --> 00:38:03.350 will also fail. 00:38:03.350 --> 00:38:06.320 Some of them will have papers that say, oh, I 00:38:06.320 --> 00:38:07.460 failed after click two. 00:38:07.460 --> 00:38:09.740 And they have to announce it to the class. 00:38:09.740 --> 00:38:11.660 And at that point, we pause. 00:38:11.660 --> 00:38:13.260 What happens? 00:38:13.260 --> 00:38:14.870 What happens when a machine fails? 00:38:14.870 --> 00:38:16.470 What does MapReduce do? 00:38:16.470 --> 00:38:18.836 Why does it do that? 00:38:18.836 --> 00:38:21.420 The spoiler for you all in MapReduce, the master 00:38:21.420 --> 00:38:23.300 will reassign that task to somewhere else. 00:38:23.300 --> 00:38:25.370 And part of that challenge is making sure 00:38:25.370 --> 00:38:28.070 that the new worker has access to the data that 00:38:28.070 --> 00:38:29.630 was on the crashed machine. 00:38:29.630 --> 00:38:33.262 MapReduce replicates data, so we talk about how that happens. 00:38:33.262 --> 00:38:35.240 And the protocol continues. 00:38:35.240 --> 00:38:38.630 The students will find that one map task, it hasn't crashed, 00:38:38.630 --> 00:38:40.950 but it's taking a very long time. 00:38:40.950 --> 00:38:43.130 And that's another good point for discussion. 00:38:43.130 --> 00:38:44.600 What does MapReduce do? 00:38:44.600 --> 00:38:46.370 Does it just wait? 00:38:46.370 --> 00:38:49.490 In practice, it restarts the task in another machine. 00:38:49.490 --> 00:38:51.950 And then, so, oh, there's a question, these two machines 00:38:51.950 --> 00:38:53.930 are running the same thing. 00:38:53.930 --> 00:38:54.597 What do we do? 00:38:54.597 --> 00:38:55.430 Who do we listen to? 00:38:55.430 --> 00:38:57.110 How do we get the data? 00:38:57.110 --> 00:39:01.280 And this is why the timing of that activity is so important, 00:39:01.280 --> 00:39:03.740 because a lot of the discussion points about MapReduce 00:39:03.740 --> 00:39:08.150 come from what we do when things are slow or when things fail. 00:39:08.150 --> 00:39:11.640 So being in control of that timing is crucial. 00:39:11.640 --> 00:39:14.780 We do have some variations on this activity though. 00:39:14.780 --> 00:39:18.050 One of our instructors this year had students count characters 00:39:18.050 --> 00:39:20.150 instead of words, same idea. 00:39:20.150 --> 00:39:24.860 But of the workers was counting Russian characters 00:39:24.860 --> 00:39:25.850 instead of English. 00:39:25.850 --> 00:39:28.490 So they just organically were much slower than the rest 00:39:28.490 --> 00:39:29.570 of the class. 00:39:29.570 --> 00:39:32.570 And that was kind of a nice way to show that. 00:39:32.570 --> 00:39:35.030 Many of our instructors have the students 00:39:35.030 --> 00:39:37.490 count types of candy instead of words. 00:39:37.490 --> 00:39:39.770 And we bring them candy. 00:39:39.770 --> 00:39:41.850 That's a big hit. 00:39:41.850 --> 00:39:44.270 In fact, after we did this recitation this year, 00:39:44.270 --> 00:39:47.120 one of our students asked why can't every recitation 00:39:47.120 --> 00:39:48.770 be like the MapReduce one. 00:39:48.770 --> 00:39:51.260 It is not clear how much the candy 00:39:51.260 --> 00:39:53.990 factored into that question. 00:39:53.990 --> 00:39:55.820 So it has taken us quite a few iterations 00:39:55.820 --> 00:39:58.640 to get that activity working. 00:39:58.640 --> 00:40:00.530 And again, I know you're not all teaching 00:40:00.530 --> 00:40:01.760 MapReduce in your courses. 00:40:01.760 --> 00:40:04.010 You're not going to take that activity to you. 00:40:04.010 --> 00:40:06.910 But the things that were interesting for us, 00:40:06.910 --> 00:40:09.950 having the students in teams for each of these roles 00:40:09.950 --> 00:40:12.590 made a huge difference, both in terms 00:40:12.590 --> 00:40:15.252 of making sure they were all doing the correct things, 00:40:15.252 --> 00:40:17.210 but just getting them more comfortable with it. 00:40:17.210 --> 00:40:19.370 I will tell you, the students go for this. 00:40:19.370 --> 00:40:20.450 They are into it. 00:40:20.450 --> 00:40:21.890 They map and reduce. 00:40:21.890 --> 00:40:26.210 They do not think it's stupid, which is very, very nice. 00:40:26.210 --> 00:40:30.080 And making sure that the master has these exceedingly detailed 00:40:30.080 --> 00:40:33.410 instructions, figuring out how to make the timing work, 00:40:33.410 --> 00:40:35.780 so that all of the things we wanted to discuss 00:40:35.780 --> 00:40:38.132 would actually happen. 00:40:38.132 --> 00:40:40.340 And so really the activity is the way I describe it-- 00:40:40.340 --> 00:40:42.710 Karen, this is how Karen does it. 00:40:42.710 --> 00:40:46.170 Karen, you're the authoritative source on how we do MapReduce-- 00:40:46.170 --> 00:40:47.210 yeah. 00:40:47.210 --> 00:40:50.300 I want to talk briefly about another system we act out 00:40:50.300 --> 00:40:51.101 because it's fun. 00:40:51.101 --> 00:40:52.850 It's one of my favorite things that we do. 00:40:52.850 --> 00:40:53.870 It's much shorter. 00:40:53.870 --> 00:40:56.240 We do this for a system called Raft. 00:40:56.240 --> 00:40:59.480 And Raft, like MapReduce is this large distributed system 00:40:59.480 --> 00:41:01.100 on a lot of machines. 00:41:01.100 --> 00:41:03.390 Timing is also important in Raft. 00:41:03.390 --> 00:41:07.350 In particular, machines set randomized timers. 00:41:07.350 --> 00:41:09.830 I picked some random number of seconds, milliseconds, 00:41:09.830 --> 00:41:11.290 whatever. 00:41:11.290 --> 00:41:14.820 And when that timer expires, they do something. 00:41:14.820 --> 00:41:17.990 So being able to act that out is actually really crucial. 00:41:17.990 --> 00:41:21.801 And we thought, gosh, how are we going to randomize them? 00:41:21.801 --> 00:41:23.300 And so one of my instructors came up 00:41:23.300 --> 00:41:26.960 with the best idea, which is to have the students spin around 00:41:26.960 --> 00:41:27.770 in a circle. 00:41:27.770 --> 00:41:30.470 So like literally, they're standing here, 00:41:30.470 --> 00:41:35.600 and then they kind of-- they do this, and it takes a while. 00:41:35.600 --> 00:41:37.970 Because he figured out that students 00:41:37.970 --> 00:41:40.220 just walk around naturally at different paces. 00:41:40.220 --> 00:41:42.240 So they've effectively randomized themselves. 00:41:42.240 --> 00:41:44.750 Someone will finish first. 00:41:44.750 --> 00:41:47.385 So I need you to imagine feel like you're walking around MIT, 00:41:47.385 --> 00:41:48.760 and you're just kind of like, oh, 00:41:48.760 --> 00:41:50.010 what's going on in this class. 00:41:50.010 --> 00:41:52.190 And you see seven or eight students up 00:41:52.190 --> 00:41:54.980 at the front of the room just very quietly spinning around 00:41:54.980 --> 00:41:57.500 in place, while all of the other students are like-- 00:41:57.500 --> 00:41:59.990 the person who finishes first wins. 00:41:59.990 --> 00:42:03.200 And so who-- who is it? 00:42:03.200 --> 00:42:06.170 Who's going to be the next leader of this election 00:42:06.170 --> 00:42:06.950 cycle in Raft. 00:42:06.950 --> 00:42:09.350 It's really-- it looks ridiculous, 00:42:09.350 --> 00:42:10.320 but it works so well. 00:42:10.320 --> 00:42:13.370 And it brings me a lot of joy. 00:42:13.370 --> 00:42:16.130 So to kind of wrap up our activities 00:42:16.130 --> 00:42:18.995 before I get to results, effectively, how 00:42:18.995 --> 00:42:20.510 has this worked for us-- 00:42:20.510 --> 00:42:22.820 again, we do all of these things a lot. 00:42:22.820 --> 00:42:25.040 This one in particular, small groups, 00:42:25.040 --> 00:42:26.660 a class wide discussion-- 00:42:26.660 --> 00:42:29.180 although acting things out, we act a lot of things out now. 00:42:29.180 --> 00:42:30.870 We've gotten pretty good at that. 00:42:30.870 --> 00:42:33.020 And the key component for us has been recognizing 00:42:33.020 --> 00:42:35.300 in all of these cases, the students 00:42:35.300 --> 00:42:37.940 are more comfortable if they're not doing something alone, 00:42:37.940 --> 00:42:40.700 if they have at least one other person on their team doing 00:42:40.700 --> 00:42:44.550 whatever it is they need to do in that recitation. 00:42:44.550 --> 00:42:48.980 And since we do something like this in every recitation, 00:42:48.980 --> 00:42:51.470 the students just come to expect it. 00:42:51.470 --> 00:42:52.660 I mean, they go for it. 00:42:52.660 --> 00:42:55.610 Sometimes, I'll come in and cover a recitation. 00:42:55.610 --> 00:42:58.070 And despite observations, I'm never 00:42:58.070 --> 00:43:01.016 exactly sure how my instructors are leaving things. 00:43:01.016 --> 00:43:02.390 And so, I always start by asking, 00:43:02.390 --> 00:43:04.970 like, hey, I'm going to put you all in groups, 00:43:04.970 --> 00:43:06.890 is that something you're used to. 00:43:06.890 --> 00:43:09.140 Every time, they're like, yeah, we 00:43:09.140 --> 00:43:12.720 do that every day, like of course we can get into groups. 00:43:12.720 --> 00:43:15.650 So again, even acting things out, they really go for it. 00:43:15.650 --> 00:43:18.260 We try to vary the types of activities 00:43:18.260 --> 00:43:20.480 that we do each week, to keep everyone engaged, 00:43:20.480 --> 00:43:22.100 to keep things interesting. 00:43:22.100 --> 00:43:26.050 It's hard to act ever system out. 00:43:26.050 --> 00:43:29.360 And there's occasions where an activity doesn't work. 00:43:29.360 --> 00:43:32.960 The first time we tried MapReduce, it was chaos. 00:43:32.960 --> 00:43:34.917 But we thought it was a cool thing. 00:43:34.917 --> 00:43:35.750 So we just iterated. 00:43:35.750 --> 00:43:37.940 We just tried it again the next semester. 00:43:37.940 --> 00:43:40.280 And in planning all of these, we're 00:43:40.280 --> 00:43:44.390 still very careful to make sure that our activities actually 00:43:44.390 --> 00:43:46.460 teach them things. 00:43:46.460 --> 00:43:48.680 Like I said, all of the recitations 00:43:48.680 --> 00:43:51.550 have technical points that they have to hit on. 00:43:51.550 --> 00:43:53.840 A MapReduce activity where they just 00:43:53.840 --> 00:43:57.950 act out the system in its normal use case, it would be fun, 00:43:57.950 --> 00:43:59.840 but they're supposed to come in knowing that. 00:43:59.840 --> 00:44:03.650 So we really want to design these activities 00:44:03.650 --> 00:44:05.940 so that we're hitting on the points we need to hit on. 00:44:05.940 --> 00:44:08.800 So there is a lot of planning that goes into this. 00:44:08.800 --> 00:44:12.700 So now, for the final part, did it work? 00:44:12.700 --> 00:44:14.520 What got better? 00:44:14.520 --> 00:44:18.640 I'll start with a few actually quantitative things. 00:44:18.640 --> 00:44:21.910 Sometimes when I tell people about this endeavor, 00:44:21.910 --> 00:44:23.720 they want to know, are grades better? 00:44:23.720 --> 00:44:25.270 Have the students' grades improved? 00:44:25.270 --> 00:44:27.320 How are they doing on exams? 00:44:27.320 --> 00:44:30.730 I don't personally think that that's the right question. 00:44:30.730 --> 00:44:33.790 There is one aspect of grading that has improved-- 00:44:33.790 --> 00:44:35.290 participation grades. 00:44:35.290 --> 00:44:38.740 We assign a pretty large portion of the final grade 00:44:38.740 --> 00:44:40.630 to participation in recitation. 00:44:40.630 --> 00:44:42.340 Again, we're asking a lot of them. 00:44:42.340 --> 00:44:44.450 We're expecting them to demonstrate certain skills 00:44:44.450 --> 00:44:46.930 in recitation, so they are graded accordingly. 00:44:46.930 --> 00:44:49.540 And we're seeing students earn higher participation grades 00:44:49.540 --> 00:44:50.260 now. 00:44:50.260 --> 00:44:52.480 I'm comfortable interpreting that as they 00:44:52.480 --> 00:44:54.070 are participating more. 00:44:54.070 --> 00:44:56.140 We are also giving them a lot more opportunities 00:44:56.140 --> 00:44:57.580 to participate. 00:44:57.580 --> 00:44:59.267 Related to that, one of the things 00:44:59.267 --> 00:45:00.850 that we've gained from this experience 00:45:00.850 --> 00:45:03.880 is just a better understanding of what it means 00:45:03.880 --> 00:45:06.490 to participate in our class. 00:45:06.490 --> 00:45:08.890 For a long time to us, it kind of 00:45:08.890 --> 00:45:11.320 meant, well, does this student answer questions 00:45:11.320 --> 00:45:13.240 when I ask them a question. 00:45:13.240 --> 00:45:15.789 But now it means, are they coming prepared to class? 00:45:15.789 --> 00:45:17.080 Are they engaged in group work? 00:45:17.080 --> 00:45:18.580 Are they listening to other students 00:45:18.580 --> 00:45:19.750 and to their instructor? 00:45:19.750 --> 00:45:22.250 Are they participating in class wide discussions? 00:45:22.250 --> 00:45:25.450 There is more opportunities for students to participate. 00:45:25.450 --> 00:45:29.030 We are much clearer about how we assign this grade. 00:45:29.030 --> 00:45:30.940 So not only have the grades improved, 00:45:30.940 --> 00:45:35.760 but the students are noticeably more relaxed about this grade. 00:45:35.760 --> 00:45:37.690 I get fewer complaints about this grade, 00:45:37.690 --> 00:45:39.130 which is great for me. 00:45:39.130 --> 00:45:41.530 And they know that if they come prepared and ready to do 00:45:41.530 --> 00:45:44.007 the work, they'll be fine. 00:45:44.007 --> 00:45:45.590 AUDIENCE: The grade for the recitation 00:45:45.590 --> 00:45:47.850 is separate from the course grade? 00:45:47.850 --> 00:45:50.800 PROFESSOR: It's part of the final grade in the course. 00:45:50.800 --> 00:45:53.020 A percentage of the final grade is from participation 00:45:53.020 --> 00:45:55.430 in recitation. 00:45:55.430 --> 00:45:59.620 So we also surveyed the students during the semester. 00:45:59.620 --> 00:46:01.720 And we were largely asking questions about 00:46:01.720 --> 00:46:05.530 do you feel comfortable in the recitations. 00:46:05.530 --> 00:46:08.380 Largely, yes, they did. 00:46:08.380 --> 00:46:10.660 In our final survey, overwhelmingly the students 00:46:10.660 --> 00:46:14.120 felt that these activities improved their engagement. 00:46:14.120 --> 00:46:16.724 So these things are all good. 00:46:16.724 --> 00:46:18.140 I don't think they really captured 00:46:18.140 --> 00:46:20.960 though what has been so great about this endeavor 00:46:20.960 --> 00:46:22.350 for our class. 00:46:22.350 --> 00:46:25.160 And so in preparing this talk, I spoke 00:46:25.160 --> 00:46:27.050 to a couple of instructors who have been 00:46:27.050 --> 00:46:28.710 with me for a number of years. 00:46:28.710 --> 00:46:31.910 So they've seen the class through its non-active learning 00:46:31.910 --> 00:46:36.620 era, through the dark times, as well as the current era. 00:46:36.620 --> 00:46:39.470 And so I stole a few quotes from them to kind of sum 00:46:39.470 --> 00:46:41.840 up what I think is made this approach better 00:46:41.840 --> 00:46:43.230 for the students. 00:46:43.230 --> 00:46:44.540 So let's turn to number one. 00:46:44.540 --> 00:46:47.120 The first is that I think the students actually understand 00:46:47.120 --> 00:46:48.650 the algorithms better. 00:46:48.650 --> 00:46:50.930 In particular, if we're acting something out, 00:46:50.930 --> 00:46:53.270 it gives them a visceral experience of the finer points 00:46:53.270 --> 00:46:54.800 and edge conditions. 00:46:54.800 --> 00:46:56.660 It's really easy to slide over things 00:46:56.660 --> 00:46:58.460 if one reads a paper quickly, and we're 00:46:58.460 --> 00:47:00.900 asking them to read a lot. 00:47:00.900 --> 00:47:03.720 Second instructor, I think that it's much better when 00:47:03.720 --> 00:47:05.550 there are pairs discussing a solution 00:47:05.550 --> 00:47:07.290 or when we're doing a game. 00:47:07.290 --> 00:47:10.020 People are more likely to notice a flaw in something I've said 00:47:10.020 --> 00:47:11.490 or in someone else's answer. 00:47:11.490 --> 00:47:13.500 And they ask better questions. 00:47:13.500 --> 00:47:15.270 Plus, I think it's reassuring either 00:47:15.270 --> 00:47:17.970 to find out that a classmate is just as confused 00:47:17.970 --> 00:47:21.021 or to find out that a classmate knows a ton about the subject. 00:47:21.021 --> 00:47:22.520 And then the last one, which I think 00:47:22.520 --> 00:47:25.520 really sums up how things have been for our students, 00:47:25.520 --> 00:47:26.590 is this one. 00:47:26.590 --> 00:47:29.090 I think the students are more engaged because they are asked 00:47:29.090 --> 00:47:31.040 to be more active participants. 00:47:31.040 --> 00:47:33.350 There's a greater sense of camaraderie between them 00:47:33.350 --> 00:47:35.790 because of their interactions. 00:47:35.790 --> 00:47:37.880 So we're seeing these students understanding 00:47:37.880 --> 00:47:40.370 the details of the systems better, while developing 00:47:40.370 --> 00:47:41.930 this sense of camaraderie. 00:47:41.930 --> 00:47:44.690 And honestly, that has been huge for us. 00:47:44.690 --> 00:47:46.940 Largely, I would say, from my perspective 00:47:46.940 --> 00:47:48.770 as the person in charge, students 00:47:48.770 --> 00:47:52.280 are buying into this class more than they used to. 00:47:52.280 --> 00:47:55.250 I get fewer complaints across the board. 00:47:55.250 --> 00:47:57.620 We're seeing them work better together 00:47:57.620 --> 00:47:59.430 in their team projects. 00:47:59.430 --> 00:48:01.190 We're seeing them be more engaged 00:48:01.190 --> 00:48:04.820 in lecture and in tutorial, even though recitations was really 00:48:04.820 --> 00:48:07.672 where this active learning push happened. 00:48:07.672 --> 00:48:09.380 And this whole thing has been beneficial, 00:48:09.380 --> 00:48:14.277 not just for the students, but also for the staff. 00:48:14.277 --> 00:48:15.860 There was a little hesitance at first. 00:48:15.860 --> 00:48:18.090 Active learning can be a hard sell. 00:48:18.090 --> 00:48:20.510 It involves more work for everyone. 00:48:20.510 --> 00:48:22.050 It takes a lot of planning. 00:48:22.050 --> 00:48:24.410 It can be a nerve-wracking way to teach if you've never 00:48:24.410 --> 00:48:25.390 taught it before. 00:48:25.390 --> 00:48:27.380 Imagine coming into a room of students 00:48:27.380 --> 00:48:30.032 you don't know very well, and say, OK, five of your 00:48:30.032 --> 00:48:31.490 are going to be on the master team, 00:48:31.490 --> 00:48:34.010 let's go, I made you all hats, right? 00:48:34.010 --> 00:48:38.670 But now, one of my instructors said, 00:48:38.670 --> 00:48:40.490 there's a modest amount of competition 00:48:40.490 --> 00:48:44.180 among the recitation instructors to work out schemes and games. 00:48:44.180 --> 00:48:47.420 So it feels like there's both a reward for applying myself, 00:48:47.420 --> 00:48:50.204 but there's also less need for perfection. 00:48:50.204 --> 00:48:51.620 It's helped that we're encouraging 00:48:51.620 --> 00:48:53.930 active learning is an approach, because it makes 00:48:53.930 --> 00:48:55.580 it feel like less of a risk for me 00:48:55.580 --> 00:49:00.020 to try some new unconventional approach that occurs to me. 00:49:00.020 --> 00:49:03.020 So I'll tell you one of the highlights for me, 00:49:03.020 --> 00:49:05.600 as a person sort of overseeing this operation. 00:49:05.600 --> 00:49:08.182 So this happened at the end of last semester. 00:49:08.182 --> 00:49:09.890 Again, that's the first time we're really 00:49:09.890 --> 00:49:11.634 putting this scheme into place. 00:49:11.634 --> 00:49:13.050 And near the end, my instructors-- 00:49:13.050 --> 00:49:14.810 I got this email from the instructors 00:49:14.810 --> 00:49:17.139 about one of the final recitations. 00:49:17.139 --> 00:49:18.680 And the recitation was about a system 00:49:18.680 --> 00:49:21.740 called DNS SEC, secure DNS. 00:49:21.740 --> 00:49:23.240 So it deals with security. 00:49:23.240 --> 00:49:27.210 DNS prevents certain types of attacks on a system called DNS. 00:49:27.210 --> 00:49:28.910 That's all you need to know DNS SEC. 00:49:28.910 --> 00:49:31.080 So the first instructor described an activity 00:49:31.080 --> 00:49:34.310 they had done, where they had groups draw diagrams 00:49:34.310 --> 00:49:36.729 of their attacks on the board in parallel 00:49:36.729 --> 00:49:38.270 and then talked through each diagram, 00:49:38.270 --> 00:49:41.680 with the students explaining the problems and the approaches. 00:49:41.680 --> 00:49:44.330 And the instructor noted that they engaged, 00:49:44.330 --> 00:49:47.330 they laughed, they collaborated, they learned more about 00:49:47.330 --> 00:49:50.480 how design figures to tell the story they wanted to tell. 00:49:50.480 --> 00:49:52.700 And I got this email, and I was so excited. 00:49:52.700 --> 00:49:55.280 Because this was not an activity that I had planned for them. 00:49:55.280 --> 00:49:57.120 This is just something they had done. 00:49:57.120 --> 00:50:00.320 And I thought, it's happened, finally it's happened. 00:50:00.320 --> 00:50:01.910 And then I get a second email. 00:50:01.910 --> 00:50:03.890 And so a second instructor responded 00:50:03.890 --> 00:50:05.600 saying that that sounded great. 00:50:05.600 --> 00:50:08.030 And what they had done, they'd had a group of students 00:50:08.030 --> 00:50:11.270 work out how DNS would work for a particular scenario, so 00:50:11.270 --> 00:50:13.040 the original system. 00:50:13.040 --> 00:50:15.950 And then two additional groups joined that first group, 00:50:15.950 --> 00:50:18.320 worked through how different attacks would 00:50:18.320 --> 00:50:19.980 affect that scenario. 00:50:19.980 --> 00:50:22.460 And then a fourth group acted as DNS SEC, 00:50:22.460 --> 00:50:25.770 and explain how DNS SEC would prevent such attacks. 00:50:25.770 --> 00:50:28.930 So to brand new ideas, both really good, 00:50:28.930 --> 00:50:30.050 both really different. 00:50:30.050 --> 00:50:31.460 And this is so exciting. 00:50:31.460 --> 00:50:33.560 And then we get to instructor number three. 00:50:33.560 --> 00:50:35.180 And I've never been able to figure out 00:50:35.180 --> 00:50:37.160 a way to summarize this email. 00:50:37.160 --> 00:50:39.950 So I have to quote it for you directly. 00:50:39.950 --> 00:50:42.320 "In my recitation, we had a marriage proposal 00:50:42.320 --> 00:50:45.290 that went bad when a romantic rival launched an attack 00:50:45.290 --> 00:50:46.910 to destroy the relationship. 00:50:46.910 --> 00:50:48.710 A second group used a different attack 00:50:48.710 --> 00:50:52.070 to induce peasants to send their taxes to a scammer. 00:50:52.070 --> 00:50:53.840 All of these attacks were then prevented 00:50:53.840 --> 00:50:55.850 by the use of DNS SEC. 00:50:55.850 --> 00:50:59.540 After each play, we did a replay with deep structure analysis 00:50:59.540 --> 00:51:03.080 that made the analogy between the play and DNS explicit. 00:51:03.080 --> 00:51:05.360 Overall, a pretty fun recitation and I 00:51:05.360 --> 00:51:07.760 was impressed with the creativity, enthusiasm, 00:51:07.760 --> 00:51:11.430 and acting talent the students displayed." 00:51:11.430 --> 00:51:12.507 So this was amazing. 00:51:12.507 --> 00:51:14.090 This was like such-- this email thread 00:51:14.090 --> 00:51:15.590 is just such a triumph for me. 00:51:15.590 --> 00:51:18.770 I have it highlighted and saved in my email. 00:51:18.770 --> 00:51:21.350 And this has continued on since last semester. 00:51:21.350 --> 00:51:23.330 I get like one of these email threads a week 00:51:23.330 --> 00:51:26.540 now, with my instructor telling me all of the cool things 00:51:26.540 --> 00:51:28.250 that they have done. 00:51:28.250 --> 00:51:32.390 So the last thing, which I hope has come through in this talk, 00:51:32.390 --> 00:51:36.020 it was a lot of work to get this to happen in 033. 00:51:36.020 --> 00:51:39.500 I cannot in good faith stand here tell you active learning 00:51:39.500 --> 00:51:42.390 is a breeze, you can just add it in, five minutes of prep time 00:51:42.390 --> 00:51:43.340 before the class. 00:51:43.340 --> 00:51:44.375 It's a lot of work. 00:51:44.375 --> 00:51:47.810 But this class is so much fun now. 00:51:47.810 --> 00:51:49.610 It's fun for me to run. 00:51:49.610 --> 00:51:51.149 It's fun for instructors to teach. 00:51:51.149 --> 00:51:52.940 I don't know how many people would tell you 00:51:52.940 --> 00:51:56.350 that their 400-person class is fun to run. 00:51:56.350 --> 00:51:57.910 But I have a great time. 00:51:57.910 --> 00:52:01.340 And the amount of enjoyment that we get out of teaching 033 this 00:52:01.340 --> 00:52:04.160 way really comes through for the students. 00:52:04.160 --> 00:52:07.250 And I think that's another component of what has helped 00:52:07.250 --> 00:52:09.990 them engage more in this class. 00:52:09.990 --> 00:52:11.790 So it has worked really well for us. 00:52:11.790 --> 00:52:14.150 I hope that you get some practical things that you can 00:52:14.150 --> 00:52:15.750 try out in your own classes. 00:52:15.750 --> 00:52:19.310 And I'm happy to take questions or to delegate questions off 00:52:19.310 --> 00:52:21.820 to Jesse and Karen.