End of Quarter Make-up Assignments

Make-up work policy is one of those classic problems of practice in teaching. There is a TON of important thinking to do, and not a lot of straightforward, always-true directions to go in. But there has got to be some serious potential value to be gained from understanding how to do meaningful make-up work. I tried something new this year, and it had a solid positive impact, so I wanted to reflect on it here.

What I've done in the past

• I usually have a "no make-up work" policy. Instead, I leave assignments and grades open for a super long time, and if kids need to make something up, they can just do the assignment they were supposed to do in the first place.
• I like this because it's a pretty straightforward workflow for me. No need to make or grade extra work. I also value the way this policy doesn't create the opportunity for students to intentionally drop an assignment, in expectation of a make-up assignment (assumed to be 'easier').
• A down-side to this policy is that sometimes the make-up work doesn't translate well from how it was executed in class to how a student would make it up, after the fact. For example, when designing tasks, I try to aim for a challenge level above "what most students can do alone" but below "what most students could figure out with a group of peers." But more often than not, when a student is completing make-up work, they're doing so individually, and so challenge level is often mis-matched. Similarly, I'll often design tasks that reward student collaboration, which again limits what a student can do if they're making up work individually.

The new thing I tried this year

• I kept my normal make-up work policy. But I also added an "End of Quarter Make-up Assignment" at the end of Q3. Here is a copy of the assignment.
• At the end of third quarter, I was looking at my grades and saw that they were surprisingly low. In particular, I saw that there were many otherwise well-performing students who experienced sudden drops from Q1/Q2 to Q3. Given that we'd done a major school-wide schedule shift Q3, and that this year was a mess in general, I figured that many students may have just fumbled a few assignments. Inspired by a colleague, I made a "Q3 Make-up Assignment" for the spring break between Q3 and Q4.
• To make the assignment, I went through all ~10 assignments in Q3 (all individual Desmos activities), and picked out the four most important assignments. I then just pasted them together as one big assignment, one big Desmos activity. I emailed students individually (using a mail-merge), letting them know about the assignment, and the positive impact it could have on their grade. I contacted their folks, letting them know that the assignment was available, and how valuable it could be for their progress in the class.
• On the last day of Q3, before spring break, I presented it in class to the subgroup of students who were not passing the class, or close to not passing. I didn't want to present it to anyone who didn't "need" it, in an effort to minimize the negative effect that make-up work sometimes has (students may rather wait for the make-up assignment).
• The assignment was due the first day back after spring break. I generally don't assign work to be completed over break. However, given that this was an extra make-up assignment, I figured it was as actually a good a time, so students could get caught up without continuing to fall behind. I ended up letting the due date roll for about a month *after* the intended due date, anyways. That said, 90% of students who completed the make-up assignment completed it over spring break anyways. (Heck, almost 50% completed it within 48 hours of being assigned.)

Important ideas when planning for this

• This is not supposed to be a major intervention for students who are majorly not on track in the whole class. A one-off assignment, done independently over break, is not going to get that job done. However, there are often some students who just need a quick "tune-up." These are students who typically can complete work independently, but maybe didn't get a few important assignments done because of executive dysfunction, or just logistics. So let me make one big straightforward assignment that has them do all that work, but hopefully meaningfully reduces the executive demand.
• Don't make new work. At most, just copy-paste the biggest chunks of important assignments from that term. This is important, because the actual yield on this "intervention" isn't necessarily gonna be that high.
• Use a mail-merge to send customized emails to students identifying their current grade, and grade if they complete the make-up assignment. A lot of times, grades and make-up assignments can be overwhelming, and students can get discouraged, even if getting caught up is super doable. If students can unambiguously see the positive impact of this assignment on something that's important to them (their grade), it may be easier for them to get motivated.
• Contact families, letting them know that I have prepared this make-up assignment for their student, when it's due, and what impact it could have on their grade. Family contact is always a little more complicated, so modify as needed. But this definitely improved the traction this assignment got.
• This assignment is optional, technically. If completed, I'll go back and update grades to reflect the newly complete work. But if a student doesn't complete the assignment, their grade won't drop further.

Important questions I still have

• When designing this make-up assignment as I have, I am assuming that some students have low grades for reasons unrelated to the actual content. I'm assuming that some students simply need me to strip away as many non-content barriers (logistics, due dates, coming to class at the right time, managing multiple assignments) as possible. In this way, I am accommodating a need that many students have for a reduced demand for executive functioning.
• How can I prepare, launch, and support this make-up assignment in a way that actually builds student capacity for executive functioning? The way I did it this year was very much a "give someone a fish" kind of support, as opposed to a "teach them to fish" support. What do I need to understand about the work, and my students, to understand how much of this particular struggle is productive? 
• Idea: instead of me reaching out to families to let them know, as a way of almost strong-arming students into getting it done, find a way to get students to bring it up with their folks on their own, so the message is coming from them. In that way, the students are managing their own accountability to their families.
• Idea: have a simple way of customizing how the assignment is put together. It'd be great if kids could see the full menu of activities, look at their own grades, and then go up and pick the 3-4 tasks they need to work on over break. I can almost see it: students walk into class, look at their individual grade report, and highlight the 3-4 most important assignments. They then go to the front of the room where the tasks are laid out in straightforward packets of some kind, and they assemble their own packet of make-up work. I guess the ultimate goal for this is to make this always available...like, there's no reason this can only happen right before breaks, or even that everyone needs to do it at the same time.
• This assignment is a "middle of the road" intervention, targeted at students who are not passing simply because they haven't kept up with assignments. This is not a good fit for students who aren't passing because there are major chunks of learning that haven't yet happened.
• Question: what's the intervention for those students? I ask this almost rhetorically--I know there isn't typically a one-shot intervention that will help students in that situation, and instead it's more about having a strong big-picture system, with lots of different layers of support.
• Question: What's the fair/effective way to determine which kids fit this intervention?
• What about the students who aren't passing, want to do this assignment independently over break, but upon attempting it realize the assignment is out of the range of what they can complete independently? That's a discouraging feeling, with negligible payoff, so I'd just as soon a student not have to experience it.
• When compiling work to put into a the Q3 make-up assignment, here's the idea I was working from: "If a student can complete all four of these activities, and do well, they should pass the quarter." I only ended up doing this after Q3. But what if I had done it after every quarter? Then I would theoretically have four big make-up assignments, each roughly "worth" a quarter of work. If a student did *only* the quarterly make-up assignments, but did them well...how would I feel about them passing the class?
• Answer: I'd feel poorly about that. I'd be discouraged that a full-year course was distilled to ~16 tasks which could be completed independently and asynchronously. I'd feel like I'd gone all in on the "transactional" nature of a lot of modern high school education. You give me a certain amount of work, I trade that for a credit. You accumulate those credits, and trade them for a degree.

Big picture take away

• This last point is an important (read: HUGE) caveat here. And it helps me to understand what the role of this intervention is. It's not a magic fix-all (unsurprising). We can maximize the positive effects, and minimize the negative effects, by targeting it at the students whose situations match what this end-of-quarter make-up assignment is meant to help with. Understanding exactly who those students are, and how to run this intervention with them, requires me to have a deep understanding of what the intervention does (and doesn't do). It also requires to me to try and understand my students as much as possible, and what they need (and don't need).

I guess I shouldn't be surprised that this big picture take away is actually not a terrible philosophy to have around interventions in general. Which is both discouraging and reassuring at the same time. At the very least, however, I am glad I tried this out this year, though I don't think this is going to be an "every quarter, every year" kind of thing. I can add this to my "teacher toolbox," in case I am faced with a similar context in the future.

Student Perception Survey (Fall 2020)

Three times a year I give my students a Student Perception Survey, to see how they're experiencing my class. I wrote in depth here (link) about what the surveys do, and how I use them. In an effort to process and reflect on my first doing of the survey last week, I thought it'd be helpful for me to publish my reflections here.

LOW-INFERENCE ANALYSIS OF RESULTS

• All together, I've got 96 responses, out of my 126 students across 5 sections.
• All 5 sections completed the survey at approximately the same rate.
• For the most part, the students completed the survey if and only if they were present on the day I gave the survey. It was open on Google Classroom, so a couple submitted it after.
• On the day we gave the survey, there were 96 students present (out of 126)
• For the 96 who completed, this is when they completed the survey
• 5 = day before (I posted to Classroom a day early)
• 73 = day we did it in class together
• 17 = 1-12 days after (within 12 days, as of this writing)
• Most students who started the survey responded to all the items, though there were a few students who skipped some questions.
• Performance by category, aggregated across all sections
• Scores
• Captivates 73%
• Challenges 78%
• Cares 73%
• Clarifies 77%
• Classroom Management 89%
• Consolidates 76%
• Confers 77%
• I notice that except for Classroom Management, everything else falls within a pretty narrow range of 73-78%.
• Performance by section, aggregated across all items
• My overall rate of teacher favorable responses in each section was 75%, 78%, 79%, 80%, 81%.
• Looking at category averages for each section
• My lowest performing category/section was 64% in Captivates for one section.
• My highest performing category/section was 93% for Classroom Management in a different section.
• Looking at performance on individual items, with data aggregated across all sections:
• Highest: 97%: Students in this class treat the teacher with respect (Classroom Management)
• Lowest: 45%: My teacher seems to know if something is bothering me (Cares)
• I had comparable performance between the questions that are positively framed ("My teacher knows when the class understands, and when we do not") and some questions are negatively framed ("When s/he is teaching us, my teacher thinks we understand even when we don't"), getting 79% and 77% teacher-favorable.
• When comparing class averages for each item, looking at the spread between different sections:
• Biggest spread: "My teacher wants me to explain my answers--why I think what I think." The five sections responded 63%, 78%, 81%, 81%, 100% --> Spread = 37%.
• Smallest spread: "My teacher in this class makes me feel that s/he really cares about me." The five sections responded 87%, 89%, 89%, 90%, 90% --> Spread = 3%.

MAKING INFERENCES

• Usually Classroom Management is one of the lower categories for me, whereas now it's especially high. Given that we are fully virtual, and there isn't so much of a "classroom" to manage, I guess this isn't surprising?
• Consider the classic situation: we're waiting for the whole class to be quiet so I can share some important info, it's taking a long time, so I start to use my body language to communicate that I'm frustrated, concerned, or disappointed. Eventually everyone quiets down, and we get started, and we move on with our lives. But that experience might trigger responses to any of these items that are not teacher-favorable:
• Students in this class treat the teacher with respect.
• Student behavior in this class is a problem.
• I hate the way that students behave in this class.
• My classmates behave the way my teacher wants them to.
• Student behavior in this class makes the teacher angry.
• It's a lot easier to mute a kid on Zoom, or push them out to the waiting room. To be honest, though, I've only had to do that once. Usually for students to be engaged in off-task behavior, there has to be some kind of off-task behavior to engage in. I'm finding that there is just less engagement, less exchange, on Zoom, and so we aren't getting disruptions from students, or negativity from me in response.
• Of the three items in the Cares section, two of them are pretty good.
• First two items
• 89%: My teacher in this class makes me feel that s/he really cares about me.
• 86%: My teacher really tries to understand how students feel about things.
• But the third item 
• 45%: My teacher seems to know if something is bothering me.
• Last year, my colleagues and I spent a lot of time thinking about this items. It raised a lot of questions around whether this item was a reflection of the teacher, at least in the same way that the other items are. And I agree it's a complicated question.
• Even though my completion rate was decent (96/126 = 76%) it's still a biased survey, because it's sampling only the students who are present, and are typically present on a random Monday. A more complete survey would capture the experiences of all the kids, including those who do not attend class or complete work consistently. This is a consequence of administering the survey in the exact same method that we deliver classwork (in class, through a link I post, etc.).

FOCUSING IN: PROBLEM OF PRACTICE

• In this first cycle of the student perception survey, I'd like to focus on the category "Clarifies." This is because it contains two lowest performing items (57% and 68%), while also containing two high performing items (89% and 91%).
• This is especially interesting because the low items and the high items seem to contradict, at least at first. Basically, it seems like my students are saying "Mr. St. checks to see if we understand, and if he thinks we don't understand he'll explain it another way....BUT, even though you are checking, you can't always tell that we don't understand."

• The flow of assessment should be that there is a learning opportunity (red), the teacher then assesses the students (yellow), and then the teacher analyzes student responses (green). At that point, the teacher needs to make the assessment that student either "got it" (blue) or "don't got it" (grey).
• Student responses point out a gap in the flow: I may be categorizing more students as having "got it", or categorizing the class as a whole as "got it", and then moving forward. So where could this perception be coming from?
• Possible explanations
• I might be setting the bar of "ok and ready to move on" lower than students are comfortable with.
• This might makes some sense. I'm feeling some pressure to move through a curriculum at a given pace, and so want to move through things. I often feel like I'm already slowing things down to be too slow (and so boring). In fact, given that only 62% of students responded favorably to "This class does not keep my attention--I get bored."
• In response, however, I have 79% of students agreeing that "In this class, we learn a lot almost every day." Since 79% > 62%, that suggests I could slow things down a bit, and trade a bit between those two?
• Students might be more ready to move forward than they feel.
• Students aren't getting papers back with scores on them. I am very regular, consistent, and frequent with using our gradebook to communicate progress and performance. But if students aren't attending to that data being provided in their gradebook, it's not going to actually lead them to adjust their perceptions of themselves and their own understanding.
• How can I help students see the progress that they ARE making? I'll often have moments where we all go into our gradebooks, look at the grades there, and then process them, trying to internalize what we need to be taking care of. But that takes a HUGE amount of time, EVERY time, and it DOESN'T feel like we usually get much out of it. Do I have evidence that this practice has little impact? During Review Week, when we did this all week, it resulted in very little overall improvement of grades, signaling little positive impact that week. Is this something where we'll just have to bank on a longer-term impact?
• My assessments might not be fully aligned to the work itself, and resulting in a higher rate of false positives. So even if students perform well enough on the assessment to signal readiness to move forward, it might not be a reliable enough indicator.
• As I mentioned above, I often feel discouraged by how slow it feels like we're moving through things, and how hard-won every little bit of progress is. It is possible that this pushes me to see what modicum of progress we HAVE made, and over-reward that with higher grades.
• The goal is to be encouraging, and to recognize that school is harder now than it has been in a long time. But the flip side of that practice is that it creates data interference, reducing the reliability with which the data signals readiness and understanding. This is a big part of the distinction between grades as generated by a traditional model, and "grades" as generated by standards-based grading (which I have committed to, at least in the way that my school has).
• Action Steps
• Be more transparent with students about the degree to which assessments indicate a need to go back over a topic, or to continue to study a topic. In particular, I can:
• When a week's work has provided evidence that we need to extend the pacing, I can communicate that with students, and so help them see how what they do impacts what we do as a class.
• "Hey folks, I was looking over our performance from last week, and it looks like we're going to need to spend a little more time working through [TOPIC]. So we've got this week's problem set that is going to look at this idea in another way."
• Be more clear (with myself and my students), about how I am grading.
• Our standards-based grading system allows for assignments that are rooted in standards, as well other assignments that are considered "practice." They are less targeted than standards-based assignments, and a bit more like the traditional grading model.
• Let the standards-based assessments reflect performance on the standards. Let the "practice" assignments reflect "practice," and if necessary use those assignments to buoy morale, while allowing standards-based grades to guide instruction.

If you have any questions, thoughts, or feelings about any of this, I'd love to hear them. I'll report back some time in February with the second round of survey results!

Designing PSETs for Remote Learning

This past spring we had to jump into remote learning. We were all figuring it out as we went along. I happened to be teaching two very different math classes at the time, which were being taught in very different ways. As such, I was able to experiment with two very different approaches.

I wrote about what I tried and learned during remote learning here. One of my preps was Integrated Math 1 (~CPM). It went fine. My other prep was a discovery-based math elective I developed (you can read about it here). This elective seemed to make the jump to remote learning much better than Math 1. I wrote about how I modified it for remote learning here. But the big idea was using weekly problem sets (PSETs). That's what I plan on doing this fall. So I want to take an opportunity here to reflect on what I think PSETs can do for remote learning specifically, and why.

What is a PSET?

• Here is a model PSET I made. Here are the major features
• Big Problem: the big complicated problem at the center of the PSET. Everyone works on it. Most of our class discussion is focused on it.
• Four Sections: Big Problem, Related Problems, "Not" Related Problems, and Going Deeper
• Variety: Different kinds of problems, of different sizes, different topics, different goals
• Content Thread: All the problems have a "Different Surface -- Same Depth". They appear different, but are all perspective on the same central Big Idea, that weaves through all the problems, like a thread.
• Students are given a PSET at the beginning of the week, and they have the week to do a week's-worth of work. They upload all their work in Google Classroom at the end of the week.

PSETs Adapt Well to Remote Learning

• More Choice → More Engagement
• Except for spending a portion of their time on the Big Problem, students can spend as much or little time as they want on whichever problems they want. If I made the PSET right, and they do a week's-worth of work, they should work with the Big Idea no matter what path they choose.
• This increases the odds that students are relatively more interested in the work they do, b/c they're picking from a bunch of problems. Moreover, just the process of picking problems will inherently build commitment...b/c they chose it!
• This also allows them to remain sensitive to their own readiness to engage in more complicated discussions out of self-preservation. If I had a PSET on exponential functions, I could totally include a problem on the COVID-19 and disease spread. But that could also be super traumatic for some students. So instead I make sure there are problems that also hit the same Big Ideas, but aren't as heavy, in case they don't want to think too much about the global pandemic. And if some kids *do* want to learn more about it, if they're ready, there it is.
• Focus on fewer bigger ideas → more efficient
• Learning has a kind of "efficiency ratio." We engage in some quantity of "doing," and some smaller quantity of it sticks as "learning." The rate at which that transfer from doing to learning is a function of a lot of factors, only some of which students and teachers can directly impact.
• Stress, new technology, different learning environment, health and wellness, and change in and of itself, are all factors that are greatly impacting students' "efficiency ratios." Kids are also being asked to quickly learn a ton of non-academic stuff. They're learning about the world, health, their own hearts, how to do emails, how to sustain relationships from afar, how to manage sickness, and a million other incredibly important things. And as far as academic learning goes, even if students are able to create space for it, it's reasonable to expect that this "efficiency ratio" is lower than normal.
• As such, it's more important than ever for is to answer the question: what are we willing to trade? Assuming we were operating anywhere near the frontier of performance and capacity for learning, we are going to need to reallocate capacity away from academic content, and toward surviving in These Uncertain Times.
• Option 1: Distance Modulation. Go into the same depth, but get through fewer ideas, modulating the distance you get based on your community's capacity.

• Option 2: Depth Modulation. Go through all the same ideas, but don't go as deep, modulating the depth you study based on your community's capacity.

• I claim that depth modulation is the way to go here. Because U.S. math education has built itself around the idea of prerequisites, and because we know the U.S. is going to push schools to push kids to get "caught up" (an oppressive and garbage expectation), it'll be a lot easier to bridge this time if kids at least have exposure.
• PSETs also have a lot of other mechanisms that allow for differentiation for readiness, and I talk a bit about them in this post. But the big idea is that it's easier for a bunch of kids to be working on the same content to different depths, as opposed to working on totally different content.
• All the problems packed into one paper → increased need/flexibility in product
• A long-term goal is that students learn how to do math "anywhere." On a whiteboard, window, piece of paper, spreadsheet...wherever is convenient and appropriate for the math itself. Part of learning how to do math is learning "where" you should do the math. Is this a small problem I can scrawl on a napkin? Is this going to make a bunch of data I'll probably want to put into a spreadsheet? Am I going to need a bunch of blank paper? Grid paper? By making it impossible to do the math *where* the problems are assigned, we force students to make a decision about when/where/how to do the math. This definitely takes up cognitive energy and work time, especially if they don't make the most efficient choice. But that's an instructional trade that feels worth making.
• This becomes especially useful during remote learning for a lot of reasons. Computers are more available than ever. So math on the computer becomes accessible anew. Not everyone knew to bring their notebook home from school. My kids this fall may not have one (we tried to pass them out at orientation, but I doubt that most kids were able to pick one up).
• Do the picture on some paper, or in your notebook, and upload a picture. Did something on desmos.com/calculator? Upload a screenshot or upload the link to share it. Made something with blocks? Take a picture and upload it. Google Classroom does decent job of collecting and archiving work. It's pretty easy to leave comments on things. I push it back for revisions? Do some work and upload pictures of that! Now we have a "paper trail" for revisions--something that's not always easy to do when passing a paper assignment back and forth.
• This is definitely going to raise the demand for teaching students how to use math tools like spreadsheets, graphing calculators, and google docs. Seems worth it though.
• You've got a week to do a week's-worth of work → honor increased demand for time flexibility
• I remember video conferencing with a student about some work a few months ago. As they were talking to me, their little sibling was running around, asking for food and attention. At one point, the kid was literally climbing my student (as two year olds are wont to do). When you're at home, it's not always as easy to get a whole hour dedicated to a task, without distraction. Throw in a dash of the year 2020 and there may be entire days where you can't set much time to work. I get it. And any adult that doesn't has to seriously do some empathy building.
• The body of mathematical knowledge isn't neatly packaged into 54-minute units of knowledge. Some things take 10 minutes to learn. Some problems take 4 hours to work through. So I'm going to give you a bunch of math to do, and you do a week's-worth of work by the end of the week. Do what you can, when you can.
• As a teacher, I am professional. And that comes with the recognition that all that matters it that I get the job done, by the time it needs to get done. I think about how disrespected I feel when PD is super prescriptive with how I spend my time. It feels important to extend that opportunity to students whenever possible. In regular face-to-face school time, that's more difficult, b/c we have 500 kids to rotate through 40 teachers, spanning 6 different courses. And very few people in power are willing to invest in schools so that they can be more humane in their scheduling. So we have to make some sacrifices for large-scale efficiency.
• Some students will need a lot of support and scaffolds for how to manage that expectation. And I'm prepared to do that. I'll recommend for some kids that the set a time for 60 minutes, sit down at the kitchen table, and crush some problems till the timer goes off. Some kids I'll say set a count-down for 5 hours, and then start it whenever they're doing math, and pause it when they stop. Again, this seems worth the instructional time it will take. Through continued conversation, reflection, and experimentation, students will learn what they need to do the learning--to get the job done. And if that's not the most transferable learning objective, I don't know what is.

PSETs do a lot of things. They're also a major area of pedagogical exploration for me for the last year and a half, for a lot of reasons. So maybe they're the hammer to my remote learning nail. But they are a remarkably flexible approach to math instruction, and now is definitely a time that demands flexibility.