Student Perception Survey (Fall 2020)

Three times a year I give my students a Student Perception Survey, to see how they're experiencing my class. I wrote in depth here (link) about what the surveys do, and how I use them. In an effort to process and reflect on my first doing of the survey last week, I thought it'd be helpful for me to publish my reflections here.

LOW-INFERENCE ANALYSIS OF RESULTS

• All together, I've got 96 responses, out of my 126 students across 5 sections.
• All 5 sections completed the survey at approximately the same rate.
• For the most part, the students completed the survey if and only if they were present on the day I gave the survey. It was open on Google Classroom, so a couple submitted it after.
• On the day we gave the survey, there were 96 students present (out of 126)
• For the 96 who completed, this is when they completed the survey
• 5 = day before (I posted to Classroom a day early)
• 73 = day we did it in class together
• 17 = 1-12 days after (within 12 days, as of this writing)
• Most students who started the survey responded to all the items, though there were a few students who skipped some questions.
• Performance by category, aggregated across all sections
• Scores
• Captivates 73%
• Challenges 78%
• Cares 73%
• Clarifies 77%
• Classroom Management 89%
• Consolidates 76%
• Confers 77%
• I notice that except for Classroom Management, everything else falls within a pretty narrow range of 73-78%.
• Performance by section, aggregated across all items
• My overall rate of teacher favorable responses in each section was 75%, 78%, 79%, 80%, 81%.
• Looking at category averages for each section
• My lowest performing category/section was 64% in Captivates for one section.
• My highest performing category/section was 93% for Classroom Management in a different section.
• Looking at performance on individual items, with data aggregated across all sections:
• Highest: 97%: Students in this class treat the teacher with respect (Classroom Management)
• Lowest: 45%: My teacher seems to know if something is bothering me (Cares)
• I had comparable performance between the questions that are positively framed ("My teacher knows when the class understands, and when we do not") and some questions are negatively framed ("When s/he is teaching us, my teacher thinks we understand even when we don't"), getting 79% and 77% teacher-favorable.
• When comparing class averages for each item, looking at the spread between different sections:
• Biggest spread: "My teacher wants me to explain my answers--why I think what I think." The five sections responded 63%, 78%, 81%, 81%, 100% --> Spread = 37%.
• Smallest spread: "My teacher in this class makes me feel that s/he really cares about me." The five sections responded 87%, 89%, 89%, 90%, 90% --> Spread = 3%.

MAKING INFERENCES

• Usually Classroom Management is one of the lower categories for me, whereas now it's especially high. Given that we are fully virtual, and there isn't so much of a "classroom" to manage, I guess this isn't surprising?
• Consider the classic situation: we're waiting for the whole class to be quiet so I can share some important info, it's taking a long time, so I start to use my body language to communicate that I'm frustrated, concerned, or disappointed. Eventually everyone quiets down, and we get started, and we move on with our lives. But that experience might trigger responses to any of these items that are not teacher-favorable:
• Students in this class treat the teacher with respect.
• Student behavior in this class is a problem.
• I hate the way that students behave in this class.
• My classmates behave the way my teacher wants them to.
• Student behavior in this class makes the teacher angry.
• It's a lot easier to mute a kid on Zoom, or push them out to the waiting room. To be honest, though, I've only had to do that once. Usually for students to be engaged in off-task behavior, there has to be some kind of off-task behavior to engage in. I'm finding that there is just less engagement, less exchange, on Zoom, and so we aren't getting disruptions from students, or negativity from me in response.
• Of the three items in the Cares section, two of them are pretty good.
• First two items
• 89%: My teacher in this class makes me feel that s/he really cares about me.
• 86%: My teacher really tries to understand how students feel about things.
• But the third item 
• 45%: My teacher seems to know if something is bothering me.
• Last year, my colleagues and I spent a lot of time thinking about this items. It raised a lot of questions around whether this item was a reflection of the teacher, at least in the same way that the other items are. And I agree it's a complicated question.
• Even though my completion rate was decent (96/126 = 76%) it's still a biased survey, because it's sampling only the students who are present, and are typically present on a random Monday. A more complete survey would capture the experiences of all the kids, including those who do not attend class or complete work consistently. This is a consequence of administering the survey in the exact same method that we deliver classwork (in class, through a link I post, etc.).

FOCUSING IN: PROBLEM OF PRACTICE

• In this first cycle of the student perception survey, I'd like to focus on the category "Clarifies." This is because it contains two lowest performing items (57% and 68%), while also containing two high performing items (89% and 91%).
• This is especially interesting because the low items and the high items seem to contradict, at least at first. Basically, it seems like my students are saying "Mr. St. checks to see if we understand, and if he thinks we don't understand he'll explain it another way....BUT, even though you are checking, you can't always tell that we don't understand."

• The flow of assessment should be that there is a learning opportunity (red), the teacher then assesses the students (yellow), and then the teacher analyzes student responses (green). At that point, the teacher needs to make the assessment that student either "got it" (blue) or "don't got it" (grey).
• Student responses point out a gap in the flow: I may be categorizing more students as having "got it", or categorizing the class as a whole as "got it", and then moving forward. So where could this perception be coming from?
• Possible explanations
• I might be setting the bar of "ok and ready to move on" lower than students are comfortable with.
• This might makes some sense. I'm feeling some pressure to move through a curriculum at a given pace, and so want to move through things. I often feel like I'm already slowing things down to be too slow (and so boring). In fact, given that only 62% of students responded favorably to "This class does not keep my attention--I get bored."
• In response, however, I have 79% of students agreeing that "In this class, we learn a lot almost every day." Since 79% > 62%, that suggests I could slow things down a bit, and trade a bit between those two?
• Students might be more ready to move forward than they feel.
• Students aren't getting papers back with scores on them. I am very regular, consistent, and frequent with using our gradebook to communicate progress and performance. But if students aren't attending to that data being provided in their gradebook, it's not going to actually lead them to adjust their perceptions of themselves and their own understanding.
• How can I help students see the progress that they ARE making? I'll often have moments where we all go into our gradebooks, look at the grades there, and then process them, trying to internalize what we need to be taking care of. But that takes a HUGE amount of time, EVERY time, and it DOESN'T feel like we usually get much out of it. Do I have evidence that this practice has little impact? During Review Week, when we did this all week, it resulted in very little overall improvement of grades, signaling little positive impact that week. Is this something where we'll just have to bank on a longer-term impact?
• My assessments might not be fully aligned to the work itself, and resulting in a higher rate of false positives. So even if students perform well enough on the assessment to signal readiness to move forward, it might not be a reliable enough indicator.
• As I mentioned above, I often feel discouraged by how slow it feels like we're moving through things, and how hard-won every little bit of progress is. It is possible that this pushes me to see what modicum of progress we HAVE made, and over-reward that with higher grades.
• The goal is to be encouraging, and to recognize that school is harder now than it has been in a long time. But the flip side of that practice is that it creates data interference, reducing the reliability with which the data signals readiness and understanding. This is a big part of the distinction between grades as generated by a traditional model, and "grades" as generated by standards-based grading (which I have committed to, at least in the way that my school has).
• Action Steps
• Be more transparent with students about the degree to which assessments indicate a need to go back over a topic, or to continue to study a topic. In particular, I can:
• When a week's work has provided evidence that we need to extend the pacing, I can communicate that with students, and so help them see how what they do impacts what we do as a class.
• "Hey folks, I was looking over our performance from last week, and it looks like we're going to need to spend a little more time working through [TOPIC]. So we've got this week's problem set that is going to look at this idea in another way."
• Be more clear (with myself and my students), about how I am grading.
• Our standards-based grading system allows for assignments that are rooted in standards, as well other assignments that are considered "practice." They are less targeted than standards-based assignments, and a bit more like the traditional grading model.
• Let the standards-based assessments reflect performance on the standards. Let the "practice" assignments reflect "practice," and if necessary use those assignments to buoy morale, while allowing standards-based grades to guide instruction.

If you have any questions, thoughts, or feelings about any of this, I'd love to hear them. I'll report back some time in February with the second round of survey results!

Designing PSETs for Remote Learning

This past spring we had to jump into remote learning. We were all figuring it out as we went along. I happened to be teaching two very different math classes at the time, which were being taught in very different ways. As such, I was able to experiment with two very different approaches.

I wrote about what I tried and learned during remote learning here. One of my preps was Integrated Math 1 (~CPM). It went fine. My other prep was a discovery-based math elective I developed (you can read about it here). This elective seemed to make the jump to remote learning much better than Math 1. I wrote about how I modified it for remote learning here. But the big idea was using weekly problem sets (PSETs). That's what I plan on doing this fall. So I want to take an opportunity here to reflect on what I think PSETs can do for remote learning specifically, and why.

What is a PSET?

• Here is a model PSET I made. Here are the major features
• Big Problem: the big complicated problem at the center of the PSET. Everyone works on it. Most of our class discussion is focused on it.
• Four Sections: Big Problem, Related Problems, "Not" Related Problems, and Going Deeper
• Variety: Different kinds of problems, of different sizes, different topics, different goals
• Content Thread: All the problems have a "Different Surface -- Same Depth". They appear different, but are all perspective on the same central Big Idea, that weaves through all the problems, like a thread.
• Students are given a PSET at the beginning of the week, and they have the week to do a week's-worth of work. They upload all their work in Google Classroom at the end of the week.

PSETs Adapt Well to Remote Learning

• More Choice → More Engagement
• Except for spending a portion of their time on the Big Problem, students can spend as much or little time as they want on whichever problems they want. If I made the PSET right, and they do a week's-worth of work, they should work with the Big Idea no matter what path they choose.
• This increases the odds that students are relatively more interested in the work they do, b/c they're picking from a bunch of problems. Moreover, just the process of picking problems will inherently build commitment...b/c they chose it!
• This also allows them to remain sensitive to their own readiness to engage in more complicated discussions out of self-preservation. If I had a PSET on exponential functions, I could totally include a problem on the COVID-19 and disease spread. But that could also be super traumatic for some students. So instead I make sure there are problems that also hit the same Big Ideas, but aren't as heavy, in case they don't want to think too much about the global pandemic. And if some kids *do* want to learn more about it, if they're ready, there it is.
• Focus on fewer bigger ideas → more efficient
• Learning has a kind of "efficiency ratio." We engage in some quantity of "doing," and some smaller quantity of it sticks as "learning." The rate at which that transfer from doing to learning is a function of a lot of factors, only some of which students and teachers can directly impact.
• Stress, new technology, different learning environment, health and wellness, and change in and of itself, are all factors that are greatly impacting students' "efficiency ratios." Kids are also being asked to quickly learn a ton of non-academic stuff. They're learning about the world, health, their own hearts, how to do emails, how to sustain relationships from afar, how to manage sickness, and a million other incredibly important things. And as far as academic learning goes, even if students are able to create space for it, it's reasonable to expect that this "efficiency ratio" is lower than normal.
• As such, it's more important than ever for is to answer the question: what are we willing to trade? Assuming we were operating anywhere near the frontier of performance and capacity for learning, we are going to need to reallocate capacity away from academic content, and toward surviving in These Uncertain Times.
• Option 1: Distance Modulation. Go into the same depth, but get through fewer ideas, modulating the distance you get based on your community's capacity.

• Option 2: Depth Modulation. Go through all the same ideas, but don't go as deep, modulating the depth you study based on your community's capacity.

• I claim that depth modulation is the way to go here. Because U.S. math education has built itself around the idea of prerequisites, and because we know the U.S. is going to push schools to push kids to get "caught up" (an oppressive and garbage expectation), it'll be a lot easier to bridge this time if kids at least have exposure.
• PSETs also have a lot of other mechanisms that allow for differentiation for readiness, and I talk a bit about them in this post. But the big idea is that it's easier for a bunch of kids to be working on the same content to different depths, as opposed to working on totally different content.
• All the problems packed into one paper → increased need/flexibility in product
• A long-term goal is that students learn how to do math "anywhere." On a whiteboard, window, piece of paper, spreadsheet...wherever is convenient and appropriate for the math itself. Part of learning how to do math is learning "where" you should do the math. Is this a small problem I can scrawl on a napkin? Is this going to make a bunch of data I'll probably want to put into a spreadsheet? Am I going to need a bunch of blank paper? Grid paper? By making it impossible to do the math *where* the problems are assigned, we force students to make a decision about when/where/how to do the math. This definitely takes up cognitive energy and work time, especially if they don't make the most efficient choice. But that's an instructional trade that feels worth making.
• This becomes especially useful during remote learning for a lot of reasons. Computers are more available than ever. So math on the computer becomes accessible anew. Not everyone knew to bring their notebook home from school. My kids this fall may not have one (we tried to pass them out at orientation, but I doubt that most kids were able to pick one up).
• Do the picture on some paper, or in your notebook, and upload a picture. Did something on desmos.com/calculator? Upload a screenshot or upload the link to share it. Made something with blocks? Take a picture and upload it. Google Classroom does decent job of collecting and archiving work. It's pretty easy to leave comments on things. I push it back for revisions? Do some work and upload pictures of that! Now we have a "paper trail" for revisions--something that's not always easy to do when passing a paper assignment back and forth.
• This is definitely going to raise the demand for teaching students how to use math tools like spreadsheets, graphing calculators, and google docs. Seems worth it though.
• You've got a week to do a week's-worth of work → honor increased demand for time flexibility
• I remember video conferencing with a student about some work a few months ago. As they were talking to me, their little sibling was running around, asking for food and attention. At one point, the kid was literally climbing my student (as two year olds are wont to do). When you're at home, it's not always as easy to get a whole hour dedicated to a task, without distraction. Throw in a dash of the year 2020 and there may be entire days where you can't set much time to work. I get it. And any adult that doesn't has to seriously do some empathy building.
• The body of mathematical knowledge isn't neatly packaged into 54-minute units of knowledge. Some things take 10 minutes to learn. Some problems take 4 hours to work through. So I'm going to give you a bunch of math to do, and you do a week's-worth of work by the end of the week. Do what you can, when you can.
• As a teacher, I am professional. And that comes with the recognition that all that matters it that I get the job done, by the time it needs to get done. I think about how disrespected I feel when PD is super prescriptive with how I spend my time. It feels important to extend that opportunity to students whenever possible. In regular face-to-face school time, that's more difficult, b/c we have 500 kids to rotate through 40 teachers, spanning 6 different courses. And very few people in power are willing to invest in schools so that they can be more humane in their scheduling. So we have to make some sacrifices for large-scale efficiency.
• Some students will need a lot of support and scaffolds for how to manage that expectation. And I'm prepared to do that. I'll recommend for some kids that the set a time for 60 minutes, sit down at the kitchen table, and crush some problems till the timer goes off. Some kids I'll say set a count-down for 5 hours, and then start it whenever they're doing math, and pause it when they stop. Again, this seems worth the instructional time it will take. Through continued conversation, reflection, and experimentation, students will learn what they need to do the learning--to get the job done. And if that's not the most transferable learning objective, I don't know what is.

PSETs do a lot of things. They're also a major area of pedagogical exploration for me for the last year and a half, for a lot of reasons. So maybe they're the hammer to my remote learning nail. But they are a remarkably flexible approach to math instruction, and now is definitely a time that demands flexibility.