Assessment and Evaluation: sacrificing complexity for granularity

I teach Math in Ontario. We have an “Achievement Chart” (see pages 28-29) which lists four categories of knowledge and skills. When we assess and evaluate student work, we separate student performance into the “TACK” categories: Thinking, Application, Communication, and Knowledge. The Chart includes criteria for each category and descriptors for different Levels of performance.

The curriculum itself is divided into Strands for each course, and these strands describe Overall Expectations and Specific Expectations (essentially the details of the Overalls).

So when evaluating student work, we evaluate Overall Expectations in the context of the four Categories of Knowledge and Skills, and we should have a “balance” between the categories (not equality, necessarily).

The truth is that I’m having some trouble with it. I posted a little while ago that I was struggling with the Thinking category, and that’s still true. But there is another issue that’s more pervasive and possibly more problematic.

Isolating skills

When trying to separate out the different components of student performance, we would often ask questions that “highlight” a particular area. Essentially we would write questions that would isolate a student’s understanding of that area.

That’s a fairly mathematical, scientific-sounding thing to do, after all. Control for the other variables, and then effect you see is a result of the variable you’re hoping to measure.

For example, we wouldn’t ask a student to solve a bunch of systems of equations which only had “nasty” numbers like fractions in them (or other unfairly-maligned number types) because we fear that a student who is terrible with the fractions will stumble over them and be unable to demonstrate their ability to solve the system of equations. So we remove the “barrier” of numerical nastiness in order to let the skill we’re interested in, solving the system, be the problematic skill.

This isn’t a great idea

But we do that over and over again, isolating skill after skill in an effort to pinpoint student learning in each area, make a plan for improvement, and report the results. And in the end, students seem to be learning tiny little skills, procedures, and algorithms, which will help them to be successful on our tests without developing the connections between concepts or long-term understanding.

We want to have “authentic, real-world problems” in our teaching so that students can make connections to the real world and (fingers crossed) want to be engaged in the learning. But authentic problems are complex problems, and by julienning our concepts into matchstick-size steps we are sacrificing meaningful learning opportunities.

What if we didn’t have to evaluate?

We’re slicing these concepts so finely because we’re aiming for that granularity. We want to be fair to our students and not penalize their system-solving because of their fraction-failings.

But if there were no marks to attach, would we do the same thing? Would we work so hard at isolating skills, or would we take a broader approach?

My MDM4U class

I’m teaching Data Management right now, and the strand dealing with statistical analysis has a lot of procedure skills listed followed by a bunch of analysis skills. If I evaluate the students’ abilities in summarizing data with a scatter plot and line-of-best-fit, do I then ask them to analyze and interpret the data based on their own plot and line? What if they mess up the plot; don’t I then have to accept their analysis based on their initial errors? Oh wait, I could make them summarize the data, then I can give them a summary for a different data set and ask them to draw conclusions from that summary! Then they’ll have the same starting point for analysis, and they can’t accidentally make the question too easy or hard!

But I’ve just messed up one of my goals, then: I’ve removed the authenticity and retained the ownership of the task. I haven’t empowered my students if I do it this way, and I’ve possibly sacrificed meaningful complexity. Worse, I’m only doing this because I need to evaluate them. I’d much rather require them to gather, summarize, and analyze data that interest them and then discuss it with them, helping them to learn and grow in that richer context.

As always…

…I don’t have answers. Sorry. I’m trying hard to make the work meaningful and the learning deep while still exposing as much detail about student thinking as I can. I’m sure in the end it’ll be a trade-off.

Advertisements

What would you do with a day to improve your teaching?

Andrew Campbell asked tonight on Twitter:

I answered a couple of times.

Then I had a related thought:

If I want to improve my teaching, I can work on something myself or I can learn from others (or both).

What would I want to learn?

A full day… 8 hours… no restrictions, no prescriptions, no requirements, no accountability, no strings… just work to improve my teaching….

It was harder than I expected, until I had the defining thought:

What do my students need me to be better at?

So instantly I landed on Assessment. That’s my area for improvement, and that’s what I would spend my day on.

The Plan

  • Before that day, decide on a specific focus (for me, probably assessment as learning, including peer assessment and self-reflection) and gather some short articles from trusted, external sources.
  • Start the day reading those articles, and writing a short, reflective blog post. The post is to share my learning and hopefully get some helpful feedback from my PLN.
  • Apply my prior knowledge and my new learning to a specific task that I’m planning for my class. If there is time, plan multiple learning opportunities using different strategies.
  • Share those tasks with my PLN and solicit more feedback.
  • Reflect, modify, reshare, etc.

Let’s be realistic – I ran out of time already, and I’ll be finishing at night.

This would be good PD

It would be wonderful to spend a day like this, although I don’t want to miss my class or my family, so it would be a really expensive day in that sense. It might be worth it, though.

What would you do?

Check out Andrew’s response (clever fella):

Improving report card comments with a checklist

It’s report card season in Ontario, and I don’t know too many people who are happy about it.

I don’t love evaluating student performance in general, and the persistent and poisonous focus on MARKS by most stakeholders in student learning is infuriating. Marks are a huge loss of information about student performance, in my rarely-humble opinion. Along with those percentage marks we have a much-less-valued-but-more-valuable evaluation of Learning Skills. My students mostly ignored those, I think.

In truth, the hero of the report card is The Mighty Comment. It has the superpowers of Explanation and Recommendation. It’s here that I can talk about what’s going on, why, and how to improve.

After all, assessment is for improving learning. Reporting a mark of 68% doesn’t do that.

So The Mighty Comment is our hope for the future, the only power that can save our students and their parents from receiving an all-but-useless document.

Let’s do it right.

I’m teaching in a high school, and we have both a provided comment bank and the latitude to write our own comments. The only rules are that we need to follow the guidelines in Growing Success and we have to keep it under 458 characters.

I read an interesting article at rs.io called The Unreasonable Effectiveness of Checklists.

Fireworks blazed across my brain. I need a checklist to make sure I’m doing what I want to do with every comment.

So I made one

The Report Card Comment Checklist (catchy name, eh?) is now live. I also included The Verbose Report Card Comment Checklist immediately after it to help explain what I mean. Please leave comments here on the blog if you can help me to improve it.

I sat with each of my students this term to review their marks, learning skills, and comments before I submitted them to my school admin team. I wanted them to know that I tried to write what I thought and that I cared about their improvement. I articulated their strengths and what I need them to do next. I asked them each to reflect on their comment (most of them needed to be prompted) and to tell me whether they thought it was fair, accurate, etc. One student found a typo (yay!) and two asked me to clarify what I meant. About five students said their comments sounded exactly like them, which makes me proud.

I have to admit that I made the checklist this evening; I may have to edit my comments a bit next week before they’re published.

You should just click the link for the complete version, but here it is anyway:

The Report Card Comment Checklist

Check each student’s report card comment and ask yourself these questions:

Strengths

  • does it include at least one strength?
  • are the strengths related to the course?
  • are the strengths worded positively?
  • do the strengths stand alone?

Next Steps

  • does it include at least one next step?
  • are the next steps related to improvement in the course?
  • if a student reads the next steps, will they know what to do to improve?
  • are the next steps worded positively?
  • do the next steps stand alone?

Language and Tone

  • did I check for spelling, grammar, etc.?
  • did I read it out loud?
  • did I listen for sarcasm and negative feeling in my voice?

The Point

  • will the student feel that I care about their success?
  • will the student “see themselves” in the comment?
  • will the student want to continue to improve?
  • will the parent understand how to help their child improve?

 

Different kinds of Thinking: Ontario Math Achievement Chart

I’m evaluating some student work today and I’m struggling with the Achievement Chart for Mathematics (see page 28). In particular, this part of the Thinking category is bothering me:

An excerpt from the math achievement chart for Ontario

Take a look at the first point in “Use of planning skills”, called “understanding the problem”, which includes “formulating and interpreting the problem” as an example of that skill.

Now look at “Use of processing skills” point “carrying out a plan”, which includes “modelling” as an example of that skill.

Are these different? In my mind (up until now, at least), “formulating and interpreting the problem” has meant representing a situation mathematically so that we can apply our other math skills to solving it. Isn’t “modelling” in the context of “carrying out the plan” sort of the same thing? Representing components of the problem mathematically? Is the difference just when it happens (i.e. formulating/interpreting is initial planning, and modelling is during the act of solving)?

I’m not trying to be pedantic here; I’m having trouble distinguishing between the different components of Thinking when I’m trying to assess and evaluate my students’ work. I could use some external thinking on this issue (and math evaluation in general, I suppose).

Please comment; I’d love to talk to you if you have ideas about this stuff.