Inconsistency in Evaluation Practices

I’ve been having some great conversations with teachers in my school about final evaluations in high school courses (i.e. exams and final culminating tasks). I see a desperate need for the discussion, so I’m hoping this might be a place for some of it. To that end, I’m sharing some of the points people having been making. 

First, some context

When two or more teachers in a school have sections of the same course, they’re encouraged to collaborate throughout the courses and are required to have consistency in the way their final 30% is evaluated. For example, if one teacher has a large culminating task for the entire 30%, another teacher of the same course shouldn’t have a 30% formal “test” exam.

This is true in lots of schools all over Ontario. It’s not a provincial policy, but it’s a very common board/school/departmental policy.

Thoughts I’ve had and heard

These are some of the points I’ve heard about this approach in no particular order. I’ll use the term “exam” to refer to any tool that is used for the final 30% component of a student’s grade, whether it’s a test, assignment, presentation, research paper, performance, etc.

  • If you have a formal exam and I have a task, students won’t get consistent marks, which matters for post-secondary entrance/scholarships.
  • How is it different from one teacher being a “hard marker” and the other teacher being an “easy marker”? Isn’t that a bigger problem?
  • If two siblings are evaluated differently, parents and siblings will all be upset that it’s not equal.
  • Two teachers in different schools/boards don’t have to align their exams; why is it required within a school?
  • You’re more likely to have a consistent mark distribution if you use the same exams.
  • Teachers should have autonomy and be permitted professional judgement as long as they’re following curriculum, Growing Success, and other policies.
  • Students need to write formal exams to prepare for university, so there shouldn’t be other forms of exams in grade 12, especially for U courses.
  • The exam is only worth 30%. The 70% term work is more valuable, but the policy doesn’t apply to it.
  • If you say my exam is easier than another teacher’s exam, you’re implying that one of us is inaccurately evaluating student understanding and performance.
  • There is no standard for the “amount of work” a student has to do for an exam.
  • We should have provincially standardized exams for senior courses for consistency and equity.
  • An open-book exam is easier than a closed-book exam.
  • An open-book exam is harder than a closed-book exam.
  • Some students need accommodations because of learning disabilities. Is it okay to give a different form of the exam for those students? Can’t other students access the same accommodations, since they aren’t modifications?
  • If school administration would approve of both exams on their own, then two teachers should be able to have different exams at the same time.
  • Not all forms of evidence of student learning are equally valid or accurate.
  • If I come to a school for semester 2, why am I restricted by what a semester 1 teacher chose to do in their class?

What do you think?

Post some comments. Let’s work on this together.

Poor research in education bothers me

When I look at educational research about instructional strategies, I’m concerned with how often the researchers ignore important controls. They confound their data and then draw invalid conclusions.

I just read some research in which students were taught the same math content but using two different approaches:

  • Group A was taught “traditionally”, which included teacher-led, direct instruction;
  • Group B was taught with a student-directed approach and a specific context.

After read the descriptions of the two groups, what conclusions could you draw if one group outperformed the other group?

Unfortunately…

…the researchers concluded that the context they used was important for student learning. They admitted that the specific context required a very different instructional approach, but they attributed the achievement differences to the “theme” of the task.

Confound it

That’s really not good enough. You can’t have two major differences between groups and then point to either difference as the cause. In fact, you can’t conclude much of anything from data like this.

Fail.

Do it right

Only change one thing at a time. If you need to change more, try a third group (here, you can have a group with a student-directed approach but without the contextual restriction). Then you’ll be able to tell if the difference is due to the approach or the approach with the context.

Late deadlines make for sleepy students

I set 11:30pm deadlines for assignments in my e-Learning course because I want to respect that students work on their courses at times outside of the instructional day. Some students take online courses specifically because of the flexibility of learning anytime, anywhere.

But most students behave like temporal gases: they will fill whatever container of time you put them in. So a side effect of that graciously long submission window is that students who could finish their assignment during the school day don’t, and they all try to cram it into their evening instead. Then they stay up far too late, which is bad for a ton of reasons.

Photo "Library visitor" by umjanedoan via Flickr (CC-BY-2.0)

Photo “Library visitor” by umjanedoan via Flickr (CC-BY-2.0)

So setting an 11:30pm deadline might have been a mistake. In the future I might set it for 3:00pm and simply expect that some folks will be a little late. At least the punctual kids will get to bed on time.

Your password stinks

Okay, your password might not stink. But it probably stinks, since most people have rotten, foul, shouldn’t-ever-be-used passwords.

Passwords aren’t fun, but until SQRL is ready you’ll need to use them. There are some super-basic rules about creating passwords that you really shouldn’t break.

Make your password unique

This is the hardest one to do, although it’s easy to understand. Don’t use the same password for more than one service.

Why not?

Suppose you have a Facebook account and an account with another service, SociallyInsecure.ca (I just made that up, and the domain isn’t registered, so feel free to grab it). SociallyInsecure.ca has a security breach (imagine) and your email address and password are now in the hands of nefarious cybernasties.

What do the cybervillains do? They head immediately over to Facebook.com to see if your email address and password from SociallyInsecure.ca will log them into your Facebook account.

Now the bad guys can read your stuff and impersonate you.

And since you used the same password for your email, they get access to all your other services too, by using the “forgot my password” links.

You could have prevented this by using a different password for each service.

Change any password someone else gave you

Often your initial password is given to you by a person or server. Change it. Otherwise, it’s possible someone knows it, or it’s printed out on a master password list somewhere. Don’t tempt anyone to use it.

Don’t use a word for a password

Those are easy to guess, and the cybercrooks try them first. Capitalizing the first letter isn’t much of a barrier either. Don’t use a word, especially monkey.

Don’t use something people will guess

Don’t use “amazon” for your Amazon.com password. Don’t use the names of your spouse or kids or dog or workplace or anything about you or the site you’re visiting.

Use a long password

Like, really long. Try more than the 8-character minimum that most services require. Try 10 characters, or 12. It takes a second longer to type it in, but it takes centuries longer for the cyberjerks to figure out.

Use multiple character types

Easy passwords use all lowercase letters, or maybe they start with an uppercase letter. Some services require a capital letter and a number, so people use things like Monkey1. Mix it up a little more than that, and include “special characters” like punctuation.

What’s a good password?

I can’t tell you a specific good password, because then it won’t be good. But here’s the sort of thing you’re looking for:

u1is4$kK5H

Urgh…

That’s a good type of password, because

  • it’s not a word
  • it’s not guessable based on what you know about me or where I’m going
  • it has lowercase and uppercase letters, digits, and symbols
  • it’s long (10 characters)

I can’t remember one of those for every service I use

You might not be able to remember dozens of those passwords (and so you’re tempted to write them down – don’t do it!), but you can remember one. So memorize it, then set up a password system for yourself using the password you memorized as a base password.

For example, maybe you modify that password based on the site you’re going to. Suppose you want a password for Amazon.com. Mix something from Amazon with something from your base password:

Amazon

u1mais4$kK5H

I took the second and third letters of Amazon (m and a) and inserted them after the first two characters of the base password.

How about for Facebook?

Facebook

u1acis4$kK5H

If you can remember the base password, you get a (nearly) unique password for each site without having to write it down.

Don’t use this exact algorithm. Come up with your own that you can remember and that you’re comfortable with.

But stop writing stinky passwords on a sticky note on your laptop.

We have to stop pretending… #MakeSchoolDifferent

I’m responding to Sue Dunlop’s challenge (which is the result of a series of challenges stretching back to Scott McLeod). I’ve only read a few of the other posts that this challenge has generated, so I apologize to anyone who already expressed these same thoughts.

  1. We have to stop pretending that it’s okay to complain about someone else instead of offering them support.
  2. We have to stop pretending that telling people to learn how to cope is an effective strategy for dealing with mental health challenges.
  3. We have to stop pretending that evaluation can be both objective and accurate when implemented by a single human.
  4. We have to stop pretending it’s acceptable and reasonable for reporting periods to dictate the pace of learning in our classrooms.
  5. We have to stop pretending that there is a single, correct solution to any one of these complex problems.
  6. We have to stop pretending that we can do this on our own.

Oops, that’s 6. Ah well.

The tagged? David Jaremy, Peter Anello, Tim Robinson, Eva Thompson, and Doug Peterson. Additional apologies if you’ve already been tagged.

Tweets from #OTRK12

Here are some tweets that I wrote, retweeted, or favourited during OTRK12. I still have to write my OTRK12 reflections in a post, and a summary of the session I hosted. Hopefully I’ll get those done this week. In the meantime, here’s OTRK12 in bite-sized chunks:

Tweets and Retweets

(Note: “the” = “they” in above tweet. You knew what I meant, right?)

Favourites

How I Use Twitter Professionally – Version 4

Two and a half years ago I wrote How I Use Twitter Professionally, then revised it with How I Use Twitter Professionally – Updated! and How I Use Twitter Professionally – Updated Again!

I guess we’ll make it an annual thing:

My tweets are public.

I’m trying to encourage conversation and collaboration, so my tweets are globally accessible. This also means I don’t make statements I wouldn’t be comfortable with anyone reading – my family, my students, my employer….

I don’t follow a lot of people.

I currently follow 370 people, of whom about 250 are actively tweeting (let’s say at least weekly). Some of these aren’t related to education; for example, I follow The LEGO Group (@LEGO_Group) and authors John Scalzi (@scalzi) and Marko Kloos (@markokloos – he has a new book out today!).

I can’t read all of the stuff they tweet. I’m relying on my tweeps to retweet the really good stuff so I have a better chance of seeing it, or to mention me if it’s something they think I ought to notice.

I accept anyone as a follower, pretty much.

Except for a few obvious accounts, I let anyone follow me. Since my tweets are public, anyone can read them (even without a Twitter account), so letting people follow me doesn’t reveal anything extra. Plus, it’s easier when you don’t have to approve people.

I don’t follow back as a courtesy.

Before I decide to follow someone, I take a look at their tweet history. Is their stream of tweets going to enhance my experience? Will I learn from them? Or will I only learn what they had for breakfast?

I’m a fan of some personal stuff on Twitter, but if you post 300 times a day just to talk without conversing, I don’t need to see it. It’s not about you, it’s just that your use of Twitter doesn’t fit with mine. I think your lifestyle on Twitter should be like the Law of Two Feet: if it’s not working for you, move on.

I don’t accept Direct Messages (DMs) from people I don’t follow.

This cuts down on the spam. Now it’s just mentions, and there aren’t too many of those. This is a good idea for anyone, so I thought I’d list it here.

I also don’t follow people who I don’t want to DM me. That especially includes students. I have my school email for that kind of communication.

I follow hashtags for a while.

I follow #OTRK12 (our annual conference in Mississauga – this week!) and #elADSB (for my Board’s e-Learning teachers). I don’t follow the very busy tags, although I sometimes apply them to my posts (#D2L, #onted, #blendedlearning, #edtech).

I try to follow the people in Northern Ontario. We face many of the same issues, and perhaps we have solutions to help each other. I like that idea.

I don’t cross post to Facebook anymore.

I tweet too much. No one on Facebook wants to read all of that stuff. The handful of FB friends who do are also Twitter users and teachers, so they just go to Twitter to find me. When I write blog posts WordPress will publicize them on Facebook, Twitter, and Google Plus, and I’m certain that’s plenty for the FB crowd.

I use Tweetdeck; it rocks.

Chrome has TweetDeck as an app; I like that I can have columns for a variety of things I want to look at. Currently I have my Twitter timeline, my Twitter Interactions, my Twitter Messages (DMs), and columns for a bunch of hashtags and lists I follow.

I say things for myself, and I say things for others.

I tweet things that I want to remember or revisit (great for “note-taking” at a session/workshop/conference). I also tweet things to inform others or start conversations. My tweets (of links and such) aren’t endorsements, but since people sometimes view them that way I try not to share stuff that I’m not at least familiar with.

I talk a lot, but not too much

I try to ask questions and help out when others ask questions. I’m proud to say I am included as an honourary member of the SGDSB educators list because I help out the teachers up there, so I think my contributions are valued.

More importantly, I’m developing relationships with these distant folks, and the growth of my PLN has helped me out in my work as well. It was very exciting last year at OTRK12 to meet people whom I knew only through Twitter, and it was surprising how natural the face-to-face interactions felt. We were already friends. So thanks, tweeps.

If you want to follow me…

I’m @bgrasley. No pressure, of course. Use Twitter however it works best for you, and don’t be upset if other people use it differently!