Handout: Evaluating your own ILOs
|The best way to become familiar with well-structured intended learning outcomes is to look at good and bad ones! Spend some time reviewing other courses and programmes within your own institutions as well as others you can find on the web. Systematically go through and evaluate those ILOs.
When we start evaluating our own intended learning outcomes, to establish how well-structured they are, we have to think about whether or not we’re actually using the right domains. Whether we’re using the right verb structures. If we only use Bloom’s taxonomy, we’re going to be very limited in the kind of active verbs that we want to use to assess possibly quite a wide range of skills.
So, in this particular example, we’ve got a number of outcomes written at various different levels, that are intended to differentiate between the kind of affective domain skills versus the metacognitive domain skills. So, the affective skills are looking at people’s value propositions, the way they relate to social and cultural knowledge. And the epistemology of their knowledge development, their self-reflection around how they are coming to terms with what knowledge actually means, how it’s made how it relates to other things. So, if you look at the verbs that I’m using here, they’re quite different between these two domains, they will then point to very different types of assessment and ultimately to very different types of learning and teaching activities.
So, it’s worth sometimes playing around with different verb structures to work out actually what it is you want to assess the students’ to be able to do at the end of the course. Obviously, I’d like all of these to be reflected in all of your courses, but sometimes if you’ve got to make a choice deciding on which domain you’re gonna work within is a good first step.
Now another way of evaluating whether or not your outcomes are well structured is whether in fact, you can start moving either the subjects or the context or the verbs between them, and whether or not they still make sense.
So, in this example, I’ve got a range of different outcomes from different disciplines. And I’d like to suggest that in some cases, not in all cases, but in some cases it will be possible to literally just reverse the context of the learning and see the outcome still makes complete sense. It’s a good check to make sure that we are not using compound verbs, that we’re not using unnecessary jargon, and that we are producing a well-structured three-part outcome.
So in this instance, I can switch from a ‘contemporary guidance in the financial services industry’ to a ‘notation system used in the financial industry’. ‘Notation system’ might not be exactly the right language, and so I might want to go back and amend that. But I could take the ‘notation system used in the context of two recital pieces’ from a music degree and actually switch to that ‘being contemporary guidance in the context of two recital pieces’.
Now they’re not perfect, but it does demonstrate that I’ve at least isolated that context. It’s independent of the subject, the subject is not necessarily contingent on the context, and so it means that I can start moving things around a little bit. If you look at all of the others, you will see, most of them are entirely applicable. You can reverse them in each case.
The one exception is the psychomotor domain, which is one of the most challenging domains to work with because it’s usually much more highly specialized and usually more highly specified. So, in this case, it wouldn’t make sense to have the ‘rehabilitation devices’ using in a ‘GIS drafted maps’, mapping system, for example, just because I don’t believe that geographical information systems need rehabilitation devices. And likewise, a range of ‘calibration point features in a typical clinical setting’ also doesn’t really make sense. So that would be the exception in the rule, but I think it does demonstrate that by looking at whether or not you can reverse contexts or reverse subjects. It just demonstrates whether or not you have a well-structured outcome. There will always be some rewording required when you start doing that obviously.
Now, if we look at the domains themselves, we sometimes find that once we’ve written a good subject, we’ve got a good context, we may think that actually the nature of the verb maybe needs to change. And we can do a very similar thing. We can move the interpersonal context up and have that now run through, I’m going to let you read that on the screen. You will see that still makes complete sense.
That was the original. / That’s the replacement.
I think it still makes sense. Let’s see if it works the other way around so we, can we take the interpersonal and lift that? Can we take the cognitive and drop it down? Yes, we can. We can have a cognitive domain that says to ‘critique the robustness of opposing arguments’ or ‘critique institutional policies’. That’s equally sound.
We can also do that between the affective and the metacognitive, in part, because they’re quite close. You can see that also works again. The psychomotor domain is the one that’s very often the hardest to use to exchange into other domains, which is one of the reasons why it’s a real challenge to write really good psychomotor outcomes.
The argument is if you have more structure, active verb, subject, context, it becomes more flexible, not less flexible.