A brief explanation (2’22”) of the central importance of designing well-structured intended learning outcomes for courses. Five domains of learning, details of which are available here: https://sijen.com/research-interests/taxonomies/ are all important in a balanced course. Structuring outcomes with an active verb, subject and context are also shared.
These resources from 2013-2017 are being shared to support colleagues new to teaching online in the face of the COVID-19 pandemic. Consultancy for International Higher Education from Simon Paul Atkinson
I persist in being a fervent advocate for the use of a broader range of domains of learning, other than the cognitive domain (Anderson & Krathwohl, 2001; Bloom, 1956), in course design.
Higher education must surely be more than just about acquiring content and intellectual skills. A broader student experience should be embedded into each course, module and programme design, beyond that of acquiring purely intellectual skills.
The ‘graduate skills’ so demanded by employers are rarely without reference to so-called ‘soft skills’ (S. P. Atkinson, 2015). These are interpersonal domain skills, conflict resolution, cross-cultural awareness, collaboration and communications are what most graduates need most (J. C. Atkinson, 2014; Bennett, 1986). Alongside these the skills, often assumed by employers, to make effective use of software pertinent to that profession are rarely taught and assessed despite being critical to be effective on the day one. These skills can be set out and assessed using psychomotor domain outcomes (Dave, 1967). Such non-intellectual attributes are often seen by faculty as beyond their remit, as are also those values, the affective domain (Krathwohl, Bloom, & Masia, 1956) and the development of personal epistemologies using a metacognitive domain (S. P. Atkinson, 2014) which form part of the maturation process within students themselves. These cannot be ‘taught’ in the strict sense, but their facilitation is entirely within the capabilities of able faculty.
Given that we know students are more likely to take anything assessed seriously, it is essential to write assessable outcomes across a range of skills. So, it is beholden on course designers to include the interpersonal domain (communications, cross-cultural awareness, conflict resolution), affective domain (values), metacognitive domain (personal epistemologies) and the psychomotor domain (manual and dexterity skills). Each has its place in all disciplines. There is clearly a craft in designing ILOs that motivate and steer learning but which are transparent and assessable. There is guidance on how to do that under the 8 Stage Learning Design Framework 8-SLDF and I am always willing to share practice with colleagues in workshops when requested.
Bibliography
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Atkinson, J. C. (2014). Education, Values and Ethics in International Heritage: Learning to Respect. Farnham: Ashgate Publishing Group.
Atkinson, S. P. (2014). Rethinking personal tutoring systems: the need to build on a foundation of epistemological beliefs. London: BPP University College.
Atkinson, S. P. (2015). Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes. Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education, 10(2), 154–177.
Bennett, M. J. (1986). A developmental approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10(2), 179–196. https://doi.org/10.1016/0147-1767(86)90005-2
Bloom, B. S. (Ed.). (1956). Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain (2nd edition). New York: Addison-Wesley Longman Ltd.
Dave, R. H. (1967). Psychomotor domain. Presented at the International Conference of Educational Testing, Berlin.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1956). Taxonomy of Educational Objectives, Handbook II: Affective Domain. New York: David McKay Company, Inc.
Four ‘Domains’ of educational objectives represented in a circular form
I think being able to visualise things is important. Faculty and learning designers need to be able to see Intended Learning Outcomes (ILOs) take shape and mant find existing lists are uninspiring. It’s not uncommon for faculty and instructional designers to get tired and weary of ILOs; they can feel restrictive, repetitive, formulaic and sometimes obstructive. In previous posts I’ve tried to suggest that the bigger picture, the challenges of effective 21st century university level learning design, make them not only useful, but also essential. If you don’t agree, don’t bother reading. I’m not going to try and persuade you. If you think there’s some truth in the argument and you want to engage with ILOs to make your teaching more focussed, your students increasingly autonomous and your graduates equipped with meaningful evidence, then I hope I have something worthwhile sharing and will welcome your thoughts.
My argument is that a module (a substantial unit of a full years undergraduate study), and the programme of which is part, should have clearly articulated outcomes in four domains:
Knowledge and understanding – or the knowledge domain
Intellectual Skills – or the cognitive domain
Professional Skills – or the affective domain
Transferable Skills – or the psychomotor domain
I’m suggesting one SHOULD expect to see a different distribution of ILOs between the outcomes in these domains depending on the focus of the module and the level of study. One might expect to see a second year anthropology module on ‘theoretical perspectives’ emphasising cognitive outcomes and a module being studied alongside it on ‘research design and techniques’ emphasising affective and psychomotor outcomes. One might reasonably expect to see more foundational ‘knowledge and understanding’ outcomes in the first year of a programme of study, and more ‘cognitive’ outcomes at the end of the programme. The lack of this ‘designed articulation’ in many modules undermines their value to the student and ultimately to faculty.
The basic principle is that an outcome should be assessable. Lots of great stuff can happen in your teaching and students’ learning that DOESN’T need to be assessed. It can be articulated in the syllabus, it just isn’t a measured outcome. A student should be able, at the end of this course of study (module or programme), to evidence that they have attained the intended learning outcomes. This evidence has been assessed in some way and the student is then able to point to the ILOs amassed throughout their programme and say “I can demonstrate that I learnt to DO this”.
Representing Taxonomies
There has been a significant shift in the language we now use from the original work in the 1950s by Bloom and colleagues. The passively descriptive language of Bloom’s Taxonomy has become the active language of Anderson and Krathwohl (Anderson & Krathwohl, 2001). The taxonomies have moved from Evaluation to Evaluate, from Analysis to Analyse. This is significant in that the emphasis has moved away from describing what the focus of the teaching is supposed to be, to the demonstrable outcomes of the learning.
The illustration above consists of four visual ‘wheels’ that I have used to discuss learning outcomes with faculty in the context of module and programme design at Massey University in New Zealand and at the LSE and BPP University College in the United Kingdom. These visual representations were inspired by work done elsewhere, on the cognitive domain in particular. The first documented example of this circular representation I have been able to find is attributed to Barbara Clark in 2002, but a great many people have since represented Bloom’s original, and the revised, cognitive domain in this way.
The circular representation has the higher level terms at the centre, proto-verbs if you will, surrounded by a series of active verbs that articulate actions an individual might undertake to generate evidence, of their ability to represent to proto-verb. The circular visualisation also serves to create a more fluid representation of the stages, or divisions, in the proto-verbs. Rather than a strict ‘step-by-step’ representation where one advances ‘up’ the proto-verbs, one might consider this almost like the dial on an old telephone, in every case one starts at the ‘foundational’ and dials-up though the stages to the ‘highest’ level. Each level relies on the previous. It may be implicit that to analyse something, one will already have acquired a sense of its application, and that application is grounded on subject knowledge and understanding. So the circle is a useful way of visualising the interconnected nature of the process. Most importantly in my practice, it’s a great catalyst for debate.
The circular representations of the domains and associated taxonomies also serve to make learning designers aware of the language they use. Can a verb be used at different levels? Certainly. Why? Because context is everything. One might ‘identify’ different rock samples in a first year geology class as part of applying a given classification of rocks to samples, or one might identify a new species of insect as part of postgraduate research programme. The verb on its own does not always denote level. I talk about the structure of ILOs in a subsequent post.
Structure of the circular representations of Educational Taxonomies
More recent representations have created new complex forms that include the outer circle illustrated here. I’ve found these rather useful, in part because they often prove contentious. If the inner circle represents (in my versions) the proto-verbs within our chosen taxonomies, and the next circle represent that active verbs used to describe the Intended Learning Outcomes (ILO) AND the Learning and Teaching Activities (TLS), the outermost circle represents the evidence and assessment forms used to demonstrate that verb. Increasingly I’ve used this to identify educational technologies and get faculty thinking more broadly about how they can assess things online as well as in more traditional settings. The outermost circle will continue to evolve as our use of educational technologies evolves. In Constructive Alignment one might reasonably expect students’ learning activity to ‘rehearse’ the skills they are ultimately to evidence in assessment (Biggs & Collis, 1982; Boud & Falchikov, 2006) and the forms to enable that are becoming increasingly varied.
Re-visioning Taxonomies
One of my favourite representations of the relationship between the knowledge dimension and the cognitive domain is from Rex Heer at Iowa State University’s Center for Excellence in Learning and Teaching (http://www.celt.iastate.edu/teaching/RevisedBlooms1.html accessed ). It’s an interactive model that articulates the relationship, as Anderson and Krathwohl saw it, rather well. My own interest, as we look to effective ILOs, is to separate out the knowledge dimension as a subject or knowledge domain and have faculty articulate this clearly for students, before reconnecting to the other domains. A process I’ll talk about subsequently.
Here are my four ‘working circles’ using adaptations of taxonomies from Anderson and Krathwohl (Knowledge and Understanding, and Cognitive), Krathwohl et al (Affective) and Dave (Psychomotor). I have adapted the Knowledge Dimension of Anderson and Krathwohl to do two things; to describe the dimension in terms of active verbs rather than as a definition of the nature of the knowledge itself, and I have incorporated a stage I believe is under represented in their articulation. I have added the ability to ‘ contextualise’ subject knowledge between the ability to specify it (Factual) and the ability to conceptualize (Conceptual). I have also rearticulated the original ‘Metacognitive’ as the ability to ‘Abstract‘. This will doubtless need further work. My intent is not to dismiss the valuable work already in evidence around the relationship between a knowledge dimension and the cognitive domain, rather it is to enable faculty, specifically when writing learning outcomes, to identify the subject, discipline or knowledge to be enabled in more meaningful ways.
These images are provided as JPG images. If you would like me to email the original PowerPoint slides (very low-tech!) so that you can edit, amend and enhance, I am happy to do so. I only ask that you enhance my practice by sharing your results with me.
I hope these provoke thought, reflection and comment. Feel free to use them with colleagues in discussion and let me know if there are enhancements you think would make them more useful to others.
Cognitive Domain – Intellectual Skills
Cognitive Domain – Intellectual Skills
Affective Domain – Professional and Personal Skills
Affective Domain – Professional and Personal Skills
Psychomotor Domain- Practical, Technical and Transferable Skills
Psychomotor Domain- Practical, Technical and Transferable Skills
Knowledge Domain – Subject and Discipline Knowledge
Knowledge Domain- Subject or Discipline Skills
The next post will illustrate the usefulness of these visualisations in drafting Intended Learning Outcomes with some examples.
………………………………………………………………………………………
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Biggs, J. B., & Collis, K. F. (1982). Evaluating the Quality of Learning: Structure of the Observed Learning Outcome Taxonomy. Academic Press Inc.
Boud, D., & Falchikov, N. (2006). Aligning assessment with long‐term learning. Assessment & Evaluation in Higher Education, 31(4), 399–413.
In my last posting I suggested that a module specification could usefully have four sections, clearly articulated, for Intended Learning Outcomes, so that a student could identify from their assessment evidence that they had met specific ILOs in a range of domains. In doing so they not only have a useful platform to identify future learning needs, but also the potential to negotiate the accreditation of prior accredited learning in a much more fine-grained and meaningful way, something I fully expect to become a significant future of international higher education accords in the next few years as institutions face up to the challenge of accredited OER schemes and credit bearing MOOCs. I believe the design of intended learning outcomes for modules and programmes will become a strategic priority.
Not everyone agrees ILOs are effective and a useful critique from Hussey and Smith is well worth reading (Hussey & Smith, 2002).
How many Intended Learning Outcomes (ILOs) one designs into a module or a programme level specification has to depend on the scope of the module or programme itself. I’m sure colleagues can adapt what I’m saying here to their own quality assurance and institutional contexts.
For the purpose of this reflection let me take a single module, worth 15 credits. In the UK context this would frequently represent one-eighth of a stage of undergraduate degree study, there being three stages each representing 120 credits. Again, in the UK context there is a strong notion of progression in higher order thinking skills between the first stage of undergraduate study (level 4) and the final stage (level 6). This progression is articulated in generic guidance that captures much of this ILO debate and in subject specific guidance drawing on the discipline communities to create ‘benchmarks’ for what be expected to be in any named award (www.qaa.ac.uk) . Level 5 would represent the second stage of undergraduate study in the UK context, the equivalent of an exit point for a Higher National Diploma or a Foundation Degree, the European Qualifications Framework Level 5 and within the EHEA (Bologna) sometimes referred to as a ‘Short Cycle’ award.
My example then is for a 15-credit module at level 5. The UK quality assurance agency does not specify periods of study for credit, but sector norms talk in terms of notional study hours and it is perhaps helpful therefore to think of 15 credits as 150 notional study hours, 30 credits as 300 notional study hours and so on.
The Domains
Before proposing a model for ‘how many’, I will briefly remind myself what these four sections, or domains, of Intended Learning Outcomes represent. They are;
Knowledge and understanding – subject domain
Intellectual Skills – or the cognitive domain
Professional Skills – or the affective domain
Transferable Skills – or the psychomotor domain
Knowledge and understanding – subject domain
The subject domain is often conflated with the cognitive domain, which is understandable as it is within Bloom’s ubiquitous taxonomy, but this does tend to confuse faculty as to the distinction between knowing and understanding a body of factual knowledge and being able to do something with that factual knowledge. The Subject domain can, and in my opinion should, be limited to defining the subject area for illustrative purposes for the student. Since the principle is that all Intended Learning Outcomes should be assessed and it is actually rather difficult to assess whether someone ‘understands’ something without having them ‘operationalize’ the knowledge, I tend not to get too hung up on the active verbs used in this domain, contenting myself it serves to contextualise what follows, but maybe I should and another post later will unpack Anderson and Krathwohl’s Knowledge Dimension in more detail.
Intellectual Skills / Cognitive domain
This domain refers to ‘knowledge structures’ building from the base of the Subject domain, the “knowing the facts”, towards high order thinking skills in which these facts become operationalized and transferable. This domain is familiar to most faculty and synonymous with the work of Bloom from the 1950s (Bloom, 1984) and the useful revisions made in 2001 (Anderson & Krathwohl, 2001).
Professional Skills / Affective domain
The affective is concerned with an individual’s values, and includes their abilities with respect to self perception through to abstract empathetic reasoning. In an extension to the early work by Bloom progressive stages take the learner from foundational ‘receiving’, through to the ‘internalization’ of personal value systems (Krathwohl, Bloom, & Masia, 1999). In the context of Higher Education programmes, particularly an era when the employability of graduates is stressed, an awareness of these professional values would do well to be built into the relevant modules.
Transferable Skills / Psychomotor domain
The psychomotor domain is less well researched and documented and this has meant a less than adequate recognition and incorporation into learning designs. Frequently tactile or technical skills become seen as ‘general skills’ or ‘transferable skills’ and there is little sense of progression. This domain refers to progressively complex manual or physical skills and so could identify the progressively complex skills of a biologist in using microscopes, or an economist using a statistics software package (Dave, 1967). I find this domain unfortunately neglected as I believe it would enhance course designs if note were taken of the practical technical skills required within disciplines and their articulation in Intended Learning Outcomes.
The Balance of Numbers
The actual balance between these domains in terms of how many Intended Learning Outcomes one might assign to them in the context of a 15 credit module will depend on the context of the module, its mode and its programme context. One might reasonably expect to see some differences in the balance of ILOs in modules in different contexts, illustrated below.
Domains
Level 5 University class taught Module
;
Work-based Level 5 Management Module
Practice-based Level 5 lab taught
Knowledge Domain
2
2
2
Intellectual Skills (cognitive)
4
2
2
Professional Skills (affective)
2
3
2
Transferable Skills (psychomotor)
2
3
4
And for those who appreciate a visual representation:
Distribution of ILOs in different domains
;
In this example each module has ten Intended Learning Outcomes but the emphasis within the module will change. Whilst it may be appropriate to stress intellectual skills (analysis, synthesis, evaluation) in a classroom based political science course for example, on might expect to see transferable skills (often described as practical, tactile or technical skills) stressed in a technical lab based course, skills such as manipulation, articulation and naturalisation of technical proficiency.
Holistic Learning
All too often Higher Education stresses the cognitive, over reliant perhaps on Bloom’s taxonomy and related work, and neglects the affective and psychomotor domains. This is has several consequences; it relegates anything that is not seen as ‘intellectual’ to a lower order of skills despite the fact that employers and students recognise and demand the need for broader skills (Mason, Williams, & Cranmer, 2006). In doing so it forces programme leaders into ‘bolt-on’ skills modules that demand additional institutional resource and student resource and frequently ill-serve the purpose. No learning design is truly student-centred if it is neglecting other domains of experience (Atkinson, 2011).
The model advocated here separates the knowledge domain and the intellectual skills, focussing the module designer on the ‘skills’ that will be acquired independent of the subject knowledge acquired. This, along with a focus on the affective and psychomotor skills, provides a framework for a module that is balanced in terms of what the student does, the context in which they do it, and correctly assessed ensures all these intended learning outcomes can be justifiably claimed in the student’s transcript.
Indeed it is not difficult to imagine a student coming to the end of the first stage of their degree, recognising that they have excelled in the psychomotor skills but struggled in the cognitive, and make module choices for future stages either to redress that balance or acknowledge their strengths and adjust choices to reflect future career path.
So how do you write learning outcomes across these four domains? That’s the subject of the next posting.
………………
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Atkinson, S. (2011). Embodied and Embedded Theory in Practice: The Student-Owned Learning-Engagement (SOLE) Model. The International Review of Research in Open and Distance Learning, 12(2), 1–18.
Bloom, B. S. (1984). Taxonomy of Educational Objectives Book 1: Cognitive Domain (2nd ed.). Addison Wesley Publishing Company.
Dave, R. (1967). Psychomotor domain. Presented at the International Conference of Educational Testing, Berlin.
Hussey, T., & Smith, P. (2002). The Trouble with Learning Outcomes. Active Learning in Higher Education, 3(3), 220–233.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1999). Taxonomy of Educational Objectives Book 2/Affective Domain (2nd ed.). Longman Pub Group.
Mason, G., Williams, G., & Cranmer, S. (2006). Employability Skills Initiatives in Higher Education: What Effects Do They Have On Graduate Labour Market Outcomes? National Institute of Economic and Social Research. Retrieved October 14th 2012 from http://ideas.repec.org/p/nsr/niesrd/280.html
