Some recent work with programme designers in other UK institutions suggests to me that quality assurance and enhancement measures continue to be appended to the policies and practices carried out in UK HEIs rather than seeing a revitalising redesign of the entire design and approval process.
This is a shame because it has produced a great deal of work for faculty in designing and administering programmes and modules, not least when it comes to assessment. Whatever you feel about intended learning outcomes (ILOs) and their constraints or structural purpose, there is nearly universal agreement that the purpose of assessment is not to assess students ‘knowledge of the content’ on a module. Rather the intention of assessment is to demonstrate higher learning skills, most commonly codified in the intended learning outcomes. I have written elsewhere about the paucity of writing effective ILOs and focusing them almost entirely the cognitive domain (intellectual skills), with the omission of other skill domains notably the effective (professional skills) and the psychomotor (transferable skills). Here I want to identify the need for close proximity between ILOs and assessment criteria.
It seems to me that well-designed intended learning outcomes lead to cogent assessment design. They also suggest that the use of a transparent marking rubric, used by both markers and students, creates a simpler process.
To illustrate this I wanted to share two alternative approaches to aligning assessment to the outcomes of a specific module. In order to preserve the confidentiality of the module in question some elements have been omitted but hopefully the point will still be clearly made.
Complex Attempt to Assessment Alignment
I have experienced this process in several Universities.
Intended Learning Outcomes are written (normally at the end of the ‘design’ process)
ILOs are mapped to different categorizations of domains, Knowledge & Understanding, Intellectual Skills, Professional Skills and Attitudes, Transferable Skills.
ILOs are mapped against assessments, sometimes even mapped to subject topics or weeks.
Students get first sight of the assessment.
Assessment Criteria are written for students using different categories of judgement: Organisation, Implementation, Analysis, Application, Structure, Referencing, etc.
Assessment Marking Schemes are then written for assessors. Often with guidance as to what might be expected at specific threshold stages in the marking scheme.
General Grading Criteria are then developed to map the schemes outcomes back to the ILOs.
Streamlined version of aligned assessment
I realise that this proposed structure is not suitable for all contexts, all educational levels and all disciplines. Nonetheless I would advocate that this is the optimal approach.
ILO are written using a clear delineation of domains; Knowledge, Cognitive (Intellectual), Affective (Values), Psychomotor (Skills) and Interpersonal. These use appropriate verb structures tied directly to appropriate levels. This process is explained in this earlier post.
A comprehensive marking rubric is then shared with both students and assessors. It identifies all of the ILOs that are being assessed. In principle we should only be assessing the ILOs in UK Higher Education NOT content. The rubric will differentiate the type of responses expected to achieve varies grading level.
There is an option to automatically sum grades given against specific outcomes or to take a more holistic view.
It is possible to weight specific ILOs as being worth more marks than others.
This approach works for portfolio assessment but also for a model of assessment where there are perhaps two or three separate pieces of assessment assuming each piece is linked to two or three ILOs.
Feedback is given against each ILO on the same rubric (I use Excel workbooks)
I would suggest that it makes sense to use this streamlined process even if it means rewriting your existing ILOs. I’d be happy to engage in debate with anyone about how best to use the streamlined process in their context.
Atkinson, S.P. (2015). Using Learning Design to Unleash the Power of Learning Analytics. In T. Reiners, B.R. von Konsky, D. Gibson, V. Chang, L. Irving, & K. Clarke (Eds.), Globally connected, digitally enabled. Proceedings ascilite 2015 in Perth (pp. 358-364 / CP:6-CP:10).
A very enjoyable presentation made this week at ascilite 2015 in Perth, Australia. Wonderful to engage with this vibrant and hospitable community. Amongst some fascinating presentations exploring the theoretical and information management dimension of learning analytics and academic analytics, my very foundational work on constructively aligned curricula and transparency in design was I believe welcomed.
I said in my paper that I believed “New learning technologies require designers and faculty to take a fresh approach to the design of the learner experience. Adaptive learning, and responsive and predicative learning systems, are emerging with advances in learning analytics. This process of collecting, measuring, analysing and reporting data has the intention of optimising the student learning experience itself and/or the environment in which the experience of learning occurs… it is suggested here that no matter how sophisticated the learning analytics platforms, algorithms and user interfaces may become, it is the fundamentals of the learning design, exercised by individual learning designers and faculty, that will ensure that technology solutions will deliver significant and sustainable benefits. This paper argues that effective learning analytics is contingent on well structured and effectively mapped learning designs.”
We hear much about the changing world of work and how slow higher and professional education is to respond. So in an increasingly competitive global market of Higher Apprenticeships and work-based learning provision I began to take a particular interest in students’ ‘graduateness’. What had begun as an exploratory look for examples of intended learning outcomes (ILO) with ’employability’ in mind ended up as this critical review published in an article entitled ‘Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes’ in the journal called Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education.
I randomly identified 20 UK institutions, 80 undergraduate modules and examined their ILOs. This resulted in 435 individual ILOs being taken by students in current modules (academic year 2014-2015) across different stages of their undergraduate journey (ordinarily in the UK this takes place over three years through Levels 4,5 and 6). This research reveals the lack of specificity of ILOs in terms of skills, literacies and graduates attributes that employers consistently say they want from graduates
The data in the table below from the full paper which describes the post-analysis attribution of ILOs to domains of educational objectives (see paper for methodology) which I found rather surprising. The first surprise was the significant percentage of ILOs which are poorly structured, given the weight of existing practice guidance and encouragement for learning designers and validators (notably from the UK Higher Education Academy and the UK Quality Assurance Agency). Some 94 individual ILOs (21.6%) had no discernible active verbs in their construction. 64 ILOs (14.7%) did not contain any meaningful verbs so could not be mapped to any educational domain. This included the infamous ‘to understand’ and ‘to be aware of’. So as a result only 276 ILOs (64%) were deemed ‘well-structured’ and were then mapped against four domains of educational objectives.
Table 8. Post-analysis attribution of ILOs to Domains of Educational Objectives
Level 4
Level 5
Level 6
Total
Knowledge(Subject Knowledge)
14
5
11
30
Cognitive (Intellectual Skills)
46
91
61
198
Affective(Professional Skills)
1
4
1
6
Psychomotor(Practical/Transferable Skills)
12
18
13
43
No Verbs
35
32
27
94
Not classifiable
23
30
11
64
Totals
131
180
125
435
Remember what I had been originally looking for were examples of ILOs that represented skills that the literature on employability and capabilities suggested should be there. These could have been anticipated to be those in the affective or psychomotor domains.
So it was rather surprising to see that of the 64% of the full sample that was codeable, sizeable percentage were cognitive (45.4%), a relatively small percentage fell into the psychomotor domain (9.8%), even less into the knowledge domain (6.8%) and a remarkably small number could be deemed affective (1.4%).
I say remarkable because the affective domain, sometimes detailed as personal and professional skills, are very much the skills that employers (and most graduates) prize above all else. These refer to the development of values and the perception of values, including professionalism, inter-personal awareness, timeliness, ethics, inter-cultural sensitivity, and diversity and inclusivity issues.
Apparently despite all the sterling work going on in our libraries and career services, employment-ready priorities within programmes and modules in higher education, are not integrated with teaching and learning practices. I suggest that as a consequence, this makes it difficult for students to extract, from their learning experience within modules, the tangible skill development required of them as future employees.
There is an evident reliance by module designers on the cognitive domain most commonly associated at a lower level with ‘knowing and understanding’ and at a higher level as ‘thinking and intellectual skills’. The old favourite ‘to critically evaluate’ and ‘to critically analyse’ are perennial favourites.
There is much more to the picture than this single study attempts to represent but I think it is remarkable not more attention is being paid to the affective and psychomotor domains in module creation.
More analysis, and further data collection will be done, to explore the issue at programme level and stage outcomes (is it plausible that module ILOs are simply not mapped and unrelated and all is well at programme level). I would also be interested to explore the mapping of module and programme ILOs to specified graduate attributes that many institutions make public.
I go on in the full paper about the relative balance of different ILOs in each of the domains depending on the nature of the learning, whether it is a clinical laboratory module or a fieldwork module or a literature-based module.
The reason I think this is important, and I have written here before, that this important (it is about semantics!), is that students are increasingly demanding control over their choices, their options, the shape of their portfolios, their ‘graduateness’, and they need to be able to identify their own strengths and weaknesses and make meaningful modules choices to modify the balance of the skills acquired in a ’practical’ module compared with those in a ‘cerebral’ one. I conclude that the ability to consciously build a ‘skills profile’ is a useful graduate attribute in itself…. which incidentally would be an affective ILO.
Full citation:
Atkinson, S. 2015 Jul 9. Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes. Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education [Online] 10:2. Available: http://community.dur.ac.uk/pestlhe.learning/index.php/pestlhe/article/view/194/281
I have no idea what the protocol is for naming versions of things. I imagine, like me, someone has an idea of what the stages are going to look like, when a truly fresh new is going to happen. For me I have a sense that version 4.0 of the SOLE Toolkit will incorporate what I am currently learning about assessment and ‘badges’, self-certification and team marking. But for now I’m not there yet and am building on what I have learnt about student digital literacies so I will settle for Version 3.5.
This version of the SOLE Toolkit 3.5.1, remains a completely free, unprotected and macro-free Excel workbook with rich functionality to serve the learning designer. In version 3.0 I added more opportunities for the student to use the toolkit as an advanced organiser offering ways to record their engagement with their learning. It also added in some ability to sequence learning so that students could plan better their learning although I maintained this was guidance only and should allow students to determine their own pathways for learning.
Version 3.5 has two significant enhancements. Firstly, it introduces a new dimension, providing a rich visualization of the learning spaces and tools that students are to engage with in their learning. This provides an alternative, fine-grain, view of the students modes of engagement in their learning. It permits the designer to plan not only for a balance of learning engagement but also a balance of environments and tools. This should allow designers to identify where ‘tool-boredom’ or ‘tool-weariness’ is possibly a danger to learner motivation and to ensure that a range of tools and environments allow students to develop based on their own learning preferences.
Secondly, it allows for a greater degree of estimation of staff workload, part of the original purpose of the SOLE Model and Toolkit project back in 2009. This faculty-time calculations in design and facilitating are based on the learning spaces and tools to be used. This function allows programme designers and administrators, as well as designers themselves, to calculate the amount of time they are likely to need to design materials and facilitate learning around those materials.
Sharing a paper today on the visualisation of educational taxonomies. I have finally got around to putting into a paper some of the blog postings, discussion, tweets and ruminations of recent years on educational taxonomies. I am always struck in talking to US educators (and faculty training teachers in particular) of the very direct use made of Bloom’s original 1956 educational taxonomy for the cognitive domain. They seem oblivious however to other work that might sit
(conceptually) alongside Bloom is a way to support their practice.
In New Zealand, whilst at Massey, I got into some fascinating discussions with education staff about the blurring of the affective and cognitive domains, significant in cross-cultural education, and this led me to look for effective representations of domains. I came across an unattributed circular representation that made instant sense to me, and set about mapping other domains in the same way. In the process I found not only a tool that supported and reinforced the conceptual framework represented by Constructive Alignment, but also a visualising that supported engagement with educational technologies and assessment tools. I hope this brief account is of use to people and am, as always, very open to feedback and comment.
I’m very grateful to those colleagues across the globe who have expressed interest in using these visual representations and hope to be able to share some applicable data with everyone in due course.
I delivered a webinar recently on technology enhanced learning. It was a 90-minute session (possibly too long) in Adobe Connect attended by some 15 faculty. Several of the evaluation comments suggested that the first third of the webinar, dealing with shared understandings of terminology and a history of the subject under discussion, was unnecessary, superfluous. I’m struck by how often in my current practice in British higher education that the contextualisation of what we do is often treated as a luxury. Pragmatism pervades everything and there is an assumption that we all know where we are, we all know what needs to be done, and the objective is simply to do it. Universities have often been accused of being ivory towers, places where people ruminate detached from reality, but there must surely be a place for a pause and a thought.
Rongomaraeroa – Te Papa’s Marae (Authors Photo 2005)
Perhaps the reason I reacted with some discomfort to the suggestion that the historical context to a discussion of technology enhanced learning was superfluous has to do with the reactions I get from colleagues on another project currently underway. The POISE project, part of the HEA Internationalisation change initiative, stands for personal orientation to international student experience. The original idea had been to establish to support individuals to identify their own epistemological assumptions, students and staff, in order that a more meaningful dialogue about adjusting to higher education study might be possible. But whilst the stress of the original project was on personal orientation the realities of implementation in the British HE context consistently stresses the student experience, the here and now, today’s problem being dealt with by today’s student support person. There is a sense in which holistic medicine, whole person medicine, has been replaced by the liberal prescription of the pragmatic pill.
I found myself turning back to the concept, the principles, of Kaupapa Māori. I am not Māori, and so these principles are necessarily engaged with at an intellectual and emotional level rather than from within, based on three periods of working in New Zealand since 1998. In 1990 Graham Hingangaroa Smith outlined six principles of Kaupapa Māori within the context of education, its implementation and research. Other theorists have expanded these concepts further in the years since. This is an evolving body of a communities’ intellectual, spiritual and inter-personal exploration of identity. In a very real sense this is ‘identity-work’. It is something we appear to do very little of in the British ‘academy’.
Kaupapa Māori principles include (but are not limited to):
Tino Rangatiratanga – The Principle of Self-determination: to sovereignty, autonomy, control, self-determination and independence, allowing for and advocating Maori control over their own culture, aspirations and destiny.
Taonga Tuku Iho – The Principle of Cultural Aspiration: asserting the centrality and legitimacy of Te Reo Māori (language), Tīkanga (customary practices, ethics, cultural behaviours) and Mātauranga Māori (wisdom).
Ako Māori – The Principle of Culturally Preferred Pedagogy: acknowledging Māori teaching and learning practices and learning preferences.
Kia piki ake i ngā raruraru o te kainga – The Principle of Socio-Economic Mediation: the need to positively alleviate the disadvantages experienced by Māori communities.
Whānau – The Principle of Extended Family Structure: acknowledges the relationships that Māori have to one another and to the world around them. Core to Kaupapa Māori and key elements of Māori society and culture, acknowledging the responsibility and obligations of everyone to nurture and care for these relationships.
Kaupapa – The Principle of Collective Philosophy: the shared aspirations and collective vision of Māori communities.
Other powerful concepts have developed within Kaupapa Māori including the principles of:
Ata – The Principle of Growing Respectful Relationships (Pohata 2005): relates to the building and nurturing of relationships, negotiating boundaries, creating respectful spaces and corresponding behaviours.
It is not only the substance, and there is undoubtedly something substantial about Kaupapa Māori, that appeals to me, rather it is the principle that behind each action, each intervention, there is a purposeful connection to a collective sense of people, of belonging. There appears to be a disconnect between the day-to-day activities of providing education in the British context and an engagement, a deliberate and conscious engagement, with the development of the individual that education is intended to form.
Kaupapa Māori, and indeed other world views that have developed independently of the western ‘scientific’ positivist paradigms, create different epistemological frameworks, different spaces within which educational discourses occur. I learnt in a very personal way in 2008 when I joined Massey University that the Māori world view does affect the way New Zealand educators, of all cultural backgrounds, see the world in which they educate. I drew a model on the whiteboard and a colleague simply asked ‘does it have to go that way up’. It is subtle, not always as evident to them as to those visiting from the outside, but it is there, a cultural ‘undertone’ that enriches and suffuses the discourse. Whilst we have done much to think about student centred learning in the UK, we often appear to mean placing the student at the centre of our machine not centering the student. We prepare them to fit into the universal mechanical rational world we anticipate needs and wants them, we do not equip them well to reshape themselves and their world. I continue to believe that understanding the context, presumptions and assumptions of any particular discipline subject or issue is an important precursor to meaning making. I believe an epistemological self-awareness is a prerequisite to a meaningful education.
Smith, G. H. (1990) ‘Research Issues Related to Maori Education’, paper presented to NZARE Special Interest Conference, Massey University, reprinted in 1992, The Issue of Research and Maori, Research Unit for Maori Education, The University of Auckland
Since October 17th 2012 [see updates] when I shared the most recent work on visualising taxonomies in a circular form and aligning these active verb patterns to particular assessment forms, I have had some great feedback – for which thank you. As a consequence, I have made a few clarifications which I hope will help those of you who want to use these visualisations in your conversations with peers or in academic educational development sessions. The biggest change has been to ‘turn’ the circles through 72′ clockwise so that the vertical denotes a “12-noon” start. I hesitate about this because it perhaps over stresses our obsession which mechanical process which isn’t my intention, but many said they would prefer this and so here it is. The second change has been to review, in the light of my own use, and some literature sources (noted on the images themselves) some of the active verbs and evidence.
I am very grateful for the feedback and hope to receive more. In answer to the question about citing this work; there is a journal article and a book chapter in the works, in the meantime please feel free to cite the blog posts. Or indeed personal correspondence at spa@sijen.com if you would like to share how these may be working for you in practice.
Click on the images to get a decent quality print version – please email if you would like the original PowerPoint slide to amend and modify.
Cognitive Domain – Circle – Taxonomy – Version 4 – November 2012 (Intellectual Skills)
Cognitive Domain – Taxonomy Circle
Affective Domain – Circle – Taxonomy – Version 4 – November 2012 (Professional and Personal Skills)
Affective Domain – Taxonomy Circle
Psychomotor Domain – Circle – Taxonomy – Version 4 – November 2012 (Transferable Skills)
Psychomotor Domain – Taxonomy Circle – after Dave (1969/71)
Knowledge Domain – Circle – Taxonomy – Version 2 – November 2012 (Subject/Discipline Skills)
This representation is perhaps the most ‘controversial’ as it represents the ‘knowledge dimension’ articulated by Anderson and colleagues as a separate domain. For the purposes of working with subject-centric academics within their disciplines as they write intended learning outcomes and assessment, I have found this a useful and sensible thing to do. I have separated out the notion of ‘contextual knowledge’ which is also not going to please everyone.
Knowledge Domain – Taxonomy Circle
I hope these representations are of some use to you in your practice. Simon (13 Nov 2012)
Four ‘Domains’ of educational objectives represented in a circular form
I think being able to visualise things is important. Faculty and learning designers need to be able to see Intended Learning Outcomes (ILOs) take shape and mant find existing lists are uninspiring. It’s not uncommon for faculty and instructional designers to get tired and weary of ILOs; they can feel restrictive, repetitive, formulaic and sometimes obstructive. In previous posts I’ve tried to suggest that the bigger picture, the challenges of effective 21st century university level learning design, make them not only useful, but also essential. If you don’t agree, don’t bother reading. I’m not going to try and persuade you. If you think there’s some truth in the argument and you want to engage with ILOs to make your teaching more focussed, your students increasingly autonomous and your graduates equipped with meaningful evidence, then I hope I have something worthwhile sharing and will welcome your thoughts.
My argument is that a module (a substantial unit of a full years undergraduate study), and the programme of which is part, should have clearly articulated outcomes in four domains:
Knowledge and understanding – or the knowledge domain
Intellectual Skills – or the cognitive domain
Professional Skills – or the affective domain
Transferable Skills – or the psychomotor domain
I’m suggesting one SHOULD expect to see a different distribution of ILOs between the outcomes in these domains depending on the focus of the module and the level of study. One might expect to see a second year anthropology module on ‘theoretical perspectives’ emphasising cognitive outcomes and a module being studied alongside it on ‘research design and techniques’ emphasising affective and psychomotor outcomes. One might reasonably expect to see more foundational ‘knowledge and understanding’ outcomes in the first year of a programme of study, and more ‘cognitive’ outcomes at the end of the programme. The lack of this ‘designed articulation’ in many modules undermines their value to the student and ultimately to faculty.
The basic principle is that an outcome should be assessable. Lots of great stuff can happen in your teaching and students’ learning that DOESN’T need to be assessed. It can be articulated in the syllabus, it just isn’t a measured outcome. A student should be able, at the end of this course of study (module or programme), to evidence that they have attained the intended learning outcomes. This evidence has been assessed in some way and the student is then able to point to the ILOs amassed throughout their programme and say “I can demonstrate that I learnt to DO this”.
Representing Taxonomies
There has been a significant shift in the language we now use from the original work in the 1950s by Bloom and colleagues. The passively descriptive language of Bloom’s Taxonomy has become the active language of Anderson and Krathwohl (Anderson & Krathwohl, 2001). The taxonomies have moved from Evaluation to Evaluate, from Analysis to Analyse. This is significant in that the emphasis has moved away from describing what the focus of the teaching is supposed to be, to the demonstrable outcomes of the learning.
The illustration above consists of four visual ‘wheels’ that I have used to discuss learning outcomes with faculty in the context of module and programme design at Massey University in New Zealand and at the LSE and BPP University College in the United Kingdom. These visual representations were inspired by work done elsewhere, on the cognitive domain in particular. The first documented example of this circular representation I have been able to find is attributed to Barbara Clark in 2002, but a great many people have since represented Bloom’s original, and the revised, cognitive domain in this way.
The circular representation has the higher level terms at the centre, proto-verbs if you will, surrounded by a series of active verbs that articulate actions an individual might undertake to generate evidence, of their ability to represent to proto-verb. The circular visualisation also serves to create a more fluid representation of the stages, or divisions, in the proto-verbs. Rather than a strict ‘step-by-step’ representation where one advances ‘up’ the proto-verbs, one might consider this almost like the dial on an old telephone, in every case one starts at the ‘foundational’ and dials-up though the stages to the ‘highest’ level. Each level relies on the previous. It may be implicit that to analyse something, one will already have acquired a sense of its application, and that application is grounded on subject knowledge and understanding. So the circle is a useful way of visualising the interconnected nature of the process. Most importantly in my practice, it’s a great catalyst for debate.
The circular representations of the domains and associated taxonomies also serve to make learning designers aware of the language they use. Can a verb be used at different levels? Certainly. Why? Because context is everything. One might ‘identify’ different rock samples in a first year geology class as part of applying a given classification of rocks to samples, or one might identify a new species of insect as part of postgraduate research programme. The verb on its own does not always denote level. I talk about the structure of ILOs in a subsequent post.
Structure of the circular representations of Educational Taxonomies
More recent representations have created new complex forms that include the outer circle illustrated here. I’ve found these rather useful, in part because they often prove contentious. If the inner circle represents (in my versions) the proto-verbs within our chosen taxonomies, and the next circle represent that active verbs used to describe the Intended Learning Outcomes (ILO) AND the Learning and Teaching Activities (TLS), the outermost circle represents the evidence and assessment forms used to demonstrate that verb. Increasingly I’ve used this to identify educational technologies and get faculty thinking more broadly about how they can assess things online as well as in more traditional settings. The outermost circle will continue to evolve as our use of educational technologies evolves. In Constructive Alignment one might reasonably expect students’ learning activity to ‘rehearse’ the skills they are ultimately to evidence in assessment (Biggs & Collis, 1982; Boud & Falchikov, 2006) and the forms to enable that are becoming increasingly varied.
Re-visioning Taxonomies
One of my favourite representations of the relationship between the knowledge dimension and the cognitive domain is from Rex Heer at Iowa State University’s Center for Excellence in Learning and Teaching (http://www.celt.iastate.edu/teaching/RevisedBlooms1.html accessed ). It’s an interactive model that articulates the relationship, as Anderson and Krathwohl saw it, rather well. My own interest, as we look to effective ILOs, is to separate out the knowledge dimension as a subject or knowledge domain and have faculty articulate this clearly for students, before reconnecting to the other domains. A process I’ll talk about subsequently.
Here are my four ‘working circles’ using adaptations of taxonomies from Anderson and Krathwohl (Knowledge and Understanding, and Cognitive), Krathwohl et al (Affective) and Dave (Psychomotor). I have adapted the Knowledge Dimension of Anderson and Krathwohl to do two things; to describe the dimension in terms of active verbs rather than as a definition of the nature of the knowledge itself, and I have incorporated a stage I believe is under represented in their articulation. I have added the ability to ‘ contextualise’ subject knowledge between the ability to specify it (Factual) and the ability to conceptualize (Conceptual). I have also rearticulated the original ‘Metacognitive’ as the ability to ‘Abstract‘. This will doubtless need further work. My intent is not to dismiss the valuable work already in evidence around the relationship between a knowledge dimension and the cognitive domain, rather it is to enable faculty, specifically when writing learning outcomes, to identify the subject, discipline or knowledge to be enabled in more meaningful ways.
These images are provided as JPG images. If you would like me to email the original PowerPoint slides (very low-tech!) so that you can edit, amend and enhance, I am happy to do so. I only ask that you enhance my practice by sharing your results with me.
I hope these provoke thought, reflection and comment. Feel free to use them with colleagues in discussion and let me know if there are enhancements you think would make them more useful to others.
Cognitive Domain – Intellectual Skills
Cognitive Domain – Intellectual Skills
Affective Domain – Professional and Personal Skills
Affective Domain – Professional and Personal Skills
Psychomotor Domain- Practical, Technical and Transferable Skills
Psychomotor Domain- Practical, Technical and Transferable Skills
Knowledge Domain – Subject and Discipline Knowledge
Knowledge Domain- Subject or Discipline Skills
The next post will illustrate the usefulness of these visualisations in drafting Intended Learning Outcomes with some examples.
………………………………………………………………………………………
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Biggs, J. B., & Collis, K. F. (1982). Evaluating the Quality of Learning: Structure of the Observed Learning Outcome Taxonomy. Academic Press Inc.
Boud, D., & Falchikov, N. (2006). Aligning assessment with long‐term learning. Assessment & Evaluation in Higher Education, 31(4), 399–413.
In my last posting I suggested that a module specification could usefully have four sections, clearly articulated, for Intended Learning Outcomes, so that a student could identify from their assessment evidence that they had met specific ILOs in a range of domains. In doing so they not only have a useful platform to identify future learning needs, but also the potential to negotiate the accreditation of prior accredited learning in a much more fine-grained and meaningful way, something I fully expect to become a significant future of international higher education accords in the next few years as institutions face up to the challenge of accredited OER schemes and credit bearing MOOCs. I believe the design of intended learning outcomes for modules and programmes will become a strategic priority.
Not everyone agrees ILOs are effective and a useful critique from Hussey and Smith is well worth reading (Hussey & Smith, 2002).
How many Intended Learning Outcomes (ILOs) one designs into a module or a programme level specification has to depend on the scope of the module or programme itself. I’m sure colleagues can adapt what I’m saying here to their own quality assurance and institutional contexts.
For the purpose of this reflection let me take a single module, worth 15 credits. In the UK context this would frequently represent one-eighth of a stage of undergraduate degree study, there being three stages each representing 120 credits. Again, in the UK context there is a strong notion of progression in higher order thinking skills between the first stage of undergraduate study (level 4) and the final stage (level 6). This progression is articulated in generic guidance that captures much of this ILO debate and in subject specific guidance drawing on the discipline communities to create ‘benchmarks’ for what be expected to be in any named award (www.qaa.ac.uk) . Level 5 would represent the second stage of undergraduate study in the UK context, the equivalent of an exit point for a Higher National Diploma or a Foundation Degree, the European Qualifications Framework Level 5 and within the EHEA (Bologna) sometimes referred to as a ‘Short Cycle’ award.
My example then is for a 15-credit module at level 5. The UK quality assurance agency does not specify periods of study for credit, but sector norms talk in terms of notional study hours and it is perhaps helpful therefore to think of 15 credits as 150 notional study hours, 30 credits as 300 notional study hours and so on.
The Domains
Before proposing a model for ‘how many’, I will briefly remind myself what these four sections, or domains, of Intended Learning Outcomes represent. They are;
Knowledge and understanding – subject domain
Intellectual Skills – or the cognitive domain
Professional Skills – or the affective domain
Transferable Skills – or the psychomotor domain
Knowledge and understanding – subject domain
The subject domain is often conflated with the cognitive domain, which is understandable as it is within Bloom’s ubiquitous taxonomy, but this does tend to confuse faculty as to the distinction between knowing and understanding a body of factual knowledge and being able to do something with that factual knowledge. The Subject domain can, and in my opinion should, be limited to defining the subject area for illustrative purposes for the student. Since the principle is that all Intended Learning Outcomes should be assessed and it is actually rather difficult to assess whether someone ‘understands’ something without having them ‘operationalize’ the knowledge, I tend not to get too hung up on the active verbs used in this domain, contenting myself it serves to contextualise what follows, but maybe I should and another post later will unpack Anderson and Krathwohl’s Knowledge Dimension in more detail.
Intellectual Skills / Cognitive domain
This domain refers to ‘knowledge structures’ building from the base of the Subject domain, the “knowing the facts”, towards high order thinking skills in which these facts become operationalized and transferable. This domain is familiar to most faculty and synonymous with the work of Bloom from the 1950s (Bloom, 1984) and the useful revisions made in 2001 (Anderson & Krathwohl, 2001).
Professional Skills / Affective domain
The affective is concerned with an individual’s values, and includes their abilities with respect to self perception through to abstract empathetic reasoning. In an extension to the early work by Bloom progressive stages take the learner from foundational ‘receiving’, through to the ‘internalization’ of personal value systems (Krathwohl, Bloom, & Masia, 1999). In the context of Higher Education programmes, particularly an era when the employability of graduates is stressed, an awareness of these professional values would do well to be built into the relevant modules.
Transferable Skills / Psychomotor domain
The psychomotor domain is less well researched and documented and this has meant a less than adequate recognition and incorporation into learning designs. Frequently tactile or technical skills become seen as ‘general skills’ or ‘transferable skills’ and there is little sense of progression. This domain refers to progressively complex manual or physical skills and so could identify the progressively complex skills of a biologist in using microscopes, or an economist using a statistics software package (Dave, 1967). I find this domain unfortunately neglected as I believe it would enhance course designs if note were taken of the practical technical skills required within disciplines and their articulation in Intended Learning Outcomes.
The Balance of Numbers
The actual balance between these domains in terms of how many Intended Learning Outcomes one might assign to them in the context of a 15 credit module will depend on the context of the module, its mode and its programme context. One might reasonably expect to see some differences in the balance of ILOs in modules in different contexts, illustrated below.
Domains
Level 5 University class taught Module
;
Work-based Level 5 Management Module
Practice-based Level 5 lab taught
Knowledge Domain
2
2
2
Intellectual Skills (cognitive)
4
2
2
Professional Skills (affective)
2
3
2
Transferable Skills (psychomotor)
2
3
4
And for those who appreciate a visual representation:
Distribution of ILOs in different domains
;
In this example each module has ten Intended Learning Outcomes but the emphasis within the module will change. Whilst it may be appropriate to stress intellectual skills (analysis, synthesis, evaluation) in a classroom based political science course for example, on might expect to see transferable skills (often described as practical, tactile or technical skills) stressed in a technical lab based course, skills such as manipulation, articulation and naturalisation of technical proficiency.
Holistic Learning
All too often Higher Education stresses the cognitive, over reliant perhaps on Bloom’s taxonomy and related work, and neglects the affective and psychomotor domains. This is has several consequences; it relegates anything that is not seen as ‘intellectual’ to a lower order of skills despite the fact that employers and students recognise and demand the need for broader skills (Mason, Williams, & Cranmer, 2006). In doing so it forces programme leaders into ‘bolt-on’ skills modules that demand additional institutional resource and student resource and frequently ill-serve the purpose. No learning design is truly student-centred if it is neglecting other domains of experience (Atkinson, 2011).
The model advocated here separates the knowledge domain and the intellectual skills, focussing the module designer on the ‘skills’ that will be acquired independent of the subject knowledge acquired. This, along with a focus on the affective and psychomotor skills, provides a framework for a module that is balanced in terms of what the student does, the context in which they do it, and correctly assessed ensures all these intended learning outcomes can be justifiably claimed in the student’s transcript.
Indeed it is not difficult to imagine a student coming to the end of the first stage of their degree, recognising that they have excelled in the psychomotor skills but struggled in the cognitive, and make module choices for future stages either to redress that balance or acknowledge their strengths and adjust choices to reflect future career path.
So how do you write learning outcomes across these four domains? That’s the subject of the next posting.
………………
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Atkinson, S. (2011). Embodied and Embedded Theory in Practice: The Student-Owned Learning-Engagement (SOLE) Model. The International Review of Research in Open and Distance Learning, 12(2), 1–18.
Bloom, B. S. (1984). Taxonomy of Educational Objectives Book 1: Cognitive Domain (2nd ed.). Addison Wesley Publishing Company.
Dave, R. (1967). Psychomotor domain. Presented at the International Conference of Educational Testing, Berlin.
Hussey, T., & Smith, P. (2002). The Trouble with Learning Outcomes. Active Learning in Higher Education, 3(3), 220–233.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1999). Taxonomy of Educational Objectives Book 2/Affective Domain (2nd ed.). Longman Pub Group.
Mason, G., Williams, G., & Cranmer, S. (2006). Employability Skills Initiatives in Higher Education: What Effects Do They Have On Graduate Labour Market Outcomes? National Institute of Economic and Social Research. Retrieved October 14th 2012 from http://ideas.repec.org/p/nsr/niesrd/280.html
MOOCs, self generated OER based curricula, kite-marking schemes, and elaborate credit transfer schemes are a reality in increasingly complex higher education sector. Students often pursuing studies from within the world of work where physical mobility of employable precludes commitment to a single campus based programme over four years require well defined, constructively aligned, module designs. Clever module design means clever programme design, clever portfolios and successful institutions. Learning design is no longer just an issue for the Quality Office; the Strategy people are beginning to care too.
The vast majority of UK Universities now are able to produce detailed module and programme specifications for their teaching programmes. Specification templates usually detail the aims and objectives, resources, indicative scheme of work, staffing and mode of delivery. They also routinely use a template to generate the Intended Learning Outcomes (ILO) for the module or programme. Frequently divided into three or four sections covering, knowledge and understanding, intellectual skills (cognitive domain), professional and practical skills (affective domain) and general transferable skills (psychomotor skills), these templates are completed with varying degrees of comprehension as module validation panels will attest.
The logic is that to achieve a well-structured and constructively aligned curricula, the module team should determine what the ILOs for the module are to be (Biggs & Tang, 2007). What will the learner be able to do at the end of the module? Having determined the ILOs the team would then determine how they would enable the student to demonstrate achievement of the outcomes and draft an appropriate assessment strategy. Then, and only then, the module design team would look at what the student needed to be able to demonstrate and work out what was needed as input. Outcomes first, assessment second, teaching inputs third.
It’s not an easy thing to do. As teachers we’re passionate about our subjects, anxious to impart what we know is important, what ‘did it for us’, and at some point in this process many faculty will ‘go native’, reach for the seminal text (or the nearest thing to it, their own book), and start thinking about what the students need to know. This can of course produce fantastic learning experiences and there are a great many exciting modules drafted on the backs of envelopes without specification templates. They don’t make for effective records of achievement however.
Accreditation of prior accredited learning has always been a challenge. An effective template for module and programme design makes a significant difference. Students should be able to identify from their transcript exactly what it is they can evidence as intended learning outcomes. I would argue further that phases in learning and teaching activity should also have notable objectives that map directly to the ILOs (See the SOLE model described in Atkinson, 2011).
So how many intended learning outcomes, how many affective, how many cognitive, how many is too many? My next post will be my reasoning on that issue.
Atkinson, S. (2011). Developing faculty to integrate innovative learning in their practice with the SOLE model. In S. Ferris (Ed.), Teaching, Learning and the Net Generation: Concepts and Tools for Reaching Digital Learners. Hershey, PA: IGI Global.
Biggs, J., & Tang, C. (2007). Teaching for Quality Learning at University: What the Student does (3rd ed.). Buckingham. GB: Open University Press.
I have experienced this process in several Universities.
