Tag: Taxonomies

Free Online CPD Course on Learning Outcomes (until 14th January 2023)

Have you got some time for professional development over the holiday period? Or do you have colleagues or design teams working on course designs over the holiday period?

Anyone who has ever tried to assess or teach to poorly learning outcomes, and then tried to defend their practices or results, will tell you that getting it right at the offset saves a huge amount of effort and heartache.

Intended Learning Outcomes are the foundations of any sound well-aligned course and programme design. Being able to create effective well-structured learning outcome is a valuable skill required of all learning designers, faculty and quality officers.

I have created a short, self-study, course hosted on a Moodle instance. The full course will take between up to 10 hours at a leisurely pace but is designed to allow you to navigate your way through it as you please. You are welcome to dip in and out. The course complements the book ‘Writing Good Learning Outcomes and Objectives’. (https://www.amazon.com/dp/0473657929/)

Join the free course entitled ‘Designing Effective Intended Learning Outcomes’ at https://sijen.net/courses

Interpersonal Domain

[See Courses on Using Educational Taxonomies]

The vast majority of employers ask that new employees, notably graduates, be effective communicators; that they should be able to work well within a team; that they take responsibility and that they are accountable for their actions. Increasingly in a global context, new employees are also expected to be culturally ‘aware’ or ‘sensitive’. A great many universities go to significant effort to promote their ‘graduate attributes’ that usually include things like ‘global citizenship‘ and ‘being an effective member of society‘.

2018 Circular Representation of the Interpersonal Domain
2018 Circular Representation of the Interpersonal Domain

These abilities or attributes, communication, conflict resolution, collaboration, and cross-culture communication, all fall within the educational taxonomy of educational objectives, described as the interpersonal domain, with some overlap to an affective domain as it denotes personal value structures. The affective certainly underpins the interpersonal. Yet it is remarkable to find an institution, certainly here in the UK, in which interpersonal domain is adequately represented in their intended learning outcomes (ILO) with any notion of progression throughout a programme.

There will be elusions to ‘being able to work together in a team’ or ‘communicate effectively’ but these are rarely articulated in the form of an assessable ILO. Surely, given its importance as a personal attribute interpersonal skills should be the central feature of at least some modules within any given programme of studies. We know students pay more attention to those skills that are directly assessed so rather than having catch-all communication-lite style ILOs we should direct address and assess such attributes. My scholarship has derived an interpersonal domain taxonomy that maps the four facets of the interpersonal, communication, conflict resolution, collaboration, and cross-culture communication. Mapped within a single domain across five progressively complex levels of competence (articulate, argue, debate, translate, interpret) all four facets are represented.

As with all of my circular visual representations the boundaries between levels is fluid and can be breached by designers based on their personal needs. There may be reasons for articulating a ‘lower-level’ ILO for conflict resolution within an ‘articulate’ range whilst at the same time having an ILO addressing cross-cultural awareness from the highest ‘interpret’ level in the same module. This visual representation is intended simply to prompt discussions within learning design teams as to the appropriate language for structural ILOs and associated assessment. I also hope that it advocates for a greater balance across the domains.

BETT 2017: Learning Design and Technology in HE

It was my pleasure today to present some of my visualisation tools to be used in higher education learning design. At the BETT2017 exhibition in London, with a small crowd of some 25 people, I shared the following presentation. With just 20 mins (plus 10 for questions) it was really simply an opportunity to emphasise that technology needs to be intelligently designed into programmes and modules in service of specified learning outcomes to be meaningful. I unveiled some recent work on an original taxonomy for the ‘Interpersonal Domain’ and shared an updated version of the Toolkit (available here)

Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes

[See Courses on Educational Taxonomies]

We hear much about the changing world of work and how slow higher and professional education is to respond. So in an increasingly competitive global market of Higher Apprenticeships and work-based learning provision I began to take a particular interest in students’ ‘graduateness’. What had begun as an exploratory look for examples of intended learning outcomes (ILO) with ’employability’ in mind ended up as this critical review published in an article entitled ‘Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes’ in the journal called  Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education.

I randomly identified 20 UK institutions, 80 undergraduate modules and examined their ILOs. This resulted in 435 individual ILOs being taken by students in current modules (academic year 2014-2015) across different stages of their undergraduate journey (ordinarily in the UK this takes place over three years through Levels 4,5 and 6). This research reveals the lack of specificity of ILOs in terms of skills, literacies and graduates attributes that employers consistently say they want from graduates

The data in the table below from the full paper which describes the post-analysis attribution of ILOs to domains of educational objectives (see paper for methodology) which I found rather surprising. The first surprise was the significant percentage of ILOs which are poorly structured, given the weight of existing practice guidance and encouragement for learning designers and validators (notably from the UK Higher Education Academy and the UK Quality Assurance Agency). Some 94 individual ILOs (21.6%) had no discernible active verbs in their construction.  64 ILOs (14.7%) did not contain any meaningful verbs so could not be mapped to any educational domain. This included the infamous ‘to understand’ and ‘to be aware of’. So as a result only 276 ILOs (64%) were deemed ‘well-structured’ and were then mapped against four domains of educational objectives.

Table 8.          Post-analysis attribution of ILOs to Domains of Educational Objectives

  Level 4 Level 5 Level 6 Total
Knowledge(Subject Knowledge) 14 5 11 30
Cognitive (Intellectual Skills) 46 91 61 198
Affective(Professional Skills) 1 4 1 6
Psychomotor(Practical/Transferable Skills) 12 18 13 43
No Verbs 35 32 27 94
Not classifiable 23 30 11 64
Totals 131 180 125 435

Remember what I had been originally looking for were examples of ILOs that represented skills that the literature on employability and capabilities suggested should be there. These could have been anticipated to be those in the affective or psychomotor domains.

So it was rather surprising to see that of the 64% of the full sample that was codeable,  sizeable percentage were cognitive (45.4%), a relatively small percentage fell into the psychomotor domain (9.8%), even less into the knowledge domain (6.8%) and a remarkably small number could be deemed affective (1.4%).

I say remarkable because the affective domain, sometimes detailed as personal and professional skills, are very much the skills that employers (and most graduates) prize above all else. These refer to the development of values and the perception of values, including professionalism, inter-personal awareness, timeliness, ethics, inter-cultural sensitivity, and diversity and inclusivity issues.

Apparently despite all the sterling work going on in our libraries and career services, employment-ready priorities within programmes and modules in higher education, are not integrated with teaching and learning practices. I suggest that as a consequence, this makes it difficult for students to extract, from their learning experience within modules, the tangible skill development required of them as future employees.

There is an evident reliance by module designers on the cognitive domain most commonly associated at a lower level with ‘knowing and understanding’ and at a higher level as ‘thinking and intellectual skills’. The old favourite ‘to critically evaluate’ and ‘to critically analyse’ are perennial favourites.

There is much more to the picture than this single study attempts to represent but I think it is remarkable not more attention is being paid to the affective and psychomotor domains in module creation.

More analysis, and further data collection will be done, to explore the issue at programme level and stage outcomes (is it plausible that module ILOs are simply not mapped and unrelated and all is well at programme level). I would also be interested to explore the mapping of module and programme ILOs to specified graduate attributes that many institutions make public.

I go on in the full paper about the relative balance of different ILOs in each of the domains depending on the nature of the learning, whether it is a clinical laboratory module or a fieldwork module or a literature-based module.

The reason I think this is important, and I have written here before, that this important (it is about semantics!), is that students are increasingly demanding control over their choices, their options, the shape of their portfolios, their ‘graduateness’, and they need to be able to identify their own strengths and weaknesses and make meaningful modules choices to modify the balance of the skills acquired in a ’practical’ module compared with those in a ‘cerebral’ one. I conclude that the ability to consciously build a ‘skills profile’ is a useful graduate attribute in itself…. which incidentally would be an affective ILO.

You can download the full paper here LINK.

Also available on ResearchGate and Academia.edu

Full citation:
Atkinson, S. 2015 Jul 9. Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes. Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education [Online] 10:2. Available: http://community.dur.ac.uk/pestlhe.learning/index.php/pestlhe/article/view/194/281

Is Higher Education lacking its affective dimension?

[See Courses on  Educational Taxonomies]

 

Affective Domain March 2015
Affective Domain March 2015

Whilst the majority of writings and reflection concerning the use of taxonomies of educational objectives remain focused around the cognitive domain, typified by Bloom (1984), there is a growing attention being paid to the affective domain, particularly in professional education. Bloom’s now famous research project which resulted in Book 2 of the ‘Taxonomy of Educational Objectives: the affective domain’ led by Krathwohl, which has been much neglected, applied scantily (and often erroneously) to practice (Krathwohl, Bloom, & Masia, 1999).

Often described in terms of feelings or emotions I think it more useful to regard the affective as being a question of degree of acceptance or rejection of values, of the evolution of value structures. It is certainly true that having one’s values challenged can result in emotional disturbance (Mezirow, 2000) but I think there is a danger to using language that describes the domain in terms of emotions, not least because it scares off academics!

In my interpretation of the affective domain, illustrated above as a circular representation, I have retained Krathwohl’s five divisions but choose to use active verbs rather than the passive. So receiving becomes to receive, responding becomes to respond, valuing becomes to value, organization becomes to organize, and the final division originally entitled characterization becomes to internalize.

Original Atkinson Descriptor Sample Verbs
Receiving To Receive Ability to learn from others Sense, experience, concentrate, attend, perceive
Responding To Respond Ability to participate responsibly , respectively as appropriate to the context Perform, contribute, satisfy, cite
Valuing To Value Ability to associate personal and collective values with contextual experiences and express value judgements Justify, seek, respect, persuade
Organization To Organize Ability to structure, prioritize and reconcile personal and others’ value systems Clarify, reconcile, integrate
Characterization To Internalize Ability to articulate one’s own values and belief systems and operate consistently within them Conclude, internalize, resolve, embody

The reason I think it helpful to think about values rather than emotions is that clearly most of higher and professional education, is concerned by changing not just how much or what students know, or even how they know and apply that knowledge, but with how they ‘feel’ about knowing. At its simplest we seek to instil a love of learning and a passion for the subject. In professional education we also seek to instil our professional values into the learning process. Whilst it is clearly very difficult to evaluate the emotional impact that learning has on students there are ways of providing formative assessment to support affective developments.

For those in the Academy who are seeking to merely perpetuate their academic DNA in their students, the latters’ changes in values may not mean a great deal to them. For those of us who teach in order to make the next generation better than we are, better able to adapt to the ever-changing world that they face outside of the Academy, then having an interest in our students affective development is extremely important.

I have argued elsewhere that the relative weighting given in learning design to the domains depends largely on the subject and the context of learning. Clearly there will be foundational modules in a degree programme in which knowledge domain learning will be dominant. I would expect much that is done in an undergraduate degree to be concerned with the cognitive domain, clearly an ability to analyse, evaluate, synopsise and synthesise represent the higher-order skills we expect from graduates on graduation. I have also argued elsewhere that psychomotor skills are also worthy of being part of higher education. But it occurs to me that much of the learning opportunity offered in our current universities neglect and equally important set of skills.

Almost all employers agree that they want to attract applicants who share their values. These oft cited idealized values are in fact widely held; the ability to work well with others, to be an effective communicator, to be an effective listener, to work independently, to take the initiative. It seems unrealistic to expect students to necessarily acquire such skills without being guided through the learning process and taught to identify their own development. And it is fair to say that certainly in the United Kingdom sector a huge amount has been added to the curriculum, with employability strategies and planning personal development (PDP) initiatives, that students do not wont for opportunities.

But I maintain that we should ingrain in our students the values we expect them to demonstrate through the disciplines themselves, not bolt them on and relieve the academics from their repsonsability. To my mind it makes sense to write intended learning outcomes to encapsulate a range of affective outcomes and align learning and teaching activities to rehearse those skills with our students.

Why not include alongside an intellectual skills outcome (cognitive) that states “by the end of this module you will be able to critique at least three different perspectives on (whatever the subject is)” another outcome, this time an affective one that says “by the end of this module you will be able to reconcile two contrasting, and contentious, perspectives on (whatever the subject is)”. There is nothing touchy-feely about the second outcome but it focuses on the students value structures, supporting their ability to structure and reconcile personal value systems in contrast to those held by others. Critiquing sounds very higher education, but to be able ‘to reconcile’ is a much needed skill in the workplace.

Bloom, B. S. (1984). Taxonomy of Educational Objectives Book 1: Cognitive Domain (2nd edition). Addison Wesley Publishing Company.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1999). Taxonomy of Educational Objectives Book 2/Affective Domain (2nd edition). Longman Pub Group.
Mezirow, J. (2000). Learning as transformation : critical perspectives on a theory in progress. San Francisco: Jossey-Bass.

Paper: Rethinking personal tutoring systems, the need to build on a foundation of epistemological beliefs.

Newly uploaded, here is the final paper that was previewed in blog postings during December 2014.

Atkinson, S. P. (2014) Rethinking personal tutoring systems: the need to build on a foundation of epistemological beliefs. BPP University Working Papers. London: BPP University.

Image of the cover of Rethinking Personal Tutoring Systems
Rethinking Personal Tutoring Systems

My argument is that in order to tailor effective support for students we must understand better their fundamental beliefs about learning; that to have a conversation about ‘our’ values we need to understand how others experience their own.

This was the purpose of the POISE project, an HEA Change Initiative and this paper is a summary of its conclusions.

There is much work to be done to make these insights more accessible to rank and file tutors in higher education but the POISE website is a start. As always I am delighted to hear about any use made of the work and to enter into a dialogue with anyone working on similar initiatives.

Adaptation of Dave’s Psychomotor Domain

internasyonal na mga workshop / International Workshops
[See Courses on  Educational Taxonomies]

I have received some interesting feedback and critique of my circular representation of Ravindra H. Dave’s psychomotor domain of educational objectives. I have been asked why I have chosen to use the circular design, to alternative verbs and to expand the definition of psychomotor activity.

Firstly the representation of the domain as a circle, which I have done across four domains elsewhere, I believe serves to make the subcategories more fluid. It contains the proto-verbs at the centre, next circle contains active verbs which also represent teaching and learning activity and the outer circle contains the nature of evidence (or assessment forms) that might demonstrate the active verbs. Using the circle one also has an inherently clock-face like visual which makes the dialling-up from the basic to more sophisticated concepts as you travel around clockwise. Maybe its most powerful function is to encourage lateral thinking on the part of learning designers, encouraging them to explore learning and teaching activities as assessment or evidence examples at the same time.

Psychomotor Domain - Taxonomy Circle - after Dave (1969/71)
Psychomotor Domain – Taxonomy Circle – after Dave (1969/71)

Secondly, I have chosen to use active verbs to describe the subcategories of the domain and so there is a clear change from:

Dave’s Original  Atkinson’s Adaptation  Descriptor
Imitation Imitate ability to copy, replicate the actions of others following observations.
 Manipulation  Manipulate ability to repeat or reproduce actions to prescribed standard from memory or instructions.
Precision  Perfect ability to perform actions with expertise and without interventions and the ability to demonstrate and explain actions to others.
Articulation  Articulate ability to adapt existing psychomotor skills in a non-standard way, in different contexts, using alternative tools and instruments to satisfy need.
 Naturalisation  Embody ability to perform actions in an automatic, intuitive or unconscious way appropriate to the context.

This is to articulate more clearly the need to describe learning outcomes as things that the students will actually ‘do’ in line with the principles of constructively aligned learning and teaching design.

The third, more less obvious change, is that I have chosen to expand the definition of psychomotor activity to incorporate a wider range of physical activities that perhaps Dave had not envisaged, particularly those involved with the manipulation of computer software, laboratory and fieldwork equipment and a range of technical equipment. I felt this was necessary because I have seen so many University courses make light of the skills developed in acquiring such expertise, as though such skills are incidental, when clearly it is the primary outcome valued by most students and employers.

For example, the specifics of the volume of water flowing through the Mississippi delta in November (Knowledge) will prove less useful than the ability to master the GIS and computational software used to document those specifics (Psychomotor).

I believe that the majority of what in the UK further and higher context is described as ‘transferable skills’ fall into the psychomotor domain and are worthy of careful attention.

Visualisation of Educational Taxonomies

Sharing a paper today on the visualisation of educational taxonomies. I have finally got around to putting into a paper some of the blog postings, discussion, tweets and ruminations of recent years on educational taxonomies. I am always struck in talking to US educators (and faculty training teachers in particular) of the very direct use made of Bloom’s original 1956 educational taxonomy for the cognitive domain. They seem oblivious however to other work that might sit
(conceptually) alongside Bloom is a way to support their practice.

Taxonomy Circles ATKINSON AUG13

In New Zealand, whilst at Massey, I got into some fascinating discussions with education staff about the blurring of the affective and cognitive domains, significant in cross-cultural education, and this led me to look for effective representations of domains. I came across an unattributed circular representation that made instant sense to me, and set about mapping other domains in the same way. In the process I found not only a tool that supported and reinforced the conceptual framework represented by Constructive Alignment, but also a visualising that supported engagement with educational technologies and assessment tools. I hope this brief account is of use to people and am, as always, very open to feedback and comment.

I’m very grateful to those colleagues across the globe who have expressed interest in using these visual representations and hope to be able to share some applicable data with everyone in due course.

Accessibility by WAH
%d bloggers like this: