Student expectations are serious constraints on imaginative assessment processes. Students are taught through their educational experience that ‘final grades matter’ so it is natural that they become fixated on the final assessment. A transparent design that closely reflects the ILOs within the teaching activity and assessment will necessarily engage students in a broader and deeper understanding of their learning journey.
Following the 8-Stage Learning Design Framework, we know at this stage what our intended learning outcomes (ILO) are. This enables us to design meaningful assessment that provides opportunities to students to evidence their learning against those ILOs. It is important to initially identify which outcomes across different domains of learning can be combined through assessment. This allows us to manage the assessment load, for both faculty and student, while ensuring all ILOs are assessed. Using taxonomy circles, we can then draft marking rubrics for the appropriate level that represent all the guidance that individual assessors and students need to guide their practice.
There is also a challenge to meet the needs for assessment, or the outcomes, sometimes even the content, dictated by external bodies. In many programmes, there is pressure to assess a range of skills and behaviours beyond subject knowledge. The challenge is to design assessments that allow students to demonstrate a range of skills (across various domains) through a single assessment. Rather than abdicate our responsibility as learning designers, this is a call to understand how better to articulate the relationship between what the intended learning outcomes of a course are, how it is being assessed and what is being experienced as learning by the students.
Drafting an assessment framework is an iterative process. Ideally one designs the assessment at the same time as one writes module ILOs, ‘tweaking’ them to give them depth and flexibility at the same time. If students are properly guided to generate well-structured evidence, it can be a fascinating and engaging process to assess them.
This post is a shortened version of a new resource posted about designing effective ILOs available here.
ILOs are the detailed explanations, written in language the students will understand before beginning the module or programme, as to what they will be able to DO when they have successfully completed the learning.
Many quality assurance structures, institutional or external, require Programme and Module Specifications contain details of the 'intended learning outcomes' (ILOs) of the programme of study. ILOs serve to provide students with a ‘checklist’ of the types of skills, attributes, abilities or competencies they should be able to evidence through successfully completing the module or programme.
Intended Learning Outcomes should not be seen as a straight-jacket for faculty. Rather, if they are well written, they should provide scaffolding for creativity in teaching and assessment.
Most teachers can identify any number of unintended learning outcomes, depending on the character of the cohort, the changing context in which learning takes place or the emergent nature of the discipline. However, the ILOs are the facets of learning that will be assessed. They should be written knowing that these are the capabilities that will be assessed, not the content knowledge.
Read more for guidance on how to structure effective ILOs that cover all dimensions of learning.
There are five key considerations we should take into account as we either work with media in our learning and teaching process or design learning using media. The relative importance of each of these depends on the level, context and nature of the module or programme but all should need to be reflected in any learning design. These five are:
Students: orientation and disposition to learning with and through media.
Staff: abilities to work with media. Their abilities to identify appropriate media and manipulate it as appropriate.
Professional needs: present and future demands of the professional context in using media appropriately.
Content & Resources: Identifying existing, or creating, effective media resources for learning.
Institutional Choices: The constraints and opportunities for learning designers to develop media.
This 33-minute presentation is from a session delivered to an internal audience of colleagues on my final day at BPP University as part of Digital Literacy Day (16 Feb 2018). Set-up as a demonstration, sharing QRs, Padlet, Twitter. After six years leading a portfolio focussed around teaching enhancement, scholarship and research and educational technology adoption, I outlined the pervasive nature of resistance to change and issued a call for recognising the urgent need to embrace sound educational technology practices. The presentation sets up the premise that our educational history has created a walled garden within which the majority of our academic practice exists. I suggested a few alternatives to existing practice, encouraging colleagues to take some small steps. These included Voicethread as a an interactive discussion platform and H5P (I misspoke in the video and said HfP!). H5P provides a degree of interactive possibilities.
My challenge to my colleagues was about who was going to design the learning experiences through educational technology. What is the relationship between instructional designers and content specialists? What about their relationships with 'classroom' practitioners?
Tertiary providers are increasingly expected to deliver 'work-ready' graduates. This is a challenge when we must acknowledge that many graduates will begin a career that does not exist today. Identifying the competency frameworks within our disciplines, and those of our professional colleagues is a good place to start. We can then identify a range of graduate attributes that will underpin our programme outcomes and inform the development of the real-world assessment.
It is important to question all of our assumptions about the context into which our learning design is intended fit. Despite the fact that you may feel you know your learning context intimately the chances are there will be some contextual evolution.How much has your discipline context evolved in recent years?.
Our learning programmes are designed to reflect our institutional specialisms and priorities, to play to our strengths. Sometimes we risk forgetting that they will be taken by real people with different dispositions, orientations and perspectives. For this reason, the first stage of my 8 Stage Learning Design Framework is concerned with profiling students. Much the same way that any product designer asks 'who is my customer going to be?', we need to do the same thing in education. I'm sharing here a process in which we describe between four (five or six works too) imaginary individuals who are as different from each other as we can imagine as our potential students. I advocate a diversity matrix to build these characters, articulating different dispositional, educational, circumstantial and cultural parameters. We then run these user-cases through four further perspectives to reassess their discipline orientation, learning orientation, personal context and social context. There are overlaps and contradictions, a creative tension that reveals the strengths and relative weaknesses of any potential programme design. See the Research pages for more of Student Profiles
This an introduction to a new resource being shared on this website, the 8-Stage Learning Design Framework, or 8-SLDF for short. The framework provides a supportive step by step process to enable faculty and course designers to develop robust and well-aligned programmes or modules. Publication of the 8-SLDF is in preparation so only brief explanations are provided but resources will be shared over time with associated commentaries. These blog posts will find a permanent home on the research pages of this site too.
I believe that the best way of ensuring that students and faculty can both engage in a meaningful, positive and fruitful learning collaboration is by designing courses well.
By well, I mean that courses that are constructively aligned, relevant to the real-world experience of students, engaging and transparent. Courses must also be cultural and socially aware. Students need to know why they are being asked to perform learning tasks and we should always have an answer. Knowing 'why' an activity matters because it is the first step in any individual's self-reflective process, their metacognition and the development of their personal epistemologies (Atkisnon, 2014). We also need to know 'why' because doing anything for the sake of it is clearly wasteful of our time and energy. We as faculty are valuable players in the relationship between our students, the discipline, our institution and the wider world. Being good at what we do makes a difference. Designing courses that enable us to be better at what we do simply makes sense.
The 8-Stage Learning Design Framework has had a long gestation. It has its foundations built through my educational development practice around the work done by John Biggs on constructive alignment (2007) and the SOLO taxonomy (1982). I then incorporated work by Anderson and Krathwohl's reworking of Bloom's cognitive domain taxonomy (2001) alongside others domain development, including the original Bloom project's articulation of the affective domain (1956), Dave's psychomotor domain (1967), and my own interpretations of Metacognitive and Interpersonal domains.
The issue of the effective materials design was inspired by the Open and Distance learning world (pre-digital), particularly by Derek Rowntree (1994) and Fred Lockwood (1994), on my collaborations with Kevin Burden around the DiAL-e Framework (2009) and my own scholarship around the SOLE Model (2011). More recently I have drawn inspiration from the work of James Dalziel and Gráinne Conole (2016), and Diana Laurillard (2012), in their learning design conceptualisations, particularly as it relates to learning activities.
The result is I believe a comprehensive, flexible and adaptable learning design framework not just for activities but for entire courses, module and programmes. It is an appropriate framework regardless of the discipline, level, context or mode of learning. It is a framework for any adult, formal, learning context.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing : a revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Atkinson, S. P. (2011). Developing faculty to integrate innovative learning in their practice with the SOLE model. In S. Ferris (Ed.), Teaching, Learning and the Net Generation: Concepts and Tools for Reaching Digital Learners. Hershey, PA: IGI Global.
Atkinson, S. P. (2014). Rethinking personal tutoring systems: the need to build on a foundation of epistemological beliefs. London: BPP University College.
Atkinson, S. P. (2015). Graduate Competencies, Employability and Educational Taxonomies: Critique of Intended Learning Outcomes. Practice and Evidence of the Scholarship of Teaching and Learning in Higher Education, 10(2), 154–177.
Biggs, J., & Collis, K. F. (1982). Evaluating the Quality of Learning: Structure of the Observed Learning Outcome Taxonomy. New York: Academic Press Inc.
Biggs, J., & Tang, C. (2007). Teaching for Quality Learning at University: What the Student does (3rd ed.). Buckingham. GB: Open University Press.
Burden, K., & Atkinson, S. P. (2009). Personalising teaching and learning with digital resources: DiAL-e Framework case studies. In J. O’Donoghue (Ed.), Technology Supported Environment for Personalised Learning: Methods and Case Studies (pp. 91–108). Hershey, PA: IGI Global.
Conole, G. (2016). Theoretical underpinnings of learning design. In J. Dalziel (Ed.), Learning design: conceptualizing a framework for teaching and learning online (pp. 42–62). New York: Routledge
Dave, R. H. (1967). Psychomotor domain. Presented at the International Conference of Educational Testing, Berlin.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1956). Taxonomy of Educational Objectives, Handbook II: Affective Domain. New York; David McKay Company, Inc.
Laurillard, D. (2012). Teaching as a Design Science (1 edition). New York: Routledge.
Lockwood, F. (Ed.). (1994). Materials Production in Open and Distance Learning. London: SAGE Publications Inc.
Rowntree, D. (1994). Preparing Materials for Open, Distance and Flexible Learning: An Action Guide for Teachers and Trainers. London: Routledge.
This morning's short coffee note identifies some of the functions, and their advantages, in establishing 'enhancement circles'. For faculty working in a less than supportive context, this works well as a collegial mechanism for personal, and collective, enhancement.
Reaching out to colleagues outside your immediate sphere, from different disciplines or institutions, you can establish a personal supportive network. I think between three and six colleagues Is about the right number. This means you can usually find someone to exchange practice with but you won't run out of people too quickly. Remember that peer observations do not necessarily have to be done in person. In theory, you could be observed by someone in a different country providing they have access to video recordings of your practice. Just make sure that you provide your observer with some specific guidance, requests of what it is you want them to comment on explicitly. Focusing on something specific in your practice will help you focus and deliver a better quality of the evaluative comments.
Remember that whilst it is important that Members of your enhancements circle or all practising at the same level of study, in other words, all undergraduate, postgraduate, pre-university level. The discipline is significantly less important in terms of identifying effective practice. I hope to hear back from some of you as to how effective this approach might be for you in your context.
I thought I would share some cross-platform videos which reflect whatever is on my mind professionally each morning. Shot in portrait for IGTV and then annotated for YouTube, they represent unscripted notes on some aspect of learning design or educational enhancement.
This one explores the value of an individual approach to personalizing reflection for academic practitioners. I urge faculty to make reflective journal notes immediately following any teaching event. This is invaluable, as is watching and listening back to your work. Combined with an SGID or in-class evaluation process you will find that any end-of-module evaluation of your teaching effectiveness should hold no surprises.
There is a wealth of literature that describes the purposes of providing feedback as part of the learning process in higher and professional education. I’m going to distil this voluminous research and scholarship into four key purposes.
Firstly, feedback for student learning is about increasing capacity for future actions. Indicating to the student how a piece of work, in-class contribution or whatever form of evidence, is provided by the students could be made better next time is about increasing their capacity. It’s human nature to think, “ok, well that’s task is completed, I passed, let's just move on”, but understanding how to do better, even in an imaginary ‘next time’, builds capacity.
This relates to the second purpose, developing self-awareness or metacognition in the student. Giving students the sense that even if there isn’t going to be another opportunity to provide evidence of learning in the same way again, there will be similar activities, tests, trails or exams and what they learn from the current feedback can be transferred into this new context.
Which leads us on to the third purpose of feedback, of developing academic skills. Poorly designed assessment might just be testing content knowledge, and it's very hard to provide meaningful feedback on such assessment. If on the other hand your assessment is well constructed, against distinct learning outcomes and using a meaningful marking rubric, then the feedback students receive should also be developing the academic abilities and skills beyond what they can recall.
The fourth and final purpose for providing feedback for student learning is to enhance the self-confidence and well-being of the student. Whether your feedback is providing confirmation of progress and success on the part of the student or providing supportive corrective guidance to a struggling student, the purpose remains the same, to bolster a positive attitude to learning, to the subject, to the practices associated with the discipline.
If you are struggling to meet these four core purposes in providing feedback to your students, you may want to think about reading a practical guidebook on providing feedback, enrolling on a professional development programme or just get together with your colleagues, and go through a course redesign or re-evaluation. You could invite a consultant to review your practices. You may find that your assessments and your in-class learning and teaching activities could be better designed to make providing meaningful feedback easier for you, and more useful for your students.
Simon Paul Atkinson
Consultancy for International Higher Education