Book on Writing Good Learning Outcomes

Introducing a short guide entitled: “Writing Good Learning Outcomes and Objectives”, aimed at enhancing the learner experience through effective course design. Available at https://amazon.com/dp/0473657929

The book has sections on the function and purpose of intended learning outcomes as well as guidance on how to write them with validation in mind. Sections explore the use of different educational taxonomies as well as some things to avoid, and the importance of context. There is also a section on ensuring your intended learning outcomes are assessable. The final section deals with how you might go about designing an entire course structure based on well-structured outcomes, breaking these outcomes down into session-level objectives that are not going to be assessed.

#ad #education #highereducation #learningdesign #coursedesign #learningoutcomes #instructionaldesign

Free Online CPD Course on Learning Outcomes (until 14th January 2023)

Have you got some time for professional development over the holiday period? Or do you have colleagues or design teams working on course designs over the holiday period?

Anyone who has ever tried to assess or teach to poorly learning outcomes, and then tried to defend their practices or results, will tell you that getting it right at the offset saves a huge amount of effort and heartache.

Intended Learning Outcomes are the foundations of any sound well-aligned course and programme design. Being able to create effective well-structured learning outcome is a valuable skill required of all learning designers, faculty and quality officers.

I have created a short, self-study, course hosted on a Moodle instance. The full course will take between up to 10 hours at a leisurely pace but is designed to allow you to navigate your way through it as you please. You are welcome to dip in and out. The course complements the book ‘Writing Good Learning Outcomes and Objectives’. (https://www.amazon.com/dp/0473657929/)

Join the free course entitled ‘Designing Effective Intended Learning Outcomes’ at https://sijen.net/courses

The Metaverse explained for university leaders: challenges for universities ahead (3/4)

Press coverage of recent cryptocurrency disruptions and the significant staff reductions at Twitter and Meta is giving pause for thought amongst investors and futurists, as well as university leaders considering the future of the Metaverse.

The fact that you may feel like you cannot keep up with the news is understandable. The collapse of the cryptocurrency platform FTX, the apparent meltdown underway at Twitter and the 11-year sentence handed down to Elizabeth Holmes for the Thanos fraud do all have something in common. The digital world is sufficiently obscured from the majority of people, sometimes deliberately, and the ‘trust train’ may have now hit to buffers, reminiscent of the end of the dot-com boom.

So what of the metaverse? I did not mean to imply that it is a fanciful dream that will never have an impact on higher education,  but I have reservations.  I received some negative feedback for comparing 3D Cinema and VR technology adoption curves. I stand by my contention that such technology developments need to take more account of user expectations, as well as their user experience. Demographic patterns play a huge part in any technological innovation. The challenge for most Universities is to decide whether they are best to invest in low-tech entry materials and approaches to build a foundation for future ‘metaverse’ technologies or to join a narrow range of institutions that are innovating around these emergent technologies.

If you are looking to review institutional strategies in the light of challenges and opportunities presented by the Metaverse, please feel free to get in touch with spa@sijen.com

The obstacles for entry are less technical than they are learning design and delivery related. Clearly having sufficient finance in place is an obstacle for some, but even for those that have the cash to spend, knowing where to invest is crucial.

The technologies already exist for building VR immersive experiences, and some a free to try out (Unity.com), and the headsets and accessories are in theory available within reach of those on a medium or high income, although with the current cost of living crisis one might anticipate that Meta’s sales expectations for its latest headset to fall short. But creating a test suite, a development platform to create VR immersive environments, requires a greater degree of investment.

If I was a betting man (and I am not), I would agree with those who believe that Zuckerberg is willing to sacrifice the social platforms (Facebook, Instagram), with their declining demographic, in favour of speculative investment in the future. The future for him is the Metaverse. However, there is no clear evolutionary path for the Metaverse. That includes the challenges mentioned previously, those of wearable technologies, computer power to sustain them, the privacy legislative framework, and the broader legal implications. There are billions around the world without internet access, millions without reliable broadband, and millions who do not have the disposable income, time, or inclination, to while away hours as a virtual avatar. It might be ‘cool’, but is it really worth the time and effort?

Legal frameworks are struggling to keep up with the rapid technological changes society faces. The European Union is possibly the most active in seeking to impose guardrails around digital technologies. Some of these are privately welcomed by the big technology companies, who lend some of their legal minds in pursuit of meaningful legislation, while other legal restrictions are resisted. Profit still comes first after all.

Esports, a growing share of the online gaming space, certainly benefits from advances in hyper-real 3D immersive technologies. A business paying to advertise inside these game spaces, whether the hoardings around a virtual pitch or track, or branding on virtual apparel, makes sense. Whether this gaming trend will fruitfully spill over into academia, I am doubtful.

What should universities do?

There are things universities should do, in my view, to ensure they are ready to react (if I’m wrong) and VR technologies become more integrated into the learning experience of a wider group of students.

They should have both a Student Charter and an Information Technology Policy that are both reviewed annually. Things move that fast. And all students and staff should be asked to reassert their commitment each year. The executive summary for both of these documents serves to enhance the digital literacy of the entire learning community.

Privacy and ‘netiquette’ are concepts that are intertwined in the experience of staff and students. I can use abusive language, within limits, and ALL CAPS to insult people on Twitter and face little in the way of challenge. If I was to stand in London’s Leicester Square and do the same thing, say exactly the same thing hurling abuse at a passerby, it would not be too long before a couple of Police officers would turn up and move me on. Failure to comply would likely result in arrest and being charged with disturbing the peace. Imagine that scenario now within a virtual world. Who is the Police? What penalties would I face, if any? The behavioural norms we associate with the real-world fall apart in the digital sphere. That is already true today given the vile abuse faced by female academics in particular.

Is your institutional policy framework designed to cope with this scenario?

A student group, registered with your Student Union, organises a virtual event, hosted on a third-party application ( ZOOM for example) using a license owned by the controversial speaker themselves. The event requires registration but this is also done by the speaker themselves, and the event is advertised without any explicit endorsement from the student group themselves through it is heavily advertised verbally and using paper flyers around campus. During the event, some students mount a protest, disrupting the event. The event attracts huge criticism and excepts of the ZOOM meeting go viral on TikTok and Telegram, with some of the student’s name and affiliations being attributed. The mainstream press seize on the event as an example of both the ‘no-platform’ policy position you hold and the ‘woke, liberal elite’ attitudes in evidence.

My advice to a recent University client was that they should run ‘war games’ scenarios with senior student leaders and Heads of Department around exactly these kinds of scenarios. Because the challenges institutions face are less about being overrun by technological developments than it is one of uncontrollable user scenarios.

And explore AR in the short term. That’s for the final blog in this short series.

If you are looking to review institutional strategies in the light of challenges and opportunities presented by the Metaverse, please feel free to get in touch with spa@sijen.com

Image credit: generated using DALL-E

Workshop review: ‘Innovating Pedagogy 2022’

Thursday 8th September I had the privilege of running an online workshop for FLANZ  to explore the potential of a range of different pedagogical approaches that might apply to different educational sectors in New Zealand and Australia.

See Transcript The Innovating Pedagogy 2022 is the 10th annual report from the Open University (UK) exploring new forms in interactive and innovative practice of teaching, learning and assessment. These innovations already exist in pockets of practice but are not considered mainstream. This collaboration between the Institute of Educational Technology at The Open University, UK, and the Open University of Catalonia, Spain, is the result of a filtering process and is compiled, based on a review of published studies and other sources. Ten concepts or themes are identified.

Hybrid models Maximising learning flexibility and opportunities. Beyond the strict curriculum delineations in Blended Learning models, Hybrid forms aim to empower the learner to optimise their own learner choices at to where, when, and how to learn. Providing flexible choices requires teachers and institutions to adjust their systemic approaches. Influencer-led education Learning from education influencers on social media platforms. Acknowledging the growth of edu-influencers, who optimise their use of social media tools to share their knowledge, experience, and passion for a range of subjects from the highly specialised to the generic. Evaluating the veracity of the message is a challenge for the learner.
Dual learning scenarios Connecting learning in classrooms and industrial workplaces. A step up from work-integrated learning models, the expectation is that course designers fully meld both formal classroom and work spaces into a coherent experience. Pedagogies of the home Understanding the home as a place for cultural learning. Not the same as home-schooling. Rather, it seeks to leverage the wider socio-cultural environment that the learner inhabits. Also recognises the burden on marginalised communities to fully participate.
Pedagogies of micro-credentials Accredited short courses to develop workplace skills. Existing approaches, snippets taken from existing programmes, fail to create an effective learning ecosystem for learners who require support to develop a patchwork portfolio meshing formal, non-formal, and informal experiences together. Pedagogy of discomfort   Emotions as powerful tools for learning and for promoting social justice. A process of self-examination that requires students to critically engage with their ideological traditions and ways of thinking about issues such as racism, oppression, and social injustice.
Pedagogy of autonomy Building capacity for freedom and independent learning. Explores the notion of incorporating informal, non-formal, and formal learning patterns into the learner’s experience, creating self-regulated learners with an emphasis on their metacognitive development and allowing them to reflect their true selves.. Wellbeing education Promoting wellbeing across all aspects of teaching and learning. Wellbeing education helps students to develop mental health ‘literacy’ by teaching them how to manage their own mental health, recognise possible disorders, and learn how, where, and when to seek help.
Watch parties Watching videos together, whatever the time or place. Leveraging the increased connectivity prompted in response to covid-19, and the move of media providers to provide educational tools, this is the notion of structured engagement around a shared viewing (or listening) experience. Walk-and-talk Combining movement and conversation to enhance learning. Not just used in service of those in need of emotional support, where the therapeutic benefits have been proven, but across a wide range of learning activities where reflection and thought would be best served by being away from the classroom and being outside and mobile.
10 Themes from the 2022 Innovating Pedagogy report

The workshop used Mentimeter as an online polling tool. Of the 25 participants, 20 regularly voted and made 659 submissions. The tertiary sector dominated, at 15, with fewer representatives from the Private Training Enterprise and Commercial L&D sectors and only one from compulsory education. Only 2 Australians participated. Despite having laboured the point in all publicity materials that it would be valuable to read the report before participating, only 8 said they had read it (or the summary), with 11 admitting they had not. Of the 17 that responded to the question about their approach to new educational technologies, 12 saw themselves as ‘progressive’, 2 as ‘radical’, and 3 as ‘pedestrian’. To get participants involved in thinking about each pedagogic approach, we ran a 2×2 square exercise, asking what the relative effort versus impact might be. See the video for responses. Following breakout groups we ranked the innovations in terms of the amount of attention participants would pay to them in the next 12 months in their personal practice (see screenshot above). The general consensus was that whilst there was nothing exceptional or radical in any of these innovations, they provided a focus for reflection and were deemed stimulating. Thank you to all who participated.


Kukulska-Hulme, A., et.al. (2022). Innovating Pedagogy 2022: Open University Innovation (No. 10). Open University.

The threat to the integrity of educational assessments is not from ‘essay mills’ but from Artificial Intelligence (AI)

The threat to the integrity of educational assessments is no longer from ‘essay mills’ and contract cheating but from Artificial Intelligence (AI).

It is not so long ago that academics complained that essay mills, ‘contract cheating’ services, and commercial companies piecing together ‘bespoke’ answers to standard essay questions, were undermining the integration of higher education’s assessment processes. The outputs of these less than ethically justifiable endeavours tried to cheat the plagiarism detection software (such as Turnitin and Urkund) that so many institutions have come to rely on. This reliance, in part the result of the increase in the student-tutor ratio, the use of adjunct markers and poor assessment design, worked for a while. It no longer works particularly well.


If you are interested in reviewing your programme or institutional assessment strategy and approaches please get in touch. This consultancy service can be done remotely. Contact me


Many institutions sighed with relief when governments began outlawing these commercial operations (in April 2022 the UK passed the ‘Skills and Post-16 Education Act 2022’ following NZ and Australian examples) and went back to the business-as-usual. For the less enlightened this meant a return to setting generic questions, decontextualised, knowledge recitation essay tasks. Some have learnt to at least require a degree of contextualisation of their students’ work, introduced internal self-justification and self-referencing, requiring ‘both sides’ arguments rather than declared positions, and applied the ‘could this already have been written’ test in advance. Banning essay mills, or ‘contract cheating’, is necessary, but it is not enough to secure the integrity of assessment regimes.

Why students plagiarise is worthy of its own post, but suffice it to say it varies greatly depending on the student. A very capable student may simply be terrible at time management and fear running out of time or feel the assessment is unworthy of them. Another student may be fearful of their ability to express complex arguments and in pursuit of the best possible grade, plagiarise. Some may simply have not learnt to cite and reference, or to appreciate that rewording someone else’s thoughts without attributing them also constitutes plagiarism. And there is that category of students whose cultural reference point, deference to ‘the words of the master’, make plagiarism conceptually difficult for them to understand.

I remember receiving my most blatant example of plagiarism and academic malpractice back in 2006. A student submitted a piece of work that included 600 words copied wholesale from Wikipedia, complete with internal bookmarks and hyperlinks. I suspect the majority of students are now sufficiently digitally literate not to make that mistake, but how many are also now in a position to do what the essay mills used to do for them, stitch together, paraphrase and redraft existing material using freely available AI text generation tools.

As we encourage our students to search the web for sources, how easy is it for them now to access some of the easily accessible, and often free, online tools? These tools include https://app.inferkit.com/demo which allows you to enter a few sentences and then generate longer texts on the basis of that origin. You can enter merely a title, of at least five words, or a series of sentences into https://smodin.io/writer and have it generate a short essay, free of references. Professional writing tools aimed at marketers, such as https://ai-writer.com, would cost a subscriber to be effective but would allow students to generate passable work. This last tool actually tells you the sources from which its abstractions have been drawn, including academic journals.

You might find it enlightening to take something you have published and put it through one of these tools and evaluate the output.

It is insufficient to ask the student to generate their own question, or even to ask the student to contextualise their own work. Some of the emergent AI tools can take account of the context. There is a need to move away from the majority of long-form text assessments. With the exception of those disciplines where writing more than a thousand words at once is justifiable (journalism, policy studies, and some humanities subjects), there is a need to make assessments as close to real-world experience as possible. It needs to be evidently the product of an individual.

Paraphrasing is a skill. A valuable one in a world where most professions do not lack pure information. The issue is to evaluate the quality of that information and then be able to reduce it to a workable volume.

I’ve worked recently with an institution reviewing its postgraduate politics curriculum. I suggested that rather than try and stop students from ‘cheating’ by paraphrasing learned texts, they should encourage the students to learn what they need to do to enhance the output of these AI tools. Using one of these tools to paraphrase, and essentially re-write, a WHO report for health policy makers made it more readable, it also left out certain details that would be essential for effective policy responses. Knowing how to read the original, use a paraphrasing tool, and being able to explore the deficiencies of its output and correct them, was a useful skill for these students.

We cannot stop the encroachment of these kinds of AI text manipulation tools in higher education, but we can make their contemporary use more meaningful to the student.


If you are interested in reviewing your programme or institutional assessment strategy and approaches please get in touch. This consultancy service can be done remotely. Contact me.


Image was generated by DALL-e



Psychomotor skills should be at the core of all learning

Any learning design framework that does not address the psychomotor skills is not worth exploring.

There is not a single discipline taught in any formal, non-formal or informal way that does not make use of some tool or technology, instrument or mechanism (aka media), at some point in the process. It makes sense that any curriculum development process needs to put the media at the forefront of its planning. Curricula need to developed around intended learning outcomes that are clearly articulated around the development of psychomotor skills.

Rather than have intellectual (cognitive) outcomes such as, (students will be able to:)

Apply transformations and use symmetry to analyze mathematical situations
Would it not be better to say
Utilise graphical representation software in order to analyse mathematical transformations and symmetry

That way the student requires the practical ability to evidence their ability to meet an intellectual skill.

Another example in a disparate discipline, let’s take theatre studies. Rather than say,

Demonstrate an understanding of all aspects of theatrical production including design and technical functions” [a real, but poorly written outcome]
Would it not be better say:
Produce design and technical specifications for a theatrical production

The learner cannot provide evidence of their ability to meet that outcome without fulfilling the weaker intellectual outcome.

The course design process then become skills focussed rather than knowledge orientated. Knowledge is acquired within a practical context. The psychomotor outcomes are not overly specific, they do not say ‘using Algosim to generate mathematical visualisations….” Or “Manage stage plans using ShowNotes….”, because in both cases stating a particular technology does not allow for future evolution of those technologies (renaming, rebranding, etc). The focus is on developing the skills, always with a light to their transferability across other tools. We should always ensure that we teach the ‘paper and pencil’ version alongside too, so the increments between origination and implementation are also evident.

The 8-Stage Learning Design Framework has as its third step the ‘Media Choices’, which requires programme and course designers to review the current (and evolving) environment into which graduates will emerge. This should incorporate a review of the tools and technologies that students are expected to use ‘on the job’. Only after this stage is complete is it appropriate to draft Intended Learning Outcomes, then assessment, and then learning & teaching activities.


See Courses on both Designing Effective Intended Learning Outcomes and the Introduction to Five Educational Taxonomies, which includes the Psychomotor domain.



Teaching about existential threats: why we need to teach concepts, not just facts.

It has now been more than four months after Russia’s invasion of Ukraine and I have been thinking how badly we need to be teaching about existential threats. I think we need to develop a curriculum that is open to contemporary real world challenges.

It has now been more than four months after Russia’s invasion of Ukraine. Like many, I have been ruminating. This post it’s about that. Or at least not directly. I have been thinking about how badly we need to be teaching about existential threats. I think we need to develop a curriculum that is open to contemporary real world challenges.

I think global education needs to adjust to new realities. The First World War, the Great War, wasn’t a World war in July 1914. It became one later. The Second World War likewise was not a world war in September 1939, although it engulfed the globe in due course. We are yet to see whether the February 2022 Russian invasion of Ukraine will prove to have been the start of the Third World War. Hopefully it will not become that kind of milestone, but i think we owe it to students to prepare them for that possibility.

Education is different now than it was in 1914 or 1939. Now we have wall to wall coverage, ubiquitous social media and real-time battlefield insights piped into childrens’, adolescents’ and adults’ television screens, tablets and smartphones. There is almost as much misinformation as there are facts on many platforms; as many well-meaning transmitters of misleading (sometimes factually inaccurate) news, as there are respectable and accredited voices. The democratisation of information is a good idea, but it assumes individual generators of that information are well-informed, critical and, if opinionated (or biased), state that upfront.

I have learnt a great deal about Ukraine in recent months. Some of it from listening to TikTokers and YouTubers. Some summarise Ukrainian news sources for us. and others share their daily lives from war torn cities. Some review Russian media sources. Some provide valuable daily summaries. Others share emotive responses to news as it happens. I am aware that I am, I believe, relatively digitally literate. I’m critical of the sources, often look up the individual commentators on other social media, check their LinkedIn profile and review past output. This last element is particularly interesting. I have been suspicious, though not dismissive, of social media account that started in late February 2022. Some are clearly chasing followers, clicks and likes. Some are clearly trying to provide what they see as a genuine information service. Being able to identify the difference is not always easy. Our students need to learn these skills.

Risk evaluation is a very difficult thing to teach. Each student will have different life experiences to make them more or less fearful of uncertainty. Those who lived through the Cuban missed crisis or the nuclear standoffs in the early 1980s may say they’ve seen it all before. What is different now is the ubiquitous nature of information, and misinformation, which is in danger of confusing students’ ability to make their own judgements.

Educators are morally obliged to teach the unseen. That includes climate change and the risk of nuclear war. There is not a discipline that cannot leverage the moment. Social sciences and humanities obviously have an edge. Physical sciences are often less flexible in terms of the curriculum. But I think it’s important that anyone teaching today pauses before delivering any concept, any idea or thought, and consider whether there is a contemporary example amongst the unseen existential threats that exist. It means sometimes abandoning our own safe assumptions, our own safe havens, and exploring things that we ourselves may see as uncertainties.

There any number of examples of elements within any curriculum that can leverage the Russia-Ukraine war. Political scientists can explore the notion of the Eurasian multipolar world view. Geographers can explore the three seas project, and Sociologists can explore the religious realignment we are now seeing amongst the Orthodox churches. Examples are endless if we focus on the concept rather than the content. Beyond any obvious historical comparisons there are lessons to be learnt across all the disciplines using contemporary examples. The learning of concepts, geopolitical perspectives, resource management and cultural power, are all more useful to the students that any specific set of facts.

How relevant the current Russian invasion of Ukraine may be perceived by faculty and their students alike will depend largely on geography. While the conflict itself is seen as a largely European ‘problem’, and it’s global economic implications are yet to be clearly felt, I would understand that these reflections are probably more relevant to my European colleagues than to many others. But the principle still stands. All concepts that form any part of the curriculum need to be based within a contemporary world context. We need to leverage the current crisis that is being seen and witnessed by students through the prism social media. In doing so we can both serve the curriculum and educate students with critical judgement about their sources of information.

Concepts not Content

I am passionate about privileging the teaching and learning of concepts rather than content. Concepts are instruments that serve to identify, define, explain, illustrate and analyse real-life elements and events, past, present or future. These are usually within the confines of a particular geography, social context and within discipline conventions, but when defined well, reach across all cultural boundaries.

There are essentially two kinds of concepts: sensory and abstract. Sensory concepts are tangible, they can be experienced through our senses. Abstract concepts are not directly experienced, they are often not visible and need to be imagined. There is a simple three step process for you to consider as you build learning with concepts: define, illustrate, and imagine.

Define:

It’s important to keep the definition of a concept at its simplest. It should be a self contained concept.

Let’s take for example the statement that Regional wars have global consequences.

We could then unpack what we means by regions, wars and global consequences. The easiest way to validate your concepts’ definition is to see how easy it is to state its opposite. Regional wars do not have global consequences.

I can already envisage an assessment task that asks students to identify regional conflicts that did not have global consequences, and then have their peers challenge them subsequently as alternative perceptions of those consequences (after some enquiry-based learning).

Illustrate:

Illustrating a concept helps learners to catagorize new knowledge, to cement that new learning in a hierarchy or order of reality. Illustrations can be examples that demonstrate the truth of the definition, or it’s opposite. An illustration that does not match the definition also serves to help learners make sense of the definition. So in this example “Regional wars can have global consequences”, I could describe the key protagonists and events that led to the war in the Middle East between Israel and Arab powers in 1956, which had profound long term implications for European loss of influence and the rise of the United States as a regional power broker.

For its opposite I could take the regional war fought between the Sahrawi Indigenous Polisario Front and Morocco from 1975 to 1991 (and involving Mauritania between 1975 to 1979), for control of Western Sahara which has had minimal global impact. Although it is still a live issue in that region.

Neither world wars, both clearly regional conflicts but with different impacts. A useful conceptual space to unpack thoughts and idea with students. Learners do not need to have detailed knowledge about the background histories of the parties to be able to develop and understanding why these two different conflicts result in different implications. The challenge for them to unpack the factors that make up the definition of the concept shared earlier, that regional wars do not have global consequences.

I am not teaching my students about the 1956 Israel-Arab war or the war in Western Sahara, I am illustrating the factors that go into making the truth of my definition self evident. Examples and non-examples both support the interpretation of concepts.

Imagine:

Imagining scenarios in which the concept might be illustrated, perhaps using analogies, can prove very effective. Interpreting analogies requires the learner to deconstruct and reconstruct the element of the concept, it supports deeper comprehension, improves retention and allows the learner to adapt then meaning of a concept into their own sociology-cultural context.

It’s important that as you construct your imagined scenario or your analogy, that you ground it in the existing, or at least conceivable, experience that your learners already have or could have. There is a danger that we forget just how culturally diverse our student cohorts are. References to popular culture, national habits and pastimes may mean something to you but are not going to be generally understood.

You could for example ask students to imagine a conflict between whichever country you are teaching in and ask how a conflict with a neighbour state might, or might not, have global consequences. I acknowledge for too many in the world this is not merely an intellectual exercise.

Changing Practice

Concepts are foundational to all new learning but we, in tertiary education, are in the habit of burying or obscuring the key concepts amidst the weight of information, and then expecting the learner to be able to think in abstract terms.

I had a lecturer recently tell me that they didn’t have time to change the example that they were using to teach supply and demand, a well developed scenario based on the oil price during the Second Gulf War. I found it very hard to believe, given that the current Russian invasion of Ukraine has had a direct impact on oil and commodity prices globally. Why, I suggested, why didn’t they ask the students to fill in the details of his scenario so that they would understand better the duplications of each factor rather than sharing a preprepared example. I suggested that might also provide an opportunity for students to talk more openly about the current threats that they may perceive impact on them personally as result of this particular war.

Strange as it may seem I think that the current war provides an important catalyst for the re-evaluation and revitalisation of much of our social science and humanities curriculum. It reminds us that there are existential threats around us, and that these should service as pivotal points of reference as we explore concepts with our students enabling them to make meaningful connections.

Students need to be encouraged to seek out sources of information with a critical eye in order to be better prepared for the unforeseen.


Photo by Антон Дмитриев on Unsplash

In-Class Evaluation (Guidance for Educators)

Transcript:

Welcome all. Please feel free to share this with your colleagues, if you think they’ll find this of interest. 

I want to talk today about why it’s important that we listen to our students during our teaching practice. I think, first of all, just want to clarify the distinction between providing feedback to students and taking evaluative comments from students.

I think sometimes the language we use is a little bit loose and that complicates things. It also devalues the sense of feedback that students are actually receiving during the course of their learning experience. I think we should always label anything we provide to students for feedback on their learning or in their learning, as feedback, and then anything we receive from them is evaluation at an institutional level of national surveys and in our individual practice. So I want to talk about the way we elicit evaluation from our students. So obviously the easiest way of doing that is to read the room. You’re looking for individual responses. You may be need to avoid focusing on that one very negative, grumpy, miserable looking student and read the room on that basis. You don’t want to imply that that is the common feeling, but if you scan the room, you’ll be able to see whether or not there is a good degree of engagement. I think we need to distinguish between that kind of very informal in session evaluation and more formal mechanisms for evaluation.

And I want to talk about an, an in session evaluation, and then I’ll talk about end of course, or end of module evaluation, in just a moment. Want to focus in this conversation about the way in which we use in class evaluation and the model that I personally prefer, and it works best in a face-to-face environment if you are teaching a series of courses, but it also works in an online environment as well.

Something called small group instructional diagnostics, S.G.I.D., which is essentially a way of eliciting from the students, some evaluative comments on what they’re learning, how they’re learning in your classroom, the structure of the questions that you ask are very important, they’re more important than how you actually ask them. Formal SGID is actually you leaving the classroom and having a colleague coming and running a session with your students for 15, 20 minutes.

And that usually would happen around a third of the way through your course. This is to elicit whether or not the students are getting what you’re intending they get, whether they’re experiencing the learning as you intend it to be, to be experienced. There are a number of different ways that you can do this.

I think it works quite well to literally just hand out pieces of paper with four blocks on and ask them to fill out, put something in each of those blocks. But it’s really important the order in which the questions are asked. 

So you’re going to ask the students what’s happening on this course that is supporting their learning, not what am I doing to support your learning? What is supporting you in your learning? What is hindering you in your learning? What could I be doing differently to support you? And fourthly, what could you be doing differently to support your own learning? It’s very important that you end with them thinking about it as a reflective exercise.

Very often, students will say that last comment. What can I, what can you be doing for your own learning? Students will very often say, oh, I need to do the pre-reading or I need to prepare better for class or so, it encourages them to take some degree of ownership over their own. It’s really important that when you collate all of those responses that you do feed back.

So if you’re teaching face-to-face or online on a weekly or, or a regular basis, some scheduled basis, it’s really important that you then say to the students, I listened to you, you suggested this is what strengthening your learning collectively, I’m going to do more of that. This is what’s hindering, I’ll try and do less of that.

That gives students a sense of participation and a sense of ownership in the learning process. That’s a very important part of getting evaluation from students. 

The more formal aspect of end of course, or end of module evaluation. It’s usually structured around whatever institutional regional or national surveys are carried out.

Historically institutions used to just do their own end of course evaluations and tutors regarded that as almost a tick box exercise. It’s become much more significant in many countries to have institutional data that is then aggregated across the, the entire piece. And I think it’s important that you recognize that those questions are not necessarily questions that you have any direct control over, but it’s very important that you are aware of what those questions are because you can then signal answers to the students through the course of the course, through the course of the learning. 

So I think it’s really important to be aware of what the end of course evaluation looks like, which you’ll probably end up administering, although you may not be able to control the questions, but you do have complete control of the in-class evaluation that happens.

So please feel free to share this video with colleagues. If you think it’s interesting, like and follow. 

Be Well

Evaluation, Assessment and Feedback (Guidance to Educators)

Transcript

Welcome all. Please feel free to share this video if you think it would be of interest to your colleagues.

I want to talk today about some of the terminological differences that we have across the English language teaching world, particularly the terms, evaluation, assessment, and feedback. In North America, the word evaluation is very often used to describe the way we measure students’ performance. In United Kingdom, in Australia and New Zealand, we generally use the term assessment. So evaluation has a different meaning in parts of English-speaking world than it does in North America. Likewise, Assessment and evaluation are sometimes used more as synonyms in the North American context. And you need to be aware of that when you read literature, if you read any of the journals, you will find that sometimes those terms are used differently to perhaps your context. So, it’s worth being aware of that.

There’s also a distinction between evaluation and feedback, which is more conceptual rather than definitional. Which is that feedback is always what we give to the student. We should always be focusing on the feedback that’s given to students on their learning and evaluation in the UK, Canada, Canada, to some extent, but certainly in Australia and New Zealand, is used to describe what they tell us about our own performances tutors, or about the course or the institution. So, they provide evaluative comment, and we provide them with feedback.

I think it’s important that we try and stick to that use of language. If only because students need to value feedback in everything they do, and it’s much easier to label things as feedback for the benefit of your students if you’re consistent in the language that you use. So, feedback is given to students. Evaluation is provided by students, and evaluation in North America is sometimes synonymous with assessment. I hope that’s of interest.

Please feel free to like, share, and follow.

Be well.

Guidance for Educators: Avoiding Burnout

Transcript:

Welcome all. Please feel free to share this video with colleagues if you think they’d find it of interest.

Want to talk today about burnout. Being a teacher, being a lecturer, being an academic, very often leads to periods of intense pressure, and very often we see colleagues burning out. They leave the profession.

They literally just find the ongoing pressure too much to bear, and I think it’s really important that we support each other, but we also have to preserve ourselves and avoid exhaustion. We do that by setting boundaries. You need to learn to say no. You need to learn that your responsibilities have a limit.

Sometimes we find ourselves wanting to, take on all of the pressures, all of the responsibilities that our students expect the entire institution to provide for them, and it’s important that we make time for relationships, for relaxation, for recreation. 

Education is a team game or a theatrical piece. It’s not a single performance. It’s not a solo game, nor are you actually a solo performer. You’re not a stand-up comedian standing alone on your stage. It’s a team effort. So if you do feel yourself under oppression. If you feel overly stressed, turn to a colleague and ask them for support. If those colleagues aren’t available for you to lean on, then you need to learn to escalate because it’s not your responsibility to run the entire college or the entire university or to run the entire school sector.

You need to be able to escalate your own stress just in order to avoid burnout. So if you feel the pressure lean on a colleague, if there’s no colleague to lead on, escalate.

Be well.

%d bloggers like this: