Evaluation, Assessment and Feedback (Guidance to Educators)

Transcript

Welcome all. Please feel free to share this video if you think it would be of interest to your colleagues.

I want to talk today about some of the terminological differences that we have across the English language teaching world, particularly the terms, evaluation, assessment, and feedback. In North America, the word evaluation is very often used to describe the way we measure students’ performance. In United Kingdom, in Australia and New Zealand, we generally use the term assessment. So evaluation has a different meaning in parts of English-speaking world than it does in North America. Likewise, Assessment and evaluation are sometimes used more as synonyms in the North American context. And you need to be aware of that when you read literature, if you read any of the journals, you will find that sometimes those terms are used differently to perhaps your context. So, it’s worth being aware of that.

There’s also a distinction between evaluation and feedback, which is more conceptual rather than definitional. Which is that feedback is always what we give to the student. We should always be focusing on the feedback that’s given to students on their learning and evaluation in the UK, Canada, Canada, to some extent, but certainly in Australia and New Zealand, is used to describe what they tell us about our own performances tutors, or about the course or the institution. So, they provide evaluative comment, and we provide them with feedback.

I think it’s important that we try and stick to that use of language. If only because students need to value feedback in everything they do, and it’s much easier to label things as feedback for the benefit of your students if you’re consistent in the language that you use. So, feedback is given to students. Evaluation is provided by students, and evaluation in North America is sometimes synonymous with assessment. I hope that’s of interest.

Please feel free to like, share, and follow.

Be well.

Guidance for Educators: Avoiding Burnout

Transcript:

Welcome all. Please feel free to share this video with colleagues if you think they’d find it of interest.

Want to talk today about burnout. Being a teacher, being a lecturer, being an academic, very often leads to periods of intense pressure, and very often we see colleagues burning out. They leave the profession.

They literally just find the ongoing pressure too much to bear, and I think it’s really important that we support each other, but we also have to preserve ourselves and avoid exhaustion. We do that by setting boundaries. You need to learn to say no. You need to learn that your responsibilities have a limit.

Sometimes we find ourselves wanting to, take on all of the pressures, all of the responsibilities that our students expect the entire institution to provide for them, and it’s important that we make time for relationships, for relaxation, for recreation. 

Education is a team game or a theatrical piece. It’s not a single performance. It’s not a solo game, nor are you actually a solo performer. You’re not a stand-up comedian standing alone on your stage. It’s a team effort. So if you do feel yourself under oppression. If you feel overly stressed, turn to a colleague and ask them for support. If those colleagues aren’t available for you to lean on, then you need to learn to escalate because it’s not your responsibility to run the entire college or the entire university or to run the entire school sector.

You need to be able to escalate your own stress just in order to avoid burnout. So if you feel the pressure lean on a colleague, if there’s no colleague to lead on, escalate.

Be well.

Guidance to Educators: Developing professional relationships

Transcript:

Welcome all, please feel free to share this video with colleagues if you think they would find it of interest. 

Let’s talk today about building professional relationships. Teaching can be quite a lonely experience. Depends a little bit on the organization that you teach in. You might be teaching in a very isolated part of the world, or you might be teaching a very specialist discipline. You might be the only person teaching that particular subject in your school, even in your area. 

And having good connections with other practitioners that understand you, understand your context, definitely do serve to lower the level of anxiety that you might feel. There’s evidence to suggest that well- connected educators do suffer less anxiety.

So reaching out now is much easier. There are any number of digital platforms that you can engage with and connect with other people. And in doing so you benefit not just that level of human connectivity, but you’re also using them as a source of new ideas, new sources, new perspectives. 

It’s very important if you do get involved in any of these platforms that you do become a contributor, as well as a consumer. That’s not just because that’s fair. It’s just, it’s also that echoing your voice is really important, using your voice to mirror the practice of others is part of the process of building those relationships.

Even if it’s just to go back to someone who’s posted something, you found a value to just say “I’ve used what you suggested. It worked very well for me” or it didn’t and I made this adjustment, and I did it this way. Having that level of feedback is really important. 

So, I’m on a number of different platforms. I’m on LinkedIn. I’m on Twitter. I’m on Instagram. I’m on Tik Tok. I’m on YouTube and I’ve got my own website, but I think the two that everyone needs to start with are Twitter and LinkedIn. Build a profile on LinkedIn, doesn’t have to be expansive, but at least something that gives people a sense of who you are as a person.

And then Twitter is a great way of just picking up ideas, sources, perspectives, re-tweeting things that you think are of interest, identifying things more widely, and posting them and share that community experience. You’d be surprised how quickly it does build and giving you a solid network to lean on will undoubtedly reduce your anxiety. 

So you might want to try some of those platforms for yourself. See how it goes. 

Let me know. Be well.

Guidance for Educators: From Simplicity to Complexity

Transcript:

Welcome all. Please feel free to share this with any of your colleagues who you think might benefit from it. Let’s talk today about complexity and simplicity. So, I think you can save yourself an enormous amount of time by carefully planning a session based on the complexity or simplicity of the argument that you’re trying to convey to your students.

It’s really important that we don’t stress the content of what we’re delivering. Rather the interrelationships of what it is we’re talking about or teaching about. Whether that’s a conceptual relationship, it might be a timeline. It might be a cause and effect, relationship, whatever the relationship is, whatever conceptually encapsulates the knowledge that’s being shared.

So, if you plan your lesson, it’ll depend a little bit on the nature of the discipline and the level which you’re teaching. But I think generally it’s quite a good idea to outline the complex picture as briefly as possible as a target that students are going to aim for, but rather than then try and make that your starting point and try and explore that you basically want to go right back to the beginning to the most simple building blocks of that complexity and build your way forward.

I think the reason to do that is it’s very easy sometimes to make assumptions that students have had the same life experiences you, or that they actually have the same linguistic ability, terminologies. That they understand the jargon, and there’s a real danger that you can trip over yourself.

If you start from a complex try and go to simple, you need to basically start with simple and build up. It’s really important to make sure that if you do think there is discipline jargon to be shared, that has to be unpicked and built into the session,  put up definitions alongside any jargon that you’re using.

So, I think it’s really important just to situate the complexity of the topic in their, in the student’s, landscape of learning, not in your own. And it’s very often, the case is that we almost, it’s not about showing off, but sometimes we literally just kind of feel that we need to start with what we’re most comfortable with, which is sometimes a very advanced level of knowledge.

So, it’s really important to just plan out your session in advance. I use a mind map to do that. I usually have a map sometimes on paper, sometimes using a bit of software that allows me. Map out the journey from simplicity to complexity. And when I do that, I can share that with my students as well.

You might want to try that, see how it goes. Please share like, and follow be well.

 

Why I am not a social-constructivist

I have never believed in social-constructivism. At least not the way the educational anthropologists’ definition of the phenomena has been distorted and contorted into current practice. Social-constructivists justifiably argue that knowledge is often constructed through social interaction. Further, they state that the social and cultural context in which that learning occurs is significant. I just don’t believe that it necessarily requires in-person encounters. And I don’t think it applies to all forms of learning and disciplines.

Atharva Tulsi at Unsplash

The fetishism of ‘group-work’, which has continued to grow since the 1980s on the back of skimming the literature about social constructivism, and further enabled through digital tools in developed economies, has been applied to nearly all disciplines and all levels. This simply doesn’t make sense. Socialisation matters for children in K-12 as they learn diverse social skills through subject-based curriculum; at least in theory. Group-work, applied to much of the University curriculum has been poorly conceived. Rich courseware should provide a  transparent socio-cultural context for its learning. It rarely does. Unless the intention is to refine and extend the processes of socialisation for University students, students can, and should, be empowered to mediate the knowledge through their own socio-cultural reality.

When I read, listen, or watch something I am engaged in learning from another human being. Often this learning is asynchronous, sometimes time-displaced to an extreme degree, but there is still evidence of a voice. How well crafted the learning is, will depend on the coherent nature of that voice, but there is always a voice. At the Open University in the early 2000s Course Teams worked hard to ensure that no matter how many course authors might contribute to a course, there was a consistent ‘voice’. I just don’t believe that it is appropriate to assume that an individual’s learning is enhanced somehow by having ‘horizontal’ conversations with others who are at the same level of learning as themselves. I agree that one can learn from others. That is not the same as saying one necessarily learns with peers.

Personally, I believe we should be designing learning experiences, and courseware, that the individual student can deploy in their own context. If learners ‘want’ to learn with others, with whānua (family/community) or colleagues, they can do so. We may want to encourage them to mobilise people around their own learning, and to build networks to support their learning journey. This would be a truer representation of their lifelong learning experience going forward.

I don’t believe we should force students to ‘come and learn with us‘. To do so is to perpetuate an arcane model of learning that reinforces notions of power and privilege. It’s a model of learning that centralises access to knowledge, and maintains the notion of gatekeepers to learning. We should empower and enable learners through our courseware, not enslave them through it.

Photo by Atharva Tulsi on Unsplash

 

A new national vocational learning strategy. What could a Te Pūkenga Ako Strategy look like?

Te Pūkenga (https://tepūkenga.ac.nz), the centralised vocational tertiary organisation in New Zealand created in the last two years, represents an exciting opportunity to create a new way of conceiving of the learner experience.

A learning experience based on learner choice, learner contexts and personalised journeys.

 

Close up fingers holding map pinDuring a recent joint ODLAA webinar,  Dr Som Naidu  provocatively suggested there were no institutions truly embracing the concept of true flexibility for learners. As President of FLANZ, I think about this all the time. What is the possibility that Te Pūkenga can do something unique?

To design and support learning across all vocational disciplines is a challenge. Some learning must be experienced, witnessed in person, and evidenced through demonstration. While much of this traditionally conceived of as in-person learning can in fact be asynchronous, captured on video and dialogue facilitated alongside, some learning requires tactile demonstration and immediate feedback. There may be some learning can be ‘single-mode’, experience, just on-campus with nothing to take home or reflect on away from the campus, although I struggle to imagine what disciplines fall into that category. Some learning might be done entirely outside of a social context, with no interaction with others, other than the authoritative voice incorporated into a text.

Current models of learning are increasingly less satisfying to contemporary generations who navigate across dynamic and fluid knowledge platforms and devices as part of their daily lives.

There is no shortage of ‘content’. Most learning is infused across a learner’s life, thoughts invading their waking hours and possibly their dreams. Designing learning journeys that are adaptable to each individual’s context is challenging for those organisations who traditionally operate on manufacturing paradigms. This is true whether the model was the individual academic as solo artisan or the large design team following an industrial process. At best, both create an imagined ‘best scenario’, an optimal pathway, at worse they generate a single restricted route through their courseware. Their conception of ‘the right way to learn’.

However, just as the world of broadcasting has changed dramatically in the last 30 years from one-way communication to a world of multiple diverse channels, citizen journalism, and expert blogging (and vlogging), so finally vocational education, at least in New Zealand, has an opportunity to change the way it creates, shares and supports learners.

There is less need for the single authoritative voice and instead there is a clear need for learners to develop autonomous learning practices, judgement and discernment, the ability to evaluate the quality and usefulness of any learning artefact.

Learning should be co-created with learners, never delivered to them as a finished product. A good place for Te Pūkenga to start would be to ask, “how do I deliver the learning experience to the learner in their own context”. That doesn’t mean turning everything into Distance Learning. Rather, it requires curriculum, programme and course designers to think about the learners’ context and design learning (materials and support) that allow them to create their own personalised, or differentiated, learning pathway.

This means Te Pūkenga might be wise to focus on establishing solid programme and course designs and navigational aids rather than on learning content. I advocate a designing around situated learning principles and then curating a range of existing learning materials, drawn from individual practitioners, professional bodies and educational providers. Te Pūkenga could choose to structure its ako strategy as being as open as possible. Encouraging learners, given a map with key milestones (assessments) and  access to curated artefacts alongside that map. Generating original learning resources then becomes only necessary when there are identifiable gaps.

Learning artefacts from which Te Pūkenga constructs its pathways should also be created as Open Educational Resources (OER). This is because the development of these learning opportunities have already been funded off the back of individual taxation and it is immoral to ask individuals to pay for them twice. There is also a strong argument for learners to be enabled to update resources, to rcontextualise them, make them suitabe for their social and cultural context, and for the next generation of learners that follow them, subject to the same quality assurance processes.

These OER learning resources require a quality framework, based on peer review, and a suitable taxonomy to ensure individual artefacts are recoverable and reusable. Learning designers who commission OER, or identify existing OER, need to do so within strict guidelines. We cannot just assume that everyone’s PowerPoints are useful out of context, but the ideal situation would be to establish key concepts and supply learners with alternatives, from visual, auditory and written interpretations and explanations. These artefacts also clearly need a curated content management system, such as one based on OpenEQUELLA.

As with any strategy, it needs to differentiate between learners’ capabilities. At lower levels of the national qualifications framework where students may require more structure, pathways may be more limited. Limited but not restricted. The system clearly needs progression built in. The focus remains on empowering the learner to take ownership of their learner journey. Part of Te Pūkenga’s stated goal is to empower learners to become competent and confident digital citizens and lifelong learners. We don’t do that by giving them a neat little bundle of a course with all the answers included. At higher levels of learning, degree level and above, part of being a contemporary learner is being able to discern the validity of sources and interrogate them.

I also conceive this system of curated OER, sitting alongside the ‘course map’, a customised version of the Mahara ePortfolio with a range of support ‘plugins’ being available. Centralised OER resources, a single course map, with minimal milestones (beyond formal assessment), and options for different levels of in-person or virtual, synchronous or asynchronous, support should be part of the strategy. Across the entire national vocational space, Te Pūkenga should then focus on supporting individuals, their whānau (community), and/or professional context where appropriate.

Empowering learners to construct their own journey has to be the foundational principle.

As Dr Som Naidu suggested, to create such an institution requires a mind shift among current leadership. In Te Pūkenga that means everyone who works within any of its subsidiary organisations needs to let go of how things are currently constructed. It requires national quality assurance agencies, in this case NZQA, to think differently. It requires educational vision and leadership, and a seismic shift in the educational paradigm. It represents a revolution in practice, not an evolution.

 

This blog also appears on LinkedIn 15 November 2021
Photo by GeoJango Maps on Unsplash

Blended by us or differentiated by learners: the future of courseware design

Ten years ago, in 2011, I wrote a blog entitled ‘there’s no such things as blended learning’, which essentially suggested that all learning experiences are blended to some extent, making the term irrelevant.

Since then, the boundaries between contexts, technologies and experiences have become even more blurred. Yet rather than discarding the blended terminology, there is simply a profusion of new terms, hybrid and hyflex, being the current vogue. Oh, and ‘flipped’, which is presented to the ill-informed as something new and radical. The problem is these terms are driven by us, as institutions, to define the nature of our course offering, rather than being conceptualised as the learner experiences them.

I am comfortable using the term ‘blend’, alongside ‘mix’, ‘selection’, ‘options’ and many synonyms when talking about courseware designed for a specific delivery context. The context of the learner is key. Any contemporary learner journey is going to involve a ‘smorgasbord’ of learning material, voices to be exposed to, individuals to share reflections with, and physical, social and cultural contexts in which learning is occurring. I can’t imagine a context in which a learner only learns through one communication mode, be it a lecture or workshop.

Learning can, and should, be as ‘flexible’ as possible. Informed by the principles of Universal Design for Learning, learning should be malleable enough by the learner to suit their evolving needs and context. Learners should be able to discard elements of the learning journey, take shortcuts rather than revisit existing learning if they choose. Equally, they should be able to explore around the edges of the path designed for them; to go ‘off-piste’ if you like.

Good learning design and good teaching encourage the learner to re-contextualise newly gained knowledge and experience in the light of previous learning. Given that each individual’s context is unique, it is essential that learners should blend their own learning experience. Learners should be enabled to make-meaning for themselves. Good teachers know this.

In practice, the terms blended, hybrid and hyflex, are really being used by institutions to define the nature of their ‘product’, rather than the nature of the learning experience. Institutions choose to package what they sell under different labels, it’s a marketing pitch, “now with added webinars” or “now with extra VLE resources available”. Some senior managers have assumed the opportunities for off-campus communication engagement in the internet era represent a new alternative pedagogy. In reality, the ‘alternative’ pedagogies have always been there. There have always been skilled faculty who reached beyond the lecture or seminar room and engaged learners in their own context. Designing courses that are suitable for open navigation is counter-intuitive for most institutions. The focus has been on designing a learning pathway, not pathways. It’s easier for institutions that way.

What has changed since 2011 is the range of communication technologies available for learners to choose, or not choose, to interact with content, experiences and each other. Courseware in my view can, and should, be designed with open navigation, open pathways, so a learner can choose how they want to arrive at a preconceived set of outcomes. We can provide an optimal route to success for the less adventurous, but choice empowers. Essentially, learners can differentiate their journey from others based on their context and personal needs. Hey, why don’t we use the term ‘differentiated learning’… although that sounds familiar. Wonder if anyone has used that term before? Forgive my sarcasm, but I do wonder whether we need to find new language to describe the aspirations for our courseware as it is experienced by learners.

If we acknowledge that everything is to some extent blended, then what term would encourage courses to be designed to enable learning journeys suitable for personalisation by the learner. Differentiated learning is the best I’ve got.

Photo credit: Photo by Elena Mozhvilo on Unsplash

How is leadership in higher education responding to changing notions of autonomy and accountability?

With the disruption to delivery models, timetables, and staff and student expectations in the last 18 months some institutions are struggling to maintain their faculty’s motivation and commitment. Some are wrestling with changing notions of autonomy and accountability.

With the disruption of delivery models, timetables, and staff and student expectations in the last 18 months, some institutions are struggling to maintain their faculty’s motivation and commitment. Universities are struggling to balance the need to provide their academic staff with more autonomy while ensuring they remain accountable.

Some academic staff still hark after the glorious days of academic self-management. The danger is that it doesn’t take much for that ‘autonomy’ to be abused; The elderly professor earning the salaries of three junior colleagues, applying fruitlessly for funds for arcane and irrelevant research, with no PhD supervision duties and no teaching, is not as rare as we like to imagine. Such individuals demonstrate to newer faculty that they can achieve career advancement by being selfish. This breeds a culture in which those with a relatively light workloads do their best to appear overburdened in order to defer requests from others to ‘pitch-in’. Most of us can identify such individuals.

The balance between academic autonomy and accountability defines the character of an institution from a faculty perspective. Autonomy and accountability are reflected in large part by how an organisation articulates leadership and management, two concepts that are frequently conflated inappropriately.

Leadership is about enabling with vision, providing clarity of purpose, illuminating the path ahead. This means communicating a clearly defined future state; a vision. Leadership does not require seniority. We often look to colleagues that we know to be skilled and confer the mantel of leadership on them. You can develop leadership skills, but usually within a specific context. A leader in one organisation at one time does not always adapt well to a different context. Some prove adaptable, but not all. Leadership is about empowering others to be more autonomous.

Management is quite different. Management is about implementing, maintaining, and curating structural processes within a given context. Everyone self-manages by this definition (calendar management, time-booking, etc). Beyond self-management, most organisations create tiers of managers to maintain policies and practices, to fulfil something externally imposed whatever legislative regulations or quality standards. Management is ensuring accountability.

We require leaders to trust the people they have responsibility for. Leaders need to provide supportive autonomy. Managers do not have to trust their people because they have tools to track them. They have instruments for accountability. It has been said that leaders make sure that the right things are done, managers make sure that things are done in the right way. 

Autonomy and accountability are two sides of the same coin. While some institutions have released faculty to get their own courses onto the institutional virtual learning environment, others had more structured approaches. In both cases, many have been unprepared for what changing models of delivery mean for accountability. Student complaints have surprised some institutions, mostly about the inaccessibility of faculty in the digital context. Students expectations need careful management. This does not need more systems to monitor faculty-student interactions, or appointing more people to watch people, and people to watch the watchers. It requires that new social-digital contracts be negotiated among all the participants and stakeholders in the University ecosystem.

Universities face challenges with some students and faculty struggling to adjust to the demands of balancing workload and practices of supporting flexible online provision. Going ‘back to normal’ for some will simply not be possible. This is a time when leaders and managers need to work together.

Managers need to hold the freeloader Professor and the ‘too busy’ junior colleague to account. Leaders need to define the future state of Universities in a language that faculty and students can make sense of. Together, they need to define, negotiate, explore and define new concepts of accountability and autonomy.

 

 

 

 

Photo by kerry rawlinson on Unsplash

University Learning and Teaching Strategies Post-Covid

Learning during the commute: Photo by Verschoren Maurits from Pexels

One characteristic of a four to five year Learning and Teaching Strategy (LTS) is that it should require a complete re-write when it comes up for renewal. Given the inevitable pace of change, any remotely ambitious strategy is likely to have several ‘not achieved’ elements when it comes up for review. If you can sign-off on a five-year strategy as ‘complete’, you weren’t trying hard enough.

Someone has recently asked me to contribute to a 2021-2025 Learning and Teaching Strategy (LTS) for a University. I have drafted and contributed to many such documents over the last 25 years, so it’s always interesting to have a glimpse into other institutions. I realized one defining characteristic of the leadership of universities today is whether they have looked at their Learning and Teaching Strategy issued before January 2020 and have thought, “Emmm, maybe we need a rethink.”

Some leadership has a long-term mindset. They have recognised the enormous effort, commitment and dedication of the majority of their faculty to adjust their practices to Emergency Remote Teaching and are supporting those same faculty to retain and enhance their best practice into the future. Others have solely focussed on their balance-sheets, student-generated income, estate costs and spend time appealing for government support. The former are concerned with investing in their future state, the later worrying about this year’s numbers.

This particular LTS is ambitious; for them. The ability for faculty to continue to support their learners regardless of whether they work remotely, across time zones, from anywhere in the world. A move away entirely from end-of-course summative assessments and exams, towards student-paced portfolio assessment regardless of the discipline. Developing practical learning experiences that can be undertaken at home, or at other institutions and work-places. There are some major structural changes that will be needed to enable these learning practices to take root. The underlying philosophy is that the contemporary University student no longer has the luxury of dedicating their entire being to live and study at University for three years. They need flexibility.

Elements within this particular 2021-2025 Learning and Teaching Strategy will not be achieved. Sometimes this is because ambitions require changes to the digital ecosystem beyond institutional control, or they are subject to the vagaries of the shifting political landscape. Given the intransigence that sometimes appears embedded in the sector, some ambitions may just require too much of people. Nonetheless, it has been satisfying to see leadership willing to embark on a strategy, knowing the best that can be hoped for is ‘partially achieved’. Which from my perspective will be an unmitigated success.

Dr Simon Paul Atkinson (PFHEA)
Learning Strategist //www.sijen.com

Photo by Verschoren Maurits from Pexels

Is the Future of Education Inevitably Going to be Digital First?

The FLANZ webinar ‘Is the Future of Education Inevitably Going to be Digital First?’, held 6th November 2020, was a conversation about how the world of higher education, in particular, has responded to the Covid-19 pandemic and whether the future looks different as a result.

Duncan O’Hara, FLANZ Vice-President, led contributors, Australian-based Professor Neil Selwyn of Monash University, and New Zealand-based Professor Giselle Byrnes of Massey University, and Dr Simon Paul Atkinson of the Open Polytechnic, through a series of questions.

All contributors agreed that the response of higher education institutions across the globe was nothing short of remarkable. A huge effort had been made, not least by technology-support and academic development units, alongside faculty, to serve students’ needs. There was a note of caution, however, that having moved so much learning online in haste, that management might perceive it as ‘job done’, a cheaper option or indeed a satisfactory learning experience for the majority of learners. The reality is that while some institutions have seized the opportunity to build-up from solid foundations and provided an enriched digital experience for their learners, others have supplied the bare minimal.

The panel largely agreed that there is no one-size-fits-all to learning. So, any decisions by institutions and policymakers need to be context-specific, putting the learner at the heart of any technology choice. 

A healthy debate was had around the issue of digital equity, ranging from access to devices, the appropriateness of those devices for the nature of the learning, network access and the disparity in digital literacy that has become emphasised in the Remote Emergency Teaching context resulting from Covid-19. The conversation turned to the Principles of the Design Justice Network (https://designjustice.org/) advocating that all too impacted on design decisions need to be enabled to share their voices. This is as true for the technology tools and platforms in use as it is for the curricula that we curate.

The struggle to ensure that the learner remains at the centre of institutional policy-making decisions was evident in the discourse. However, the openness of the dialogue, and the questions and comments shared by participants, show great promise for the Australasian region, with all of its heterogeneity, that positive solutions are at hand.

#

This webinar was part of the inaugural Australasian Online Distance Learning Week #AODLW2020 run by FLANZ in association with ODLAA and EDEN, it’s Australian and European equivalents.

%d bloggers like this: