Skip to content

Te Pūkenga (https://tepūkenga.ac.nz), the centralised vocational tertiary organisation in New Zealand created in the last two years, represents an exciting opportunity to create a new way of conceiving of the learner experience.

A learning experience based on learner choice, learner contexts and personalised journeys.

 

Close up fingers holding map pinDuring a recent joint ODLAA webinar,  Dr Som Naidu  provocatively suggested there were no institutions truly embracing the concept of true flexibility for learners. As President of FLANZ, I think about this all the time. What is the possibility that Te Pūkenga can do something unique?

To design and support learning across all vocational disciplines is a challenge. Some learning must be experienced, witnessed in person, and evidenced through demonstration. While much of this traditionally conceived of as in-person learning can in fact be asynchronous, captured on video and dialogue facilitated alongside, some learning requires tactile demonstration and immediate feedback. There may be some learning can be ‘single-mode’, experience, just on-campus with nothing to take home or reflect on away from the campus, although I struggle to imagine what disciplines fall into that category. Some learning might be done entirely outside of a social context, with no interaction with others, other than the authoritative voice incorporated into a text.

Current models of learning are increasingly less satisfying to contemporary generations who navigate across dynamic and fluid knowledge platforms and devices as part of their daily lives.

There is no shortage of ‘content’. Most learning is infused across a learner’s life, thoughts invading their waking hours and possibly their dreams. Designing learning journeys that are adaptable to each individual’s context is challenging for those organisations who traditionally operate on manufacturing paradigms. This is true whether the model was the individual academic as solo artisan or the large design team following an industrial process. At best, both create an imagined ‘best scenario’, an optimal pathway, at worse they generate a single restricted route through their courseware. Their conception of ‘the right way to learn'.

However, just as the world of broadcasting has changed dramatically in the last 30 years from one-way communication to a world of multiple diverse channels, citizen journalism, and expert blogging (and vlogging), so finally vocational education, at least in New Zealand, has an opportunity to change the way it creates, shares and supports learners.

There is less need for the single authoritative voice and instead there is a clear need for learners to develop autonomous learning practices, judgement and discernment, the ability to evaluate the quality and usefulness of any learning artefact.

Learning should be co-created with learners, never delivered to them as a finished product. A good place for Te Pūkenga to start would be to ask, “how do I deliver the learning experience to the learner in their own context”. That doesn’t mean turning everything into Distance Learning. Rather, it requires curriculum, programme and course designers to think about the learners’ context and design learning (materials and support) that allow them to create their own personalised, or differentiated, learning pathway.

This means Te Pūkenga might be wise to focus on establishing solid programme and course designs and navigational aids rather than on learning content. I advocate a designing around situated learning principles and then curating a range of existing learning materials, drawn from individual practitioners, professional bodies and educational providers. Te Pūkenga could choose to structure its ako strategy as being as open as possible. Encouraging learners, given a map with key milestones (assessments) and  access to curated artefacts alongside that map. Generating original learning resources then becomes only necessary when there are identifiable gaps.

Learning artefacts from which Te Pūkenga constructs its pathways should also be created as Open Educational Resources (OER). This is because the development of these learning opportunities have already been funded off the back of individual taxation and it is immoral to ask individuals to pay for them twice. There is also a strong argument for learners to be enabled to update resources, to rcontextualise them, make them suitabe for their social and cultural context, and for the next generation of learners that follow them, subject to the same quality assurance processes.

These OER learning resources require a quality framework, based on peer review, and a suitable taxonomy to ensure individual artefacts are recoverable and reusable. Learning designers who commission OER, or identify existing OER, need to do so within strict guidelines. We cannot just assume that everyone’s PowerPoints are useful out of context, but the ideal situation would be to establish key concepts and supply learners with alternatives, from visual, auditory and written interpretations and explanations. These artefacts also clearly need a curated content management system, such as one based on OpenEQUELLA.

As with any strategy, it needs to differentiate between learners’ capabilities. At lower levels of the national qualifications framework where students may require more structure, pathways may be more limited. Limited but not restricted. The system clearly needs progression built in. The focus remains on empowering the learner to take ownership of their learner journey. Part of Te Pūkenga’s stated goal is to empower learners to become competent and confident digital citizens and lifelong learners. We don’t do that by giving them a neat little bundle of a course with all the answers included. At higher levels of learning, degree level and above, part of being a contemporary learner is being able to discern the validity of sources and interrogate them.

I also conceive this system of curated OER, sitting alongside the ‘course map’, a customised version of the Mahara ePortfolio with a range of support ‘plugins’ being available. Centralised OER resources, a single course map, with minimal milestones (beyond formal assessment), and options for different levels of in-person or virtual, synchronous or asynchronous, support should be part of the strategy. Across the entire national vocational space, Te Pūkenga should then focus on supporting individuals, their whānau (community), and/or professional context where appropriate.

Empowering learners to construct their own journey has to be the foundational principle.

As Dr Som Naidu suggested, to create such an institution requires a mind shift among current leadership. In Te Pūkenga that means everyone who works within any of its subsidiary organisations needs to let go of how things are currently constructed. It requires national quality assurance agencies, in this case NZQA, to think differently. It requires educational vision and leadership, and a seismic shift in the educational paradigm. It represents a revolution in practice, not an evolution.

 

This blog also appears on LinkedIn 15 November 2021
Photo by GeoJango Maps on Unsplash

Ten years ago, in 2011, I wrote a blog entitled ‘there’s no such things as blended learning’, which essentially suggested that all learning experiences are blended to some extent, making the term irrelevant.

Since then, the boundaries between contexts, technologies and experiences have become even more blurred. Yet rather than discarding the blended terminology, there is simply a profusion of new terms, hybrid and hyflex, being the current vogue. Oh, and ‘flipped’, which is presented to the ill-informed as something new and radical. The problem is these terms are driven by us, as institutions, to define the nature of our course offering, rather than being conceptualised as the learner experiences them.

I am comfortable using the term ‘blend’, alongside ‘mix’, ‘selection’, ‘options’ and many synonyms when talking about courseware designed for a specific delivery context. The context of the learner is key. Any contemporary learner journey is going to involve a ‘smorgasbord’ of learning material, voices to be exposed to, individuals to share reflections with, and physical, social and cultural contexts in which learning is occurring. I can't imagine a context in which a learner only learns through one communication mode, be it a lecture or workshop.

Learning can, and should, be as ‘flexible’ as possible. Informed by the principles of Universal Design for Learning, learning should be malleable enough by the learner to suit their evolving needs and context. Learners should be able to discard elements of the learning journey, take shortcuts rather than revisit existing learning if they choose. Equally, they should be able to explore around the edges of the path designed for them; to go ‘off-piste’ if you like.

Good learning design and good teaching encourage the learner to re-contextualise newly gained knowledge and experience in the light of previous learning. Given that each individual’s context is unique, it is essential that learners should blend their own learning experience. Learners should be enabled to make-meaning for themselves. Good teachers know this.

In practice, the terms blended, hybrid and hyflex, are really being used by institutions to define the nature of their ‘product’, rather than the nature of the learning experience. Institutions choose to package what they sell under different labels, it’s a marketing pitch, “now with added webinars” or “now with extra VLE resources available”. Some senior managers have assumed the opportunities for off-campus communication engagement in the internet era represent a new alternative pedagogy. In reality, the ‘alternative’ pedagogies have always been there. There have always been skilled faculty who reached beyond the lecture or seminar room and engaged learners in their own context. Designing courses that are suitable for open navigation is counter-intuitive for most institutions. The focus has been on designing a learning pathway, not pathways. It’s easier for institutions that way.

What has changed since 2011 is the range of communication technologies available for learners to choose, or not choose, to interact with content, experiences and each other. Courseware in my view can, and should, be designed with open navigation, open pathways, so a learner can choose how they want to arrive at a preconceived set of outcomes. We can provide an optimal route to success for the less adventurous, but choice empowers. Essentially, learners can differentiate their journey from others based on their context and personal needs. Hey, why don’t we use the term ‘differentiated learning’… although that sounds familiar. Wonder if anyone has used that term before? Forgive my sarcasm, but I do wonder whether we need to find new language to describe the aspirations for our courseware as it is experienced by learners.

If we acknowledge that everything is to some extent blended, then what term would encourage courses to be designed to enable learning journeys suitable for personalisation by the learner. Differentiated learning is the best I’ve got.

Photo credit: Photo by Elena Mozhvilo on Unsplash

With the disruption to delivery models, timetables, and staff and student expectations in the last 18 months some institutions are struggling to maintain their faculty’s motivation and commitment. Some are wrestling with changing notions of autonomy and accountability.

With the disruption of delivery models, timetables, and staff and student expectations in the last 18 months, some institutions are struggling to maintain their faculty’s motivation and commitment. Universities are struggling to balance the need to provide their academic staff with more autonomy while ensuring they remain accountable.

Some academic staff still hark after the glorious days of academic self-management. The danger is that it doesn't take much for that 'autonomy' to be abused; The elderly professor earning the salaries of three junior colleagues, applying fruitlessly for funds for arcane and irrelevant research, with no PhD supervision duties and no teaching, is not as rare as we like to imagine. Such individuals demonstrate to newer faculty that they can achieve career advancement by being selfish. This breeds a culture in which those with a relatively light workloads do their best to appear overburdened in order to defer requests from others to 'pitch-in'. Most of us can identify such individuals.

The balance between academic autonomy and accountability defines the character of an institution from a faculty perspective. Autonomy and accountability are reflected in large part by how an organisation articulates leadership and management, two concepts that are frequently conflated inappropriately.

Leadership is about enabling with vision, providing clarity of purpose, illuminating the path ahead. This means communicating a clearly defined future state; a vision. Leadership does not require seniority. We often look to colleagues that we know to be skilled and confer the mantel of leadership on them. You can develop leadership skills, but usually within a specific context. A leader in one organisation at one time does not always adapt well to a different context. Some prove adaptable, but not all. Leadership is about empowering others to be more autonomous.

Management is quite different. Management is about implementing, maintaining, and curating structural processes within a given context. Everyone self-manages by this definition (calendar management, time-booking, etc). Beyond self-management, most organisations create tiers of managers to maintain policies and practices, to fulfil something externally imposed whatever legislative regulations or quality standards. Management is ensuring accountability.

We require leaders to trust the people they have responsibility for. Leaders need to provide supportive autonomy. Managers do not have to trust their people because they have tools to track them. They have instruments for accountability. It has been said that leaders make sure that the right things are done, managers make sure that things are done in the right way. 

Autonomy and accountability are two sides of the same coin. While some institutions have released faculty to get their own courses onto the institutional virtual learning environment, others had more structured approaches. In both cases, many have been unprepared for what changing models of delivery mean for accountability. Student complaints have surprised some institutions, mostly about the inaccessibility of faculty in the digital context. Students expectations need careful management. This does not need more systems to monitor faculty-student interactions, or appointing more people to watch people, and people to watch the watchers. It requires that new social-digital contracts be negotiated among all the participants and stakeholders in the University ecosystem.

Universities face challenges with some students and faculty struggling to adjust to the demands of balancing workload and practices of supporting flexible online provision. Going 'back to normal' for some will simply not be possible. This is a time when leaders and managers need to work together.

Managers need to hold the freeloader Professor and the 'too busy' junior colleague to account. Leaders need to define the future state of Universities in a language that faculty and students can make sense of. Together, they need to define, negotiate, explore and define new concepts of accountability and autonomy.

 

 

 

 

Photo by kerry rawlinson on Unsplash

Learning during the commute: Photo by Verschoren Maurits from Pexels

One characteristic of a four to five year Learning and Teaching Strategy (LTS) is that it should require a complete re-write when it comes up for renewal. Given the inevitable pace of change, any remotely ambitious strategy is likely to have several ‘not achieved’ elements when it comes up for review. If you can sign-off on a five-year strategy as ‘complete’, you weren’t trying hard enough.

Someone has recently asked me to contribute to a 2021-2025 Learning and Teaching Strategy (LTS) for a University. I have drafted and contributed to many such documents over the last 25 years, so it’s always interesting to have a glimpse into other institutions. I realized one defining characteristic of the leadership of universities today is whether they have looked at their Learning and Teaching Strategy issued before January 2020 and have thought, “Emmm, maybe we need a rethink.”

Some leadership has a long-term mindset. They have recognised the enormous effort, commitment and dedication of the majority of their faculty to adjust their practices to Emergency Remote Teaching and are supporting those same faculty to retain and enhance their best practice into the future. Others have solely focussed on their balance-sheets, student-generated income, estate costs and spend time appealing for government support. The former are concerned with investing in their future state, the later worrying about this year’s numbers.

This particular LTS is ambitious; for them. The ability for faculty to continue to support their learners regardless of whether they work remotely, across time zones, from anywhere in the world. A move away entirely from end-of-course summative assessments and exams, towards student-paced portfolio assessment regardless of the discipline. Developing practical learning experiences that can be undertaken at home, or at other institutions and work-places. There are some major structural changes that will be needed to enable these learning practices to take root. The underlying philosophy is that the contemporary University student no longer has the luxury of dedicating their entire being to live and study at University for three years. They need flexibility.

Elements within this particular 2021-2025 Learning and Teaching Strategy will not be achieved. Sometimes this is because ambitions require changes to the digital ecosystem beyond institutional control, or they are subject to the vagaries of the shifting political landscape. Given the intransigence that sometimes appears embedded in the sector, some ambitions may just require too much of people. Nonetheless, it has been satisfying to see leadership willing to embark on a strategy, knowing the best that can be hoped for is ‘partially achieved’. Which from my perspective will be an unmitigated success.

Dr Simon Paul Atkinson (PFHEA)
Learning Strategist //www.sijen.com

Photo by Verschoren Maurits from Pexels

The FLANZ webinar ‘Is the Future of Education Inevitably Going to be Digital First?’, held 6th November 2020, was a conversation about how the world of higher education, in particular, has responded to the Covid-19 pandemic and whether the future looks different as a result.

Duncan O’Hara, FLANZ Vice-President, led contributors, Australian-based Professor Neil Selwyn of Monash University, and New Zealand-based Professor Giselle Byrnes of Massey University, and Dr Simon Paul Atkinson of the Open Polytechnic, through a series of questions.

All contributors agreed that the response of higher education institutions across the globe was nothing short of remarkable. A huge effort had been made, not least by technology-support and academic development units, alongside faculty, to serve students’ needs. There was a note of caution, however, that having moved so much learning online in haste, that management might perceive it as 'job done', a cheaper option or indeed a satisfactory learning experience for the majority of learners. The reality is that while some institutions have seized the opportunity to build-up from solid foundations and provided an enriched digital experience for their learners, others have supplied the bare minimal.

The panel largely agreed that there is no one-size-fits-all to learning. So, any decisions by institutions and policymakers need to be context-specific, putting the learner at the heart of any technology choice. 

A healthy debate was had around the issue of digital equity, ranging from access to devices, the appropriateness of those devices for the nature of the learning, network access and the disparity in digital literacy that has become emphasised in the Remote Emergency Teaching context resulting from Covid-19. The conversation turned to the Principles of the Design Justice Network (https://designjustice.org/) advocating that all too impacted on design decisions need to be enabled to share their voices. This is as true for the technology tools and platforms in use as it is for the curricula that we curate.

The struggle to ensure that the learner remains at the centre of institutional policy-making decisions was evident in the discourse. However, the openness of the dialogue, and the questions and comments shared by participants, show great promise for the Australasian region, with all of its heterogeneity, that positive solutions are at hand.

https://hail.to/flexiblelearningNZ/article/YVWxb5d

This webinar was part of the inaugural Australasian Online Distance Learning Week #AODLW2020 run by FLANZ in association with ODLAA and EDEN, it’s Australian and European equivalents.

We need to continue to move away from seeing tertiary education as the imparting knowledge and see it rather as developing the skill of all students to be able to decide which learning pathways best suits their context, prior experience and aspirations. One of the consistent messages I try and instil in others' practice is the importance of the social context in which the student inhabits.

In November 2018 I contributed to an EDEN online webinar talking about 'Innovative Education' as part of the 2018 European Distance Learning Week. Here is my presentation, entitled "Designing Pathways: which way to innovation?"

I have been rather busy of late, so this is a rather delayed post from an interview done in June 2020 with SRCE, the Croatian National e-Learning Centre. The Interview appears under the title "Student je središte obrazovnoga procesa", which translates as 'The student is at the centre of the educational process'. Below is an English version of the extended interview.

Screenshot 2020-11-08 at 19.16.03.png

SK: How does one transfer face-to-face (f2f) courses to an online environment? 

Carefully! The difficulty is that the honest answer is "it depends". It depends on several factors. Whether your institution already has a robust infrastructure to support learning online, is there already a virtual learning environment in place, and how well is it being used and supported? What are the relative skill sets of the technologists, educational technologists and academics? Has the institution already predetermined a learning and teaching strategy that defines the nature of the blend of online and face-to-face provision, or has it just been left to develop organically as the technologies have become more and more widespread? 

There a strategic challenge for institutions, beyond the influence of most front line teachers. I think the challenge is that language, certainly in English, is very changeable and often misused. Managers mix up phrases like the 'flipped classroom' and 'blended', or they associate 'online' with 'remote' learning. I think it is worth investing some time at an institutional level to agree on what you mean by the terms you use. Otherwise, people can talk across each other and misunderstand both each other's context and meaning.

Institutionally, we need to think about the human element. Recognising that the skills to design learning and teaching online, whether to overcome the short-term lack of classroom contact or for longer-term distance learning purposes, are not the same skills required of face-to-face teachers. Most online distance and flexible learning organisations (ODFL) have created many specialisms within their workforce. Many traditional institutions still expect individuals to play a range of unique roles and so moving these online is increasingly challenging. We need to confront the fact that the change of environment means a dramatic change in roles, and that won't fit everyone's personal or career aspirations.

SK: And for an individual teacher's perspective, what should they be thinking about?

I would suggest that the first thing anyone should do when moving a course from face-to-face to online is to be honest with themselves. It's an opportunity to look for quality enhancement. In the short term you may be compelled to move existing presentation material online, the resources you might have delivered in a lecture, and wrap around that some discussion forum to substitute what you might have done in a seminar. Readings are less of an issue. But in the medium term, and in the long term, we need to recognise that designing learning for online delivery is an entirely different experience from face-to-face. So I think there is a perfect opportunity to dismantle your existing course, challenge your assumptions, and go right back to the basic design elements within your courses.

I think it is also important to avoid taking a deficit mindset around online, by which I mean to think that online or distance learning is 'worth' less than traditional face-to-face learning. The evidence suggests the opposite but what is true is that they are significantly different experiences from a student's perspective.

Most courses have been designed around course outcome statements. Ordinarily, these are written statements as to what the student will be able to 'do' once they've completed the course and these guide both assessment, learning activities and content. If your institution doesn't follow this approach, it's not too late to do it for yourself. Writing good learning outcomes provides a structure for both the learner and the tutor. If you've got learning outcomes already in place for your course, go right back to them and re-conceptualise your learning approach, knowing the context in which your students are now learning.

There is an assumption that when the student is on campus and sitting in the classroom, they are a captive audience, you can monopolise their time for the one or two hours that they are timetabled. In an ODFL context, students' time is out of your reach. That means the learning itself needs a degree of motivation and structure built into it. The consequence of this is that you design learning that requires the learner to build on their own experience, to situate their learning wherever they are, and seek out authentic learning opportunities. I believe as a result, ODFL can produce much better learning than many face-to-face instances where there is the danger that given the captive nature of the student, they can be subjected to content lead learning design. So my key message is to be as honest as you can be about the quality of the learning experience rather than being overly concerned with the quality of your content. Focus on the learner experience.

SK: So what practical steps can teachers do to achieve that?

First, I think the best teachers are reflective practitioners. They are continuously asking themselves whether they are serving the best interests of the learners from a learning perspective, not from a subject content perspective. The reality is the vast majority of factual knowledge delivered through our courseware is available to students free if they know how to find it. What we do as educators is we synthesise, organise and order this knowledge into meaningful experiences. Some of our colleagues may have been teaching for 30 years in a face-to-face environment and now teach online for the first time, some of them I know feel like they're teaching for the first time! The COVID-19 pandemic is a significant opportunity, despite unfortunate circumstances, to confront everything about what we do. Teachers need to be aware of their own strengths and weaknesses. So keep a journal, reflective notes, annotate your presentations, make quick observations after each teaching encounter. Get into the habit of challenging yourself as to what you could have done differently and what you plan to try differently next time. 

Second, teachers need to know that they are not expected to do everything by themselves. Indeed, I don't think anyone can do everything. There is a reason ODFL organisations have so many interwoven specialisms. I think it's important to recognise that learning design, course design, is neither a science nor is it a dark art. If it were a science, big organisations would've already found a way of making an engine where you put in your requirements and of course comes out the other end, but despite their best efforts that appears not to be on the horizon. Likewise, good course design and good teaching, with some very rare exceptions, are best done in broad daylight, and with peer review, rather than behind the closed doors of a dark lecture theatre. One of the positive effects of the expansion, or massification, of higher education in Europe and globally is that it has compelled greater transparency in learning and teaching practices. So teachers need to know enough about themselves, so they know when to ask for help.

Third, and it's a related point, is when there is no immediate support if your institution just doesn't have the staff to guide you in developing good online learning, reach out. Look for support from many international organisations that have provided resources to support you and your colleagues. I'm aware of resources that are shared by EDEN, ICDE, UNESCO and others. I think it's important that teachers know that they are not alone and that regardless of the discipline, whether its undergraduate dance or postgraduate chemistry, someone, somewhere will be trying to solve the same challenges you face. I, like many others, share resources openly on blogs and institutional websites. Use them.

SK: How can teachers ensure that an online course is not simply a repository of learning materials?

Good learning design would never result in a simple repository of learning materials. It just can't happen, that's a contradiction. It's the equivalent of a face-to-face teacher handing out a reading list for their and doing nothing else. Good learning design puts the learner experience at the heart of its design; resources complement the experience; they don't replace it. I think sometimes we associate with having students physically present in our classrooms means that they are engaged. We assume that they are listening and that they're paying attention. We may have designed in-class activities, and the students may be actively engaged with them, but a lot of our teaching is still very passive. Sometimes when teachers move recorded lectures, presentations, PowerPoints and notes online but are not being 'present' to facilitate any engagement with these resources, they are left wondering whether they have any value. They may wrap a discussion forum around resources and engage the student in a meaningful dialogue amongst themselves and with the tutor, but when this dialogue fails to happen that can be very frustrating. Some students engage some don't, and there is no immediate way of resolving that I'm afraid. We would have to explore questions of motivation, discipline-fit, literacies and a range of other things which I think is a bit beyond the scope of this conversation.

However, I will say that good learning design would suggest that rather than providing a resource and attempting to get the student to engage with it, it is better to envisage the learning experience and use resources only when it is appropriate. There is nothing wrong with students spending time reflecting, looking for other sources of information to validate what is being given to them or creating online resources collegially or individually. We need to get away from this myth that we, as teachers, are the source of all wisdom. Some students will experience poor learning during this COVID-19 emergency that we all face. But it's important to recognise that that is not implicit in the ODFL experience, it is likely to result from several assumptions and presumptions that have informed less than ideal practices. Moving courses online is challenging many personal assumptions that we have.  

SK: Let's talk a little more about assessment in an online environment. How it differs from f2f and how to do it? 

Gosh, that's a huge question. In some ways, the design approach shouldn't be any different. Every institution should have an assessment strategy, and arguably every programme and every course needs to have a refined version of that strategy written into it. Assessments, regardless of the mode in which they run, should be valid and reliable, but they should also be authentic, durable and situated. Just to explain my language here; valid in that they assess the outcomes and not content, reliable in that you could expect to repeat the assessment with future cohorts and get comparable results. Authentic because it is appropriate for discipline and level, durable because ideally, we wouldn't want to keep re-writing assessments each time we deliver a course, and situated in that the assessment allows the student to personalise their evidence.

If you start on that basis, and you look at whatever the learning outcomes are for your course, re-conceptualising assessment becomes less daunting. It should be less about how you adapt your existing assessments to deliver them online; rather, it should be to take an online approach to assess your outcomes. I recognise that for some colleagues, the conventions within their disciplines are incredibly rigid. They may feel uncomfortable about even suggesting that the sacred cow that is the 'final exam' may not even be necessary. I understand that. But, there is a strong argument to be made where assessing learning outcomes differently represents a quality enhancement opportunity.

SK: So, how does a teacher decide on the assessment method for online? 

I would say first that people need to be thinking about 'originating' not 'replicating'. Forget about trying to assess the same way online. The learning outcomes dictate the assessment method. Without them, I suspect it's just about what's realistic within your institution. If your outcomes require factual recall and memorisation, then the assessment method might be as straightforward as a multiple-choice questionnaire embedded inside the virtual learning environment. Any higher-order thinking skills, to discuss, to debate, to analyse, to evaluate or to critique, are likely to require long answer text forms. The question is how authentic that assessment can be. Many institutions still have a tendency to run exams, I think often just because they're configured to do so rather than because it is actually assessing any defined outcomes. Unless you're assessing someone in policy studies or journalism who is required to show an ability to write under pressure, exams risk assessing merely the ability to sit exams or memory rather than any higher cognitive functions. I think it's important to think about what it is the students actually providing evidence of.

One of my biggest professional frustrations has always been with teachers who say they want their students to be able to identify, research and critically evaluate a range of complex sources in order to synthesis or analyse a situation, and then they set a three-hour exam based around the content in the course. Assessment is undoubtedly one of the most challenging areas of learning design but incredibly satisfying when you get it right!

Clearly, outside of the social sciences, there are other assessment challenges. In the performing arts one may have inferred, but not written into an outcome, the ability to perform in front of an audience for example. In the long run, one may need to rethink the way those outcomes are written, but in the short term, it is entirely possible for students to provide recordings of performances in their own time, in their own space. Some might object to allowing students to perform something multiple times and submit their best effort. I don't see that as a problem unless the learning outcome suggests that the student only gets one high-pressure opportunity. Again, I think it depends on what it is you're assessing.

I think it's important that we try to avoid thinking about how to replicate what we do face-to-face online and go back to the fundamentals of our learning design. We can arguably make adjustments to most of our face-to-face intentions by adopting an ODFL mindset.

SK: In your view does online assessment provide an equally relevant assessment as that in f2f? 

Relevant is an interesting choice of word. I suspect what you are alluding to is the fear that many colleagues have about plagiarism or academic misconduct away from the exam context. You cannot just post a 'standard' exam online and not expect students to go into their social media groups and exchange information, download answers from the Internet, or even buy a response. The alternative might be running online proctored exams, but I'm not in favour of that. Unless writing within strict time limits like a journalist is an outcome, I think it's unnecessary. Instead, I think we are better to be thinking about the assessment from where the student is standing. We need to conceive of the learner, online and distant, living away from campus, and having access all the course content. They don't live on the moon! They are part of society, so we need to design learning, and assessment, that encourages them to personalise their own context.

I want to unpack the word relevance. Whether the assessment is valid and reliable is about whether the skills you want the students to provide evidence of is captured through your assessment design, and if you repeat the assessment, the results are likely to be equivalent. So designing valid assessment determines its relevance. I think it's important wherever possible to design assessment that is authentic and situated. Authentic means that the evidence the student is expecting to provide of their learning should be as close to real-world experiences as possible. It always surprises me that institutions go to substantial lengths to create scenarios and case studies when the real-world is just outside the students' windows! Situated means that the learner should be required wherever practicable, to draw on experiences and realities in their own context. This means they could be asked to create a case-study based on something within their social context. What could be more relevant?

As I've said, I think it is arguably easier to provide situated and authentic learning to students who are studying away from the campus and away from an existing cohort. If a student has to situate their learning, and therefore their assessment, it is often easier for teachers to become familiar with an individual's student's pattern of learning. Where a teacher is aware of their students' strengths and weaknesses, they will recognise evidence of academic misconduct more readily. Many institutions now teach at a scale that means individual teachers are not assessing their own students directly. I think if we design assessments to be authentic and situated; it is possible to design out plagiarism. I suspect that is the holy grail in assessment design. Ensuring that students can make the learning, and therefore also their assessment, as meaningful to their own lives as possible. Forcing students to provide context for their learning means cutting and pasting from the Internet becomes more of a chore than it is an advantage.

SK: What final words would you like to share with colleagues here in Croatia?

To say that I respect the commitment and tenacity of teachers, and I admire how they have coped under extreme pressure to do what's best for their students. To say that I think being reflective, understanding your strengths and weaknesses, asking for help when required. Thinking from the Online Distance Flexible Learning (ODFL) position back towards the campus, rather than the other way around, is the way to tackle the personal challenges ahead. Also, if I might add I have fond memories of my visits to Zagreb and hope to visit Croatia again.

On April 27th I had the pleasure of sharing a virtual platform with Alan Tait (Open University UK), hosted by Tim Read (UNED, Spain), for a webinar entitled 'How to Engage and Support Students Online'.

This is the EDEN (European Distance Education Network) community's response to the demands put on staff to teach remotely, many for the first time, with very little notice.  The webinar series, with all webinars available on the EDEN Youtube Channel covers a wide range of perspectives.

My contribution (starting at 27'50") was to highlight some of the tools and approaches that are widely available to ensure students and faculty can continue to make learning happen. I focussed on the notion of engaging students and advocated that everyone needs to adopt a learning model or approach that serves to make sense of the chaos. My own approach uses the SOLE model but any model would help structure responses at an individual and institutional level.

Here are the resources:

an Adobe Presenter version of the PowerPoint that I shared (on a separate page)

Screenshot of Adobe Presenter

A PDF version of the presentation with full notes here

A link to the EDEN YouTube recording of the full webinar available on the EDEN channel https://youtu.be/mw-6066s1vQ. 218 Individuals attended the ZOOM meeting with 50 colleagues joined via YouTube Livestream.

 

 

I believe it is important to design learning from the learners perspective. That means learning that is both relevant, meaningful and motivating but also that is realistic and feasible within an agreed timeframe. This is a very brief explanation for those new to designing courses of how to work out "how much is enough?"

I believe we should calibrate our learning to take account of the 'notional study hours' or NSH (alternatively referred to as 'Notional Student Hours').

The calculation may vary from the country by country. In tertiary institutions in the United Kingdom, New Zealand, South Africa and other jurisdictions, a single academic credit equates to 10 hours of student learning. In the European Credit Transfer Scheme, one credit equates to 25 to 30 hours. My explanations below take the UK/NZ model and assume that a single course, worth 15 credits has an NSH value of 150 hours. A full-time student would be expected to study eight such courses in a year, 120 credits in the first year, 120 in the second and so resulting in 360 credits for a three year degree.

Remember that the NSH is the total students are expected to study to earn their credits, NOT the amount of time you have to be guiding them. Work out what time each week you are expecting students to spend on independent study (without any guidance from you) and what time you are responsible for guiding them on. This last number that is most important to faculty designing courses.

Here is a simplified list of actions that all Faculty might want to enact:

  1. Review the course documentation (check level and benchmark statements from national or regional quality assurance agencies)
  2. Remind yourself of any assumptions made as to prior learning
  3. Remind yourself of the learning outcomes for your course
4. Remind yourself as to the credit weighting and work out for your course NSH 1 Credit = 10 hours NSH

15 Credit= 150 hours NSH

5. Remind yourself of the number of hours expected to be guided, as opposed to independent study. Institutions sometimes have different interpretations of national guidance. Usually, they see a decline in the number of guided hours as you go up the level. First-year undergraduates receiving more guided hours (65%) than masters students for example (33%)
6. Remind yourself of the assessment hours allocated to your course. It is not uncommon to deduct a number of hours for overall assessment tasks, these are usually included in the independent study hours. So say we deduct 30 hours off the 150 hours for this 15 credit course.
7. Then do a calculation of the number of weeks over which your course is expected to run and divide the NSH of the course by the number of weeks. This will give you the number of notional study hours (NSH) for your course per week We would then take the remaining 120 hours, work out what percentage of that was appropriate for guided learning hours (@ first year let's say 120 x 0.65 = 78)

Divided by the number of weeks in a  course (say 12) that would mean in this example we would be expected to provide learners with (78/12) 6.5 hours of guided learning.

You need to work through an example based on guidance from your own quality assurance colleagues to ensure you stay in tune with regional or national guidelines.

What is essential is that you do not see the guided learning hours as time spent directly with students. It includes anything you direct a student to watch, read or listen to. Any activities you instruct them to undertake as well as any online resources you choose to provide.

It is very often the case that we are 'over-teaching' in our on-line courses. Being aware of the NSH for your course is a good place to start.

In this short video (5'52"), Simon touches on three basic principles of programming assessments. The first is that it should be programme wide, the second that assessing outcomes not content provides future flexibility, and the third that summative (or credit-bearing) assessments do not have to be final or terminal assessments. Assessment is one of the most difficult areas for faculty to become comfortable with. Most will have experienced badly designed assessment themselves and their expectations of their academic managers, programme leaders and their students are often low. This is a shame because well-designed assessment can be a pleasure for both students and faculty.

These resources from 2013-2017 are being shared to support colleagues new to teaching online in the face of the COVID-19 pandemic.

%d bloggers like this: