Category: Tools and Technology

The threat to the integrity of educational assessments is not from ‘essay mills’ but from Artificial Intelligence (AI)

The threat to the integrity of educational assessments is no longer from ‘essay mills’ and contract cheating but from Artificial Intelligence (AI).

It is not so long ago that academics complained that essay mills, ‘contract cheating’ services, and commercial companies piecing together ‘bespoke’ answers to standard essay questions, were undermining the integration of higher education’s assessment processes. The outputs of these less than ethically justifiable endeavours tried to cheat the plagiarism detection software (such as Turnitin and Urkund) that so many institutions have come to rely on. This reliance, in part the result of the increase in the student-tutor ratio, the use of adjunct markers and poor assessment design, worked for a while. It no longer works particularly well.

If you are interested in reviewing your programme or institutional assessment strategy and approaches please get in touch. This consultancy service can be done remotely. Contact me

Many institutions sighed with relief when governments began outlawing these commercial operations (in April 2022 the UK passed the ‘Skills and Post-16 Education Act 2022’ following NZ and Australian examples) and went back to the business-as-usual. For the less enlightened this meant a return to setting generic questions, decontextualised, knowledge recitation essay tasks. Some have learnt to at least require a degree of contextualisation of their students’ work, introduced internal self-justification and self-referencing, requiring ‘both sides’ arguments rather than declared positions, and applied the ‘could this already have been written’ test in advance. Banning essay mills, or ‘contract cheating’, is necessary, but it is not enough to secure the integrity of assessment regimes.

Why students plagiarise is worthy of its own post, but suffice it to say it varies greatly depending on the student. A very capable student may simply be terrible at time management and fear running out of time or feel the assessment is unworthy of them. Another student may be fearful of their ability to express complex arguments and in pursuit of the best possible grade, plagiarise. Some may simply have not learnt to cite and reference, or to appreciate that rewording someone else’s thoughts without attributing them also constitutes plagiarism. And there is that category of students whose cultural reference point, deference to ‘the words of the master’, make plagiarism conceptually difficult for them to understand.

I remember receiving my most blatant example of plagiarism and academic malpractice back in 2006. A student submitted a piece of work that included 600 words copied wholesale from Wikipedia, complete with internal bookmarks and hyperlinks. I suspect the majority of students are now sufficiently digitally literate not to make that mistake, but how many are also now in a position to do what the essay mills used to do for them, stitch together, paraphrase and redraft existing material using freely available AI text generation tools.

As we encourage our students to search the web for sources, how easy is it for them now to access some of the easily accessible, and often free, online tools? These tools include https://app.inferkit.com/demo which allows you to enter a few sentences and then generate longer texts on the basis of that origin. You can enter merely a title, of at least five words, or a series of sentences into https://smodin.io/writer and have it generate a short essay, free of references. Professional writing tools aimed at marketers, such as https://ai-writer.com, would cost a subscriber to be effective but would allow students to generate passable work. This last tool actually tells you the sources from which its abstractions have been drawn, including academic journals.

You might find it enlightening to take something you have published and put it through one of these tools and evaluate the output.

It is insufficient to ask the student to generate their own question, or even to ask the student to contextualise their own work. Some of the emergent AI tools can take account of the context. There is a need to move away from the majority of long-form text assessments. With the exception of those disciplines where writing more than a thousand words at once is justifiable (journalism, policy studies, and some humanities subjects), there is a need to make assessments as close to real-world experience as possible. It needs to be evidently the product of an individual.

Paraphrasing is a skill. A valuable one in a world where most professions do not lack pure information. The issue is to evaluate the quality of that information and then be able to reduce it to a workable volume.

I’ve worked recently with an institution reviewing its postgraduate politics curriculum. I suggested that rather than try and stop students from ‘cheating’ by paraphrasing learned texts, they should encourage the students to learn what they need to do to enhance the output of these AI tools. Using one of these tools to paraphrase, and essentially re-write, a WHO report for health policy makers made it more readable, it also left out certain details that would be essential for effective policy responses. Knowing how to read the original, use a paraphrasing tool, and being able to explore the deficiencies of its output and correct them, was a useful skill for these students.

We cannot stop the encroachment of these kinds of AI text manipulation tools in higher education, but we can make their contemporary use more meaningful to the student.

If you are interested in reviewing your programme or institutional assessment strategy and approaches please get in touch. This consultancy service can be done remotely. Contact me.

Image was generated by DALL-e

Why is adaptive learning not personalised learning

Lots of blogs are declaring the ‘new’ trends in e-learning for 2022. There is nothing truly new in most of these. The difficulty with trends is they frequently draw from the ‘thought-osphere’, rate than from emergent practice. One of these trends predicted for 2022 is ‘adaptive learning’.

banner image of road network at night

This ‘trend’ has been around for well over a decade and still cannot gain significant traction. Few would disagree that learning should be personalised to the largest degree possible to ensure motivation and relevance for the learner. Adaptive learning is the notion that we can tailor learning for the individual based on past performance and the ultimate direction. A little like the way an intelligent GPS navigation system for your car can suggest alternative routes when you take a wrong turn.

Educators have long implemented adaptive learning strategies in face-to-face contexts, usually as differentiated teaching practices. Teachers will tell you that for this strategy to be effective, class sizes need to be small and that you need to know your students at a human level. Knowing not just that they struggle with numerical concepts, but also that they enjoy music, or sports, so that we can pose exemplars as problems in a language that is meaningful to that learner. Differentiated teaching is highly personalised.

In contrast, adaptive learning systems, which are based on the interactions individuals have with computer based learning activities, simply cannot consider the societal influences, the personal likes and dislikes, of the individual. Sophisticated adaptive systems allow the learner to be presented with alternative phrasing of a problem if they appear to misunderstand instructions, or to be presented them with simpler versions of tasks if the initial one appears to be too difficult.

Adaptive systems are limited to the domain of knowledge acquisition, learning stuff, and some lower cognitive skill development. Essentially, adaptive learning is an old-style computer-based training (CBT) course on steroids. Facts and cognitive processes are important, and delivering up examples that are appropriately positioned to stretch the learner is a good thing. We should all understand, however, that the technology lags way behind the aspirations of teachers. It is a far cry from personalising learning.

Personalised learning is best enabled by situating learning experiences within the social context in which the learner lives and works. Allowing the learner to design their own responses to assessment tasks, to generate personalised evidence of their learning and relate their learning to their real lives. True personalised learning is essential for four domains of educational objectives, affective, psychomotor, interpersonal and the intrapersonal (meta-cognitive), for the factual knowledge ‘stuff’ and some cognitive skills there are, or may be, adaptive learning systems.

Unfortunately for budget-holders and institutional leaders, personalised learning requires a staff-student ratio that defies current budgets and, my particular interest, carefully crafted curriculum, programme and course design. For some, it appears to be worth investing in ‘automated’ learning systems, sold on the promise of responding directly to the student’s needs. Systems hyped by the vendors frequently underrepresented the investment needed in designing alternative branching scenarios and associated questions. Most vendors promise banks of questions based on relatively simple algorithms. Until computing power is significantly increased, answers and questions can be truly automated through AI systems, and systems can draw accurate student profiles based on social media and shared data (a worrying possibility for many) adaptive systems will remain a limited tool for specific contexts.

These contexts include mathematics and most ‘hard’ sciences. Where there is a required base of factual knowledge that is widely regarded as uncontested, adaptive learning systems can provide a marginally more engaging version of rote learning. It may even provide some ability to prompt the learner to transfer knowledge from one context to another beyond pure memorisation. I contest its applicability is still limited to the cognitive domain.

Still, the hyping of adaptive systems continues and they remain on most 2022 trends list. Clearly, one trend that I confidently predict for 2022 is that technological determinism, the concept that technology is intimately related to our social development, will continue to feature in the ‘trends’ blogosphere.

Photo by Ishan @seefromthesky on Unsplash

Teaching through Webinars (10’17”)

This ten-minute video (10’17”) is a series of screen captures from live synchronous webinars taught using Adobe Connect (2015). It is annotated to give you some sense of how to manage interactivity, manage your tone, reflect on the importance of personal presence and to make use of the visual nature of the webinar interface. These examples are taken from a postgraduate teaching qualification but the ‘content’ is irrelevant. While it is not intended to be a blow-by-blow explanation of how to construct your webinars, once you have access to a webinar room, Connect, Collaborate or other solution, this might give you some ideas as to how you could adapt your teaching practice for this form of synchronous distance teaching. #highered #teaching #webinars

These resources from 2013-2017 are being shared to support colleagues new to teaching online in the face of the COVID-19 pandemic.

Preparing for Webinar Teaching (6’18”)

This six-minute video (6’18”) is entitled ‘Best Practices in Preparing Online Materials for Webinar facilitation.” It is essential guidance for novice webinar teachers. There are 8 tips for preparing your webinar so you do not end up asking questions into the dark abyss and hear nothing back from your students. This screencast was generated for colleagues using AdobeConnect but it is suitable regardless of whether you are using this, or Blackboards Collaborate, ZOOM or any webinar context. Originally recorded in 2015. #elearning #AdobeConnect #webinar #higher #teaching

These resources from 2013-2017 are being shared to support colleagues new to teaching online in the face of the COVID-19 pandemic.

Effective Discussion Forums – disrupting assumptions with VoiceThread

An online discussion forum should be an effective way of engaging students in careful and considered reflection, yet often they represent time-consuming and frustrating experiences for faculty. Getting students to share thoughts and ideas, balancing contributions and knowing when to stimulate, moderate or step-back can be challenging. I’ve long found the advice to faculty, much of it still rooted in Computer Mediated Conferencing (CMC) models of the early mid 1990s-mid 2000s, unresponsive to the context of learning and the changing nature, and expectations, of learners.

It is remarkable how quickly Gilly Salmon’s Five-Step-Model (Salmon, 2000) became for the majority what Stephen D. Brookfield might describe as a paradigmatic assumption (Brookfield, 1995). The need for familiarity with the tool or context, a first-step, is itself now profoundly complex. Learners have hugely differing understandings of the function, and etiquette, ‘within‘ a given online communication tool. One of Brookfield’s lovely example of ‘hunting assumptions’ is the illustration of a common-sense assumption that to circulate around the classroom having assigned group tasks shows engagement, interest and commitment, whilst he suggests it may well be interpreted as a lack of trust and distorts learner responses to the task in hand. How quickly have we adopted Salmon’s notion of responsiveness, encouraging faculty to respond to each student posting, as common-sense, our paradigmatic assumption. Salmon herself is not to blame for this, the context of her original model had a very different digital landscape to underpin it, that notion of personal presence less well articulated, learners experiences of commenting, rating, sharing, reviewing in a myriad of different online contexts is something ‘new’.

So this week, while I watch colleagues around the globe initiate MOOCs and discuss global OER standards, I set myself a more modest task. I wanted to explore what we thought worked well in online threaded text-discussion and why there appeared to see an enduring negativity, from students and staff, about the dreaded ‘discussion board’.

Screenshot from asynchronous staff workshop in VoiceThread
Screenshot from asynchronous staff workshop in VoiceThread

I set up a one week asynchronous online continuing professional development (CPD) workshop for faculty and learning support staff discussion effective thread text-based discussions online (we have Blackboard 7). The workshop was to run during the working week, from Monday 9am to Friday 5pm, with a ‘new’ topic added each day and an encouragement for colleagues to ‘dip-in-and-out’. What was different from our usual institutional practice was we did it in Voicethread (www.voicethread.com).

A VoiceThread is a collaborative, multimedia slide-shot that holds images, documents, and videos and allows people to navigate slides and leave comments in four different ways. Users can use text, voice using their computer microphone, upload a pre-recorded audio file or capture a video on their webcam. In the US they can also use the telephone.

Users can also draw on the slides while commenting, allowing them to annotate a diagram, image or schematic; they can also zoom-in to the image for more detail. Users can also create multiple identities, allowing them for instance to take on a group leadership role whilst remaining a member of the group, or adopt more playful and creative personalities. VoiceThreads can be embedded in the existing VLE or another web page and can also be archived. It is primarily a web browser based tool, now also available on iOS mobile devices and available in an accessible screen reader version or very low bandwidth version, and users have no software to install.

The tool itself has strengths and weaknesses and, whilst I declare it is a tool I’ve used and evaluated previously (Burden & Atkinson, 2010), I am not seeking to promote it. Rather what I wanted to do was to take faculty away from their existing assumptions about discussion threads and have the conversation in a very different context. As we explored each day a different theme it proved remarkable how some contributions followed the ‘discussion board’ convention, whilst others playfully sought to exploit the new technological opportunities the environment afforded them. One colleague made a series of short, positive and responsive contributions in response to others, what might be seen as rather appropriate netiquette in online discussions. However, because the comments were all appended to the end of a timeline, those responses (unless clearer ‘tagged’ as such) appeared as ‘hanging interjections’ without context. None of the 79 contributions during the week, except for my own, used anything but text which I felt was disappointing.

In coming weeks I will analyse the pattern and nature of the responses in more detail, and critique my own model of facilitation in this context, but what has emerged immediately is how quickly some ‘assumptions’ have been set and are subsequently modelled even when users find themselves in a different communication context. This is perhaps one of our biggest challenges as educational technologists and developers, or instructional designers, is to recognise our quickly solidifying paradigmatic assumptions and move beyond them. The digi-ecology is in constant flux and we need to consistently challenge how we do what we do.

Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. John Wiley & Sons.

Burden, K., & Atkinson, S. (2010). De-Coupling Groups in Space and Time: Evaluating New Forms of Social Dialogue for Learning. In L. Shedletsky & J. E. Aitken (Eds.), Cases on Online Discussion and Interaction (pp. 141–158). Hershey, Pennsylvania: IGI Global.

Salmon, G. (2000). E-moderating : the key to online teaching and learning. London: Kogan Page.

Space, Tools and Voice: effective classroom practice

Teaching quality interventions are always a challenge for an Academic Developer, walking a thin line between sanction and support, between reassurance and patronizing. Yesterday I ran a small workshop with new classroom tutors teaching Statistics. The challenges of teaching a dozen international students new statistical concepts at 5pm on a wet windy Friday evening in London needs little unpacking.
I sought to provide some tools for staff to think about their role, to reflect on how they might meet discipline goals. We focussed not on ‘tips and tricks’ but on the fundamentals of space and voice, and how to develop engagement with both.
We explored some classroom layouts, including the one we were in, and identified the social conventions of space that determine the learning and teaching styles that are adopted. It’s a nice activity which prompts participants to identify three different disciplines being taught in three layouts illustrated when in fact all three real examples are from the same course on the same day. It prompts the question ‘am I using this learning space effectively or does it constrain, and indeed determine, how I teach?’
The tools available in the space, projector, whiteboard, PowerPoint, Prezi, etc were then discussed in relationship to this space. One of the questions I like to ask is how would you teach if the familiar tools fail you (in this case, the decorators say have removed the whiteboards). On this occasion I was walking the talk because my PowerPoint presentation had no means of being projected on rhe day and I had rearranged the seating to be able to run the session differently. Useful demonstration however unwelcome! It was a nice exploration of the nature of the whiteboard as a ‘canvas’. I suggested students might be encouraged to photograph the whiteboard (the mobile phonecam being almost ubiquitous) to support the idea that this was a group creative process which rewarded engagement not simple the instruction from the board.
We then explored , with some amusing examples, the nature of the English language as a stressed language and the relationship, interplay, between intonation and connotation. This is always fun, not least because I get to remind myself why I never went into acting!
This combination of being more spatially aware, of using the tools with engagement as the intent not information delivery, and of simple appreciating the power of the human voice, will hopefully develop confidence and a sense of each tutors’ unique abilities to communicate.
It was a fun session to run and one I hope I get to run again soon and I would love to run it overseas too.

Too wild a concept for publication: a date for 3V model

Looks like this book chapter with Kevin Burden on the conceptual modelling of emerging technologies is finally going to see the light of day. I note the publisher’s site now has chapter details, download prices and chapter ISBNs. So after a long wait, it’s going to happen:

D'Agustino Book
Adaptation, Resistance and Access to Instructional Technologies: Assessing Future Trends In Education

Atkinson, S., & Burden, K. (2011). Using the 3V Model to Explore Virtuality, Veracity and Values in Liminal Spaces (Pages 199-215). In S. D’Agustino (Ed), Adaptation, Resistance and Access to Instructional Technologies: Assessing Future Trends In Education. Hershey, PA: Information Science Publishing.

It’s been an interesting process for this one. We had a journal article rejected and I was beginning to wonder if this was just too ‘left field’ and whether anyone would engage with it as an idea. I’m still convinced that the 3V concept has an interpretive and evaluative value but it needs a professional conversation and that means at least getting it ‘out there’ in a form that can be referenced in the hope dialogue follows. Here’s hoping.

Digital technologies and their role in Higher Education

I attended a seminar run through the London Learning Lab yesterday focused on the future of education and the implications for ethical research. You can read more about it here. By chance I came across this presentation by Helen Beetham which she gave at Greenwich University recently. It covers a vast range of issues related to digital technologies in Higher Education but a lot of what she is saying is pertinent to what we are interested in achieving with the DiAL-e framework. Have a watch and listen if you get a chance!

Visual Rhetoric: Prezi explanantion

Yesterday I posted some early thoughts on how visual rhetoric might be important to us in thinking about how we communicate in the teaching process, not just what we have to say but HOW we say it.

I said I’d have a crack at a PREZI presentation to illustrate my point – and here it is 🙂

See-Sick in Prezi: What place for visual rhetoric?

I have begun writing a paper on visual rhetoric. I sat on the 7:31 commuter train to St.Pancras and watched commuters, hunched over their laptops, working in PowerPoint. Their screens filled with words, varieties of fonts, and formatting tricks in abundance. These comprehensive essays in landscape, perhaps to be printed and distributed but more likely projected illegibly for a bewildered business audience later that day reminded me again of the fundamental misuse of a very powerful and effective technology. The same day I showed my wife a Prezi presentation that I was preparing for a workshop the following day. Her comment was that it made her feel seasick as I moved fluidly, but somewhat distractedly, from one block of text to another. I suggested the term ‘see-sick’.

So I began to consider the power of these visual tools in our classrooms and the very superficial understanding that I, and I suspect the majority of my colleagues have, of their use. I turned to Aristotle.

Aristotle identified three branches of rhetoric: judicial, epideictic and deliberative.  Judicial rhetoric is concerned with justice and injustice, the defence or advocacy of charge or accusation. Epideictic rhetoric refers to speech or writing in praise or blame. Perhaps the most familiar notion of rhetoric is that of deliberative, in which speech or writing attempts to persuade others to take or not to take some defined action.

Much of our teaching is the incitement to learners to do something, to take an action. Teaching may in many circumstances be considered deliberative rhetoric, an invitation on the part of the student  (as reader, listener or participant) to pause and consider in response to a carefully timed performance and managed argument, the pace and rhythm control, the deliberate self interruption, punctuated silence, exclamations, questions, punctuating gestures. The teacher’s role is not simply to highlight an argument but to ensure that if a vote were cast the learner might make an appropriate judgment.

Teaching in face-to-face contexts supported by presentational technologies, the ubiquitous PowerPoint or some more contemporary form of visual media, requires a new mastery of rhetoric – that of visual rhetoric. This branch of rhetorical studies that concerns itself with the persuasive use of images, in isolation or in harmony with words, is a powerful tool in the classroom.

We live in an intensely visual world, surrounded by images in advertising, music, news information and educational media. Arrangements of words, in tag clouds, Word Clouds,  or PowerPoint arrangements are visual objects. Text projected on the wall is either a visual representation, discursive, provocative, motivating or informative, or it is ‘just words’. Not every projected arrangement of light and dark on the classroom wall is easily inferred as a visual object, as visual rhetoric, “(W)hat turns a visual object into a communicative artefact–a symbol that communicates and can be studied as rhetoric–is the presence of three characteristics. In other words, three markers must be evident for a visual image to qualify as visual rhetoric. The image must be symbolic, involve human intervention, and be presented to an audience for the purpose of communicating with that audience.” (Smith, 2005, p. 144)

Kostelnick and Roberts in “Designing Visual Language: Strategies for Professional Communicators”, detail six canonical criteria through which to interpret the rhetorical impact, primarily of written text. These six are: arrangement, emphasis, clarity, conciseness, tone and ethos.

  • Arrangement – is the organisation of visual elements to demonstrate structure (and relationships)
  • Emphasis – differentiates elements giving some prominence through changes in size, shape and colour.
  • Clarity – avoiding unnecessary elements to assist the reader in ‘decoding’ quickly and completely the ‘message’
  • Conciseness –appropriately succinct designs that serve a specific audience need
  • Tone – the writer/presenter/designer’s tone provides evidence of their attitude to the subject
  • Ethos – developing the trust of the audience

These six visual criteria provide a helpful starting point in beginning to see images as objects for visual rhetoric and appropriate interpretation.  (Kostelnick & Roberts, 2010)

Since Zaltman suggests that thoughts occur as images, which are essentially visual, there is a direct inverse relationship between the power of the visual to provoke an emotional non-verbal reaction, a thought. Research by Joy and colleagues using the Zaltman Metaphor Elicitation Technique suggest a range of interesting relationships between viewers sense of space and depth, related directly to the positioning of objects, to the juxtaposition (overlapping, transparency, distortion) of images in support of a narrative. They conclude, “(U)ltimately, images and words are separate building blocks in the telling of stories but the two amplify each other. Researchers need to enrich and supplement the abstractions that accompany visuals with the details and particulars that accompany the verbal.” (Joy, Sherry Jr., Venkatesh, & Deschenes, 2009, p. 566)

Back in 2001, I did (what I still think was ) some interesting work with Nicola Durbridge at the Open University’s Institute of Educational Technology looking at how to overcome some of the restrictions of text-based discussion boards, the ‘drudgery’ of CMC (Computer Mediated Conferencing). I explored a simple ‘visual metaphor’ of a classroom so that individuals posting items to a forum did so ‘spatially’ as icons rather than simply adding the posting to the list. (Atkinson, 2001). I would be keen to extend now in looking at Prezi and its use of visual rhetoric.

A Prezi that explores visual rhetoric is here.

 

 

……

Atkinson, S. (2001). Re-Tooling Online. In Book of Abstracts (pp. 154-156). Presented at the Online Educa Berlin 2001, Berlin: ICEF Berlin GMBH.

Joy, A., Sherry Jr., J., Venkatesh, A., & Deschenes, J. (2009). Perceiving images and telling tales: A visual and verbal analysis of the meaning of the internet. Journal of Consumer Psychology, 19(3), 556-566. doi:10.1016/j.jcps.2009.05.013

Kostelnick, C., & Roberts, D. D. (2010). Designing Visual Language: Strategies for Professional Communicators. Longman Publishing Group.

Smith, K. L. (2005). Handbook of visual communication: theory, methods, and media. Routledge.

Accessibility by WAH
%d