A specter is haunting higher education
Fall semester after the generative AI revolution
Quick note: I’m adding more AI news items at the end of these posts. Please let me know if that’s useful to you.
It’s been a wild and chaotic summer, both for me personally as well as for the world. And it’s only July’s first week! I suspect this makes the fact of fall classes starting up in just a few weeks seem daunting and also a bit surreal.
I write today to draw attention to one of the great challenges facing academia during this swiftly onrushing semester. Those classes will take place one year after the generative AI revolution took off. Faculty will be teaching, students studying, and staff supporting while new AI is at its highest powers to date, and when the technology is so widely available as to be commonplace.
What does this mean for higher education? Are we ready to grapple with AI as our students use it throughout their college or university experience?
Today… I think the answer is “generally, no.” Gathering information on higher education teaching practices is famously challenging, but I can find little evidence that post-secondary classes are meaningfully changing to account for the generative AI revolution in students’ hands.
I’m not the only one who thinks this. Other professors have shared similar observations. Ian Bogost wrote a grim warning about the problem in May, focusing on how easy it is for students to cheat in writing assignments, and how badly detection protocols presently are:
Reports from on campus hint that legitimate uses of AI in education may be indistinguishable from unscrupulous ones, and that identifying cheaters—let alone holding them to account—is more or less impossible.
Last week Ethan Mollick analyzed the problem in detail, breaking down a series of class functions which AI is reshaping, from essay writing to reading and problem sets. He dubbed the fall 2023 teaching situation “the Homework Apocalypse.”
In this post I’d like to explore that apocalyptic model. For reasons of space, I’ll leave off analyzing student cheating motivations or questioning the entire edifice of grade-based assessment. I’ll save potential solutions for another post.
Let’s dive into the practical aspects of teaching to see why Mollick and Bogost foresee such a dire semester ahead.
Reading From textbooks to primary source literature and scholarly articles, instructors require students to read. Here AI steps up to reduce the student workload. Users can upload entire texts or, for certain applications, a URL or description of the source, then ask for a summary. That asking can take many forms, from the classic executive summary or Cliff’s Notes to an outline, bullet points, major ideas highlighted, and so on.
In a sense this is not new. Students have avoided, missed, or scrimped on readings for centuries. Faculty have used many techniques to encourage them to read and also to catch them out when they haven’t. Yet now they have customizable shortcuts available. Creative users might find ways to get their summarizes pointed towards their individual career paths or to the instructor and class insofar as they understand them. How can instructors address this, especially if they lack knowledge of the technology and/or don’t have the time to devote to it?
And this is just one part of the problem.
Essay writing and writing in general This is the most popular topic discussed, from what I’ve observed. Academics, journalists, and other observers have correctly noted that the leading chatbots (Bard, Bing, ChatGPT) can quickly produce decent amounts of text to varying degrees of quality. Like some of you, no doubt, I’ve experimented with having these applications generate all kinds of prose (and some verse). This is the simplest version of AI’s challenge to teaching.
I wrote “simplest” because this is really a quick and dirty chatbot use. Like IO9’s recent article debacle, a user can ask AI once and get a single result, then run with it to some degree of potential disaster. This is especially a problem when looking for references or recent topics. People - students - will do this to some degree. Better results come from iteration, by refining one’s initial query, asking the bot to tinker, revise, and improve. Additionally, we can follow what I think of us the “cyborg collaborator” model of exporting the text, then editing and amending it ourselves. (See below for a recent Grammy award cyborg policy.) If readers are interested in my methods, I can follow up with another post; for now, suffice to say that students have a range of possibilities for using AI to churn out writing assignments.
Think about the range of classes this impacts. Writing and composition classes, of course, but also anything requiring extended writing: history, political science, sociology, journalism, communication, and so on. The broader the range of writing across the curriculum, the greater the scope of AI action.
Note, too, the sheer range of writing assignments which can fall under this rubric. Mid-term papers and end of term essays are the clear targets, but take-home exams aren’t immune. In fact, in-class exams when students have access to AI can also see ChatGPT et al in play, depending on how proctoring is established, if an honor code is present (and heeded), and if there are any software or network obstacles.
Digitally-native writing surely falls into this category. Discussion posts, blog posts, content for a wiki/Google Doc - all are capable of receiving texts extruded by AI. Indeed, if there’s writing with any degree of student autonomy, we should expect at least some AI-enabled cheating. And maybe more than “some.”
Now recall what Bogost and others have observed. AI detectors have very high failure rates. That is, all too often they identify human writing as bot emissions and vice versa. The software has other problems, such as expressing bias against non-native language writers. As for humans, well, our ability to correctly determine a paper’s human or digital source will be sorely tested, especially for instructors and support staff who lack training in this, the time to do it well, or just the inclination to do so.
Scientific writing I have been surprised to notice that so many discussions of AI and student cheating focus on essays for humanities classes and related functions (the hoary college essay). Student writing for science classes has gotten relatively scant attention, so let’s focus on that one of CP Snow’s Two Cultures.
Can ChatGPT, Bard, and Bing write lab reports? On the face of it, it seems unlikely, since such assignments depend on students actually doing the work of experimentation, measurement, and assessment. Yet we know that ChatGPT can produce a good amount of scientific prose, albeit with some weaknesses. We can also imagine students using chatbots to create fake yet plausible-sounding experimental results. “Please give me ten temperature readings for the following conditions, with random variations <2 degrees C.” More ambitious, even entrepreneurially minded students could use AI to create lab report templates to help set up the experiments and the final results. In other words generative AI may give students the ability to create portions of their assignments around the actual experimental work.
One limitation to this idea comes from Andy McMillan, who notes that many labs involve two or more people:
[T]hey need to report data that is unique but shared with lab partners there's a check. Interpreting that data requires more specific reference than currently possible. Even without ai assistance a common error is to report what they expect and not what data actually showed so at best it could replicate a poor report. Though i'd say with humanities it's also likely to give a superficial paper as well[…]
This is a good point, as it would require coordination to cheat, which will be an obstacle for some students. Others will take the risk and approach their partners.
How many science faculty are in a position to monitor such behavior? As with our previous point about essay writing, how many instructors will have the understanding, the time, and the inclination to vet each report?
Computer science and related fields AI-generated code is a major issue in the technology world. An open question is, to what extent can code-creating software replace human coders? Already the post-industrial “learn to code” mantra looks shaky.
Computer science classes and others which ask students to write programs are now also susceptible to AI-generated trickery. If we can return to our earlier theme of entrepreneurial thinking, we might imagine students in these classes viewing such uses of AI as professional training, or even producing cheating strategies for their classmates. I would expect some healthy debates about comparing Bard to a code library, for example.
Some computer scientists have been focused on this point for a while. For example, a warning from Radboud University:
So text creation with AI, ah, assistance is really an across the curriculum problem.
What about other kinds of generative AI?
Art and related classes Before ChatGPT leaped onto the world stage in late 2022 there was a set of impressive image creating applications. DALL-E sprung from ChatGPT’s shop. Midjourney took to Discord. A series of other AIs appeared. Shouldn’t we expect students, when asked to create images for an assignment, to consider software as an ally or replacement?
This cheating area might not be so fruitful, since each AI app tends to have a clear style. Additionally, they have well known problems in creating passable hands and text, so we can imagine a quick anti-cheating assignment: “Draw someone holding up a legible sign.” Yet we run into versions of the problems identified earlier. Will all art and image-requiring faculty have the knowledge to detect such fraud? Will they have the time and the attitude to pursue it?
PowerPoint presentations This assignment is enormously widespread, either formally (“Please prepare a slideshow for your presentation”) or implicitly (students giving a presentation and deciding to use .ppt because they’re familiar with it/see it as a job skill they need to have). We can readily envision students using chatbots to produce slide text and image creators to add art. This may, in fact, be allowed to some degree, depending on the assignment and instructor.
But there is actually a subcategory of generative AI app devoted solely to this purpose. Microsoft’s Office tool is now being infused with AI. Further, there’s a small startup, inevitably called Slides.GPT, which generates slide decks on demand. I had it create one last month for a keynote speech about the future of the web. For this Substack I asked it to create a presentation about academic cheating - and its guardrails kicked in to stop me! The site claimed it was a forbidden topic. So I had it make a more generic show about challenges facing faculty this fall, and it did so, albeit without mentioning academic integrity and AI.
As with chatbots, we can imagine students quickly ordering up a slideshow and turning it in without iteration or cyborg collaboration, and probably getting flagged. We can also envision students being more creative. And once more we run into the challenges faculty face in grappling with the problem.
To sum up: generative AI gives students many abilities to behave unethically with college class assignments across a breathtaking range of curricula and task types. I’ve sketched out broad categories and some examples, and I’m sure readers can think of others.
Please note that all of the preceding depends on generative AI as it exists in early July 2023. Things will change over the next six months. Image uploading functions are rumored to appear. New applications to assist students in cheating may well surface to varying degrees of openness. The arms race of generative AI detection is still going on, so we should also expect new applications to appear as well as developments in the present ones. Policy decisions may shape access to generative AI. Business decisions can also alter the forms of AI we can use. Open source and nonprofit instances may well grow.
Where does this AI-enabled cheating scenario leave us?
Fall 2023 classes - which start in a handful of weeks, remember - may see enormous swathes of their assignments in question due to new, AI-driven academic integrity challenges. What proportion of the university class experience will be faked? How much student work will stem from silicon? Potentially, the majority of post-secondary student work will be… questionable.
Where does that leave the idea of a series of class experiences constituting a curricular experience for a given student? To what extent with graduate schools and employers look askance - or even more askance - at graduates? Might teaching enter a crisis akin to the replication problem bedeviling some scientific research? Perhaps worse, will this AI “apocalypse” add another layer to the piling up of skepticism of and outright attacks on higher education?
Facing these possibilities seriously might take some imagination, but I am willing to bet we will need to do so. Confronting them, we can ask: what choices do faculty and institutions have now? How can we redesign our teaching to address them? What can we learn from academics already conducting this work?
I will share some responses in the following posts.
In recent AI, academia, and the future news:
A new article, co-authored by a colleague of mine, arguing for ways of using AI to teach health care. Cooper University’s hospital started using AI to detect abdominal aneurisms.
The National Institutes of Health (NIH) determined that peer reviewers should not use AI in assessing scholarship, because of potential privacy problems. The Grammy music awards will welcome AI-generated content, but won’t recognize software as creators.
Baidu launched an upgraded version of its ERNIE AI.
Cloud computing firm Snowflake partnered with chipmaker Nvidia to allow users to create their own, custom AI.
Science fiction writer and futurist Bruce Sterling considers the various ways new AI appears to be monstrous.
Two authors sued ChatGPT for copyright violations.
Immigrants play an unusually large role in staffing and leading American AI companies, according to a new Axios study.
(thanks to David Staley, Ruben Puentedura, Ceredwyn Alexander, and more for feedback on this post)
Two things, well, maybe three. First, as faculty, we need to explore gen AI and become proficient before attempting to decide what to do about it. My own example - starting early December, I am investing at least an hour a day, every day, in learning about AI and using it. As a result, I am now able to do my grant work in a team of one instead of 3-4 people. I wish I can add a couple more people with similar skills but I cannot find them. Also, I can now tell what AI can and cannot do for you. You need to be able to direct it, and prime it thoughtfully every step of the way in order to get a meaningful outcome. Theoretically, this is what we need to teach our students to do. Second of all, I tried different approaches in Spring 2023 courses to prevent misuse of chat gpt. So far, detailed rubrics with requirements for students to do some tasks that the early versions of GPT could not do worked well. But by Fall, I feel that my chatgpt proofing will not stand; I can bypass them with more specialized AI now. My students will certainly discover it. This brings us to number three. In academia, we should move to competence-r based training and assessment. Like in my MBA capstone, with a semester or two long projects built around a practical case, students are required to jump all the hoops and use whatever assistance they can get but in the end it is their final oral presentation for the entire school is what really matters the most. There are other things. Students that do not use AI will be at a disadvantage. Perhaps, AI should be taught and required as a professional competency.
It's ironic that Slides GPT stopped you from creating a presentation on cheating... that seems to be a misapplied filter (presumably, there are few students presenting on that topic for class!)