Generative AI

Recap: Beyond ChatGPT – New Tools to Augment Your Research

· Nov 10, 2023 ·

by Derek Bruff, visiting associate director

Earlier this week, I had the chance to host another event in the series on generative AI organized by CETL and AIG. Wednesday’s event was titled “Beyond ChatGPT: New Tools to Augment Your Research,” and it featured speakers Marc Watkins, academic innovation fellow and lecturer of writing and rhetoric and someone who seems to always be three steps ahead of me on generative AI, and Kellye Makamson, lecturer in writing and rhetoric and formerly reluctant adopter of AI in teaching. Marc provided some updates on generative AI technologies available to UM instructors, and Kellye shared examples of her use of a particular AI tool (Perplexity) in her writing courses. Below you’ll find a few highlights from the session.

First, Marc’s updates, which you can also find on this Google doc:

Bing Chat Enterprise is now available to UM faculty and staff. This version of Bing Chat runs on GPT-4, which is noticeably more powerful than the earlier version that powers the public version of Bing Chat. It also has an integration with DALL-E for image generation and, since it’s available through UM’s agreement with Microsoft, comes with Microsoft data protection. This means you can use it to analyze data that you shouldn’t share with other tools for privacy reasons. To access Bing Chat Enterprise, visit Bing.com and sign in with your UM credentials.
UM’s Blackboard Ultra now has an “AI design assistant” available to UM instructors. This assistant can quickly build out your course shell with modules and descriptions and images, and it can generate quiz questions and rubrics based on your course content. Anthology is the company that provides Blackboard, and you can read more about the new AI design assistant on their website. Marc said that the
folks at the Faculty Technology Development Center (better known as FTDC) can assist instructors with getting started with this new AI tool.
Microsoft will soon be releasing an AI assistant called CoPilot for their Office 365 programs, which includes Word and Excel and lots more. In the demo video, CoPilot is seen reading meeting notes from a user’s OneNote files along with a Word document about a proposal that needs to be written. Then CoPilot generates a draft proposal based on the meeting notes and the proposal request. It looks like CoPilot will be able to do all kinds of practical (and sometimes boring) tasks! UM hasn’t decided to purchase CoPilot yet, since there’s an additional per-user charge, but Microsoft will be on campus to demo the new AI tool as part of Data Science Day on November 14th.
Just this past Monday, OpenAI, the company behind ChatGPT, made a bunch of announcements, including a cost decrease for ChatGPT Plus (the pro version of the tool that runs the more powerful GPT-4), a new GPT-4 Turbo that runs faster and allows for much more user input, and a way to create your own GPT-powered chat tools, among other things. One of the sample “GPTs” purports to explain board and card games to players of any age! We’ll see about that.

Marc also recommended checking out Claude, a generative AI chatbot that’s comparable to ChatGPT Plus (the one that runs on GPT-4) but (a) free and (b) has a large “context window,” that is, allows greater amounts of user input. You can, for instance, give it a 30,000 word document full of de-identified student feedback and ask it to analyze the feedback for key themes. (Note that Claude is provided by a company called Anthropic, which is a different company from Anthology, the folks that make Blackboard. Don’t be confused like I was.)

After these updated, Kellye took us on a deep dive into her use of the AI tool Perplexity in her writing courses. See her slides, “AI- Assisted Assignments in Student Learning Circles,” for more information, but what follows is my recap.

Kellye attended the AI institute that Marc organized this past summer. She came in hesitant about using AI in her teaching and was a little overwhelmed at variety and power of these technologies at first, but now she is actively experimenting with AI in her courses. She has also become accustomed to feeling overwhelmed at the pace of change in generative AI, and she uses this to have empathy for her students who are also feeling overwhelmed.

Kellye uses a small-group structure in her courses that she calls student learning circles. These are persistent groups of 3-5 students each that meet during class time weekly throughout the semester. She called these class sessions “authority-empty spaces” since she encourages the students to meet around the building without her. She’s available in her office and by email for assistance during these class sessions, but she’s encouraging student autonomy by removing herself from the front of the room.

One of the AI-supported activities in her first-year composition course involves “stress testing” claims. She opens this activity by considering a common claim about learning styles, that each student has a specific way of learning (verbal, visual, and so on) in which they learn better. She’ll ask her students if they know their learning style, and most report being visual learners, with verbal learners in a distant second. Then she’ll ask Perplexity, a generative AI tool like ChatGPT but with better sources, “Are learning styles real?” Of course, there’s piles of research on this topic and all of it refutes the so-called “matching hypothesis” that students learn best when the instructional modality matches their learning style. It becomes clear from Perplexity’s response, replete with footnotes, that the claim about learning styles is questionable.

Then Kellye turns her students loose on a list of claims on various topics: crime rates, campus mental health, pandemic learning loss, and much more. First students work in their groups to analyze a given claim. What do they know about? Do they agree with it? What assumptions are baked into the claim? Then the student groups ask Perplexity about the claims, using whatever prompts they want. Then Kellye provides students with specific prompts for the claim, ones aimed at uncovering the assumptions the claim makes. The students enter these prompts into Perplexity and then analyze the results.

Here’s an example. One of Kellye’s claims reads, “With record crime rates across the nation, cities should invest in robust community programs designed to increase awareness of prevention methods to keep residents safe.” One group of students noted in their pre-Perplexity analysis that there are already a lot of such community programs that don’t seem to be lowering the crime rates, so the claim needs more work around the types of programs to be launched, perhaps matching them with particular kinds of crime. When the group asked Perplexity about the claim, the bot said something similar, noting that such programs need to be targeted to types of crime. But then Kellye provided the group with her prompt: “Are crime rates at an all-time high?” Perplexity quickly provided lots of data indicating that, in fact, crime rates are far lower than they’ve been historically. There was an assumption baked into the claim, that crime rates are at record highs, that neither the students nor Perplexity picked up!

I find this activity fascinating for a couple of reasons. One is that it shows how hard it can be to “stress test” a claim, that students need opportunities to learn how to do this kind of work. The other is that the AI chatbot wasn’t any better than the students in identifying faulty assumptions baked into a claim. Perplexity did a great job backing up its statements with real sources from the internet (that students could follow and analyze for credibility), but it only answered the questions it was given. What you get from the AI depends on what you ask, which means it’s just as important to be asking good questions with an AI as it was without an AI.

It’s possible that other AI tools might be better at this kind of questioning of assumptions. Bing Chat, for instance, will often suggest follow-up questions you might ask after it answers one of your questions. On other hand, I’ve found that the quality of sources that Bing Chat uses are often low. Regardless, I love Kellye’s activity as a way to teach students how to think critically about the claims they encounter and how to think critically about the output of an AI chatbot.

I’ll end with one more note from Kellye. She was asked how her students react to using generative AI in her course. She said that several of her students had a hard time believing her when she said it was okay that they use these tools. They had received clear messages from somewhere (other instructors?) that using generative AI for coursework was forbidden. But once these students started experimenting with Perplexity and other similar tools, they were impressed at how helpful the tools were for improving their understanding and writing. Kellye also noted that when students are working in groups, they’re much more likely to question the output of an AI chatbot than when they’re working individually.

This week’s event was our last in series on AI this fall, but stay tuned for more great conversations on this topic in the spring semester.

CETL in the News – September 2023 Roundup

· Oct 1, 2023 ·

In a recent Inside Higher Ed blog post, John Warner writes that teaching is a wicked problem, that is, a situation where the nature of the problem and the tools for solving it are constantly changing. (This is “wicked” in the sense of tricky, not evil!) Warner argues that tackling this wicked problem requires a different kind of educational research than what is typically valued in higher ed: qualitative research. “In short,” Warner writes, “we gotta go qualitative over quantitative in a big way. As a wicked problem, creating valid quantitative studies related to instruction often requires either ignoring or sanding away many of the complexities that inevitably exist in teaching.”

As an example of the kind of qualitative research he’s calling for, John Warner cites Unmaking the Grade, the newsletter written by Emily Donahoe, CETL associate director of instructional support. Emily has been using this platform to chronicle her experiments with ungrading in her courses. Warner appreciates the nuance Emily brings to her newsletter: “Read entry to entry, the experiment takes on a narrative form, which not only makes for more compelling reading but also provides a lens for Donahoe to reflect on what’s happening in her class. We see the layers of complexity at play in the teaching experiment.”

If you haven’t been reading Emily’s newsletter, you can read all of her posts at Unmaking the Grade.

Meanwhile, CETL visiting associate director Derek Bruff continues to make the rounds on podcasts talking about generative AI and its impact on teaching and learning this fall. His latest appearance is on the Limed: Teaching with a Twist podcast from Elon University’s Center for Engaged Learning. Host Matt Wittstein interviewed Elon strategic communications professor Jessica Gisclair about her goals for teaching with and about AI this fall, then talked with a panel of students and faculty, including Derek, about possible approaches for meeting those goals. You can listen to the entire conversation here, or search for “Limed: Teaching with a Twist” in your favorite podcast app.

Recap: Teaching in the Age of AI (What’s Working, What’s Not)

· Sep 21, 2023 ·

by Derek Bruff, visiting associate director

Earlier this week, CETL and AIG hosted a discussion among UM faculty and other instructors about teaching and AI this fall semester. We wanted to know what was working when it came to policies and assignments that responded to generative AI technologies like ChatGPT, Google Bard, Midjourney, DALL-E, and more. We were also interested in hearing what wasn’t working, as well as questions and concerns that the university community had about teaching and AI.

We started the session with a Zoom poll asking participants what kinds of AI policies they had this fall in their courses. There were very few “red lights,” that is, instructors who prohibited generative AI in their courses and assignments. There were far more “yellow lights” who permitted AI use with some limitations. We had some “green lights” in the room, who were all in on AI, and a fair number of “flashing lights” who were still figuring out their AI policies!

Robert Cummings and I had invited a few faculty to serve as lead discussants at the event. Here is some of what they said about their approaches to AI this fall:

Guy Krueger, senior lecture in writing and rhetoric, described himself as a “green light.” He encourages his writing students to use AI text generators like ChatGPT in their writing, perhaps as ways to get started on a piece of writing, ways to get additional ideas and perspectives on a topic, or tools for polishing their writing. Guy said that he’s most interested in the rhetorical choices that his students are making, and that the use of AI, along with some reflection by his students, generated good conversations about those rhetorical choices. He mentioned that students who receive feedback on their writing from an instructor often feel obligated to follow that feedback, but they’re more likely to say “no, thanks” to feedback from a chatbot, leading to perhaps more intentional decisions as writers. (I thought that was an interesting insight!)
Deborah Mower, associate professor of philosophy, said she was a “red light.” She said that with generative AI changing so rapidly, she didn’t feel the time was right to guide students effectively in their use or non-use of these tools. She’s also teaching a graduate level course in her department this fall, and she feels that they need to attend to traditional methods of research and writing in their discipline. Next semester, she’ll be teaching an undergraduate course with semester-long, scaffolded projects on topics in which her students are invested, and she’s planning in that course to have them use some AI tools here and there, perhaps by writing something without AI first and then revising that piece with AI input. (It sounds like she’s planning the kinds of authentic assignments that will minimize students’ use of AI to shortcut their own learning.)
Melody Xiong, instructor of computer and information science, is a “yellow” light in the 100-, 200-, and 400-level courses she’s teaching this fall. She teaches computer programming, and while she doesn’t want students to just copy AI output for their assignments, she is okay with students using AI tools as aids in writing code, much like they would have used other aids before the advent of AI code generators. She and her TAs offer a lot of office hours and her department provides free tutoring, which she hopes reduces the incentive for her students to shortcut the task of learning programming. She also knows that many of her students will face job interviews that have “pop quiz” style coding challenges, so her students know these are skills they need to develop on their own. (External assessments can be a useful forcing function.)
Brian Young, scholarly communication librarian, and also described himself as a “green light” on AI. In his digital media studies courses, generative AI is on the syllabus. One of his goals is to develop his students’ AI literacy, and he does that through assignments that lead students through explorations and critiques of AI. He and his students have talked about the copyright issues with how AI tools are trained, as well as the “cascading biases” that can occur when the biases in the training data (e.g. gender biases in Wikipedia) then show up in the output of AI. One provocative move he made was to have an AI tool write a set of low-stakes reading response questions for his students to answer on Blackboard. When he revealed to his students that he didn’t write those questions himself, that launched a healthy conversation about AI and intellectual labor. (That’s a beautiful way to create a time for telling!)

Following the lead discussants, we opened the conversation to other participants, focusing on what’s working, what’s not, and what questions participants had. What follows are some of my takeaways from that larger conversation.

One faculty participant told the story of a colleague in real estate who is using generative AI to help write real estate listing. This colleague reports saving eight hours a week this way, freeing up time for more direct work with clients. This is the kind of AI use we’re starting to see in a variety of professions, and it has implications for the concepts and skills we teach our students. We might also find that generative AI can save us hours a week in our jobs with some clever prompting!

Several faculty mentioned that students are aware of generative AI technologies like ChatGPT and that many are interested in learning to use them appropriately, often with an eye on that job market when they graduate. Faculty also indicated that many students haven’t really used generative AI technologies to any great extent, so they have a lot to learn about these tools. One counterpoint: Deborah Mower, the “red light,” said that her graduate students have been content not to use AI in their course with her.

Course policies about AI use vary widely across the campus, which makes for a challenging learning landscape for students. I gather that some departments have leanings one way or another (toward red or green lights), but most course policies are determined by individual instructors. This is a point to emphasize to students, that different courses have different policies, because students might assume there’s a blanket policy when there is not.

This inconsistency in policy has led some students to have a fair amount of anxiety about being accused of cheating with AI. As AIG’s Academic Innovation Fellow Marc Watkins keeps reminding us, these generative AI tools are showing up everywhere, including Google Docs and soon Microsoft Word.

Other students have pushed back on “green light” course policies, arguing that they already have solid processes for writing and inserting AI tools into those processes is disruptive. I suspect that’s coming from more advanced students, but it’s an interesting response. And one that I can relate to… I didn’t use any AI to write this blog post, for instance.

A few participants mentioned the challenge of AI use in discussion forum posts. “Responses seemed odd,” one instructor wrote. They mentioned a post that clearly featured the student’s own ideas, but not in the student’s voice. Other instructors described posts that seemed entirely generated by AI without any editing by the student. From previous conversations with faculty, I know that asynchronous online courses, which tend to lean heavily on discussion forums, are particularly challenging to teach in the current environment.

That last point about discussion posts led to many questions from instructors: Where is the line between what’s acceptable and what’s not in terms of academic integrity? How is it possible to determine what’s human-produced and what’s AI-produced? How do you motivate or support students in editing what they get from an AI tool in useful ways? How can you help student develop better discernment for quality writing?

One participant took our conversation in a different direction, noting the ecological impact of the computing power required by AI tools, as well as the ethical issues with the training data gathered from the internet by the developers of AI tools. These are significant issues, and I’m thankful they were brought up during our conversation. To learn about these issues and perhaps explore them with your students, “The Elements of AI Ethics” by communication theorist Per Axbom looks like a good place to start.

Thanks to all who participated in our Zoom conversation earlier this week. We still have a lot of unanswered questions, but I think the conversation provided some potential answers and helped shape those questions usefully. If you teach at UM and would like to talk with someone from CETL about the use of AI in your courses, please reach out. We’re also organizing a student panel on AI and learning on October 10th. You can learn more about this event and register for it here. And if you’d like to sign up for Auburn University’s “Teaching with AI” course, you can request a slot here.

Update: I emailed Guy Krueger, one of our lead discussants, and asked him to expand on his point about students who have trouble starting a piece of writing. His response was instructive, and I received his permission to share it here.

I mentioned that I used to tell students that they don’t need to start with the introduction when writing, that I often start with body paragraphs and they can do the same to get going. And I might still mention that depending on the student; however, since we have been using AI, I have had several students prompt the AI to write an introduction or a few sentences just to give them something to look at beyond the blank screen. Sometimes they keep all or part of what the AI gives them; sometimes they don’t like it and start to re-work it, in effect beginning to write their own introductions.

I try to use a process that ensures students have plenty of material to begin drafting when we get to that stage, but every class seems to have a student or two who say they have real writer’s block. AI has definitely been a valuable tool in breaking through that. Plus, some students who don’t normally have problems getting started still like having some options and even seeing something they don’t like to help point them in a certain direction.

CETL in the News – August 2023 Roundup

· Sep 1, 2023 ·

Generative AI technologies like ChatGPT and Midjourney are posing new challenges (and maybe opportunities) for higher education this fall. The University of Mississippi is ahead of that curve thanks to pre-ChatGPT explorations of AI technologies by faculty in Writing & Rhetoric and elsewhere. As a result, CETL staff have useful things to say about teaching with and without AI, and that’s the focus of this month’s CETL news roundup.

Just this week, Robert Cummings, executive director of academic innovation, was interviewed by Biloxi, Mississippi, television station WLOX about AI’s impact on higher education this fall. You can watch his four-minute interview here.

Back in July, CETL visiting associate director Derek Bruff was interviewed about teaching and AI, as well. Derek was featured in Jeff Young’s EdSurge piece “Instructors Rush to Do ‘Assignment Makeovers’ to Respond to ChatGPT” about ways faculty are updating assignments for the current AI landscape. Derek was also interviewed by Lauren Coffee for her Inside Higher Ed report “Professors Craft Courses on ChatGPT with ChatGPT,” which looked at new courses on the books about generative AI this fall.

More recently, Derek appeared on the popular Teaching in Higher Ed podcast hosted by Bonni Stachowiak in Episode 481, “Assignment Makeovers in the AI Age.” Bonni has been producing her podcast weekly for almost ten years, and it’s a fantastic resource for the higher education community. If you’d like to listen to it with colleagues, you might try CETL’s new Podcasts & Puzzles get-togethers!

Policies and Practices for Generative AI in Fall Courses

· Aug 15, 2023 ·

by Derek Bruff, visiting associate director

Last Friday, CETL co-sponsored an online workshop titled “Generative AI on the Syllabus” with our parent organization, the Academic Innovations Group (AIG). Bob Cummings, executive director of AIG, and I spent an hour with 170 faculty, staff, and graduate students exploring options for banning, embracing, and exploring generative AI tools like ChatGPT, Bing Chat, and Google Bard in our fall courses.

After starting the session providing a brief overview of the current landscape of generative AI tools, I asked participants to provide a few words describing how they were feeling about generative AI and its impact on teaching and learning. The resulting word cloud (seen below) is full of words like excited, curious, and intrigued, but also apprehensive, concerned, and overwhelmed. It was clear to me that the instructors present on Friday hadn’t completely figured out their approach to generative AI for the fall.

Bob and I then shared the CETL Syllabus Template, a document that has suggested syllabus language, information about UM policies, and links to course design resources. My CETL colleague Emily Donahoe led a small team of us this summer in developing a new section of that template focused on generative AI. If you’re still working on your AI policy for the fall, I high recommend opening that document and scrolling to page 9 for some thoughtful options to consider. (Please note: We have released AI section of the Syllabus Template under a Creative Commons license so that those outside of the University of Mississippi can use and adapt it as they like.)

For example, if you’re leaning toward prohibiting the use of AI text generators in your course, you might use the suggested syllabus language under the heading “Use of Generative AI Not Permitted”:

Generative AI refers to artificial intelligence technologies, like those used for ChatGPT or Midjourney, that can draw on a large corpus of training data to create new written, visual, or audio content. In this course, we’ll be developing skills that are important to practice on your own. Because use of generative AI may inhibit the development of those skills, I ask that you refrain from employing AI tools in this course. Using such tools for any purposes, or attempting to pass off AI-generated work as your own, will violate our academic integrity policy. I treat potential academic integrity violations by […]

If you’re unsure about whether or not a specific tool makes use of AI or is permitted for use on assignments in this course, please contact me.

You’ll need to fill in those ellipses with your own words, but this is a good start on a conversation with students about use of generative AI in their learning process in your course.

If you’re more open to student use of generative AI tools in your course, then read the section titled “Use of Generative AI Permitted (with or without limitations).” That section offers language to talk about the ways AI tools can support or hinder learning, appropriate uses of generative AI tools in your course, and options for disclosing one’s use of an AI tool on an assignment (e.g. the APA’s newly issued recommendations). That section also lists a number of AI tools as a reminder that we’re not just talking about ChatGPT here.

The recommendation section of the syllabus template expands on that idea:

As you craft your policy, please keep in mind that students may encounter generative AI in a variety of programs: chatbots like ChatGPT; image generators like DALL-E or Midjourney; writing and research assistants like Wordtune, Elicit, or Grammarly; and eventually word processing applications like Google Docs or Microsoft Word. Consider incorporating flexibility into your guidelines to account for this range of tools and for rapid, ongoing developments in AI technologies.

There’s also a caution about the use of AI detection tools:

Please be aware, too, that AI detection tools are unreliable, and use of AI detection software, which is not FERPA-protected, may violate students’ privacy or intellectual property rights. Because student use of generative AI may be unprovable, we recommend that instructors take a proactive rather than reactive approach to potential academic dishonesty.

After discussing the CETL Syllabus Template and its new language about AI, I shared a few ideas for revising assignments this fall in light of the presence of generative AI tools. I walked through an “assignment makeover” for an old essay assignment, a makeover that I detailed on my blog Agile Learning last month. In that post, I suggest six questions to consider as you rethink your assignments for the fall:

Why does this assignment make sense for this course?
What are specific learning objectives for this assignment?
How might students use AI tools while working on this assignment?
How might AI undercut the goals of this assignment? How could you mitigate this?
How might AI enhance the assignment? Where would students need help figuring that out?
Focus on the process. How could you make the assignment more meaningful for students or support them more in the work?

There were two big questions that emerged from the Q&A portion of the workshop. One, is there any practical way to determine if a piece of student work was ghostwritten by ChatGPT? Answer: No, not really. All the AI detectors are unreliable to one degree or another. Two, how might we teach students about the limitations of AI tools, like the fact that they output things that are not true? Answer: One approach is to have students work with these tools and critique their outputs, like these divinity school faculty did in the spring. (I think that example is my favorite pedagogical use of ChatGPT that I’ve encountered thus far.)

What’s next for this topic? Bob and I shared a few possibilities for UM instructors:

Auburn University has opened its online, asynchronous “Teaching with AI” course to faculty across the SEC, which includes UM faculty. This course is a time commitment (maybe 10 to 15 hours), but it’s full of examples of assignments that have been redesigned for an age of AI. Reach out to Bob Cummings if you’re interested in taking this course.
Marc Watkins, lecturer in writing and rhetoric and now also an academic innovation fellow at AIG, has also built a course available to UM instructors. It’s called “Introduction to Generative AI,” and it offers a lot for faculty, staff, and students interested in building their AI literacy, something Marc recommended in his recent Washington Post interview. To gain access, just contact me.
This past summer, the UM Department of Writing & Rhetoric offered an AI summer institute for teachers of writing. At some point this fall, they’re planning to offer the institute again for UM faculty, but with a broader scope. Keep an eye out for announcements about this second offering.
Here at CETL, our staff of teaching consultants are available to talk with UM instructors about course and assignment design that integrates or mitigates the use of generative AI. You’re welcome to contact me or any of our staff with questions.
Finally, we’re planning two more events in this CETL/AIG series on teaching and AI, both on Zoom later this semester. “Teaching in the Age of AI: What’s Working, What’s Not” is scheduled for September 18th, and “Generative AI in the Classroom: The Student Perspective” is scheduled for October 10th. See our Events page for details and registration.