4.1: How do we tell uses of AI that help learning from ones that hurt?

Everyone knows chatbots can be used to cheat. They can also be used to support learning. Sometimes it’s easy to tell the difference: if AI does all the work and and we do nothing, we don’t learn. If we get AI to make up hard questions, and we have to come up with answers, we do learn. But often it’s not that clear. 

There are so many forms of AI assistance. In her Vox.com video AI can do your homework. Now what? Students and teachers grapple with the rise of the chatbots, journalist Joss Fong gives us a taste of myriad possibilities:

• Answers to a homework question. 
• Background information on a topic. 
• Definitions or explanations of a concept. 
• Sources to find more information. 
• Summaries of readings and lectures. 
• Study guides for an exam. 
• Ideas for how to respond to an assignment. 
• Instructions for solving a problem. 
• An outline for a paper or presentation. 
• Examples, analogies, and counterarguments. 
• A draft of a paper or a discussion post. 
• A script for a presentation. 
• Feedback on their work. 
• A revision of a text to improve it. 
• A revision of a text to change its word count

So what’s a good idea, and what’s not? Here, I want to suggest one principle and one rule of thumb that could guide us in the maze of options.

A principle: focus on how AI use affects your thinking

As Fong points out in the video, it can feel like the point of school is to produce work that meets criteria. Really, of course, the point is what happens in our brains when we do the work. When I develop guidelines for AI use, I try to judge how each possible application might help or hurt the learning goals of the assignment. I’m more interested in mental processes than in abstract ideas about what constitutes cheating. 

Self-monitoring learning

Figuring out whether a particular use of AI helps us learn often requires a lot of self-awareness. When it’s not obvious whether it’s helping more than it’s hurting, you’ll need to act like something of a scientist: experiment, reflect, and adjust your practices. Ask yourself,

• Did using AI in that way help me get through a stuck place? 

• What other strategies could I have tried? 

• Did the AI use make me feel more or less engaged? 

• What could I do again or do differently next time?

Of course, your teacher may be able to guide you. (See How do I know which use of AI is allowed and which isn’t?)  But they may not have thought of every use, and they may leave some decisions to you. 

The value of friction

The tricky thing about monitoring our learning is that we tend to underestimate how much we’re learning when we’re frustrated. Joss Fong discusses a study by Deslaurier et al. where students who listened to a polished lecture rated their learning high but actually did worse on the test. Students who engaged in active learning with the same teacher were frustrated and ranked their learning low but did better on the test. 

As Fong observes, “Whenever we try and judge if a learning experience is productive or not, the strongest metacognitive cue that we use is perception of fluency. Fluency is when information is going down easy. It’s well presented, it’s organized, it’s convenient.” But, as she explains, “Education researchers have this term ‘desirable difficulties,’ which describes this kind of effortful participation that really works but also kind of hurts. And the risk with AI is that we might not preserve that effort, especially because we already tend to misinterpret a little bit of struggling as a signal that we’re not learning.”

So it’s worth remembering that a frustrating path might be more fruitful than an easier AI-assisted path. Fong suggests using AI in ways that allow us to do harder work. Rather than using “a chatbot to avoid reading a challenging text… you could use it to work through that text and help you get more out of it.“ We don’t need to be perfect at this; we just need to pay attention and keep trying. Gradually, we’ll get better at self-awareness and strategy.

A rule of thumb: use AI for tutoring-style assistance

To reduce the complexity of these decisions, it helps to have a rule of thumb to turn to. If you’re considering a specific use of AI, ask yourself, would an ethical tutor agree to do this?

Generally speaking, tutors expect that student work should be student ideas and student words. They don’t complete part of the assignment, but they do give feedback, examples, and explanations. Tutors get training to help them make the call on gray areas: is it okay to suggest a new outline for your essay or should they ask you questions to help you come up with one on your own? 

When you go to a tutor, they decide what’s helping and what’s overhelping. With a chatbot, you have the burden and freedom of drawing that line. You might not know exactly where to draw it, but you probably have some sense of what a human tutor will and won’t do. If you can’t see a human tutor rewriting your sentences to make them sound sophisticated, it’s probably not a good idea to ask AI to rewrite them.

For more guidance on using AI for tutor-style alternate explanations, examples, quiz questions, or feedback, see AI for Tutoring-Style Assistance. (And remember, this tutor is unreliable and possibly biased.)

AI for task completion? Not unless the teacher has designed for it

What about using AI to complete part or all of an assignment? For example, what about asking AI to write a first draft? Some argue that since we’ll likely use AI in future workplaces, we should use it to do homework. However, as we’ve noted, the purpose of schoolwork is to change our brains. (There’s no other great reason to add to the pile of completed assignments in the world.) The purpose of work is to get things done for someone who pays us. 

Some teachers design assignments to invite AI collaboration. They identify learning goals students can achieve by prompting AI and responding to what it gives. Then they invite students to use AI to complete some part of the assignment. Students check, revise, and add their own words and ideas. Ultimately, they are responsible for the quality of the result. 

If your teacher has not explicitly designed for this, however, then using AI to actually do what the teacher assigned will likely get in the way of learning. Ask yourself, 

• What is the assignment supposed to teach?
• What might I learn if I do what I was going to ask the AI to do? 

It’s safer to stick with tutoring-style assistance.

Further reading

• AI can do your homework. Now what? Students and teachers grapple with the rise of the chatbots, produced by Joss Fong for Vox.com
• Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom by Louis Deslauriers, Logan S. McCarty, Kelly Miller, and Greg Kestin
• AI Assessment Scale by Mike Perkins, Leon Furze, Jasper Roe, and Jason MacVaugh
• Student Use Cases for AI by Ethan Mollick and Lilach Mollick
• AI Study Buddy from Northwestern University
• Principles for Using AI in the Workplace and Classroom by Joel Gladd

Discussion