Effective Altruism Forum
EA Forum

Hide table of contents
Comment Permalink
Zach Stein-Perlman6
4
0

I think this type of misuse is an emerging AI alignment problem.

Misuse can be important or interesting, but the word “alignment” should be reserved for problems like the problem of making systems try to do what their operators want, especially making very capable systems not kill everyone.

Reply
See in context
by Marshall
4 min read 2

8

AI safetyEducationLarge Language Models
Frontpage

Summary

  • I'm concerned about a lack of discussion on this forum about the use of AI to cheat on academic writing. I think this type of misuse is an emerging AI alignment problem. 
  • This is adding to the substantial challenges faced by teachers. At least in the US, they are already disrespected, poorly paid, and "leaving the profession in droves." 
  • I provide 7 tips for teachers to optimize their essay prompts to help them recognize ChatGPT output.
  • I suggest that AI companies should provide free tools to teachers to help them detect and counteract cheating.

Notes: Crossposted here. I've added an EA-specific summary for the forum. This is not an attempt to synthesize AI-enabled cheating as a cause area. I'd file this more under "some advice for dealing with this new problem and some suggestions for AI companies." 

Introduction

Like so many people, I’ve been surprised and baffled by ChatGPT. It generates coherent responses to a huge array of prompts. The technology threatens the written assignment as a means of developing and assessing the type of “21st century” skills that are (at least for now) essential in so many jobs. 

The thing is this: ChatGPT has some glaring weaknesses. It produces a lot of material that is just flat out wrong. It even (very helpfully) fabricates citations and quotes. These telltale errors and fabrications are easy to see because you, the teacher, are knowledgeable about the essay topic. The key is to make your job easier by getting ChatGPT to throw as many errors as possible. That way, if students do try to use this AI to write their essays, they need to spend so much time researching and fixing the errors to avoid getting caught that it’s not even worth the trouble.

ChatGPT readily admits to fabricating a citation…and rhapsodizes about academic integrity. Source: ChatGPT from OpenAI.

Seven ways to optimize for nonsense ChatGPT output

Here are seven strategies that I’ve found useful for getting the best kind of garbage out of ChatGPT. I recommend implementing all seven together. Rather than explain each in detail, I provide some examples here.

  1. Ask about recent developments – ChatGPT usually refuses to opine on anything that isn’t covered in its training data, which runs through 2021. If you phrase the question correctly, it may still answer but will refer to older developments. For example, when asked about medications recently approved by the FDA, it references approvals from the 2010s.
  2. Request multiple citations – Many (though not all) of the citations generated by ChatGPT are complete fabrications, even though they look real. If you search for the sources or follow the links, you’ll usually find nothing. This is telltale.
  3. Ask for quotes from the citations – Similarly, ChatGPT will generate what appear to be quotes from sources when the quotes actually don’t exist in the real world. Even if you ask for “direct quotes” it will do this! Another dead giveaway.
  4. Tell it to take a stand – ChatGPT will often refuse to take a stand, it will hedge its answers, or it will make very general statements. For example, with health-related information, it usually throws in a very Mayo Clinic-esque disclaimer about consulting a health professional. How many students do that?
  5. Request specific information – The more detailed the response from ChatGPT, the more likely it is to be rife with errors. You’ll recognize many of these errors right away.
  6. Ask it to compare and contrast – This is a version of getting specific, which reveals more factual errors. But it also gives ChatGPT the opportunity to make more general statements that betray a lack of real understanding of the question.
  7. Test your prompt in ChatGPT – I’m hesitant to recommend this, because it gives OpenAI more training data, but you want to be sure the responses you get back are awful. Use this as an opportunity to optimize your prompt to maximize the AI’s errors.

It’s not a coincidence that many of these tips for writing prompts seem like good practices in the pre-AI era. Remember that ChatGPT has mastered an illusion: we’re impressed by the legibility and coherence of its writing and we assume this means that it really understands what it is talking about, the way a human student can.

What OpenAI (and other AI behemoths) should do

I’m worried that companies will recognize these faults and try to “fix” them. Let’s face it: this technology takes the job of teachers – who are already disrespected, underpaid, and under enormous strain – and makes it worse. This is a clear and present AI alignment problem. I think these companies should be very hesitant about making AI an even better cheating machine without also giving teachers tools to address new problems caused by AI technologies. Here’s one suggestion:

  • create a tool that checks an essay response for similarity to historical ChatGPT output. 
  • ensure this tool is easy to use and commit to making it free to teachers as long as ChatGPT (or its successors) can conceivably be used for essay cheating.

I’m sure there are logistical and privacy concerns. I’m also sure that these companies can figure out how to surmount these concerns or develop alternative tools for teachers – they certainly have the resources.

Some notes and disclaimers

  • This is current as of mid-to-late December of 2022. It seems possible (even likely) that OpenAI will further improve ChatGPT to enable “higher quality” cheating, so be sure to follow step 7 and field test any essay prompt in the software itself.
  • This is most applicable to science-related prompts for advanced secondary and tertiary-level students. I am sure some of the same ChatGPT weaknesses can be exploited in other fields and for students at other levels. I would welcome additional tips and adaptations to these tips.
  • This should not be taken as a defense of thoughtless regurgitation of facts as a pedagogical approach. That said, facts are important in science, and the types of errors that ChatGPT makes betray its lack of understanding of how facts relate to each other in complex systems. We want students to be able to use careful research and writing to develop that understanding.

8

0
0

Reactions

0
0

More posts like this

Comments2


Sorted by
New & upvoted
Click to highlight new comments since:
Zach Stein-Perlman6
4
0

I think this type of misuse is an emerging AI alignment problem.

Misuse can be important or interesting, but the word “alignment” should be reserved for problems like the problem of making systems try to do what their operators want, especially making very capable systems not kill everyone.

Reply
david_reinstein2
0
0

This is helpful at the margin

But my experience over 15 years of teaching at UK universities is that “marking student research/take home essays ” was already a highly problematic and inaccurate means of evaluation. Extremely time consuming to judge, especially with students who struggle with the English language. (And you are asked to not penalise them for this). Students can and do pay others to write it for them.

the availability of chat Gpt makes this substantially more difficult.

My advice (to the educational system) would be a combination of:

  1. Try to work with a smaller number of intrinsically motivated students. Focus less on evaluation. This is particularly true for research intensive subjects.

  2. If you need to do evaluation, arrange proctored/Invigilated exams

Reply
More from Marshall
65
New cause area: training health workers to prevent newborn deaths
Marshall
· · 17m read
4
4
55
Requesting feedback: proposal for a new nonprofit with potential to become highly effective
Marshall
· · 9m read
9
9
79
HealthLearn: Impact Evaluation and Next Steps
2 authors
· · 12m read
1
1
View more
Curated and popular this week
Paul Present
 ·  · 28m read
 · 
Note: I am not a malaria expert. This is my best-faith attempt at answering a question that was bothering me, but this field is a large and complex field, and I’ve almost certainly misunderstood something somewhere along the way. Summary While the world made incredible progress in reducing malaria cases from 2000 to 2015, the past 10 years have seen malaria cases stop declining and start rising. I investigated potential reasons behind this increase through reading the existing literature and looking at publicly available data, and I identified three key factors explaining the rise: 1. Population Growth: Africa's population has increased by approximately 75% since 2000. This alone explains most of the increase in absolute case numbers, while cases per capita have remained relatively flat since 2015. 2. Stagnant Funding: After rapid growth starting in 2000, funding for malaria prevention plateaued around 2010. 3. Insecticide Resistance: Mosquitoes have become increasingly resistant to the insecticides used in bednets over the past 20 years. This has made older models of bednets less effective, although they still have some effect. Newer models of bednets developed in response to insecticide resistance are more effective but still not widely deployed.  I very crudely estimate that without any of these factors, there would be 55% fewer malaria cases in the world than what we see today. I think all three of these factors are roughly equally important in explaining the difference.  Alternative explanations like removal of PFAS, climate change, or invasive mosquito species don't appear to be major contributors.  Overall this investigation made me more convinced that bednets are an effective global health intervention.  Introduction In 2015, malaria rates were down, and EAs were celebrating. Giving What We Can posted this incredible gif showing the decrease in malaria cases across Africa since 2000: Giving What We Can said that > The reduction in malaria has be
LintzA
 ·  · 15m read
 · 
Cross-posted to Lesswrong Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: 1. Updates toward short timelines 2. The Trump presidency 3. The o1 (inference-time compute scaling) paradigm 4. Deepseek 5. Stargate/AI datacenter spending 6. Increased internal deployment 7. Absence of AI x-risk/safety considerations in mainstream AI discourse Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on. In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!  Implications of recent developments Updates toward short timelines There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months). We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achi
Rory Fenton
 ·  · 6m read
 · 
Cross-posted from my blog. Contrary to my carefully crafted brand as a weak nerd, I go to a local CrossFit gym a few times a week. Every year, the gym raises funds for a scholarship for teens from lower-income families to attend their summer camp program. I don’t know how many Crossfit-interested low-income teens there are in my small town, but I’ll guess there are perhaps 2 of them who would benefit from the scholarship. After all, CrossFit is pretty niche, and the town is small. Helping youngsters get swole in the Pacific Northwest is not exactly as cost-effective as preventing malaria in Malawi. But I notice I feel drawn to supporting the scholarship anyway. Every time it pops in my head I think, “My money could fully solve this problem”. The camp only costs a few hundred dollars per kid and if there are just 2 kids who need support, I could give $500 and there would no longer be teenagers in my town who want to go to a CrossFit summer camp but can’t. Thanks to me, the hero, this problem would be entirely solved. 100%. That is not how most nonprofit work feels to me. You are only ever making small dents in important problems I want to work on big problems. Global poverty. Malaria. Everyone not suddenly dying. But if I’m honest, what I really want is to solve those problems. Me, personally, solve them. This is a continued source of frustration and sadness because I absolutely cannot solve those problems. Consider what else my $500 CrossFit scholarship might do: * I want to save lives, and USAID suddenly stops giving $7 billion a year to PEPFAR. So I give $500 to the Rapid Response Fund. My donation solves 0.000001% of the problem and I feel like I have failed. * I want to solve climate change, and getting to net zero will require stopping or removing emissions of 1,500 billion tons of carbon dioxide. I give $500 to a policy nonprofit that reduces emissions, in expectation, by 50 tons. My donation solves 0.000000003% of the problem and I feel like I have f
Recent opportunities in AI safety
23
Last week to apply for the Futurekind AI Fellowship! (deadline: April 1)
Jay Luong
· · 1m read
0
0
27
Out of This Box: The Last Musical (Written by Humans) - Crowdfunding!
GuyP
· · 7m read
0
0
39
Australians for AI Safety Launches New Election Campaign — Here's How You Can Help
Luke Freeman 🔸
· · 3m read
5
5
View more