In 2024, I started writing a policy report to respond to a neglected educational issue for a class project. Reliance on AI use and loss of critical thinking became my main concerns to learn and write about. Though I finished the class with a good grade, it was still just a Class Paper, which didn't deliver any direct impact to any community.
After speaking with 30+ encouraging and impact-directed minds in two EAG for my project, I learned so much more about new approaches to making impactful projects.
While rebrushing this experience of conducting a policy report on AI risks in higher education as a master's student for the application to MATS, many advises received from impact-driven seniors made me realize there were many ways I could have done differently to advance the impact of the project.
Thus, I am sharing the postmortem to potentially help any students who are finishing a policy report to organize their papers more effectively and strategically to push for impact beyond grades and completion.
Summary & Main Takeaways
- Positive thinking & Belief in the larger community
- When seeing a novel social issue, but no one around you is interested in the issue, do not deny your interests and your own potential. Do not turn into a negative self-fulfilling prophecy.
- Finding the right mentor:
- When seeing a novel social issue, find seniors who are interested in the topic instead of over-consuming time on persuading uninterested seniors.
- Prepare some artifacts (proposal, preliminary drafts, theory of change) when you approach the seniors. You gain more feedback on the project materially.
- Artifact production
- Publishing at any level is helpful to advance your ideas and gain feedback to iterate.
Full Story
The Context & Process
Selecting Topics
At the end of 2024, after long hours of using Generative AI (GAI) for understanding papers and improving arguments for classes, I started feeling that I was relying on GAI to make decisions. I then started raising questions: is GAI use causing reliance? Is GAI causing loss of critical thinking? What are the feasible ways to detect the issue and solve the problem? How are the teachers in schools reacting to this problem?
Meanwhile, I was requested to draft a policy report responding to an urgent education issue and recommending policy solutions to senators by analyzing the existing federal policies during the class Federal Policy Institute. I briefly brainstormed about some global-level educational pandemic: mental health issues, absenteeism, phone use, and maybe the risks of AI.
My personal experience raised questions about the risks of AI. Though the adoption was only clustered among the workforce and some higher education institutions, I thought the unstoppable trend of AI growth would ultimately penetrate schools and families. Thus, I decided to dive into ‘the risks of AI' as the subject of policy analysis.
Process of Completion
Spending the first week, I reviewed some literature and experimental results on the uses and risks of AI in the classroom, such as the piece in Nature Human Behaviour and the piece on PNAS. Though most evidence published then was still preliminary and contested, the evidence showed GAI use poses high risks to students, resulting from the misuse of GAI in the classroom (loss of critical thinking, reliance, etc.）.
After confirming the risks, I kept searching for the existing policies implemented in the classroom. Within Higher Education Institutions (HEI), I searched for the existing regulations on AI use in HEI. McDonald and coauthors (2025) showed that only 41% of 116 R1 schools implemented guardrails on AI use in the classroom.
Then I spent a few weeks (20+ per week) searching through the initiated and passing bills through Congress. I saw fewer proposed bills on AI safety and AI risks. All these statistics altered me: the policymakers were not paying attention to the risks of AI in the classroom and were not paying attention to this rising crisis. Without the proper regulations, the misuse of AI would only exacerbate the declining learning outcomes in the classrooms.
At the end of the policy report, I finished my analysis and policy recommendations.
Beyond the Project
Beyond the project, the unawareness of policymakers and academic groups on AI risks surprised me. This pushed me to think that the AI risks need to be better understood to prevent greater harm. I thought about advocating for the issue, but I denied my thought because I am an international student. I thought about publishing this as an OpEd, but I ducked for not knowing where to do so, and didn't think there was enough evidence showing the direct negative impact.
Finally, I thought about producing more rigorous evidence on the impact of AI tools in the classrooms. This was the route that I went down.
I thought that understanding the students' use was the first step of setting guardrails. Therefore, I wanted to have an assessment tool to analyze the conversations students hold with GenAI. To fulfill this goal, I learned about 'quantitative ethnography.' But it ended up not relevant to my initial idea, so I ended up exploring ways to assess the educational conversations.
Reflection
What was wrong? What would I do differently?
Looking back, I think I’ve achieved the goal within the project: answered the set of questions that I began with, and analyzed the existing regulations and policies on higher education and the Congress.
Beyond finishing the project, I think I could have made a real impact by perfecting the project and making it a real policy report instead of an ordinary class project.
On the empirical evidence and producing more valid evidence, I could've reached out to the researchers who are already experimenting. Rather than sticking with the researchers who have no interest in the questions and nudging them to do so, I could've just reached out to the researchers caring about 'AI risks' or 'AI safety' for mentorship. From working with some seniors with more expertise in conducting experiments and producing outputs, I wouldn't have wrapped up my experimental project too early.
On the Op-Ed intention, I could've just published that policy report on any platform to raise awareness. It doesn't have to be on any prestigious platform. My vision was too narrow to those few large think tanks and platforms. Producing the paper is already the process of making an impact. Plus, only through producing artifacts can I then iterate on the quality of the writing and ideas.
On the advocacy of the issue, I didn't pay enough attention to developing a risk pathway and theory of change to advocate the importance of the issue. Missing a robust risk pathway, the topic is naturally less attractive and persuasive for the professors who are unaware of the issue. Lacking a theory of change, the policy recommendation is naturally hardly persuasive.
Most importantly, I was not brave enough to believe that I could stand up to the challenge.
Overall, thanks for reading through. Please leave a comment if you have any additional advice on advancing a class project to a real-impact project. This includes tactics to find an advisor, matched mentors, and advancing the ideas as an entry-level professional in the field. Also, please leave any questions if you have!