knowledge-kitchen / research / education-automation

class: center, middle # Working Group on AI in Education - Status Report (2025 September 26) --- # Agenda 1. [Intro](#intro) 1. [Broad categories of use](#categories) 1. [For instructors](#instructors) 1. [For students](#students) 1. [Next steps](#todo) --- name: intro # Intro --- template: intro ## Who - Amos Bloomberg (Clinical Professor, Computer Science) - Pascal Wallisch (Clinical Professor, Data Science and Psychology) Both of us have been using AI tools in our teaching and have been experimenting with AI course assistants. --- template: intro ## Remit Develop recommendations for the use of AI course assistants and other uses of AI in education. -- > The overall goal is to inform decision-making at Courant/CS/CDS, and in particular requests for resources (faculty time commitment/compensation. Software development, computing support, software licenses etc) > > -- _Denis_ --- template: intro ## Our progress So far... -- - 9/14 - Denis shared our remit, goals, and initial materials to review -- - 9/25 - We have reviewed these materials and developed an understanding current uses of AI in education --- name: categories # Broad categories of use -- ## for Instructors Instructor use cases for AI: - Course design and generation - Grading and feedback - Verification of student work - Student support / AI course assistants - Administrative tasks --- template: categories ## for Students Student use cases for AI: - Completing assessments - Researching / getting help - Content summarization - Content personalization - Generating study materials --- name: instructors # For Instructors --- template: instructors ## Course design and generation Generating curriculum maps, lesson plans, notes, quizzes, lecture slides, assignments, etc. can be done with general-purpose LLM or specialized models fine-tuned for education, e.g. -- - Google’s [LearnLM](https://cloud.google.com/solutions/learnlm) (promo video [here](https://www.youtube.com/watch?v=NTECA6ct55w), educator-targeted description [here](https://blog.google/outreach-initiatives/education/google-learnlm-gemini-generative-ai/)) - fine-tuned generative AI models for education within Google Gemini helps teachers develop lesson plans and assessments and “_meet every student where they are_”. Now integrated into Googlel Classroom and Google Workspace for Education. -- - List of [similar projects](https://www.fullestop.com/blog/generative-ai-in-education-use-cases-benefits-and-challenges) using LLMs fine-tuned for education. -- - Amos’ [slide generating experiment](https://github.com/bloombar/slide-machine) (deployed [here](https://bloombar.github.io/slide-machine/)) --- template: instructors ## Course design and generation Agatha Christie [teaches from the grave](https://www.bbcmaestro.com/courses/agatha-christie/writing).  --- template: instructors ## Grading and feedback Assessing student work and providing meaningful feedback to students at scale, e.g. -- - [Gradescope](https://www.gradescope.com/) - uses AI to help instructors grade paper-based, digital, and code assignments faster; helps provide more consistent feedback from instructors. -- - [ChatGPT for Education](https://chatgpt.com/business/education) - assists with grading and providing feedback on student work. Instructors can use it to generate feedback comments, assess the quality of student writing, and grade assignments based on predefined rubrics. -- - [Edstem](https://edstem.org/) - helps manage courses, assignments, and student interactions, including AI tools for grading and providing feedback on student submissions. --- template: instructors ## Verification of student work Verifying student work meets academic integrity standards, e.g. -- - [Turnitin](https://www.turnitin.com/products/ai-writing-detection) - AI writing detection tool that helps educators identify potential instances of AI-generated content in student submissions. -- - [GPTZero](https://gptzero.me/) - AI detection tool that helps educators identify AI-generated text in student work. --- template: instructors ## Student support / AI course assistants Helping answer student questions and providing additional resources with **AI course assistants** and similar tools, e.g. -- - ChatGPT for Education - can be used to create AI-powered chatbots that assist students with course-related questions. -- - **AI course assistants** - typically LLM chatbots trained on specific course materials. Many instances of these currently being developed (e.g. Amos' [course assistant bot deployed to Discord](https://github.com/bloombar/discord-channel-manager).) Often instructed to behave in a "Socratic" manner, i.e. guide students towards discovering Truth, rather than answer questions directly. -- - **Integration plugins**, i.e. apps to deploy these assistants into 3rd-party platforms currently used by students to complete their work (e.g. VS Code extensions, extensions to the Ed platform, extensions to code debuggers, chatbot apps installed into Discord and Slack, etc) -- - [Some research](https://link.springer.com/chapter/10.1007/978-3-031-98459-4_28) suggests a combination of human/gen-AI tutoring may be most effective for student learning outcomes. --- template: instructors ## Student support / AI course assistants ### Pros -- - **Available 24/7** (unlike humans) -- - Provides **personalized help** (i.e. emulates 1-on-1 tutoring) -- - **Conserves resources** (less resource intensive than hiring actual tutors) -- - **Students less embarrassed** to ask questions of a bot -- - **Instructors can curate** generative AI responses, compared to students simply using generalized systems like ChatGPT, GitHub Copilot, Claude, Gemini, etc. -- - **Frees humans** to focus on more complex pedagogical issues -- - Potential for **multi-channel deployment** (e.g. web-based, code editor extension, Discord/Slack bot, etc) to meet students wherever they are --- template: instructors ## Student support / AI course assistants ### Cons -- - Potential dissemination of **false, misleading, or outdated information**... requires constant supervision -- - **May “over-answer”**, i.e. offer solutions, despite system instructions to the contrary -- - Any limitations imposed are **easily bypassed** using ChatGPT, GitHub Copilot, etc. -- - Some feedback and some research suggests **human tutoring still is important** -- - Use of generative AI in education **may [reduce brain activity](https://teaching.cornell.edu/generative-artificial-intelligence#Learning)** --- template: instructors ## Student support / AI course assistants ### Further considerations -- - **Hosting**: use of hosted SaaS solution versus local homegrown install -- - **Training**: prompt engineering, fine-tuning, RAG (or some combination) -- - **Copyright**: training requires copyright permission for source material. (e.g. [UC San Diego AI Tutor](https://today.ucsd.edu/story/this-bespoke-ai-tutor-helps-students-learning) was trained on textbook written by the instructors.) -- - **Ownership**: some faculty may object to training department/university generative AI on their course materials without additional compensation (see [AAUP report on ownership debate](https://www.aaup.org/reports-publications/aaup-policies-reports/topical-reports/defending-freedom-innovate-faculty)), with the Contract Faculty Union currently negotiating with the university about this (see the union's current negotiating positions on [intellectual property](https://docs.google.com/document/d/1482n7uj9f1oklYN5Iz8O3yohOkV76Tf6/edit) and [generative AI](https://docs.google.com/document/d/1wY2lvYvymmA27R61eX3TmA7KEdc0eLZF/edit); administration's position [on IP](https://drive.google.com/file/d/11ik4dsXWHsL_G2pFuczDWGeqF8-eOKss/view) is to claim an irrevocable license to all scholarship and all course materials created by T- and C-Faculty for any use forever.) --- template: instructors ## Student support / AI course assistants ### Further considerations (continued) - **Privacy** and **FERPA compliance**: must redact personally-identifiable information from any training data and ensure FERPA compliance in usage -- - **Maintaining control**: many articles mention a preference to deliberately incorporate AI into courses rather than outright forbid it or leave it up to students to use on their own. -- - **Pedagogical approach**: Use of services like OpenAI API may influence in what manner, style, and with what language any material is taught by a bot, even if asking it to follow a “_Socratic_” style. They have their own proclivities and biases which may run contrary to an instructor’s own style. -- - **Cost**: training data must be regularly updated, systems maintained, dependencies upgraded, features added, bugs fixed, etc... --- template: instructors ## Administrative tasks Helping instructors write emails and other administrative things, e.g. -- - Many tools to help **draft emails and other written communications** (e.g. [GPT for Gmail](https://workspace.google.com/marketplace/app/gpt_for_gmail/802100925247) and [Grammarly](https://www.grammarly.com/).) -- - A variety of tools are available to **transcribe, translate, and summarize lecture and meeting videos**, facilitating instructor review and dissemination of information (e.g. [Otter.ai](https://otter.ai/).) --- template: instructors ## Administrative tasks Does AI-generated email deserve an AI-generated response?  --- name: students # For Students -- ## Completing assessments Students use generative AI to **assist with or fully complete assessments** on their behalf. -- - Code generation tools (e.g. [GitHub Copilot](https://github.com/features/copilot).) -- - Written text generation (e.g. [ChatGPT](https://chat.openai.com/).) -- - Image, video, audio, and other media generation (e.g. [DALL-E](https://openai.com/dall-e-3), [Midjourney](https://www.midjourney.com/), [Suno](https://suno.ai/), etc.) -- - Feedback on work-in-progress (e.g. UC San Diego's [AI Tutor](https://today.ucsd.edu/story/this-bespoke-ai-tutor-helps-students-learning) and Harvard's [CS50.ai](https://cs50.readthedocs.io/cs50.ai/) - both can "see" and speak to student code in editor and explain errors in terminal.) -- We know student use is [ubiquitous](https://www.chronicle.com/article/how-are-students-really-using-ai), perhaps more so than web search, and students who do not use these tools may be at a disadvantage in grading. --- template: students ## Researching / getting help Students use generative AI to **research and hone understanding of course materials**. -- - General-purpose LLMs (e.g. ChatGPT, Claude, Gemini, etc.) -- - Education-specific LLMS trained on specific course data (ChatGPT for Education, Google LearnLM, etc) -- - Google Workspace for Education (used by a large majority of US schools) now includes [AI tools for teens](https://blog.google/outreach-initiatives/education/workspace-gemini-classroom-iste-2024/) including guard-rails when students use its Gemini chatbot. Also includes "Learning Coach" built on top of Google's [LearnLM](https://blog.google/outreach-initiatives/education/google-learnlm-gemini-generative-ai/) education models. -- - LearnLM and [ChatTube](https://chattube.io/) let students "ask questions" of video content like they do with text content and search results. --- template: students ## Content summarization Generation of summary course materials (cheat sheets, CliffsNotes, podcasts, etc), e.g. -- - Google's [NotebookLM](https://notebooklm.google.com/?original_referer=https:%2F%2Fsearch.brave.com%23&pli=1) aggregates documents, summarizing them in written and/or **podcast form** and providing a chat interface for querying the docs. -- - [GPT4All](https://www.nomic.ai/gpt4all) - open source locally-hosted LLM that can be trained on specific documents to provide a chat interface for querying them. -- - Monash University has a nice [guide to AI summarization](https://www.monash.edu/student-academic-success/build-digital-capabilities/create-online/effectively-summarising-with-generative-artificial-intelligence). --- template: students ## Content personalization **Course materials can be transformed** into different formats, languages, and complexity to suit students of different ages, different abilities, or different learning styles, improving accessibility, eg. -- - Google’s "[Learn Your Way](https://learnyourway.withgoogle.com/)" experiment (technical report [here](https://services.google.com/fh/files/misc/ai_augmented_textbook_final.pdf) ), transforms text content to make it more personalized to the abilities and needs of the particular student, using "a range of presentation forms and assessment components." -- - See MIT Fluid Interfaces Group’s "[Generative AI for Personalized Learning & Self-Development](https://www.media.mit.edu/projects/generative-ai-for-personalized-learning/overview/)" project (not much of interest yet). -- - Some overview of research in this area [here](https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2024.1460651/full). --- template: students ## Generating study materials Students can use generative AI to create **custom study materials** such as flashcards, practice problems, quizzes, etc., e.g. --- name: todo # Next steps ## To do - Speak with colleagues in Math and elsewhere also researching this - Develop recommendations for Courant/CS/CDS leadership