The Impact of AI Feedback on Student Assessment Practices and Outcomes
Presentation and links for NZARE 2024
Tomorrow, I am presenting some research at NZARE, and I thought I’d run through my presentation and share the links for anyone interested (this also lets me check that it’s not too long!). The research tested an idea about a pre-assessment feedback GPT that I designed. The video will explain in detail, and the links that I mention in the video are also below. I’ll write the full paper up for full publication next year but please contact me if you’d like any further detail.
Impact of AI Feedback on Student Assessment Practices - Watch Video
Link to the slides from the presentation
Link to the student-facing GPT
Link to the AI reflection guide
Here is the submitted abstract:
The Impact of AI Feedback on Student Assessment Practices and Outcomes
This study examined the impact of using a pre-assessment submission Generative Pre- trained Transformer (GPT) on student learning and assessment practices. The GPT was available on the assessment page and allowed students to choose to receive generative AI feedback before making their final submission. In addition, students filled out a reflective form to evaluate their use of generative AI. Data was collected through pre- and post-surveys using the UTAUT scale, as well as AI literacy and critical reflection questions. Interactions with the GPT were recorded and transcribed for qualitative analysis. After the intervention, revised assignments were evaluated using rubrics, and survey data measured student experiences, engagement, AI literacy, and critical reflection. The implementation of this tool promoted transparent AI use and improved the quality of student submissions. Additionally, students' AI literacy increased. These findings contribute to the existing literature on AI in education and provide practical recommendations for integrating AI in learning and assessment. Implementing a GPT is a simple and practical first step in supporting the integration of generative AI into assessment.
