Enhancing Writing Skills Through AI-Driven Feedback: A Study on the Effectiveness of ChatGPT in Reflective Writing Practice

Abstract

Formative feedback during writing sessions is crucial for improving students’ writing skills. However, educators often struggle with the heavy workload and limited time available to provide such feedback. Research has shown that the ethical use of AI tools like ChatGPT can alleviate this burden, allowing instructors to offer timely and effective feedback. This study aimed to assess the effectiveness of ChatGPT’s feedback in enhancing writing skills, evaluate student perceptions and experiences, and analyze qualitative insights on its impact. A convenience sample of 192 first-year undergraduates comprising of high, intermediate and low English proficiency levels from a Sri Lankan state university participated in a writing intervention consisting of 30 hours of synchronous lectures, which covered pre writing techniques and nine patterns of paragraph structures. In the final 45 minutes of each 2-hour session, students wrote an individual paragraph of the specific type being taught. They also spent nearly 30 hours practicing and revising their work based on feedback generated by ChatGPT, and 10 hours reflecting on their progress and recording key insights from each round of revision, all outside of class hours. Using the provided guidelines, students were required to revise their paragraphs twice using ChatGPT: first, focusing on paragraph structure, grammatical accuracy, and vocabulary appropriateness; second, refining the unity and development of the paragraphs. For both revisions, students received detailed, line-by-line explanations of the changes and the rationale behind them, promoting reflective learning. As a final step, participants compiled their first and second drafts of paragraphs and reflections into a portfolio. Immediately after the intervention, they also completed a 15-item Likert-scale questionnaire. The research employed a mixed-methods design. Quantitative data collected from the questionnaire, which included eight items measuring the effectiveness of ChatGPT’s feedback on writing skills and eight items exploring students’ perceptions and experiences with the tool, were analyzed using descriptive statistics. The percentages and mean values indicated that ChatGPT feedback significantly improved students’ writing skills, and participants held a notably positive perception of its benefits. Additionally, thematic analysis of qualitative data from the student reflections identified six major themes: Elevated writing, Academic (future) pursuits, Enhanced learning experience, Transformative journey, Built confidence, and Successful. Overall, the qualitative data received through portfolio reports revealed that revising paragraphs using ChatGPT feedback was highly beneficial in developing their writing proficiency. These findings suggest that ChatGPT feedback had a positive and substantial impact on students’ writing development. Given that ChatGPT is a relatively new AI tool, further research on its effective application in academic settings is needed. The findings of this study suggest that using ChatGPT as a writing assistant can be highly advantageous for both students and teachers. This research encourages educators to integrate AI tools like ChatGPT into their pedagogical practices in an ethical manner, aligning with the advancements in technology.