Document Type

Article

Publication Date

2025

DOI

10.1007/s12528-025-09444-6

Publication Title

Journal of Computing in Higher Education

Volume

Article in Press

Pages

22 pp.

Abstract

Considering both the transformative opportunities and challenges presented by generative AI (GenAI) in academic writing, effectively integrating GenAI into the academic setting becomes a significant need requiring prioritization. Yet, there is limited understanding regarding the nature of interactions between different types of students, what behavioral patterns students exhibit during a student-GenAI interaction (SAI) on a given task, and how these different SAI patterns relate to the actual writing task performance. This study, therefore, aimed to identify SAI patterns of academic writing tasks depending on students’ level of AI literacy and examine the differences in academic writing performance between the identified SAI patterns. Drawing from the combination of three data sources, including think-aloud protocols, screen-recordings, and chat histories between 36 Chinese graduate students and a GenAI writing system, epistemic network analysis (ENA) was used to reveal the distinctive SAI patterns of students with different levels of AI literacy. The study found that students with a high level of AI literacy exhibited a collaborative approach to SAI, actively accepting GenAI’s suggestions and engaging GenAI in meta-cognitive-related activities such as planning, whereas students with a low level of AI literacy demonstrated much less interaction with GenAI in completing their writing tasks, instead choosing to ideate and evaluate independently. In addition, the Wilcoxon rank-sum (Mann-Whitney U) test was conducted to assess the writing task performance of the two AI literacy groups. Findings revealed statistical differences in all evaluation rubrics (content, structure/organization, expression). This study offers implications for the design and implementation of GenAI agents in writing tasks and the pedagogy of GenAI-assisted instruction.

Rights

© 2025 The Authors.

This article is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original authors and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Data Availability

Article states: "The data supporting the findings of this study are available from the corresponding author upon reasonable request. We recognize the importance of transparency and reproducibility in scientific research, and therefore, we are committed to sharing our data with interested parties for further analysis and verification. However, we reserve the right to assess the reasonableness of each request based on factors such as the purpose of the request, the ability of the requester to handle sensitive data responsibly, and any potential conflicts of interest. We encourage the use of our data for non-commercial research purposes and will make every effort to accommodate reasonable requests in a timely manner."

Original Publication Citation

Kim, J., Lee, S.-S., Detrick, R., Wang, J., & Li, N. (2025). Students-generative AI interaction patterns and its impact on academic writing. Journal of Computing in Higher Education. Advance online publication. https://doi.org/10.1007/s12528-025-09444-6

ORCID

0000-0002-3365-7354 (Kim)

Download

Share

COinS