Abstract

GenAI is now capable of producing journal articles that look like human work, raising the question of whether the scholarly credibility might be misinterpreted as evidentiary validity. This research investigates AI-generated scholarship by shifting attention from the detection of AI-generated content to its critical appraisal and evaluation. To achieve this, a split-paper audit procedure was applied to a fully AI-generated mixed-methods manuscript that was independently reviewed by three university faculty members. Findings indicate that the model can simulate academic style and coherence with a high degree of surface plausibility; however, this superficial rigor was combined with ungrounded content, unsupported contextualization, selective reporting, and inconsistent methodological details. Reviewers’ comments also highlighted the formulaic language and narratives that confirmed their prior expectations, and questioned whether prompt structures may be channeling outputs towards pre-existing assumptions and masking areas where evidence is scant. The study defines synthetic scholarship as an epistemic condition where plausibility may supplant verification, and suggests a transparent audit process to facilitate the assessment of AI-mediated scholarship. It argues that ensuring research integrity in the era of generative AI will rely on more than just disclosure of generative AI use; it will also require auditable trails to assist in examining, verifying, and substantiating claims.

IPC Classification

Keywords