By [vc_row css=”.vc_custom_1592900783478{margin-right: 0px !important;margin-left: 0px !important;}”][vc_column css=”.vc_custom_1592900766479{margin-right: 10px !important;margin-left: -10px !important;}”][vc_column_text]Several papers that have recently been published in peer-reviewed journals display obvious signs of having been written by the AI tool ChatGPT. This has sparked a heated online debate about the transparency of research communication and academic integrity in cases where AI is used in the academic writing process. In this blog article, Kim Tung Dao discusses the ethical implications of using AI for academic writing and ponders the future impact of AI in academic research, urging for a balance between the efficiency of AI tools and research integrity.[/vc_column_text][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][vc_single_image image=”28106″ img_size=”full” alignment=”center”][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][vc_column_text]
Used for everything from streamlining everyday tasks to revolutionizing industries, artificial intelligence (AI) has come to profoundly affect our lives in the past few decades. The emergence of new forms of AI in recent years has led to a heated debate in academia about whether students should be allowed to use AI tools — usually large language models (LLMs) such as ChatGPT — in their writing. And if they are permitted, a related question is to what extent they should be used, especially in higher education.
A new issue related to the rise of LLMs is now rearing its head within the realm of scientific research: the publication of LLM-generated content in peer-reviewed journals. This worrying trend reflects not only the rapid advancements in LLMs’ ability to replicate human work but also gives rise to discussions on the ethics of research (communication) and research integrity.
More and more researchers are attempting to leverage generative AI such as ChatGPT to act as a highly productive research assistant. It is very tempting to have an LLM compose content for you, as these AI-generated pieces often exhibit sophisticated language, conduct statistical analyses seamlessly, and even discuss new research findings expertly. The line between human- and machine-generated content is blurring. In addition, these LLMs work tirelessly and quickly, which can be considered highly beneficial for human scholars.
However, beneath the surface of effectiveness and efficiency lies a complex labyrinth of ethical concerns and potential repercussions for the integrity of scientific research. Publishing academic research in journals remains the most popular way for many researchers to disseminate their findings, communicate with their peers, and contribute to scientific knowledge production. Peer reviewing ensures that research findings and truth claims are meticulously evaluated by experts in the field to sustain quality and credibility in the formulation of academic theories and policy recommendations. Hence, when papers with AI-generated content are published in peer-reviewed journals, readers can’t help but question the integrity of the entire scientific publishing process.
There is a big difference between receiving assistance from generative AI and allowing it to generate entire or significant parts of research texts without appropriate supervision and monitoring. These can entail smaller tasks such as proofreading AI-generated content before its distribution/publication but can also play a much more critical role in ensuring the originality and significance of AI-enhanced research. This is why this article seeks to reflect on the abuse of AI in the writing of academic texts by researchers and provides commentary on the insufficiency of the current peer-review system. I also try to initiate a thoughtful discussion on the implications of AI for the future of research.
Falling through the cracks
The latest volume of Elsevier’s Surfaces and Interfaces journal recently caught the attention of researchers on X (Twitter), as one of its papers has evidently been written by ChatGPT. The first line of the paper states: “Certainly, here is a possible introduction for your topic: […].” Any ChatGPT user knows that this is the typical reply generated by the LLM when it responds to a prompt. Without any expertise in AI or other related fields, a common ChatGPT user with normal common sense can therefore tell that this sentence and at least the following paragraph, if not many others, has been generated by ChatGPT.
But this paper is certainly not the only one in this new line of LLM-generated publications. ChatGPT prompt replies have been found in other papers published in different peer-reviewed journals and are not limited to any specific fields of science. For example, a case report published in Radiology Case Reports (another Elsevier journal) includes a whole ChatGPT prompt reply stating “I’m very sorry, but I don’t have access to real-time information or patient-specific data, as I am an AI language model. I can provide general information about […], but for specific cases, it is essential to consult with a medical professional […].”
Hallucinating information
What is more worrisome is the quality, integrity, and credibility of scientific research conducted by these LLMs, as ChatGPT has the tendency to hallucinate information and draws on seemingly non-existent citations and references to support the texts it generates. For example, in a forum discussion where contributors talked about detecting AI-generated content in academic publications, one contributor pointed out that they could not find the references cited in a paper titled “Automatic Detection of Coagulation of Blood in Brain Using Deep Learning Approach”. Several other cases are mentioned in the discussion thread.
Besides likely contributing to the publication of false or unevidenced information, the use of LLMs in the writing up of scientific research also highlights the failure of peer reviewers to catch or question these practices, showing either their carelessness or their irresponsibility. The peer-review system has long served as the gatekeeper of scholarly knowledge, aiming to uphold high standards of quality, integrity, and credibility that are part and parcel of academic research and publishing. But with obvious evidence of LLM-generated content being included in papers published in peer-reviewed journals, it might be time to start questioning the transparency and accountability inherent in the peer-review process. When a peer-review publication starts with a ChatGPT’s typical prologue, it’s reasonable to wonder how such article was reviewed.
A call for responsible use
AI is not all bad. Clearly, it can be a powerful assistant to researchers in the research process, used for anything ranging from brainstorming, developing research strategies, coding, analyzing empirical results, and language editing to acting as a competently critical reviewer to provide useful and helpful feedback for excellent improvement. But to work with this powerful assistant, researchers still need to have a solid knowledge of the research topic, make significant decisions on the research strategy, and, most importantly, ensure that the research is an original contribution to the literature and can be applied. Relying heavily on AI to finish a research project without understanding the foundation and the essence of the research is plainly ethical contamination and fraudulent behavior.
AI is not a scientific researcher — and might never be
Beyond the immediate finger-pointing at the peer-reviewed system and research practices, the increasing influence of AI in research outputs carries broader implications for the role and integrity of human researchers, the nature of scientific discovery, and the social perception of AI. Even if the potential for deception and manipulation is ignored, AI-generated research outputs might still lack genuine insights, critical analysis, and might fail to take into account ethical considerations without human guidance. Moreover, in order for research outputs to be meaningful for human life and society, they need to be validated by human researchers.
We don’t necessarily need to fear AI; we do need to fear the improper use of AI, and we need to play an active role in preventing this from happening. Thus, instead of fearing being replaced by AI, human researchers should start acknowledging its abilities and using it to shape our projects. Let’s board this technological advancement ship to escalate our research efficiency and accelerate the speed of scientific discovery. But let us remain cautious. We are responsible for ensuring that AI contributes to instead of compromises scientific knowledge production.
Writing this post with the help of ChatGPT 3.5 (which I used to improve my language), I can’t help but recall the question I was asked when receiving my doctoral degree: “Do you promise to continue to perform your duties according to the principles of academic integrity: honestly and with care; critically and transparently; and independently and impartially?”
I promise.[/vc_column_text][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][vc_column_text]Opinions expressed in Bliss posts reflect solely the views of the author of the post in question.[/vc_column_text][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][vc_column_text css=”.vc_custom_1713430703942{margin-top: 0px !important;}”]
About the author:
Kim Tung Dao is a recent PhD graduate of the International Institute of Social Studies. Her research interests include globalization, international trade, development, and the history of economic thought.
[/vc_column_text][/vc_column][/vc_row][vc_row][vc_column css=”.vc_custom_1596795191151{margin-top: 5% !important;}”][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][vc_column_text]
Are you looking for more content about Global Development and Social Justice? Subscribe to Bliss, the official blog of the International Institute of Social Studies, and stay updated about interesting topics our researchers are working on.
[/vc_column_text][vc_column_text][newsletter][/vc_column_text][vc_separator color=”custom” accent_color=”#a80000″ css=”.vc_custom_1594895181078{margin-top: -15px !important;margin-bottom: 10px !important;}”][/vc_column][/vc_row]
Discover more from Bliss
Subscribe to get the latest posts sent to your email.
Peter van Bergeijk
18 April 2024Thanks for a very informative blog. What about the refereeing procedures. It is tempting to let AI do the refereering?
I asked chatGPT two questions: write a positive and a negative report.
This is what I got:
PUBLISH
This paper presents a comprehensive analysis of the increasingly prevalent phenomenon of using large language models (LLMs), such as ChatGPT, in academic research and the ethical implications associated with it. The author provides a clear overview of the debate surrounding the use of AI tools in academic writing and extends this discussion to the realm of scientific research publication, particularly focusing on the emergence of LLM-generated content in peer-reviewed journals.
The author effectively highlights the benefits and challenges of utilizing LLMs in research, acknowledging their potential to enhance productivity and efficiency while also raising concerns about the integrity of scientific research and the peer-review process. The examples provided, such as the identification of ChatGPT-generated content in published papers, serve to underscore the gravity of the issue at hand and prompt readers to consider the implications for research ethics and scholarly integrity.
One strength of the paper is its balanced perspective on the role of AI in research. While recognizing the value of AI as a research assistant, the author aptly cautions against overreliance on LLMs and emphasizes the importance of maintaining human oversight and critical judgment in the research process. This nuanced approach contributes to a more nuanced understanding of the complex relationship between AI and human researchers.
Furthermore, the paper makes a compelling argument for the need to reassess current practices in peer review to address the challenges posed by the integration of AI-generated content into scholarly publications. By questioning the transparency and accountability of the peer-review process in light of recent developments, the author prompts readers to consider potential reforms that may be necessary to uphold the quality and credibility of academic research.
Overall, this paper provides a thought-provoking exploration of a timely and important issue in academic research. The clarity of writing, cogent arguments, and well-supported evidence contribute to its strength as a scholarly contribution. I recommend its acceptance for publication, as it offers valuable insights that will enrich discussions on the ethical use of AI in research and stimulate further inquiry into this evolving field.
REJECT
This paper presents an analysis of the use of large language models (LLMs) in academic research, focusing on the ethical implications of incorporating LLM-generated content into peer-reviewed journals. While the topic is undoubtedly relevant and timely, the paper suffers from several significant shortcomings that undermine its scholarly merit.
Firstly, the paper lacks depth in its exploration of the ethical concerns surrounding the use of LLMs in research. While it briefly touches on issues such as integrity and credibility, it fails to engage with the broader ethical debates in the field. Additionally, the paper’s reliance on anecdotal evidence, such as Twitter discussions and forum posts, undermines the rigor of its analysis and detracts from its credibility as a scholarly work.
Furthermore, the paper’s argumentation is overly simplistic and lacks nuance. It presents a dichotomous view of AI as either a beneficial research assistant or a threat to academic integrity, without adequately considering the complex interplay between human researchers and AI tools. The author’s insistence on portraying LLMs as mere assistants fails to acknowledge their potential to contribute meaningfully to the research process, thus oversimplifying a nuanced issue.
Moreover, the paper’s critique of the peer-review process is unsubstantiated and relies on speculative assertions rather than empirical evidence. While it raises valid questions about the ability of peer reviewers to detect AI-generated content, it fails to provide concrete examples or data to support its claims. As a result, the paper’s critique comes across as unfounded and lacking in scholarly rigor.
Overall, while the topic of the paper is relevant, its execution is lacking. The paper would benefit from a more thorough analysis of the ethical implications of using LLMs in research, supported by empirical evidence and informed by a nuanced understanding of the complexities involved. Additionally, the paper’s critique of the peer-review process should be grounded in empirical research rather than speculative conjecture. As it stands, the paper does not meet the standards of quality and rigor expected of scholarly work in this field.