Emergence of meta-research: a meeting report on the 10th International Congress on Peer Review and Scientific Publication

Soo Young Kim<img src='/image/flag/kr.gif' style='width:21px;height:13px;margin-top:4px;margin-left:7px;vertical-align:text-top' alt=''/>

doi:10.6087/kcse.388

Articles

Page Path: HOME > Sci Ed > Volume 13(1); 2026 > Article

Meeting Report Emergence of meta-research: a meeting report on the 10th International Congress on Peer Review and Scientific Publication: Soo Young Kim; Science Editing 2026;13(1):51-54.
DOI: https://doi.org/10.6087/kcse.388
Published online: January 19, 2026

Department of Family Medicine, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea

Correspondence to Soo Young Kim hallymfm@gmail.com

• Received: October 15, 2025 • Accepted: October 22, 2025

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

prev next

1,071 Views
31 Download
1 Crossref
1 Scopus

Full Article

Download PDF

Notes

Meeting: 10th International Congress on Peer Review and Scientific Publication

Date: September 3–5, 2025

Venue: Chicago, IL, USA

Theme: Improving the quality and credibility of scientific communication

The International Congress on Peer Review and Scientific Publication, held every 4 years, is one of the most authoritative international gatherings dedicated to research publishing. First launched in 1989 in Chicago, the congress convened for its 10th iteration in 2025, once again in Chicago. Under the theme “Improving the Quality and Credibility of Scientific Communication,” researchers, editors, publishers, and research ethics experts from around the world gathered to discuss the reliability and innovation of the scholarly communication ecosystem.

The meeting adopted a hybrid format, attracting approximately 400 in-person and 80 virtual participants, including 8 attendees from Korea. The program ran intensively from 8:00 AM to 5:00 PM each day and was structured into four plenary sessions, each consisting of one plenary lecture followed by four to five oral presentations. Poster sessions were held both onsite and online (102 onsite and 31 online). I presented a poster entitled “Reporting Study Design in Korean Medical Journals,” which examined more than 300 Korean medical articles indexed in KoreaMed in 2023. The study assessed whether explicit reporting of study design, strongly recommended by reporting guidelines, is practiced and, when present, whether the design label used is appropriate (Fig. 1).

The scientific program covered a wide spectrum of topics related to publishing. Among these, two themes stood out as central: meta-research and artificial intelligence (AI). Meta-research (i.e., “research on research”) was a concept I explored in depth for the first time at this congress. It examines how science is produced, reported, reproduced, disseminated, and evaluated, with the goal of analyzing, critiquing, and ultimately improving the research enterprise itself. Although still relatively unfamiliar in Korea, meta-research has expanded rapidly worldwide over the past 10 to 15 years, with a notable turning point following the establishment of the Meta-Research Innovation Center at Stanford (METRICS) in 2014.

Across the 3 days, multiple sessions addressed meta-research focusing on methods, reporting, reproducibility, bias, and research evaluation. As in many academic meetings today, AI featured prominently. AI-related research was organized into three sessions—manuscript authoring and peer review, quality assessment, and problem detection—reflecting vigorous activity across editorial and publishing workflows. Because many presentations offered notable insights, I provide concise one-line summaries of selected contributions in Table 1.

Overall, the congress offered a valuable opportunity to engage with contemporary debates in publishing and editorial practice, particularly concerning meta-research and AI. The insights gained from this meeting are expected to inform practical strategies for addressing similar challenges within the Korean scholarly community.

Notes

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Funding

No financial support was received for this work.

Data availability

Data sharing is not applicable as no new data were created or analyzed in this article.

Supplementary materials

No supplementary materials were provided for this article.

Fig. 1.

Poster presentation on the current status of reporting study designs in articles published in Korean medical journals.

Table 1.

Major presentation topics at the 10th International Congress on Peer Review and Scientific Publication

Presentation	Key findings
September 3
Morning 1. Use of AI in manuscript preparation and peer review
Isamme AlFayyad et al.	Only 5.7% of BMJ Group submissions voluntarily disclosed AI use, suggesting substantial under-reporting.
Roy Perlis et al.	Among manuscripts and peer-review reports submitted to JAMA Network journals, only 2.7% declared AI use.
Vincent Yuan et al.	Of 423 medical AI studies published in 2023, 35% overstated their performance and 56% selectively emphasized significant findings, indicating widespread “spin” and selective reporting.
Morning 2. Authorship and research integrity
Ana-Catarina Pinho-Gomes et al.	A comparative analysis of retraction reasons for women vs. men authors in biomedicine suggested gender-linked differences in the causes.
Tim Kersjes	“Paper mills” manipulate peer review by deploying fabricated identities.
Coromoto Power Febres et al.	Authorship changes during submission may signal threats to research integrity.
Nicholas DeVito et al.	Notifications via “RetractoBot” to authors citing retracted work markedly reduced subsequent citations of those retracted articles.
Afternoon 1. Diversity and research environments
Michael Mensah et al.	Analysis of equity, diversity, and inclusion issues raised by JAMA Network reviewers showed persistent concerns about bias and unequal opportunities.
Noémie Aubert Bonn et al.	Indicators derived from the UK Research Excellence Framework 2021 can differentiate the quality of institutional research environments.
Afternoon 2. Research misconduct and integrity
João Phillipe Cardenuto et al.	Characteristics of problematic images in retracted articles revealed frequent patterns suggestive of manipulation.
Ahmad Sofi-Mahmudi et al.	Across 167 countries, retraction counts correlated with democracy indices, indicating an association between political freedom and integrity in scholarly publishing.
Renee Hoch et al.	Lessons from PLOS One highlighted sustainable practices for maintaining high ethical standards amid large-scale threats.
September 4
Morning 1. Bias and reporting guidelines
Jae Il Shin et al.	“Immortal time bias” is common in systematic reviews and meta-analyses and can materially distort effect estimates.
Yiwen Jiang et al.	Designating the same endpoint as primary versus secondary can yield different effect estimates, underscoring the need for careful design and interpretation.
Yulin Yu et al.	Repurposing datasets beyond their original intent in AI research can compromise interpretability and reproducibility.
Nicola Di Girolamo et al.	Phrases such as “to our knowledge” are pervasive in biomedical literature and may overstate originality or uncertainty.
Morning 2. Peer-review models
Charvi Rastogi et al.	Maintaining reviewers' anonymity may improve objectivity and candor in deliberations.
John Carpenter et al.	Applying a double-anonymous, distributed review model to ALMA telescope proposals improved fairness and transparency.
Elena Álvarez-García et al.	Open review altered review length and deepened content.
Afternoon 1. Editorial and publishing models
Christos Kotanidis et al.	Submitted abstracts often diverge substantially from the final published abstracts.
Afternoon 2. Peer-review timelines and incentives
Emilie Gunn et al.	Extending review deadlines beyond 2 weeks did not increase reviewer acceptance but did lengthen turnaround time; testing “reviewer deadline” prompts improved timeliness and compliance, with uncertain effects on quality.
Christopher Cotton et al.	A randomized experiment in medical journals indicated that monetary incentives modestly increased reviewer participation.
September 5
Morning 1. AI for quality and reporting assessment
Lan Jiang et al.	Large language models can automate checks for adherence to reporting guidelines in RCTs.
Fangwen Zhou et al.	Using SHAP to interpret LLM classifiers clarified which features drive automated RCT quality assessments.
Xiangji Ying et al.	A GPT-based system can automatically detect outcome switching in trials registered on ClinicalTrials.gov.
Morning 2. Open Science I
Ayu Putu Madri Dew et al.	A randomized trial showed that Open Science checklists significantly improved reproducibility.
Benjamin Speich et al.	High rates of non-registration, discontinuation, and non-publication persist in RCTs from Switzerland, the UK, Germany, and Canada, underscoring the need for greater transparency.
David Blanco et al.	Among manuscripts submitted to The BMJ, multiple factors influenced inadequate trial registration, registration defects, and publication outcomes.
Afternoon 1. Open Science II
Aidan Tan et al.	Meta-research on compliance with journal data sharing policies showed low real-world adherence and ongoing deficits in data transparency.
Robert Thibault et al.	Funder mandates substantially increased sharing of data, code, protocols, and research materials.
Kyobin Hwang et al.	Meta-research confirmed very low compliance with journal data sharing policies.
Afternoon 2. AI for peer-review quality and problem detection
M. Janina Sarol et al.	LLMs can effectively detect citation errors in biomedical literature.
Bojan Batalo et al.	An automated system can identify “hype” to mitigate exaggerated claims.
Vishisht Rao et al.	Methods to detect LLM-generated peer reviews were evaluated (reported false-positive rate, 0).
Fares Alahdab et al.	LLM-assisted reviews may be competitive with, or comparable to, human reviews in quality and comprehensiveness.

AI, artificial intelligence; RCT, randomized controlled trial; SHAP, Shapley Additive Explanations; LLM, large language model; GPT, generative pretrained transformer.

Figure & Data

References

Citations

Citations to this article as recorded by

How editors perceive the use of generative artificial intelligence in writing academic papers: a narrative review
Sun Huh
Journal of the Korean Medical Association.2026; 69(2): 111. CrossRef

PubReader
Cite

Cite

export Copy Format

Close

XML Download

Figure

Emergence of meta-research: a meeting report on the 10th International Congress on Peer Review and Scientific Publication

Fig. 1. Poster presentation on the current status of reporting study designs in articles published in Korean medical journals.

Fig. 1.

Emergence of meta-research: a meeting report on the 10th International Congress on Peer Review and Scientific Publication

Presentation	Key findings
September 3
Morning 1. Use of AI in manuscript preparation and peer review
Isamme AlFayyad et al.	Only 5.7% of BMJ Group submissions voluntarily disclosed AI use, suggesting substantial under-reporting.
Roy Perlis et al.	Among manuscripts and peer-review reports submitted to JAMA Network journals, only 2.7% declared AI use.
Vincent Yuan et al.	Of 423 medical AI studies published in 2023, 35% overstated their performance and 56% selectively emphasized significant findings, indicating widespread “spin” and selective reporting.
Morning 2. Authorship and research integrity
Ana-Catarina Pinho-Gomes et al.	A comparative analysis of retraction reasons for women vs. men authors in biomedicine suggested gender-linked differences in the causes.
Tim Kersjes	“Paper mills” manipulate peer review by deploying fabricated identities.
Coromoto Power Febres et al.	Authorship changes during submission may signal threats to research integrity.
Nicholas DeVito et al.	Notifications via “RetractoBot” to authors citing retracted work markedly reduced subsequent citations of those retracted articles.
Afternoon 1. Diversity and research environments
Michael Mensah et al.	Analysis of equity, diversity, and inclusion issues raised by JAMA Network reviewers showed persistent concerns about bias and unequal opportunities.
Noémie Aubert Bonn et al.	Indicators derived from the UK Research Excellence Framework 2021 can differentiate the quality of institutional research environments.
Afternoon 2. Research misconduct and integrity
João Phillipe Cardenuto et al.	Characteristics of problematic images in retracted articles revealed frequent patterns suggestive of manipulation.
Ahmad Sofi-Mahmudi et al.	Across 167 countries, retraction counts correlated with democracy indices, indicating an association between political freedom and integrity in scholarly publishing.
Renee Hoch et al.	Lessons from PLOS One highlighted sustainable practices for maintaining high ethical standards amid large-scale threats.
September 4
Morning 1. Bias and reporting guidelines
Jae Il Shin et al.	“Immortal time bias” is common in systematic reviews and meta-analyses and can materially distort effect estimates.
Yiwen Jiang et al.	Designating the same endpoint as primary versus secondary can yield different effect estimates, underscoring the need for careful design and interpretation.
Yulin Yu et al.	Repurposing datasets beyond their original intent in AI research can compromise interpretability and reproducibility.
Nicola Di Girolamo et al.	Phrases such as “to our knowledge” are pervasive in biomedical literature and may overstate originality or uncertainty.
Morning 2. Peer-review models
Charvi Rastogi et al.	Maintaining reviewers' anonymity may improve objectivity and candor in deliberations.
John Carpenter et al.	Applying a double-anonymous, distributed review model to ALMA telescope proposals improved fairness and transparency.
Elena Álvarez-García et al.	Open review altered review length and deepened content.
Afternoon 1. Editorial and publishing models
Christos Kotanidis et al.	Submitted abstracts often diverge substantially from the final published abstracts.
Afternoon 2. Peer-review timelines and incentives
Emilie Gunn et al.	Extending review deadlines beyond 2 weeks did not increase reviewer acceptance but did lengthen turnaround time; testing “reviewer deadline” prompts improved timeliness and compliance, with uncertain effects on quality.
Christopher Cotton et al.	A randomized experiment in medical journals indicated that monetary incentives modestly increased reviewer participation.
September 5
Morning 1. AI for quality and reporting assessment
Lan Jiang et al.	Large language models can automate checks for adherence to reporting guidelines in RCTs.
Fangwen Zhou et al.	Using SHAP to interpret LLM classifiers clarified which features drive automated RCT quality assessments.
Xiangji Ying et al.	A GPT-based system can automatically detect outcome switching in trials registered on ClinicalTrials.gov.
Morning 2. Open Science I
Ayu Putu Madri Dew et al.	A randomized trial showed that Open Science checklists significantly improved reproducibility.
Benjamin Speich et al.	High rates of non-registration, discontinuation, and non-publication persist in RCTs from Switzerland, the UK, Germany, and Canada, underscoring the need for greater transparency.
David Blanco et al.	Among manuscripts submitted to The BMJ, multiple factors influenced inadequate trial registration, registration defects, and publication outcomes.
Afternoon 1. Open Science II
Aidan Tan et al.	Meta-research on compliance with journal data sharing policies showed low real-world adherence and ongoing deficits in data transparency.
Robert Thibault et al.	Funder mandates substantially increased sharing of data, code, protocols, and research materials.
Kyobin Hwang et al.	Meta-research confirmed very low compliance with journal data sharing policies.
Afternoon 2. AI for peer-review quality and problem detection
M. Janina Sarol et al.	LLMs can effectively detect citation errors in biomedical literature.
Bojan Batalo et al.	An automated system can identify “hype” to mitigate exaggerated claims.
Vishisht Rao et al.	Methods to detect LLM-generated peer reviews were evaluated (reported false-positive rate, 0).
Fares Alahdab et al.	LLM-assisted reviews may be competitive with, or comparable to, human reviews in quality and comprehensiveness.

Table 1. Major presentation topics at the 10th International Congress on Peer Review and Scientific Publication

AI, artificial intelligence; RCT, randomized controlled trial; SHAP, Shapley Additive Explanations; LLM, large language model; GPT, generative pretrained transformer.