| Home | KCSE | Sitemap | Contact Us |  
Science Editing > Volume 8(1); 2021 > Article
Yi and Huh: Korean editors’ and researchers’ experiences with preprints and attitudes towards preprint policies

Abstract

Purpose

This study investigated editors’ and researcher’s experiences with preprints and their attitudes towards preprint policies in Korea.

Methods

From December 30, 2019 to January 10, 2020, a Google Forms survey was mailed to members of the Korean Council of Science Editors and the Korean Federation of Science and Technology Societies. The 16 survey items included two demographic items, six items on experience with preprints, five 5-point Likert-scale items on attitudes towards preprints, and three items on advantages and disadvantages.

Results

Out of 365 respondents, 56 had deposited their manuscripts on preprint servers, while 49 stated that they allowed preprints in their journals. More than half of the respondents expressed favorable attitudes towards prioritizing preprint deposition, promotion of open access, rapid feedback on preprints, earlier citations, and evidence of research work. Responders in engineering had more experience with the concept of preprints, and were more likely to have heard about preprint servers and preprint deposition by other researchers, than those in medicine. Half of the editors disagreed with the need for preprints, for reasons including a lack of scientific integrity, stealing ideas/scooping data, priority issues regarding research ideas, and copyright problems.

Conclusion

The above results showed that preprints are still not actively used in Korea. Although experiences with preprints were not widespread, more than half of the respondents showed favorable attitudes towards preprints. More of a consensus should emerge for preprint policies to be accepted by editors in Korea.

Introduction

Background/rationale: Preprints refer to pre-published papers uploaded to a public server by the author. The first preprint server, arXiv (https://arxiv.org/), appeared in 1991 as a repository of physics manuscripts. After arXiv, preprint servers expanded to other subject areas, including biomedicine, mathematics, social sciences, nutrition, and agriculture. The Center for Open Science established Open Science Framework (OSF) Preprints in 2016 (https://osf.io/preprints), as an open source-based preprint server. The OSF Preprints infrastructure collects data from preprint servers and provides integrated search services. As of July 2020, 26 preprint servers have provided records for 2.2 million registered records. When publishing a preprint, the author retains copyright on the paper and expects it to be submitted as a paper. However, this is not always the case. It is estimated that approximately 20% of the preprint published in arXiv are not published in journals [1]. It is not uncommon for preprints deposited in preprint servers to be cited before they are published in a journal [2].
Several articles have described the introduction of preprint policies by editors, and 60.8% of 171 major academic journals were found to have a stated preprint policy [3]. Of the top 100 journals in clinical medicine, 86 permitted preprints, while 13 journals permitted separate screening and one journal rejected preprints [4]. Of the 383 SCIE-listed academic journals in Asia, 30 (7.8%) described preprint policies and 28 (7.3%) received preprint submissions. Of the 76 SCIE journals in Korea, 10 (13.1%) currently describe preprint policies, and 10 (13.1%) receive preprint submissions [5]. As shown by these findings, many Korean editors have not yet introduced preprint policies.
Objectives: This study investigated editors’ and researchers’ experiences with preprints and attitudes towards preprints through a survey questionnaire, with a particular focus on the proportion of editors and researchers with experiences of preprints and their attitudes toward preprints. Differences in responses were analyzed according to respondents’ research field and role in journal publishing. Perceived advantages and disadvantages of preprints were also identified.

Methods

Ethics statement: This survey was anonymous. Individuals’ sensitive information was not included, so the requirement to obtain written informed consent was waived. Subjects were not vulnerable individuals such as students.
Study design: This was a survey-based descriptive study.
Setting: For 12 days from December 30, 2019, to January 10, 2020, a survey was sent through Google Forms to a total of 1,201 people whose e-mails were listed as members of the Korean Council of Science Editors (KCSE) and the Korean Federation of Science and Technology Societies. Email recipients were asked to resend the survey to the members of each society. Therefore, the total number of targeted individuals could not be estimated.
Participants: Participants were editors, society members, or staff of the two organizations. Because the number of respondents could not be estimated in advance, the response rate could not be calculated. Only the response data from 365 respondents were analyzed.
Variables: The survey included two demographic items (academic field and role in journal publishing), six nominal items on experiences with preprints, five 5-point Likert scale ordinal items measuring attitudes towards preprints, and three items on preprint needs. The questionnaire is presented in Suppl. 1. Of the six nominal items, five were included as variables. One item (B4_1) was a supporting item for another (B_4).
Validity and reliability test of the measurement tool: Except for the categories of research field and role in journal publishing, all items elicited information on experiences with or attitudes towards preprints. The item content was discussed with three board members of the KCSE (Kihong Kim from Ajou University, Soo-Young Kim from Hallym University, and Tae-Seol Seo from the Korea Institute of Science and Technology Information). They agreed with the validity of these items for the survey on preprints. Reliability was tested for the five items of 5-point Likert scale using DBSTAT ver. 5.0 (DBSTAT Co., Chuncheon, Korea). The Cronbach alpha coefficient was 0.8576 (df= 360, significance level [α]= 0.05, t= 1.9666).
Data source: Data were responses from participants to the survey form.
Bias: There was no bias in the selection of participants. Participation was strictly voluntary, and that aspect of the sampling was outside the authors’ control.
Study size: This was not an experimental study. The sample size could not be estimated before the survey. For the comparative analysis based on journal category and participants’ role, a post hoc analysis was done. G*Power was used for the post-hoc analysis, using the generic chi-square test [6]. The power (1-β error probability) was 0.985, given α= 0.05, df= 2, and the noncentrality parameter λ= 20.
Statistical methods: A frequency distribution analysis of responses to each item and a comparative analysis of experiences with and attitudes towards preprints were done according to participants’ research fields and roles. DBSTAT ver. 5.0 (DBSTAT Co., Chuncheon, Korea) was used for the statistical analysis. To compare experiences with preprints according to research field, an analysis was conducted using the chi-square test or log-likelihood test (for the item of having heard of preprint deposition by others). For the chi-square test of the item on whether society journals allowed preprint submissions, responses of “do not know” were removed. Attitudes towards preprints according to research field were compared using the Kruskal-Wallis test. The chi-square test was used to compare experiences with preprints according to participants’ role. Because there were few participants outside of the fields of medicine and engineering, participants from other fields were integrated into the natural science group. Therefore, three research fields were used for the comparative analysis. Missing values were excluded from the statistical analysis.

Results

Participants’ characteristics

A total of 365 people responded to the survey. Respondents’ research fields were divided into eight areas: medicine accounted for 202 (55.3%), followed by 117 in engineering (32.1%), 26 in natural sciences (7.1%), eight in agricultural and fisheries research (2.2%), two in humanities, seven in the social sciences, one in artistic and sports research, and two in interdisciplinary science. Regarding roles in journal publishing, there were 118 editors, including editors-in-chief and editorial board members (32.3%), 229 researchers (62.7%), and 18 staff who worked in journal publishing, including manuscript editors, society staff, and company employees (4.9%). The distribution of respondents according to research field and role is available in Suppl. 2. Raw response data from respondents are available in Dataset 1.

Main outcomes

Experience with preprints: The responses to questionnaire items about experiences with preprints were as follows: 243 respondents (66.6%) stated that they were familiar with the concept of preprints, 127 (34.8%) had heard about preprint servers such as arXiv or bioRXiv, 56 (15.3%) had deposited a manuscript on preprint server, 49 (13.4%) said that their society journals allowed preprint submissions, and 122 (33.4%) had heard of preprint deposition by other researchers (Fig. 1). Of the 138 respondents who stated that their society journals did not allow submissions of preprints, 10 (7.3%) said that their society journals planned to allow authors to submit preprints.
The chi-square test showed associations between research field and the three items of experience with preprints. Respondents from engineering and other fields showed significantly higher responses than those from medicine for three items: the concept of preprints (P= 0.0001), having heard of preprint servers (P < 0.0001), and having heard of preprint deposition by other researchers (P= 0.0195). However, no significant association was found between research field and deposition of manuscripts to preprint servers (P=0.6761); and society journals’ allowance of preprint submissions (P=0.2912).
The chi-square test showed associations between participants’ role and three items on experience with preprints. More editors and staff were familiar with the concept of preprints than researchers (P = 0.0022). Likewise, more editors and staff than researchers had heard about preprint servers (P = 0.0309). More researchers than editors and staff stated that their society journals allowed preprint submissions (P = 0.0004). However, no significant association was found between participants’ role and the deposition of manuscripts to preprint servers (P= 0.1704). The same was found for having heard about preprint deposition by other researchers (P= 0.5603).
Attitudes towards preprints: Five items identified attitudes towards preprints. Respondents showed markedly more positive attitudes towards three items (prioritizing preprint deposition [62.7%], promotion of open access [63.6%], and counting preprints as evidence of research work [60.1%]) than towards two other items (rapid comments from researchers [51.5%] and earlier citations than non-preprint articles [53.2%]) (Fig. 2).
The Kruskal-Wallis test showed no association between research field and prioritizing preprint deposition (P= 0.6409), promotion of open access (P = 0.994); and earlier citations than non-preprint articles (P= 0.547). However, respondents in engineering and medicine showed more positive attitudes towards rapid comments from researchers (P = 0.0060) and preprints counting as evidence of research work (P= 0.0057).
The Kruskal-Wallis test showed no significant difference in the median values of the following three items dealing with attitudes towards preprints according to participants’ role: prioritizing preprint deposition (P= 0.2351), promotion of open access (P= 0.0994), and earlier citations than non-preprint articles (P= 0.0547). However, researchers and staff had more favorable attitudes towards researchers’ rapid comments than editors (P= 0.0060), and researchers and staff had more favorable attitudes towards preprints counting as evidence of research work than editors (P= 0.0057).
The necessity of accepting preprints in journal publishing: Of the 365 respondents, 230 persons (63.8%) responded “yes” regarding the necessity of accepting preprints in journal publishing, while 132 (36.2%) replied “no.” No significant association was found according to respondents’ research field (P=0.7760) (Fig. 3). A higher proportion of researchers than editors or staff responded positively towards the necessity of accepting preprints (P=0.0012) (Fig. 4). The reasons given were as follows: “I have already acknowledged the need for preprints” (n=102) and “I can acknowledge the need for preprints during this survey” (n=137).
Out of 132 responders who said that preprints were not acceptable, 73 provided reasons for not accepting preprints (Dataset 2). The responses were classified into the following five categories: scientific integrity (31), stealing ideas/scooping data (19), priority of ideas (11), copyright issues (6), and others (12). The most frequent concern was scientific integrity. These respondents worried about manuscripts not undergoing peer review. In the physical sciences, incorrect results may not be harmful to people’s health. However, in the medical field, information from clinical trials without peer review may be harmful if the information is not sound. The second concern was stealing ideas or scooping data. The third was disputes regarding the priority of ideas. If an author insists on the priority of an idea after depositing a preprint without concrete evidence, it might hinder the publication of data with the same idea. The fourth issue was copyright problems. Insisting on copyright over preprint content may cause difficulties for other researchers who deal with the same content. Other opinions were as follows: confusion in citing work, causing trouble in the publishing ecosystem, difficulty in peer review, duplicate publications, no change or information on journals’ preprint policies in Korea, and preprints not being necessary due to online publishing.

Discussion

Key results: Fewer than half of participants had experience with preprints, except for familiarity with the concept of preprints. There was a low acceptance of preprint submissions by society journals (13.4%). Differences were found according to the research field for three items regarding experience: the concept of preprints, having heard about preprint servers, and having heard about preprint deposition by others. More participants from engineering and other fields had experience with these items than those from medicine. More editors and staff than researchers were familiar with the concept of preprints and had heard about preprint servers. More than half of the participants showed a positive response to all five items on attitudes towards preprints. Researchers and staff had more favorable attitudes than editors towards preprints receiving immediate comments and counting as evidence of research work. Editors were more likely to be against accepting preprints than researchers and staff.
Interpretation: The above results may provide basic data on experiences with and attitudes towards preprints among editors, researchers, and staff who work in journal publishing. However, it is difficult to conclude from the above results whether participants’ level of experience and attitudes were high or low because no comparative studies have been conducted in other countries. The finding that participants from engineering and other fields had more experience with preprints than those from medicine can be explained by the active use of preprint servers in the physical sciences, including physics, chemistry, and engineering. In the medical field, preprint servers are still not popular, especially among the society journals in Korea [5]. MedRXiv (https://www.medrxiv.org/) was launched in June 2019 and bioRXiv (https://www.biorxiv.org/) began in November 2013, whereas arXiv (https://arxiv.org/) started in August 1991.
Researchers in the engineering and natural science fields are believed to have already encountered preprints, although no significant difference was found in the proportion of researchers who had deposited manuscripts to preprint servers. More editors and staff than researchers were familiar with the concept of preprints and preprint servers. This may have resulted from their attendance at the editors’ workshop held by the KCSE. The KCSE recently presented a training course on preprints in January 2020. Furthermore, preprints have been introduced through the official journal of the KCSE, Science Editing, which is distributed to member editors.
Participants’ attitudes towards preprints were more favorable than the levels of experience with preprints. An interesting question is why researchers and staff had more favorable attitudes than editors towards preprints enabling colleague researchers’ immediate comments and towards preprints counting as evidence of research work. This finding is difficult to interpret, and it may have been just a phenomenon. Out of 365 participants, 132 (36.2%) did not want to accept the submission of preprints, and editors were more reluctant to accept preprints as a matter of policy. Out of the 117 editors, 58 (50.0%) did not express favorable attitudes towards accepting preprints, and this might be the key finding of this survey. Journal policies usually depend on the editor’s decisions. It is challenging to adopt a preprint policy that accepts preprint submissions. Many reasons for non-acceptance were presented. In particular, out of 74 medical editors, 40 (54.1%) were against accepting submissions of preprints. Many concerns were raised about preprints, primarily including scientific integrity, stealing and scooping ideas, and priority of ideas.
Comparison with previous studies: Chiarelli et al. [2] examined the behavior of researchers, including various perspectives, in interviews based on a sampling approach between Octobor 2018 and January 2019. That study conducted detailed interviews with 38 stakeholders in the United States. In their work, they agreed with the broad definition that preprints are the result of peer review and research that can be submitted before publication. However, significant disagreement was found regarding the details of preprints’ meaning and effectiveness. Since the study of Klebel et al. was a qualitative study based on structured interviews, it is challenging to compare their findings to those of the present study. According to Klebel et al., the main concerns were related to the lack of quality assurance and the “Ingelfinger rule,” which prohibits the earlier announcement of article content in mass media before publication in a journal. A common concern identified by Klebel et al. and in the present study was scientific integrity, or the quality of content. Many editors surveyed in this study worried about the scientific integrity of preprints.
It is not easy to find other country-level survey studies on preprints in the literature databases, including PubMed and the Web of Science Core Collection. A survey was conducted by 80 Sanofi-Synthelabo researchers on preprints and eprints. The researchers regarded preprints or e-prints as unreliable, although they recognized the advantage of being able to modify their work through depositing it to preprint or eprint servers [7]. Soderberg et al. [8] surveyed 3,759 researchers across a wide range of disciplines in 2019 and showed that 69.8% of them had favorable attitudes towards preprints. However, 15.2% were opposed to preprints. Only 51% of respondents from the medical field were in favor of the use of preprints. Out of 3,759 respondents, 29.85% had submitted a preprint at least once. That survey primarily dealt with the use of cues on preprint services, but some basic data are comparable with the present survey. In the present study, 15.3% of the participants had deposited a manuscript to a preprint server, and more than 50% of the respondents showed favorable attitudes toward preprints. Thus, Korean researchers’ and editors’ experiences of depositing manuscripts to the preprint server are relatively inactive compared to researchers elsewhere in the world.
Penfold and Polka [9] also mentioned “getting scooped” as a reason for not accepting preprints. This opinion was also verified in the present survey as a major issue.
Limitation: This study was conducted to investigate the current perceptions of Korean editors and researchers regarding preprints. However, it was based on voluntary participation, not a random sampling, which may have adequately represented the whole population. Among the respondents, those in the medical field accounted for 55.3%, meaning that the findings predominantly reflect the opinions of researchers in the medical field. In the future, it is necessary to conduct a study with random sampling.
Suggestion: It is not mandatory for all journals to accept preprints—that is, depending on the journal’s policy, editors may or may not receive preprint submissions. Large international commercial publishers usually allow preprint submissions and the citation of preprints, so it is time to think about what policies are needed in society journals in Korea. Editors should create a written preprint policy stating whether preprints are reviewed or rejected and whether preprints can or cannot be cited.
Generalizability: The results represent the opinions of Korean editors and researchers; as such, the findings are not expected to be generalizable to editors and researchers from other countries.
Conclusion: Only 15.3% of respondents reported having published preprints, which means that they are relatively inexperienced with preprints. Half of the editors had negative views about introducing preprinted policies. Respondents who were against the need for preprints were most concerned about scientific integrity, the theft of research ideas, or the possibility of scooping data. It is necessary to hold discussions at the editor’s council level on how to address these concerns when introducing a preprint policy.

Conflict of Interest

Sun Huh has been the President of the Korean Council of Science Editors since January 2020, but has no role in the decision to publish this article. No potential conflict of interest relevant to this article was reported. This work was done as a research project of the Korean Council of Science Editors. However, this article’s opinion is not the Korean Council of Science Editors’ official opinion, but that of the authors.

Notes

Funding

The authors received no financial support for this study.

Notes

Data Availability

Dataset file is available from: the Harvard Dataverse at: https://doi.org/10.7910/DVN/KQLZNV.

Dataset 1. Raw response data to survey questionnaire from participants

Dataset 2. Raw response data on the reasons for not accepting preprints and the categorization of responses

Supplementary Material

Supplementary file is available from: the Harvard Dataverse at: https://doi.org/10.7910/DVN/KQLZNV

Suppl. 1.

The survey questionnaire including 16 items eliciting information on Korean editors and researchers’ experiences with preprints and attitudes towards preprint policies
kcse-223-suppl1.txt

Suppl. 2.

Distribution of respondents according to the research field and their roles in journal publishing
kcse-223-suppl2.xlsx

Responses to five items on experiences with preprints from 365 editors and researchers in Korea.
kcse-223f1.jpg

Fig. 1.

Responses to five items on attitudes towards preprint policy from 365 editors and researchers in Korea.
kcse-223f2.jpg

Fig. 2.

Responses to the necessity of accepting preprint submissions according to the research field of 365 editors and researchers in Korea.
kcse-223f3.jpg

Fig. 3.

Responses to the necessity of accepting preprint submissions according to the respondent’s role in journal publishing among 365 editors and researchers in Korea.
kcse-223f4.jpg

Fig. 4.

References

1. Johnson R, Watkinson A, Mabe M. The STM report: an overview of scientific and scholarly publishing [Internet]. The Hague: STM; 2018 [cited 2020 Jul 1]. Available from: https://www.stm-assoc.org/2018_10_04_STM_Report_2018.pdf


2. Chiarelli A, Johnson R, Pinfield S, Richens E. Preprints and scholarly communication: an exploratory qualitative study of adoption, practices, drivers and barriers [version 2; peer review: 3 approved, 1 approved with reservations]. F1000Res [Preprint] 2019;8:971. [cited 2020 Jul 1]. Available from: https://doi.org/10.12688/f1000research.19619.2
crossref

3. Klebel T, Reichmann S, Polka J, et al. Peer review and preprint policies are unclear at most major journals. PLoS One 2020;15:e0239518. https://doi.org/10.1371/journal.pone.0239518
crossref pmid pmc

4. Massey DS, Opare MA, Wallach JD, Ross JS, Krumholz HM. Assessment of preprint policies of top-ranked clinical journals. JAMA Netw Open 2020;3:e2011127. https://doi.org/10.1001/jamanetworkopen.2020.11127
crossref pmid pmc

5. Choi YJ, Choi HW, Kim S. Preprint acceptance policies of Asian academic society journals in 2020. Sci Ed 2021;8:10–17. https://doi.org/10.6087/kcse.224
crossref

6. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175–91. https://doi.org/10.3758/bf03193146
crossref pmid

7. Miranda GF, Ginestet J. The attitude of pharmaceutical industry research scientists to browsing and publishing on internet preprint and e-print servers. Ther Innov Regul Sci 2002;36:831–7. https://doi.org/10.1177/009286150203600412
crossref

8. Soderberg CK, Errington TM, Nosek BA. Credibility of preprints: an interdisciplinary survey of researchers. R Soc Open Sci 2020;7:201520. https://doi.org/10.1098/rsos.201520
crossref pmid pmc

9. Penfold NC, Polka JK. Technical and social issues influencing the adoption of preprints in the life sciences. PLoS Genet 2020;16:e1008565. https://doi.org/10.1371/journal.pgen.1008565
crossref pmid pmc

Editorial Office
The Korea Science & Technology Center 2nd floor,
22 Teheran-ro 7-gil, Gangnam-gu, Seoul 06130, Korea
TEL : +82-2-3420-1390   FAX : +82-2-563-4931   E-mail : kcse@kcse.org
Copyright © Korean Council of Science Editors. All rights reserved.           Developed in M2PI