Compliance of “Principles of transparency and best practice in scholarly publishing” in academic society published journals

Data collection

To investigate how the Principles are applied in academic society journals, a list of Science Citation Index Expanded journals was downloaded from the Clarivate Analytics website in July 2018. Among 9,058 journals, 984 that had “ASSOC,” “SOC,” or “SOC” in their address information were extracted. Furthermore, because the researchers could manually distinguish the journals from South Korea, 104 South Korean journals were collected. Duplicated data from all extracted data were excluded. Thereafter, 65 journals that were not offered in English, 135 that were using platforms from commercial publishers, and 41 with access problems were also excluded. Finally, a total of 781 journals were selected and manually combed for adherence to the Principles. All the journal’s websites were accessed between October 1 and November 26, 2018 by the three researchers.

To increase the reliability of data collection, the researchers collected data from 30 of the same websites as a beta-test and came to a consensus regarding the items that they found difficult to measure before starting actual data collection. Then, to ensure that the researchers conducted the data collection with consent, 10% of the data (80 journals’ homepages) were randomly selected and checked again by another researcher so that the journals were examined under the same standard.

Data analysis

Based on collected data, the 16 best practices were divided into four different categories: basic journal information; publication ethics information; copyright and archiving information; and profit model information, as shown in Table 2 [4]. In the process of checking each practice, the 16 practices were specified with sub-items, and a total of 33 items were checked manually through journal websites. The arrangement of items was changed during this process. The definitions were drawn directly from the “Principles of transparency and best practice in scholarly publishing” [4] to elaborate on each specific sub-item in categories.

Table 2.

Sixteen original principles sub-divided into 33 items and categorized into four different subjects [4]

In addition, to evaluate each sub-item with a yes response, the responses from 781 journals were divided into four scales: 0% to 25% for rarely practiced, 26% to 50% for poorly practiced, 51% to 75% for adequately practiced, and 76% to 100% for well practiced.

Moreover, to account for regional distribution, countries were categorized into one of six continents (Africa, Asia, Europe, North America, South America, and Oceania) using address information from the journal list (Table 3).

Table 3.

Based on each journal’s address, collected data were categorized into one of six continents

Basic journal information

Among those 14 specific items of basic journal information, 11 items were well or adequately practiced on the websites, as shown in Fig. 1. Readership and author fees were classified as poorly practiced items because only 50% or fewer of the journals stated the items on their homepages.

Fig. 1.

Fourteen items in the basic journal information category (X-axis) were searched on 781 journals’ websites for whether the items were indicated clearly and counted as “yes” or “no." ISSN, International Standard Serial Number.

In the case of methods of peer review, specific data were collected as shown in Table 4 (Dataset 1). Among 781 journals, 185 stated their detailed review process. The most common type was single-blind, followed by double-blind. However, 596 journals did not provide specific methods of peer review.

Table 4.

Specific types of peer review methods as indicated on 781 journals’ websites

Based on a continent distribution analysis, two poorly practiced items were analyzed in Table 5. The readership item was indicated by only 33% of total journals. More of the journals in Europe and Asia did not follow the readership item than in North America.

Table 5.

Two poorly practiced items in basic journal information

Publication ethics information

Among the ten items, six were found to be adequately practiced (Fig. 2). In the publication ethics information category, there was no item that was well practiced. In particular, COPE’s guideline; data sharing and reproducibility; and post-publication discussion were poorly practiced, with only 26% to 50% of journals indicating these items on their websites.

Fig. 2.

Ten items in the publication ethics information category (X-axis) were searched on 781 journals’ websites for whether the items were indicated clearly and counted as “yes” or “no.” COPE, Committee on Publication Ethics.

Three poorly practiced items were analyzed for continent distribution analysis, as shown in Table 6. The findings showed that COPE’s guideline; data sharing and reproducibility; and post-publication discussion were not clearly indicated on a large number of journal homepages. Data sharing and reproducibility was indicated among only 8% of Asian journals but 36% of North American journals. Post-publication discussion was indicated among 6% of Asian journals compared to 61% of European journals.

Table 6.

Three poorly practiced items in publication ethics information

Copyright and archiving information

Among the five items collected, copyright and licensing information was proven to be well practiced, and Creative Commons and open-access items were adequately practiced (Fig. 3). Policies on the posting of accepted articles with third parties, subscription, and archiving were classified as poorly practiced. Number of journals which allowed open access to the electronic edition and required a subscription for the print edition was 288.

Fig. 3.

Five items in the copyright and archiving information category (X-axis) were searched on 781 journals’ websites for whether the items were indicated clearly and counted as “yes” or “no.”

Two poorly practiced items were analyzed by continent distribution, as shown in Table 7. Policies on the posting of accepted articles with third parties and archiving were mostly found to be not clearly indicated in this category. The “yes” percentage in Asia, especially for policies on the posting of accepted articles on the third party, was comparatively lower than for other continents.

Table 7.

Two poorly practiced items in copyright and archiving information

Profit model information

Among the three items in this category, all were classified as poorly or rarely practiced (Fig. 4). In particular, revenue sources were rarely practiced. No more than 20% of journals indicated this information on their homepage. The profit model information category can be characterized as a poorly practiced category overall.

Fig. 4.

Three items in the profit model information category (X-axis) were searched on 781 journals’ websites for whether the items were indicated clearly and counted as “yes” or “no.”

This category was poorly or rarely practiced overall, as shown in Table 8. Two items, advertising and direct marketing, were poorly practiced items, and the revenue source item was seldom practiced.

Table 8.

Three poorly or rarely practiced items in profit model information

Geographic variation in compliance

European journals were complying well with most best-practice guidelines compared with their Asian and North American counterparts. In particular, Asian journals rarely practiced “data sharing and reproducibility,” “policies on posting accepted articles on third-party websites” and “post-publication discussion.” “Archiving” were mostly found to be not indicated in North America. Especially for this item, the “yes” percentage was higher among Asian than North American journals. Considering the growing number of research funders who are asking to mandate open access policies in European countries, the perception of such policies is less favorable in Asian countries. Regardless of continents, all items from the profit model information category were poorly or rarely practiced.

Basic journal information

In the basic journal information category, the readership item was most significantly poorly practiced by journals. Academic publishing depends greatly on trust because authors and readers believe that editors will conduct a fair peer-review process as per the information provided on the journal homepage. The items, not only “readership” but also “no guarantee of manuscript acceptance,” are easily revised on the journal homepage. These items could be the first improvement target to meet the best-practice guidelines.

Good management of these contents will foster a sustainable publishing infrastructure, which will benefit all stakeholders in the publishing process [5]. Because the most critical journal information is described in this section, journal editors should pay close attention to implementing these basic journal information items well.

Publication ethics information

The proportion of poorly practiced items is higher in the publication ethics category compared to basic journal information. In particular, “data sharing and reproducibility” and “post-publication discussion” are rarely practiced in Asia compared to Western countries. A great deal of research has been conducted on data sharing from the perspective of researchers; however, few studies have been performed on compliance with this policy from the perspective of journal editors. According to a recent study by Kim et al. [6] in a survey of 100 Korean scholarly journals, only 13 stated that they had already adopted a data sharing policy.

Data-sharing policies may differ according to the journal’s situation. Such a policy can have a positive effect because it enables more in-depth research using already existing data. PLOS has explicitly stated that data must be made publicly available, and rare exceptions must be agreed with the editor [7]. Vines et al. [8] proved that journals with an archiving policy have 25 times higher data availability than those with no such policy. Stating the data sharing policy in the author guidelines makes the authors actively participate in data archiving, which is highly germane to the success of an open data policy. Hence, the detailed adoption level should be announced on the journal homepage.

Copyright and archiving information

Policies on the posting of accepted articles with third parties are the most poorly practiced item in this category, and Asian journals performed comparatively worse than other continents in this regard.

SHERPA/RoMEO [9], which provides a journal deposit policy, could be a good channel to use for the enhancement of policies on the posting of accepted articles on the third party. SHERPA/RoMEO is not yet commonly used in Asian countries; among Asian journals, more attention needs to be paid to how to use the SHERPA/RoMEO site effectively. Our result is similar to that of a study by Kim and Choi [10], who analyzed the digital standards of Asian journals registered in Directory of Open Access Journals, indicating that 85% of the journals (1,689) had no digital deposit policy. Because the deposit policy differs depending on publishers and journals, most authors are confused over how to deposit their articles when the deposit is mandatory. Hence, Asian journal editors need to give more attention to improving this policy.

Profit model information

Revenue sources from the profit model information category should also be improved by the academic society journals. Traditionally, academic societies’ publishing remains the most valued function, and revenue from publishing is usually used to support society members [11]. Business models or revenue sources (e.g., subscriptions, advertising) should be clearly stated on the journal homepage to avoid misleading authors on ethical feasibility. They should be separated from the journal content and the authors and should also not influence the editorial decision. According to a Trakya University analysis, three guidelines (marketing, intellectual property, data sharing) were not met by any 10 journals which were investigated [3]. In particular, because most Asian journals are published by non-profit organizations such as academic societies, journal editors’ awareness of the profit model is low. This result also showed that these business model components should receive more attention compared to other parts because none of the three items are well practiced, regardless of country.

Limitation

The limitation of this study is that the journal homepage was the only part that was checked and examined to collect the data. Even though the study included an “information for authors” section or an author guideline on the homepage, the absence of that information on a journal homepage may not mean that a journal is not following good practice. Additionally, because of the limited resources, only the journals that clearly stated that they were published by a society were considered when collecting the raw data. Further study could be recommended that would also include surveys or interviews with journal editors regarding best practice compliance.

Conclusion

This study was focused only on academic society journals that are registered on the Science Citation Index Expanded. As such, it would not be representative of the overall global peer-reviewed journal status. Because most small-society journals are operated by independent academic societies, society journals have difficulties in competing with the commercial publishers that invest a great deal in enhancing their infrastructure and publishing system. Society journal editors should continuously evaluate their journals regarding the “Principles of transparency and best practice in scholarly publishing” to keep up with a rapidly changing publishing environment. This research would be useful for editors of society journals when they upgrade or renovate their journal homepage to meet the best practice guidelines.

It would also help publication associations to understand how to help small societies meet the global standard. A positive step would be for society journal editors to receive regular training to keep pace with rapidly changing scholarly publication trends. Especially in Asia, the editor is less trained, and difficulties are involved in attaining proper information on this matter. Regional editors’ associations, such as the Korean Council of Science Editors and Council of Asian Science Editors, could be the best channel to provide information on the latest best practice guidelines. Currently, Korean Council of Science Editors in Korea conducts regular editor training on how to apply the guideline effectively. If similar regional editors’ associations can be created in other regions to actively share the latest information, more academic society journals can get the right information to meet the international publishing standards.