Predatory publishing in Scopus: evidence on cross-country differences

Following a pressure by publisher Frontiers and biased post-publication peer review, Scientometrics has retracted my paper written jointly with Vít Macháček on „Predatory publishing in Scopus: evidence on cross-country differences“ that has been published in this journal, one of the leading scientific journals in the field, earlier this year. I strongly object against the retraction, because it is based on false accusations, invalid arguments and most importantly because no convincing evidence on a serious problem that undermines the main findings of the paper has been presented to me. In my view, the retraction is driven by business and political, not academic, interests, does not meet generally accepted guidelines for a retraction and cannot withstand an independent enquiry. Scientometrics has seriously mishandled this case.

The main purpose of this blog is threefold. First, this post gives me an opportunity to present the facts and my side of the story in order to prevent speculations on what has really happened and others from spinning the case in whatever direction they choose. Not commenting on it could be interpreted as hiding from a public scrutiny and admitting guilt, which could not be further from the truth. Second, an independent review of this case will be initiated under the framework of the Committee on Publication Ethics (COPE) sometimes soon, during which I will have to remain completely silent about it, hence this could be my last chance to speak for quite some time. Third, I must admit that writing this post has a therapeutic value for me. I have never been under so much stress in my work life before as due to this blunder lately. Spilling it out here will hopefully help me to sleep a little bit more peacefully in the coming weeks.

So here are the hard facts on how the retraction has unfolded on a timeline, which should be more than sufficient for you, if you are willing to invest a little bit of your time in reading it, to make up your own mind about this matter (you are strongly encouraged to read the supporting documents that are available for download under the links below, especially if something remains unclear, as these contain essential details, present the arguments and explain the context):

• 6th May, 2021 – Dr. Frederick Fenter, the Chief Executive Editor of publisher Frontiers, sends a letter to Prof. Wolfgang Glänzel, the Editor-in-Chief of Scientometrics, requesting that our article “must be immediately retracted” (available for download under the “obtained by Retraction Watch” link here).
• 12th May 2021 - The Editor-in-Chief of Scientometrics forwards the letter to us, informs us that an investigation has been initiated and that “the basic concern regards the use of “Beall’s List” as data source.”He asks us to respond in one week.
• 19th May 2021 - We (me and my co-author) send a detailed response to the letter by Frontiers, in which we argue that “Beall’s lists have become the dominant source of data in this line of research” and express our hope that “Scientometrics would protect this line of scientific inquiry and fend off this attack once and for all” (available for download under the “May 19 letter” link here).
• 2nd July 2021 - The Editor-in-Chief of Scientometrics informs us that they “have received two post-publication peer reviews” of our article, which he is “summarizing to us” (we have never seen their complete texts). He gives us a deadline of two weeks to respond.
• 12th July 2021 - We send a detailed response, which carefully addresses comments raised in the post-publication peer review point by point. We express a concern that the post-publication peer review is unusually aggressive, biased and does not provide fair assessment of our article (available for download, including the reviewer’s points, under the “responded to each criticism” link here).
• 17th August 2021 - The Editor-in-Chief of Scientometrics informs us that they have decided to retract our article and that “the decision has been taken on the basis of several post-publication peer reviews”. No reflection on our response to the reviewer’s points has been provided to us. We only got seven days to respond whether we agree with the retraction or not.
• 23rd August 2021 – We respond that we do not agree to this retraction, because it is “driven by political, not academic, interests”, because it is unjustified and that we request an independent re-assessment.
• 23rd August 2021 – We send a letter to members of the Distinguished Reviewers Board of Scientometrics (and other relevant stakeholders) – in total 68 people – who we perceive as an inner circle of the journal, in which we explain the case, express our dissent with the retraction and request an inquiry (see the letter here).
• 6th September 2021 – Retraction note tagged to our article is published online in Scientometrics, by which the retraction becomes effective and public. The note claims that the results of our paper are unreliable due to a lack of a control group and languages (see the note here).
• 6th September 2021 – Retraction Watch informs us that they are writing a post on the retraction and asks us for a comment. I reply the next day and the post is published shortly afterwards. Note that the post makes available the key background documents linked above that we have shared with the Reviewers Board on 23rd August 2021, but which we did not provide to the Retraction Watch by ourselves (see the post here).
• 8th September 2021 – We send a second letter to the members of the Distinguished Reviewers Board of Scientometrics (and other relevant stakeholders) informing them that the retraction note is published, in which we reiterated our dismay with the decision and asked them for a formal statement (see the letter here).
• 21st September 2021 – We received a letter of support signed by 36 members of the Distinguished Reviewers Board and the Price Medal Laureates Board of Scientometrics, which states that they disagree with the retraction of our paper, because there are doubts whether ‘due process’ has been strictly adhered to and because they believe there are no valid grounds for retracting our paper. Consequently, they advise us to appeal to the Facilitationand Integrity Subcommittee of COPE (we do not have the liberty to make this letter publicly available at this point).

For the sake of completeness, it should be noted that we also exchanged several short emails with the Editor-in-Chief of Scientometrics between 30th August and 8th September, in which we enquired about the permission to reprint our paper elsewhere, called for publishing the background documentation for the sake of transparency as an attachment to the retraction note and whether there is an internal appeal procedure that we can formally invoke to challenge the retraction, which however did not lead anywhere, thus they are not listed separately on the timeline above. In addition, we communicated with colleagues and experts in the field about it privately and there was an attempt for mediation between us and the journal organized by one of the top senior professors in the field, both of which, of course, must remain strictly confidential, but let me use this opportunity to express our utmost gratitude for their efforts and support.

In my own reading of what most likely happened is that Springer Nature – the publisher of Scientometrics - has made a cynical calculation that giving in to the pressure by Frontiers would be less expensive for them than protecting contributors to their journal. The Editor-in-Chief of Scientometrics has decided to play along with this, thus doing what is best for the publisher, not the journal or the research community. Other explanations than business interests – ownership connections between Frontiers and Springer Nature and/or possibly legal threats by Frontiers - make little sense because no credible academic reasons to retract our paper have been presented to me. In fact, that Frontiers and Springer Nature are connected by ownership, which is no secret, represents a potential conflict of interest for processing this case that should have been declared in the first place but was not at any point. Note that not declaring a conflict of interest of this kind is actually one of the main reasons for retracting a published paper. Should not this alone be a reason for retracting the retraction?

The retraction note is outrageous in itself. You are strongly recommended to read it – only two short paragraphs - to realized how profoundly this has been messed up. Even if some of our findings were unreliable and statements problematic, as claimed but disputed by us, this could be fixed by issuing a mere correction, not retracting the whole paper. The two key arguments about a lack of a control group and languages are confused and invalid. The control group point has been put forward by someone who does not understand comparative research at the country level and regression analysis in social sciences. The absurdity of the argument about languages is clearly explained by the comment made by “Zoë the Scribe” (September 8, 2021 at 4:12 am) to the RetractionWatch post, so let me just refer you to that. Hence, the justification of the retraction cannot survive a scrutiny by an independent expert knowledgeable about the subject matter, who actually understands the paper. It is hard to believe that a journal of this calibre puts its reputation on a line by publishing something like this.

The letter by the Chief Executive Editor of publisher Frontiers that initiated the retraction contains a litany of allegations and insults (available for download under the “obtained by Retraction Watch” link here). As already noted above, his only substantial argument is the claim that Beall’s lists are not legitimate sources of data for scientific inquiry. Our response letter is therefore focused on addressing this claim (available for download under the “May 19 letter” link here). Essentially, we point to the fact that Beall’s lists have been the dominant source of data in empirical inquiry on predatory publishing and that they have been used in a number of published papers, including in top journals like Science, Research Policy, Journal of Informetric and at least three other papers in Scientometrics (see a full list of them in our response letter; one more paper that is quite similar to ours has come out in Scientometrics in the meantime). Hence, we did not use some new, fringe and untested dataset, but only followed the suit of a long line of published papers that used Beall’s lists in essentially the same way as we did by ourselves. Why using this data source should suddenly become a reason for retracting a published paper?

Nevertheless, I agree that Beall's lists need to be used with caution, which we actually do in our paper. We acknowledge a number of limitations of this data and try to do something about those that we can address by ourselves. We even directly test for the possibility that Frontiers does not fit in given the evidence that we have in hand. Ironically, when we presented a draft version of our paper at the 17th International Conference on Scientometrics and Infometrics (ISSI) 2019 in Rome, we were asked publicly by one of the participants whether we have any affiliation to Frontiers, apparently because of a suspicion that we are overly favourable to this publisher in our analysis. As the result, we added to the published paper Footnote 16 that rules out any connection of us to them. Beall’s lists have become outdated and perhaps should not be used in future research anymore. But I do not believe that published research using Beall's lists that has been subjected to rigorous pre-publication peer review should be retracted. Scientometrics should have issued an editorial stating its reservations to Beall’s list, if indeed this is their view on this matter, not retract our paper. Science does not progress one retraction at a time. Or does it from now on?

For me, the most appalling part of the whole story is how unfairly we have been treated during the post-publication peer review and that the journal did not pay any attention to our response (available for download, including the reviewer’s points, under the “responded to each criticism” link here). If this review was supposed to be a 'fair trial', we deserved a 'due process', in which we were treated as 'innocent' until proved otherwise. But the journal did not confront us with convincing evidence that we were guilty of any wrongdoing. The findings of our paper do not become unreliable just because somebody claims so. Only asserting that is not sufficient to justify a retraction. The journal should have provided direct evidence that proves beyond reasonable doubt that our findings are unreliable. If the journal deems that there is a problem because our analysis did not include a control group (whatever this means in the context of our paper), they should have showed to us how results using such control group would lead to significantly different conclusions. If the journal claims that our results are inconclusive due to some sort of problem with languages, they should have showed to us how a correct approach would provide different results, thus demonstrating our error. But they did not bother.

After a submission for publication a journal editor has the authority to desk-reject any paper that she/he does not deem worthy of further scrutiny. But after a paper goes through a proper pre-publication peer-review and gets published, the journal editor cannot decide to retract the paper merely based on opinions, speculations and claims on what could possibly be wrong about it or just because it 'feels good' to do so. According to COPE retraction guidelines, to which Scientometrics has signed up, journal editors should consider retracting a publication in several instances, including: “They have clear evidence that the findings are unreliable, either as a result of major error (eg, miscalculation or experimental error), or as a result of fabrication (eg, of data) or falsification (eg, image manipulation)”; the other instances are plagiarism, copyright infringement or a failure to disclose a major conflict of interest, for example. None of these legitimate reasons for retraction hold here. Our paper therefore does not meet COPE’s guidelines for retraction.

Moreover, besides the actual content, there are some worrying procedural inconsistencies in the post-publication peer review process that also deserve mentioning here. On 2nd July 2021, the Editor-in-Chief of Scientometrics wrote to us that there were “two post-publication peer reviews”, but then on 17th August 2021 he already referred to “several post-publication peer reviews”. However, we have not seen and did not have a chance to respond to any of these additional reviews that were apparently obtained in the meantime, not even mentioning the fact, as already noted above, that we never received a reply to our detailed response to the initial two reviews. As if the journal editor did not really care about our view. As if it did not matter for the retraction decision to get to the bottom of the problem.

It is no secret that journal editors (or evaluators of any kind by that matter) have a big sway in deciding on fate of a paper by choosing the peer reviewers. Of course, we will never know who were the post-publication peer reviewers, because their identity will remain confidential for obvious reasons. However, one does not need to know the specific names in order to sense a foul play in this regard, because the comments are sometimes more than enough to judge. Because we had a very bad feeling about it, we wrote in the introductory part of the response letter to the post-publication reviewer’s comments the following: “The post-publication peer-reviewers seem to be strongly critical about the use of Beall’s lists and hence also inevitably about the evidence presented in our paper – one can even say that their comments are unusually aggressive. But this is not representative to the overall stance of the scientific community to this matter. It would be fair to consult reviewers who are ex-ante neutral, who do not have predetermined opinions in this regard, who in particular did not publish papers critical of Beall’s lists by themselves in the past, and who could therefore provide assessment of the content and relevance of our paper that is representative and free of emotions.” (available for download under the “responded to each criticism” link here).

Some comments below the Retraction Watch post suspect one particular person (see the comment by Kyle Siler, September 7, 2021 at 12:50 pm). On the one hand I hesitate to propagate this speculation here, but on the other hand the possibility that it was actually him is crucial to reflect on in order to understand this case. So let me not reproduce his name, but let me give you my honest opinion about it. I have to admit that it also crossed my mind immediately after reading the post-publication review that this could have been written by that person, because in his papers he has displayed a rather unique language style of demeaning work of other academics who happen to hold a different view from him and because he is an outspoken critic of Beall’s lists, on the topic of which he has published a number of articles. And this is exactly why, we wrote in the response to the peer review the lines that are cited in the previous paragraph. But regardless of whether it was him personally, my best guess of the odds of which is fifty-fifty, it surely was somebody with a very similar profile, mindset and thus expected outcome. If this was indeed the case, the post-publication peer review was a premeditated murder of our paper.

Sadly, hence, I cannot help thinking that the only purpose of the post-publication peer review was to produce a pretext for the retraction. Springer Nature has probably made a decision to retract our paper already soon after the initial request by Frontiers has arrived and gave clear instructions to the Editor-in-Chief of Scientometrics to execute it. From then on they were only looking for a hammer to nail it. At first, they perhaps thought that they can get it over with by calling Beall’s lists an illegitimate source of data and retracting our paper on the base of that. After receiving our response on 19th May 20, however, they realized that if they were to justify the retraction this way, they could create a precedent, on the base of which they might have to retract several other papers published in Scientometrics alone (and possibly other journals); hence triggering a domino effect with a potential to stir up a wave of bad press coming back at them. Instead, therefore, they decided to orchestrate the post-publication peer review to come up with other excuses. Even though they did not really find that there is anything seriously wrong with our paper, they could not care less and went ahead with the retraction anyway, thinking that they are so powerful that they can do whatever they want with impunity.

If the post-publication peer review was organized in a good faith with the aim to find the truth and make a balanced decision, why not have a second more careful look at it, if the authors repeatedly alert the journal to the fact that it is biased, incompetent and untrue and that they make it adamantly clear that they disagree with the retraction? Why didn’t they tap at that point into the immense intellectual capital of the Distinguished Reviewers Board of Scientometrics for counsel (which they apparently did not given what we know now)? Why didn’t they do that even after we explicitly called for it in our first letter to the board? Why didn’t they provide reply to our detailed response to the reviewer’s points and wait for our reaction before the retraction note was published online? Why didn’t they discuss and explain their arguments to us but steamed ahead with the retraction without any further deliberation? Could it be that they did not allow anybody at least partly impartial to look into it simply because the ugly truth about why, how and by whom this has been done would easily come out? Will the truth ever come out?

Sacrificing our paper and its two co-authors as scapegoats to appease business interests is reprehensible. I work as a Deputy Director for Research at one of the top research institutes in the field of Economics in Europe. I have published more than thirty papers in respectable international scientific journals, including top field journals. My research integrity has never been questioned. I was never investigated for any misconduct or wrongdoing before. But this retraction shatters my reputation. Essentially, for all practical purposes of research evaluation this marks me as a convicted academic criminal. As the result, my prospects for acquiring highly competitive research funding or academic positions are bleak. For my co-author Vít Macháček, a Ph.D. student for whom this is his first paper ever published, the retraction could turn out to have disastrous consequences for his research career. Retracting a paper is no joking matter; there must be grave reasons, serious misconduct or error proven, which was not the case here. For everybody out there the scary part is that if this could happen to us, it can happen to anyone!

As noted above, the letter of support for us is signed by 36 members of the Distinguished Reviewers Board and the Price Medal Laureates Board of Scientometrics, which is about half of them. Although we do not have the liberty to reveal the full content of the letter and its signatories individually at this point, by implication one can easily guess that the list contains the best experts in the field, including top professors. In fact, their combined number of publications and citations recorded in Web of Science is 3,189 and 111,780, respectively (as of September 26, 2021), which admittedly is a formidable academic firepower! And they clearly express their dissent with the retraction and doubt that we have been treated fairly. Note that signing such letter must not have been easy for anybody, so there are probably many more who think the same but did not decide to come forward for various reasons. Of course, there is a number of people outside of this relatively narrow group who are equally angry over the retraction. We feel immense support in the research community and we are very grateful for that!

So what happens next? We may have lost a battle but not the war just yet. We will keep on fighting to clear our names. Even retraction can be retracted and we will do whatever we can to that aim. As suggested to us by the letter of support, our next immediate step is to submit an appeal to the Facilitationand Integrity Subcommittee of COPE for an independent ruling, which should be accepted by both Frontiers and Springer Nature, as they are institutional members of this organization. I will be updating this post below, if new significant development arises, but not before the COPE review is finished, because after it is initiated I am required not to discuss the case publicly, while the review is ongoing. By corollary, I apologize to journalists and others who might be tempted to ask me for my view in the meantime, as I will have no other option than send back a “no comment” reply. I am confident that we have a strong case for a successful appeal. Yet I am also well-aware of the fact that we take on one of the largest and most powerful publishers of scientific literature in the world. Cross your fingers for us!

Autor: Martin Srholec

Text vyšel na autorově blogu