November 22, 2022, #22 Serge Horbach, Michael Ochsner, and Wolfgang Kaltenbrunner wrote for a blog from the Centre for Science and Technology Studies (CWTS) at Leiden University: “The idea for this blog post emerged in the context of a special issue with the online journal Frontiers in Research Metrics and Analytics.”¹ The authors admitted to some initial uneasiness because of “previous criticism of Frontiers’ approach to scholarly publishing“.¹ In the end the opportunity outweighed their concerns and they went ahead. Anyone who has been an academic editor knows that managing the peer reviewer process is hard for non-standard topics. Frontiers has its own required process for peer review: “Reviewers are selected by an internal artificial intelligence algorithm on the basis of keywords automatically attributed by the algorithm based on the content of the submitted manuscript and matched with a database of potential reviewers, a technique somewhat similar to the one used for reviewer databases of other big publishers.”¹ The authors discovered that the process resulted in unqualified reviewers, and the fast-pace of the standard review process meant that reviews were due in seven days, though the deadline could be extended to up to 21 days on request. The goal of quick review and quick publication matches the wishes of many authors who feel pressured to publish. One of the ways Frontiers has of speeding up the review process is that “editors are encouraged to accept manuscripts as soon as they receive two recommendations for publication by reviewers (regardless of how many other reviewers recommend rejection).”¹ Exceptions to the rules were possible, but discussing exceptions apparently meant onerous email discussions. When the issue was ready, the three editors prepared an editorial for the issue that contained their own critical reflections on the reviewing process. When they submitted it, they “received a message informing us that our text was not in accordance with the guidelines of Frontiers.”¹ Frontiers promised to find a solution to allow their comments, but after Frontiers failed for months to respond to messages, the authors turned to an official university blog, Leiden Madtrics to share their experiences in the hope that their “blog post can trigger the open scholarly debate”.¹ 1: Horbach, Serge, Michael Ochsner, and Wolfgang Kaltenbrunner. 2022. ‘Reflections on Guest Editing a Frontiers Journal’. 31 October 2022. https://www.leidenmadtrics.nl/articles/reflections-on-guest-editing-a-frontiers-journal.

November 29, 2022, #23 Macháček and Srholec begin their article in MIT’s Quantitative Science Studies (2022) by noting that: “Predatory publishing represents a major challenge to scholarly communication.”¹ They describe predatory publishing as journals in which “[a]uthors are motivated to pay to have their work published for the sake of career progression or research evaluation”¹. In return the journals largely ignore the peer-review process, or simplify the reviewing to make it quick and mechanical. In this paper the authors sought to discover the geographical distribution of authors in predatory journals as defined in Beall’s 2016 list. One difficulty lies in identifying which journals to include. Beall’s lists are old, and, as the authors note, they “very likely to suffer from English bias.”¹ In preparing the study, the authors used a three step process in which they first tried “matching the lists of standalone journals and publishers by Beall (2016) with records in the Ulrichsweb (2016) database”¹. Then they sought data about the author’s home countries, and they “downloaded the total number of indexed articles by country from Scopus”¹. The analysis used “evidence from the period between 2015 and 2017”¹. They found that “Kazakhstan and Indonesia appear to be the most badly affected, with roughly every sixth article falling into the suspected predatory category. They are followed by Iraq, Albania, and Malaysia … South Korea is by far the worst among advanced countries. All countries on the top 20 list, excepting only Albania, are indeed in, or [physically] very near, Asia and North Africa.”¹ Since language could play a role, the authors checked that too: “English-speaking countries do not display significantly higher propensities towards suspected predatory publishing than Francophone areas or countries speaking other languages.”¹ Prosperity and income represented another factor that the authors considered: “The worst situation is in middle income countries, many of which recognize the role of research for development, and therefore strive to upgrade.”¹ In discussing weaknesses of their analysis, the authors admit that: “[a] major limitation of this study is that we can only speculate that the way in which research is evaluated in each country makes the primary difference…”¹ Ideally the analysis would have a metric about local pressures to publish, but that would have required a different research project. 1: Macháček, Vít, and Martin Srholec. 2022. ‘Predatory Publishing in Scopus: Evidence on Cross-Country Differences’. Quantitative Science Studies 3 (3): 859–87. https://doi.org/10.1162/qss_a_00213.

December 13, 2022, #24 Olavo B. Amaral wrote in Nature on 23 November 2022 that peer review could be fixed by breaking it into stages: “All data should get checked, but not every article needs an expert.”¹ He focuses especially on data quality, and argues that “For most papers, checking whether the data are valid is more important than evaluating whether their claims are warranted…. Undetected errors or fabricated results will permanently damage the scientific record.”¹ He wants to distinguish between basic checks, including the availability of data, and the reliability of the statistical calculations. He raises the question of whether an expert really needs to check every paper, especially since they often do not have the time to do a thorough job, and he recommends using automated tools for some tasks. “In 2015, researchers in the Netherlands developed statcheck, an open-source software package that checks whether P values quoted in psychology articles match test statistics. SciScore — a program that checks biomedical manuscripts for criteria of rigour such as randomization, experiment blinding and cell-line authentication — has screened thousands of COVID-19 preprints.”¹ He noted, however, that such programs do not work if the data are insufficiently standardized, and that merely “checking data cannot guarantee that they were collected as reported, or that they represent an unbiased record of what was observed.”¹ Reproducibility represents a much-discussed topic that garners less academic credit than new articles. Nonetheless, he noted some positive developments: “reproducibility hubs such as the QUEST Center at the Berlin Institute of Health at Charité have been set up to oversee processes across multiple research groups at their institutions.”¹ Recognition is a problem: “These systematic efforts will not become integral to the scientific process unless institutions and funding agencies grant them the status currently enjoyed by journal peer review.”¹ His core concern is that “peer review drains hundreds of millions of hours from researchers but delivers little.”¹ The real problem may not actually be with peer review, but with our unrealistic expectations about what it can accomplish in its current form. 1: Amaral, Olavo B. 2022. ‘To fix peer review, break it into stages’. Nature: World View. 23 November 2022. https://doi.org/10.1038/d41586-022-03791-5.

January 10, 2023, #25 Many universities and many countries use Journal Impact Factors (JIFs) when assessing the quality of publications for individual scholars, for departments, and for the universities as a whole, and this assessment often has a direct effect on funding. Many administrators like the simplicity of using JIFs as a grading system. An important question is to what degree JIF represents a reliable indicator of quality, and a group of authors decided to investigate. The study was largely British and funding came from “Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland.”¹ The authors warn that: “results are limited by considering a single period (2014-18)“¹ and are “restricted to results from a single country.”¹ The authors relied on “the UK REF”¹ because it “is almost an ideal case in the sense of large-scale expert judgements by people explicitly told [to] ignore the reputation of the publishing journal,”¹ as they note “individual sub-panel members in some UoAs [Units of Assessment] may have disregarded this advice or have been subconsciously influenced, based on their own perceptions of their fields.”¹ The study, which is available on the arXiv preprint server, found that: “[t]he social sciences investigated had weak or moderate correlations”¹, and the social sciences included “library and information science (r=0.528, r=0.267, n=71)”¹. The authors claim that their results “add weight to the evidence that journal impact associates with article quality at least a small amount in all areas of scholarship.”¹ They note, however, that the arts and humanities could be an exception. They also warn that the “correlations are very weak (0.11) to moderate (0.43) for broad fields…. Weaker correlations may reflect non-hierarchical subjects, where journal specialty is more relevant than any journal prestige.”¹ For the iSchools, this is particularly interesting because Information Science methodologically is extremely broad. The results suggest that JIF is not irrelevant as an evaluation tool, but is far from reliable in our field and “confirms that journal impact is not ever an accurate proxy for the quality of individual articles.”¹ 1: Thelwall, Mike, Kayvan Kousha, Mahshid Abdoli, Emma Stuart, Meiko Makita, Paul Wilson, and Jonathan Levitt. 2022. ‘In Which Fields Do Higher Impact Journals Publish Higher Quality Articles?’ https://doi.org/10.48550/arXiv.2212.05419.

January 17, 2023, #26 Mike Rossner wrote a recent guest post for Scholarly Kitchen about why “Publishers Should Be Transparent About the Capabilities and Limitations of Software They Use to Detect Image Manipulation or Duplication”¹. Rossner was “Managing Editor of The Journal of Cell Biology” and now has his own company called Image Data Integrity, Inc. His post discusses the STM Integrity Hub, which offers software to detect image manipulation. Automating the complex process of detecting image manipulation is a longstanding goal of both publishers and universities. Rossner writes that “[i]n the past decade, numerous software applications have been developed for the automated detection of image manipulation/ duplication. These applications present the possibility of screening images at a scale that is not practical with visual inspection, and their use has the potential to protect the published literature in ways that were not previously possible. Several of them are now commercially available.”¹ While this is good news, skepticism remains. Rossner writes: “In my opinion, visual inspection remains the gold standard for screening images for manipulation/duplication within an individual article or for image comparisons across a few articles, especially when a processed image in a composed figure can be compared directly to the source data that were acquired in the lab.”¹ He argues that the test data and the test results need to be made public. He cautions that any testing needs independent verification because it is “not unheard of for entities with a vested interest in a product to test it themselves.”¹ He recommends that “the validation data for software designed to protect the [public health and safety record] should at least be made public….”¹ This kind of transparency is a goal that the scholarly world broadly supports, but at least two challenges remain. One is to define the exact nature of the transparency, since software developers are not typically eager to give away secrets. The second is to ensure that the software can distinguish among different kinds of manipulations, since removing a scratch is significantly different than pasting external elements into an image. 1: Rossner, Mike. 2023. ‘Guest Post — Publishers Should Be Transparent About the Capabilities and Limitations of Software They Use to Detect Image Manipulation or Duplication’. The Scholarly Kitchen. 10 January 2023. https://scholarlykitchen.sspnet.org/2023/01/10/guest-post-publishers-should-be-transparent-about-the-capabilities-and-limitations-of-software-they-use-to-detect-image-manipulation-or-duplication/.

January 24, 2023, #27 ChatGPT has made headlines in the press, but as yet the number of papers offering a systematic analysis of its capabilities is small. Gary Marcus is a Prof. Emeritus at New York University, and a well-known critic of AI developments, writes in his post “Scientists, please don’t let your chatbots grow up to be co-authors: Five reasons why including ChatGPT in your list of authors is a bad idea”¹. He continues: “The worst thing about ChatGPT’s close-but-no-cigar answer is not that it’s wrong. It’s that it seems so convincing.”¹ He goes on to argue: “ChatGPT has proven itself to be both unreliable and untruthful. It makes boneheaded arithmetical errors, invents fake biographical details, bungles word problems, defines non-existent scientific [phenomena, stumbles] over arithmetic conversion, and on and on.”¹ A more neutral analysis can be found in a recent paper on the arXiv. In “How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection”² by a Chinese team Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. The authors “collected tens of thousands of comparison responses from both human experts and ChatGPT, with questions ranging from open-domain, financial, medical, legal, and psychological areas.”² The authors are more cautious about their opinions. They are particularly interested in building a data set to examine ChatGPT’s effectiveness: “The human evaluations and linguistics analysis provide us insights into the implicit differences between humans and ChatGPT, which motivate our thoughts on LLMs’ [Large Language Models] future directions.”² Nonetheless they conclude that: “On the English datasets, the F1-scores for human answers are slightly higher than those for ChatGPT without any exceptions…”² The situation is different for different data sources: “On the Chinese datasets, the F1-scores of humans and ChatGPT are comparable with no significant difference. This suggests that the difficulty in detecting ChatGPT depends on the data source.”² None of this is conclusive, of course. Anyone who gets a free account at “https://chat.openai.com” can experiment for themselves. What is ultimately interesting about ChatGTP is not whether it is so good that it can replace human authors, but rather how successful it is at making the question plausible. 1: Marcus, Gary. 2023. ‘Scientists, Please Don’t Let Your Chatbots Grow up to Be Co-Authors’. Substack newsletter. The Road to AI We Can Trust (blog). 14 January 2023. https://garymarcus.substack.com/p/scientists-please-dont-let-your-chatbots. 2: Guo, Biyang, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. ‘How Close Is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection’. arXiv. http://arxiv.org/abs/2301.07597.

February 1, 2023, #28 On 19 January 2023 the “Scholarly Kitchen” hosted a discussion about “The Dea(r)th of Social Media? Assessing ‘Twexit’”. The participants were a mix of library directors, professors, publishers, and consultants. The introduction to the posts lays out the problem, “... the rocky tenure of Twitter’s new CEO and what many see as his open embrace of disinformation may portend a dramatic shift away from what’s been a decade plus-long bulwark of the industry.”¹ There are services that could replace Twitter, but have not so far. Angela Cochran writes: “...there is the question of whether one should continue to push out content on a platform that is actively turning a blind eye to anti-science content and promising to not moderate hate speech.”¹ Karin Wulf writes “… I’m starting to meet and connect with new people and to learn about new projects and events in my field and related ones through Mastodon…”¹ Mastodon is a newer and more distributed social media platform, “though it’s not what Twitter was, it’s already become quite valuable as a professional space.”¹ Rick Anderson is cautious about predicting the death of Twitter, and writes that he continues to use Twitter despite concern about the owner’s politics: “I would hope that people assess my politics based on what I say and do, rather than on who owns the platform I happen to occupy at a given moment.”¹ Another of the authors, Lisa Hinchliffe, writes that she joined “Mastodon in 2019” and notes that the people she interacts with on social media “are now on Twitter and Mastodon…”¹. She feels that Twitter “is still delivering far greater value than Mastodon”¹, and finds “the distributed nature of Mastodon a weakness and the promises of the seamlessness of server migration are greater than what was delivered in my experience.”¹ David Crotty writes that he has planned to stay on Twitter as long as possible, but that the company’s decision to begin “blocking third party apps from accessing their API … means that my usual tools, Tweetbot and Twitterific no longer function.”¹ Explanations about this policy change are hard to get because Twitter no longer has a Communications department. He argues also that from the corporate viewpoint “the real customers are those paying for advertisements.”¹ In a sense, of course, everyone who uses Twitter is supporting it because the advertisers care about the number or readers. The iSchools have never been heavy users of Twitter, but we have an account. The question is, whether to continue to use it. 1: Crotty, Karin Wulf, Angela Cochran, Rick Anderson, Lisa Janicke Hinchliffe, David. 2023. ‘The Dea(r)Th of Social Media? Assessing “Twexit”’. The Scholarly Kitchen. 19 January 2023. https://scholarlykitchen.sspnet.org/2023/01/19/the-dearth-of-social-media-assessing-twexit/.

February 14, 2023, #29 On 7 February 2023 Jonathan Bailey wrote an entertaining article called Falsifying Attribution for a Bad Pun. The issue is in fact serious: “A 2013 survey by iThenticate found that Misleading Attribution to be one of the most serious plagiarism and attribution problems in research, just behind verbatim plagiarism and complete plagiarism.”¹ The story in the article shows how easily even serious scholars can make mistakes. The person who falsified the attribution was George Gamow, a noted physicist who wrote Mr. Tompkins in Wonderland (1940) as a means of explaining laws of physics by presenting a world in which basic facts like the speed of light were significantly lower to make it easier to perceive the effects of approaching the speed of light. Gamow did not think he was committing an integrity violation when “before publishing the paper in the April 1949 journal Physical Review, Gamow decided to add the name of his friend and fellow physicist Hans Bethe. This made the authorship of the paper read 'Alpher, Bethe, Gamow', a play on the Greek letters alpha, beta and gamma.”¹ Gamow’s excuse is that Bethe got the pun and accepted the change, even though he had not contributed to the actual paper. Today with the pressure on scholars to increase their publication count as much as possible, wrong attributions have become serious issues, and can lead to situations where innocent errors can have complex consequences. Authorship, as Bailey writes, “is also very fuzzy. Determining who has earned an authorship credit on a paper can be difficult and, because of the importance of publishing papers, there’s a great deal of pressure to include as many names as possible.”¹ Commissions trying to decide whether a scholar has committed malpractice often prefer to err by making rules too simplistic, which makes their lives easier, even when reality is more complex. As to Gamow’s integrity error, Bailey adds: “no real harm was done as the truth both was and is widely known. But, that being said, I wouldn’t recommend trying this again in the 2020s. I doubt this joke would be nearly as funny the second time…”¹ 1: Bailey, Jonathan. 2023. ‘Falsifying Attribution for a Bad Pun’. Plagiarism Today (blog). 7 February 2023. https://www.plagiarismtoday.com/2023/02/07/falsifying-attribution-for-a-bad-pun/.

February 21, 2023, #30 On 13 February 2023 Amber Dance wrote an article in Nature called: “Stop the peer-review treadmill. I want to get off; Faced with a deluge of papers, journal editors are struggling to find willing peer reviewers.”¹ Anyone who has been a journal editor or conference organiser or peer reviewer knows that peer review takes substantial time and energy, not just for the reviewers, but for those reviewing the peer-reviews. The scale may surprise many scholars. Dance quotes Balazs Aczel at Eötvös Loránd University in Budapest: “Using a data set covering more than 87,000 scholarly journals, Aczel and his colleagues estimated that researchers globally, in aggregate, spent the equivalent of more than 15,000 years on peer review in 2020 alone.”² The problem with aggregate numbers is that they are so large that they obscure the individual problem of balancing time for peer-review with writing, teaching, and other necessities of academic life. A standard complaint is that peer review is unrewarded work. A few for-profit journals will pay for reviews, and a few will offer a free online subscription. This may not, however, be an ideal solution: “Non-profit journals might not be able to compete for reviewers if commercial rivals paid. And researchers eager for an easy pay cheque might churn out lower-quality reviews.”¹ Another proposal is to expand the reviewer pool. Dance quotes “Bernd Pulverer, head of scientific publications at EMBO Press in Heidelberg, Germany,”¹ as saying: “We’re not using enough early-career researchers…”¹ Taking this suggestion risks criticism that the reviewers lack experience, though it is unclear to what degree an eager post-doc will do a worse job than a full professor pressed for time. A practical idea is to use computers to check statistical results and various mechanical tasks that humans tend to overlook. Preprints are also a popular suggestion that some journals use as a first step in the reviewing process. As Dance notes, preprints “don’t replace the publications that scientists need to populate their CVs.”¹ The iSchools operate no journals at present, but peer-reviewing is a core part of accepting papers for conferences, for giving dissertation awards, and for giving out the iSchools research grants. Its complexities and flaws cannot be ignored. 1: Dance, Amber. 2023. ‘Stop the Peer-Review Treadmill. I Want to Get Off’. Nature 614 (7948): 581–83. https://doi.org/10.1038/d41586-023-00403-8. 2: Aczel, Balazs, Barnabas Szaszi, and Alex O. Holcombe. 2021. ‘A Billion-Dollar Donation: Estimating the Cost of Researchers’ Time Spent on Peer Review’. Research Integrity and Peer Review 6 (1): 14. https://doi.org/10.1186/s41073-021-00118-2.

March 7, 2023, #31 On 15 February Ludo Waltman and his co-authors published a blog article in Leiden Madtrics about “Experimenting with Open Science Practices at the STI 2023 Conference”¹. The organisers of the 2023 Science, Technology and Innovation Indicators conference (STI 2023) decided to try “two open science experiments”¹: the first opened the publication and peer review process, and the second involved “[r]eflecting on open science practices”¹. Publishing the article in an openly accessible blog may also represent a continuation of the reflections. The scholarly community has been discussing the idea of more open peer-review processes actively for years, as reflected in past news features (see for example: “Fixing Peer Review”, 13 December 2022). What is new here is the degree to which the process is itself analysed as an experiment. They emphasise a number of benefits, including speeding up dissemination, increasing the value of peer review by making the comments available to everyone who reads the articles, helping authors by giving them “feedback and credit more rapidly,”¹ and by giving more recognition to those reviewers who allow their identity to be made public. The authors also note some concerns: “A common objection against preprinting is that preprints may present inaccurate results because they are published before peer review. … Another concern about preprinting is that journals might be reluctant to publish articles that have already been published as a preprint. However, very few journals still have such a policy.”¹ The authors are especially concerned about how doctoral students might regard the experiment. As they note: “a closed ‘review, then publish’ model, peer review may be biased against these researchers, lowering their chances of getting their work published.”¹ The authors also recognize that doctoral students ”may feel uncomfortable both about their own work being critiqued publicly and about publicly critiquing the work of others, in particular the work of more senior colleagues.”¹ As part of the reflections aspect of the experiment, the STI conference organisers will survey participants, and will report on the results of the survey in a blog (presumably Leiden Madtrics). Broadening the discussion about open science methods seems appropriately integral to the openness of open science itself. NOTE: the iConference so far uses traditional double-blind peer reviewing for all papers with established scholars as reviewers. 1: Waltman, Ludo, Rong Ni, Kwun Hang (Adrian) Lai, Marc Luwel, Biegzat Mulati, Ed Noyons, Thed van Leeuwen, Leo Waaijers, and Verena Weimer. 2023. ‘Experimenting with Open Science Practices at the STI 2023 Conference’. Leiden Madtrecs. 15 February 2023.https://www.leidenmadtrics.nl/articles/experimenting-with-open-science-practices-at-the-sti-2023-conference.

March 14, 2023, #32 Julian Hamann and Leopold Ringel have written a blog post on “University rankings and their critics – a symbiotic relationship?”¹ where they examine why “university rankings have proven a resilient feature of academic life”¹. Rankings have different significance in the various regions and countries. For many US universities a high ranking leads to more students and more tuition income, but even in countries where the universities do not charge tuition (Germany, for example) the status of a high ranking matters. Nonetheless, as the authors note, a number of “prestigious law schools and medical schools announced their withdrawal from the U.S. News & World Report rankings.”¹ One critique is that rankings “produce and consolidate inequalities, instill opportunistic behavior by those trying to anticipate ranking criteria, and infringe upon the independence of higher education and science.”¹ Another criticism is “that the measures in use are too crude and simplistic, methodology and data not transparent enough, and neither validity [n]or reliability sufficient.”¹ The rankings producers like US News have several defences. One is to “downplay their own influence”¹, which is somewhat disingenuous. The second defence “claims a broad demand for evaluations of university performance, demonstrates scientific proficiency, or temporalizes rankings by framing them as always needing improvement.”¹ The demand for the rankings is to an extent created by the rankings producers themselves, and it is arguably also the byproduct of a competitive society that views academic performance in sports terms with winners and losers. The authors talk about “[c]onstructive conversations between rankers and critics”¹ They describe the dialog between producers and critics as a “discursive phenomenon”¹, and argue that “university rankings develop a quality we have coined discursive resilience: the ability to engage with critics in a productive way in order to navigate a potentially hostile environment.”¹ The authors do not view the withdrawals from the rankings as suggesting the “end of the ranking regime”¹, but only as an “opportunity to build discursive resilience.”¹ It could also become an opportunity for universities to rethink the utility of ranking systems in general, and to ask whether this kind of competition promotes science in the broadest sense. 1: Hamann, Julian, and Leopold Ringel. 2023. ‘University Rankings and Their Critics – a Symbiotic Relationship?’ Impact of Social Sciences (blog). 6 February 2023. https://blogs.lse.ac.uk/impactofsocialsciences/2023/02/06/university-rankings-and-their-critics-a-symbiotic-relationship/.

March 21, 2023, #33 ChatGPT has dominated the tech news in ways that cannot escape notice. Craig Griffin writes in The Scholarly Kitchen that he has “enjoyed playing around with the capabilities of ChatGPT.”¹ Just for fun he asked ChatGPT to turn a scholarly article from IWA Publishing’s Water Science and Technology journal into a Haiku: “Açaí endocarp, Removes heavy metals well, Nature’s solution.”¹ He also asked ChatGPT to turn the article into a sonnet. Here are the first two lines: “Nature’s gift, the açaí endocarp, A fibrous, porous, laminar mass, With voids and fissures, it doth embark On a journey to cleanse and surpass. Heavy metals in water, a plight, A danger to all living things, But with this biosorbent, they take flight, Leaving the water pure and clean.”¹ The style transformation is impressive. When Griffin asked ChatGPT to summarise the article, he writes: “The conclusion from ChatGPT is fact heavy, but not incorrect. To be sure, the authors’ conclusion is better written, more easily understood, and brings in points (such as the fact that further study is needed) however, the ChatGPT conclusions are accurate and useable.”¹ He notes “It’s conceivable that ChatGPT could be used to summarize complex article concepts into more accessible and consumable formats. Beyond that, the power to identify patterns in large information sets could truly be transformative, by ingesting thousands of papers on a topic and generating a meta-analysis in minutes.”¹ Some tasks are easier for ChatGPT than others. Griffin writes: “I asked the AI which is better, Siri or Alexa, to which it replied that it doesn’t have opinions as an AI model”¹. Griffin kept asking what ChatGPT meant by “its personal opinion” with the result that “in the span of 8 messages ChatGPT flip flopped 4 times.”¹ Griffin does not make light of the risks. He notes that: “ChatGPT has also created a turbo-charged weapon for plagiarism, fake analysis, and horsepower for the paper mills. It’s entirely plausible that ChatGPT will be (or is already) being used to crank out ‘papers’ that no researcher ever touched.”¹ AI hype is nothing new and is often exaggerated, but ChatGPT and similar tools appear to offer a level of substance that ought not be ignored. 1: Griffin, Craig. 2023. ‘Guest Post — ChatGPT: Applications in Scholarly Publishing’. The Scholarly Kitchen. 14 March 2023. https://scholarlykitchen.sspnet.org/2023/03/14/guest-post-chatgpt-applications-in-scholarly-publishing/.