Women Reviewers

While there have been a few good-press stories about women in science lately, with the viral blog post about a woman’s experience at Uber and today’s story about this issue in The Conversation, I thought it was finally time to write up another post on the topic of women and bias in publishing.

Specifically for geoscience and readers of JGR Space Physics, there were two recent Eos articles or relevant, one on scientists at the Women’s March on Washington and another on the obstacles facing women in our field and another. This second article is an especially worthwhile read, including parts of particular interest to scientific publishing. AGU HQ staff wrote an article that just appeared in Nature last month about gender bias in reviewing, finding that women do not serve as reviewers as much as they write first-authored papers. For researchers in their 20s, this gap doesn’t exist, but it slowly widens, almost monotonically, with each additional decade of age.

As you can see from the paper title:

lerback_nature_banner

I’ve described the article’s findings too generally. The title nicely links the finding to the cause. That is, the gap is not the fault of the researchers; the review-request decline rate is identical between men and women. It is the fault of the editors, who send out the review requests, and manuscript corresponding authors, who suggest potential reviewers. The proportion of women in these two categories (those getting review requests and those listed as potential reviewers) is lower than the proportion of women in the field. We need to do better. I need to do better.

Science did a related study looking at the proportion of women authors of their papers, finding that it is substantially lower than the proportion of women among potential authors. So, it isn’t just geosciences, but across science as a whole, that a gender bias in publishing exists.

How can we change this? In addition to me and the other editors getting our requests in line with the research population, I have one idea to share here for all of you.

Manuscript corresponding authors: please think about your list of potential reviewers before signing in to GEMS to submit your paper. GEMS asks you for lots of information and you should think about all of these questions before sitting down to submit, but I especially encourage you to put some effort into the potential reviewer list. If we do it “on the fly” while in the process of submitting, then the usual suspects of senior people, often men, will most likely come to mind. These people are often busy and decline. Please spend some time on this list and think about the full range of potential reviewers. It will help you because it helps us find highly qualified reviewers that much faster.

Subscribing to journal e-alerts

Here is one more post in the series about my estimate of the 2015 JGR-Space Physicsspecific Journal Impact Factor and the need for citing recent articles. This could be my last on this for a while. I’m off to the AGU Editors in Chief meeting tomorrow, so there will probably be new topics to discuss here from that meeting.

How do we know about the recently-published literature? How do we put our new studies in the proper context of the latest research in that field? How do we prepare ourselves to be good reviewers and know about the full range of papers that the author should be citing?

Simply relying on our memory will systematically bias our recollection of what papers to cite to the “famous” and “classic” papers, which are usually the older ones. They are the ones we have seen cited more times in the past and the ones we have probably read several times during our career. Unless we make the recent papers fresh in our minds, we will probably not think about them.

On this note, a big thing that I think we can (and should) be doing is subscribing to journal table of content alerts. More than this, we need to be giving them a quick scan for relevant new papers in our fields of expertise and clicking on those articles. I like to download them and collect these PDFs in a “Papers to Read” folder. Most of the time, the contents of this unread-papers-folder expands with time simply expands with time but every now and then I read through one of these papers and move it to my “Electronic Reprints” folder. Okay, clearing out this folder often comes in little binge reading sessions, usually when I am with my laptop but isolated from the internet. At this point, though, I’ve seen the paper at least twice and I hope that something about it sticks in my head, ready to be recalled the next time I am talking about that topic.

Nearly all journals have TOC e-alerts now. At JGR Space Physics, it’s over on the right-hand side on the main website:

content_alert

            The other big thing we should be doing is not relying on our memory to populate the cited references in the Introduction and Discussion section of a new manuscript, but conducting a rigorous literature search for relevant papers. I wrote a post on that last fall.

Should we do more for our JIF?

In my last post I presented my estimate of the 2015 Journal Impact Factor for JGR-Space Physics. The number is below the all-sections JGR impact factor by about half a point.   I also showed that this section-specific impact factor has been lower than the all-section value for, well, as far back as I calculated it (~10 years). While I am not that concerned, it is a little troubling to think that space physics, as a field, isn’t as good as other fields, like atmospheric science or astrophysics, at citing recently published papers in our new studies.

The ultimate responsibility for this is with the authors of papers. Each of us should be a conducting literature search with every new paper we write, including citation of relevant papers that either build up to the question addressed or place the findings in the context of existing knowledge, in the Introduction and Discussion sections, respectively. As I have written before, please do this with every new paper you submit.

In addition to this, should we who gate-keep and publish the papers, meaning the Editors, reviewers, AGU, or Wiley, be doing more to increase the impact factor of JGR-Space Physics? I guess we could, but it seems a bit unethical and manipulative, as mentioned in the Physics Today article I highlighted earlier this month. We can do something, though, especially the reviewers.

Reviewers, as the expert assessors of the quality of the work, are the best people to be addressing this issue. They should include an examination of the citations in the manuscript, especially the Introduction and Discussion sections, and determine if the study properly motivates the study with respect to existing knowledge of the topic as well as places the findings into the context of other similar or competing findings from other studies.

At the reviewer instructions at GEMS, AGU brings this up to reviewers in two places. First, it is asked of reviewers in the question set they must answer when submitting their review: “Is the referencing appropriate?” GEMS only provides three answers to choose from: yes; mostly yes, but some additions are necessary; and no. By asking the question, though, it really is just prompting the reviewer to think about this aspect of the paper and encourage suggestions of additional relevant papers to cite. The second place is in the question set for the reviewer to think about in the formal review: ” Does this paper put the progress it reports in the context of existing published work? Is there adequate referencing and introductory discussion?” Again, making sure that the reviewer assesses this aspect of the paper.

See the reviewer instructions for more details on this. There is also a 2011 Eos article about writing a good review. This Eos article has a spiffy flow chart about the review process:

peerreviewguide-flowchart

It suggests that you read the manuscript up to 3 times. The article states that reviewers are not there to catch “to catch every typo, missing reference, and awkward phrase.” I agree. The reviewer should, however, catch glaring omissions of clearly relevant studies.

This idea of you-don’t-have-to-force-citation-of-everything is reflected in the GEMS questions to reviewers. Neither of these questions listed above explicitly ask the reviewer to look for citations to recent articles, nor is there a requirement to cite some minimum number of recent articles. I am glad, because I think that would be stepping over the line of ethical acceptability. In the process of thinking about all relevant literature on which the manuscript builds, though, the reviewer should also consider the recently-published studies as well as the older, and perhaps better known and more familiar, studies.

Defining Plagiarism

Happy Halloween; one of the most bizarre holidays ever invented (in my opinion).

To go with my last post, I’d like to continue the conversation on plagiarism. Lots of people are talking about this topic, , and I have several times before. Here’s a graphic on the usage of the word “plagiarism” in the last 200 years:

plagiarism_usage_googlebooks

How did I make this plot? Google has a site that does this.

Here’s a definition of plagiarism from Dictionary.com:

plagiarism: an act or instance of using or closely imitating the language and thoughts of another author without authorization and the representation of that author’s work as one’s own, as by not crediting the original author:

            It is not just “language” but “thoughts” as well. AGU can and does check for language overlap, and I and the other editors of JGR Space Physics occasionally send manuscripts back to authors for revision before review to have them rewrite text that is too close to already published papers.

Checking for “idea overlap” is very difficult. The closest that we can come to this is if an editor or reviewer notices that references are missing to key studies of direct relevance. If it is published, then you should give those authors credit for the ideas that they have discussed.

So, I have two pitches to the community.

Authors: please include references all relevant papers. Conduct a literature search at AGU’s EASI database, Harvard’s ADS astronomy abstract service, or Google Scholar. You have lots of resources for this. This is an important step in the scientific method that greatly helps to refine your message to what it truly new and original in your study.

Reviewers: please scrutinize the references, especially in the Introduction and Discussion sections, to ensure that key papers are being cited. It’s one of the questions we ask of you (“Is referencing appropriate?), hoping that this spurs you to read the manuscript with this issue in mind.

Because it’s almost election time here in America, grabbed some hat images and I made up some baseball cap designs that I think we all should be wearing, figuratively if not literally.

citations_election_hats_small

Reviewer Awardees for 2015

In re-reading my post from earlier this week, I went back and checked and realized that I did not have a post listing the awardees of the 2015 Editor’s Citation for Excellence in Scientific Refereeing. Each year, AGU’s journal editors get to select a few people for this award. By a few, I mean a few: up to 1% of the total number of manuscripts submitted to the journal that year. For JGR Space Physics, we had 1190 manuscript submitted in 2015, so we were able to select 12 people for this award.

2015refereeingexcellence_small

            This is an amazingly hard decision because so many people write outstanding reviews. Plus, there is the perennial decision of how to weight various criteria, like how many reviews someone did, their average time to submit a review, their highest or average rating (yes, we rate referees on every review), or the importance of a single review to the decision on a particular manuscript. Plus, at JGR Space Physics, we make this a group editorial decision, so all 5 of us deliberate and vote on the list.

For 2015, our 12 awardees are (in alphabetical order):

  • Eric Christian
  • Ingrid Cnossen
  • Xueshang Feng
  • Ryochi Fujii
  • Manuel Lopez Puertas
  • Paul O’Brien
  • Minna Palmroth
  • Natalia Perevalova
  • Viktor Sergeev
  • Kazue Takahashi
  • Bruce Tsurutani
  • Angelos Vourlidas

THANK YOU VERY MUCH for your outstanding service to the journal and to the research community.

I’ve said it before but it needs to be said again: I would also like to thank all of the 1,506 people that served as reviewers for JGR Space Physics in 2015. AGU rules limit our awardee number to 12, but I am grateful for the time and effort put in by every single one of you. Thanks!

Peer Review Week

Did you know that there is an event called “Peer Review Week”? Apparently, it’s a conference, half in-person, half virtual. The second annual one of these was just held last month. This year’s theme was “Recognition for Review.”   I found it interesting to read the blurbs about the conference speakers.

On this note, AGU has been exploring some options for better recognition of peer reviewers. The main recognition is the Reviewer Excellence Award, for which Editors select a very tiny handful of peer reviewers for recognition each year. We are only allowed to pick a number equal to 0.1% of the total number of new submissions to the journal. For JGR Space Physics, that’s 10-12 people; not a lot. They get their name in Eos and a special reception at the Fall AGU Meeting. AGU also passes on the number of reviews each person did to their ORCID account, and this aggregate information is then a verified documentation and recognition of your service.

On a related note, Noah Diffenbaugh, the EiC of GRL, wrote a recent Editors’ Vox article, “Stuff My Reviewers Say.” He brings up a very good point that most reviewing work is uncredited and unknown to nearly everyone, except the author and editor. I would like to echo his comment that nearly all reviews are constructive and provide helpful advice for making the science better. By “science” I mean any aspect of the study, from the historical perspective in the introduction, setting up the hypothesis, the description and choices made in the methodology, the presentation of the results, the discussion of the findings, and the summary of the work in the Abstract or Conclusions. Reviewers do a lot of work to make our research community function.

I’ve said it before, but thanks again for all of your hard work out there for JGR Space Physics. The journal could not exist without the thousands of hours a year invested by the research community to assess each other’s work and provide high-quality vetting before acceptance.

And, then, of course, there is this. Here’s a particularly funny one:

funny_review_quote

Don’t Cite Unpublished Work

The title of this post really says it all. Here’s a quote from a document at the Editor Portal (sorry, I don’t have a public link for it), “AGU journals do not allow references to unpublished journal articles.” This includes JGR Space Physics.

nounpublishedrefs

            Like the requirement of having open access to the data (observational or numerical) used to develop findings in a study, all scientific understanding on which the study is based (i.e., the cited literature) needs to be available. This does not mean freely available (the paper could be behind a subscription paywall) or even easily available (for instance, print only in an old monograph), but available somehow. Citing unpublished articles, especially the promissory note of “manuscript in preparation,” is forbidden.

Let me make an important clarification to this: unpublished articles cannot be cited in accepted or published AGU journal articles. At initial submission, citing papers that are “under review” or “accepted” is allowed. You need to provide a copy of the unpublished paper as a supplemental document so that the Editor and reviewers can see it and assess the worthiness of the reference. If they are not supplied, then reviewers can and should ask to see such references and the corresponding author should be ready to provide it.  This means authors should confirm with the authors of the cited yet unpublished paper that it is okay to cite their paper and provide it to the Editor and reviewers.

On submission of any revisions, however, these other papers must have progressed to the level of being available online or in print. If not, then they should be removed and the manuscript revised to accommodate that change in referenced literature. If they are still in the revised manuscript, then AGU staff will ask the authors for a justification about the citation and will consult with the Editor about how to handle it. It could be that the other paper is close but not quite through to acceptance. If this can be verified, then we will probably let that through. It could be a companion paper or another paper in the same special section. Again, this is probably okay. If we let it remain, however, and the citing paper is accepted before the cited paper is available, then AGU/Wiley will hold the citing paper until the publication of that other one. If you must cite that paper, then your paper will wait until the other is available. If two papers mutually cite each other, AGU will coordinate publication. They will even coordinate with other publishers, like they did with the MAVEN special section in GRL last November, which came out simultaneously with 4 related papers in Science, all released in phase with a press conference.

For AGU journals, being “in press” means being available. AGU posts nearly all papers at the journal website within 3-4 days of acceptance. Other journals may or may not do this, though, so “in press” is not a guarantee that you can cite the paper. Like I said, AGU will contact other publishers and coordinate, release. Who knows, this might even expedite publication and availability of that other paper.

Finally, citation of some non-DOI references is allowed, especially those that are permanently archived. One example of special relevance to JGR Space Physics is the arxiv.org preprint service. Citing a paper there is allowed, even if it doesn’t have a corresponding peer-reviewed version available yet. Posting to arxiv.org is allowed because these papers are “permanently available” at this site. In the end, it is up to the Editor, in consultation with the authors and reviewers, to decide if the citation to a paper at arxiv.org (or similar service) is acceptable.

This has been enforced at GRL for a while but is relatively new for JGR Space Physics. If you start to see emails from AGU staff asking about these references to unpublished work, now you know why.

 

 

More On Bro Culture

This summer, since my Women in Space Physics post, I have been attuned to writings about women in the workforce, especially STEM fields, and on the prevalence of “bro culture.” Like the From the Prow blog post from AGU President Margaret Leinen about building diversity in our community. There was also the New York Times opinion piece about “bro talk” on Wall Street and how it keeps women out of the conversation and insidiously pushes them out of that workforce. And, on the even darker side, who can forget that guy in the news stating that if his daughter was harassed at work, then she should just find another job or even switch careers. Okay, back to the light side: here is an awesome and inspiring collection of profiles of women in planetary science.

Just recently, another of AGU’s blogs, Dan’s Wild Wild Science Journal, had a post about the lack of women in the STEM workforce. It recapped a PLOS One article that argues for increased tutoring through Calc I and II in early college, especially for those that didn’t have calc in high school. As a former introductory chemistry tutor at my undergrad institution, I think this is a fantastic idea. College can be hard, especially adjusting during that first term or two, and a broad support system, including tutoring for those tough introductory classes, is critical for maximizing student success. Here’s one of the important charts from the article:

journal.pone.0157447.g001

The article focuses on increasing the second-to-last data point on this graph. They show evidence that tutoring in Calc I/II could significantly alter the drop in STEM students during those first few years of college. I completely agree.

What they didn’t really discuss is the far bigger drop off between the senior year of high school and freshmen wanting to major in STEM. That’s where the “number of people in the STEM pipeline” drops off the most. In addition, back in middle school and high school, the lines between male and female diverge the most, with the male interest curve rising and the female curve dropping.

As a final topic here, I’d like to recommend a book. Among the many comments I received was a link to a book, “Now What Do I Say?” by Anne Krook. It’s a how-to book, filled with hypothetical (or, sadly, very real) examples of sexist comments and questions women might encounter, and good advice on how to respond in these situations. She compares this process to disaster planning, “the options for addressing risks as you plan for a disaster are to prevent, mitigate, prepare, and accept risks.” Krook offers lots of advice on how to address the risk of inappropriate interactions at work via all 4 of these avenues. My e-book copy is highlighted on nearly every page. There is too much good advice in this book to be condensed into a paragraph on a blog, so I won’t try. I will say this: the 10 workplace commandments section alone is worth the book price. I sat up the night before the GEM/CEDAR Workshop finishing this book, and then was at the meeting for so short of a time that I barely got to talk about it with people. Whether you are male or female, I strongly urge you to read this book.

Krook-book-cover-for-biz-card-2

Getting back to editing JGR Space Physics, I strongly urge you to remove gendered wording in reviews and, especially, responses to reviews. It is now acceptable to use the singular they in formal writing, which is a good alternative to guessing the gender of an anonymous person. I still see “he” and “him” in correspondence and, really, unless your reviewer signs it, you will probably guess incorrectly about who your writing about.

 

Reviewer Selection Editorial

The Editors of JGR Space Physics just published an Editorial on our reviewer selection process. The big point in this article is that we often use the Areas of Expertise, a menu of space physics topics within GEMS specific to this journal, for identifying experts within the community who might serve as qualified reviewers of a manuscript.

I wrote a month ago about our expansion of this list. We now have 18 more items in the list, bringing the total to 73. These new Areas of Expertise will only help us if members of the research community update their GEMS profile and click on whichever of the new topics falls within their specialty. Here’s the full list, with the new ones in green:

Areas of Expertise v3.jpg

            Updating your Areas of Expertise selections in GEMS helps you as a reviewer because the Editors will be better at sending you papers within your specialty. Without this, we either have to know you very well or we have to guess a bit based on the papers that you have authored or reviewed in the past. Filling this out will hopefully cut down on the number of times we request a review from you for something outside of your comfort zone.

Also, from a communal perspective, the more people that fill out the Areas of Expertise, the higher the quality of reviews that you should have on your submitted manuscripts. With high participation of researchers selecting their Areas of Expertise, then all of the manuscripts will be better matched with specialists in that field.

Yes, filling this out means that you might get asked to review more often. But, as seen in our statistics for 2015 and for 2014, the average number of reviews per reviewer per year is ~2.5, so we are trying not to overwork you. If you feel overworked as a reviewer, then you always have the option to decline our request.

So, I encourage you read the Editorial and then log in to GEMS for JGR Space Physics and check out the new Areas of Expertise.

Decision Notifications

For a few months now, GEMS has been sending final decision notifications to reviewers. As of this month, GEMS is now including coauthors on final decision emails for all manuscripts. I just got one (as coauthor of a now-accepted paper in JGR Space Physics), so I know that the new feature is working. Both of these are small changes but ones that hopefully will allow the reviewers and coauthors to be aware of the final decision on a manuscript with which they have been involved.

email-data-blue-ss-1920-800x450

            For the reviewers, you are providing expert advice for the editor to take into account in the decision about the manuscript. Your recommendation for that decision may or may not be heeded. And, while your review comments are always forwarded, your decision recommendation is usually conveyed to the author, as that line in the review can be deleted by the editor if desired. Plus, we usually get two or even three reviews for each paper. While we greatly value your input, it is our responsibility to make the judgment on the fate of each of our assigned manuscripts. So, my comment to the reviewers: if you have any questions about the final decision on a manuscript for which you served as a reviewer, then please reply to that notification email (i.e., to jgr-spacephysics@agu.org) and ask us. Without revealing names of other reviewers, we will fill you in on the rest of the story about the manuscript.

For the coauthors, you should be in contact with the corresponding author and should normally be aware of the current status of all papers on which you are listed as an author. However, we understand that sometimes the corresponding author forgets to forward the final decision letter to the coauthors, leaving them in uncertainty about the status of the manuscript until they perhaps see it in the table of contents alert. Except under unusual circumstances, coauthors should not contact the Editor. Rather this should be done through the corresponding author. So, my comment to coauthors: if you have any questions about the final decision on a manuscript, then please contact the corresponding author and direct all questions back to the editor or the journal through that person.

I hope that you find the additional communication from AGU to be useful.