Even More Year-End Stats for 2018

I have one last chart to share from the set for the JGR Space Physics editorial board meeting earlier this month. This is a chart of the authors of JGR Space Physics articles in 2015 and 2016, counting each individual person separately. Two things are shown on this chart: the bars show the number of articles with authors from that country publishing in these two years (scale on the left), while the line shows the citations to papers authored by people from that country in 2017 (the Journal Impact Factor calculation window), with values on the right axis (ICW = in-window citations).

JGRA-Dec2018-OutputByCountry

Yes, JGR Space Physics is dominated by authors from the United States. China has about one-third the number of authors and it goes slowly down from there.

What I like about this is the relative flatness of the citations-per-paper line. It’s 3.00 for the US and hovers between 2.5 and 3.0 for all countries in the top 12 (down through Finland). A cynic might suggest that we should publish more papers with authors from Norway and Belgium, as they are near 3.3 average citation value.

Hey, I’m of Norwegian descent. We even made a big batch of lefse this past weekend. I should look into the average citations of my papers and see how they stand up to my counterparts in my ancestral homeland. But I digress…

Note that this chart is for a single year of citations to papers published in the two previous years. The ordering of the countries along the x axis shifts quite a bit from year to year, as does the average citations-per-paper, especially for those towards the right of the scale. This is a snapshot of early citations to recent publications.

In general, I am happy with the flatness of this line. Without doing more analysis into it, I think it means one of two things: (a) we publish with international author lists so these numbers are not independent or (b) we cite papers independent of the author’s country of origin. I hope both are true.

Happy New Year!

 

Advertisements

More Year-End Stats for 2018

For the 2017 Journal Impact Factor, JGR Space Physics increased ever so slightly from 2.7 to 2.8 (rounding to two sig figs). Remember, this is only the second year of the 7 JGR journal titles having their own JIF scores, before that they all shared one, which hovered in the ~3.3 range. Also remember how the JIF is calculated – it is only citations in 2017 to papers published in 2015 and 2016. So, let’s take a closer look at citations to papers in that window.

Here is a line from a chart that Wiley staff put together for us:

JGRA-Dec2018-CitationsPerPaper

These are citations in 2017 to papers published in JGR Space Physics during 2015 and 2016. By the way, I am not revealing a trade secret about the journal here; you could do this yourself if your institution has a subscription to the Clarivate Analytics Web of Science database.

Just under 20% of the papers had zero citations, over half of the papers had 1-3 citations, just over 20% had 4-7 citations, and ~6% had 8 or more citations. Only one paper surpassed the 40 citation mark within the 2017 JIF window, by Kurth et al.

I am not concerned that 300 papers had no citations. Nearly all of these will get cited. Back at the beginning of this blog, I analyzed the Lost Papers and they are only a few percent, even just 5 years after publication. I am certain that this number will drop dramatically in the next few years for this collection of papers.

While I would like more papers to be in the 4-7 range than the 1-3 range, this will take a monumental shift in how the space physics community reads journal articles and cites them. Look at the chart I had in this post – our research community cites papers an average of ~3 times per year for many years and the citations for an average paper grow fairly linearly. I made that chart in January 2017, so the first column (citations to papers published in 2015) is citations of papers from 1.0 to 2.0 years old (so, on average, papers that were ~1.5 years old). After that, it grows at roughly ~3 citations per year for the next 10 years. Furthermore, JGR Space Physics has a citation half life of over 10 years, meaning that 10 years after publication, the average paper can expect its number of citations to more than double as the years progress, with over 60 eventual citations. That’s a lot!

Special side note if you are feeling bad that your decades-old papers do not have 60 citations: remember that these are averages, not medians, and that citation count is a positive definite value with a long-tailed skew. So, most papers will actually have fewer than 60 citations. I have not plotted the histogram and run the numbers; maybe I’ll do that some time.

My point is that space physicists don’t cite papers the way that the creators of the JIF expect researchers to cite papers. The JIF creators expect researchers to cite a paper soon after publication and then the paper declines in importance and relevance with time, perhaps even being forgotten at some point. This might be true in some fields, but not in space physics. To me, it does not seem that we cite articles that way.

Again, remember how the JIF is calculated: it is citations in the year 2017 to papers published in 2015 and 2016. Take the average: a mid-2017 paper citing a paper published at the end of 2015. It is only 1.5 years past its publication date, the same as the first column in my chart in the previous post. So, the average should be ~3. And guess what? To one significant digit, that is the JIF of JGR Space Physics.

If the journal is growing in size, then the paper count is lopsided towards young papers, reducing the citations from this expected average. So, going all the way back to the number of papers with 1-3 citations versus 4-7 citations, I am not surprised that the numbers are what they are. Let me say it one more time – this is how space physicists cite the literature.

Keep on doing what you do, space scientists. Keep reading the literature and citing those papers most relevant to your work: that is, those that you think build the case for a new study; those that contain the descriptions and example uses of the methodology you choose; and those that place proper context around the significance, originality, and limitations of your findings. If those are new papers, so be it. If those are decades-old papers, that’s fine too.

I look forward to seeing your manuscript submissions in the coming year.

 

Year-End Stats for 2018

For our JGR Space Physics editorial board meeting that we had during the Fall AGU Meeting a couple weeks ago in Washington, DC, AGU, Wiley, and me compile a bunch of stats about the journal. I’d like to share a few of those with you over the next few posts.

In GEMS, my editor powers allow me to make reports about the journal workflow. This is one of them:

JGRA-2018-admin-summary-report

The “2018 to Date” is through today, Friday, December 28, so it probably won’t change by much. Maybe the manuscripts will go up by a few, and I just assigned a bunch of manuscripts to other editors, so the number of reviewers could also go up by several, but the other numbers won’t change much. I took over as EiC of JGR Space Physics at the beginning of 2014, so this shows a quick summary of journal stats during my term.

First, let’s look at the second row, “Number of Manuscripts Received.” Just for perspective, this was under 1000 before 2014. Note that this is the number of “new” manuscripts, not revision submissions with an “R” added to the end of the manuscript number. You were writing many more papers every year, increasing the submissions by ~100 manuscripts a year for several years. This was also due to several large special sections in the journal, with the really big Measurement Techniques in Solar and Space Physics, lots of papers in the ULF special section, several large ones focused on Van Allen Probes results and other inner magnetosphere special sections, a big one for MAVEN, and several on space storms, like the St. Patrick’s Day Storms and storms in the Van Allen Probes era. We had a big one for MMS in 2017 in there too. This last year, we’ve had a few special sections, but so many and not as large.

There are several rows that I really like on this chart. The “Percentage of Manuscripts Sent for External Review” has remained steady, right near ~90%. Similarly, the “Receipt to First Decision” time has hovered near ~40, and the “Receipt to Acceptance” has actually dropped in recent years. Finally, I like that the “Acceptance Rate” has remained steady near ~70% throughout my term.

Note that the “N/A” values for “publication” are because the manuscript shifts to Wiley for that phase and is no longer in GEMS. Those numbers are typically 3-4 weeks.

My last post was in mid-August, four months ago. When I started this blog, I told myself that it was an extra thing I’d do for the community and I would not apologize for a hiatus. So I won’t.

DORA and the JIF

No, this isn’t a young Latina’s adventure story about mantequilla de maní (or crema de cacahuate, or one of the other translations for peanut butter). DORA is the Declaration on Research Assessment and is a call to action to put less reliance on the Journal Impact Factor (JIF). This is timely because the new JIFs for 2017 were just released this week.

DORA - logo@2x_withblack

            I particularly like the “general recommendation” of DORA:

  • Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

It is a simple yet powerful message – the JIF (or any other journal metric) is not a measure of the quality and impact of any individual paper in that journal.

For 2017, the JIF for JGR Space Physics is 2.752, which is ever-so-slightly higher than last year’s JIF. A few notes about it:

  • Remember that the JIF is calculated only from citations in one year to papers published in the previous two years. It is an average of a highly skewed, non-Guassian, positive-definite distribution of a very small subset of full journal content.
  • I like to only quote the JIF to two significant figures, so this year’s value is 2.8, which, due to rounding, appears as a small improvement over last year’s 2.7 value.
  • There are other journal metrics out there and I haven’t yet seen these values for 2017.
  • The JIF for Space Weather is 2.9, the first time that that journal’s JIF is bigger than the JIF for JGR Space Physics. Way to go, Space Weather!
  • AGU’s few-year-old journal Earth and Space Science received its first JIF this year, coming in at 3.2. Awesome job, ESS!
  • The journal had significant growth in terms of papers published from 2015 to 2016, up by 42, which is more than 5%. The Scholarly Kitchen just had a post a couple of weeks ago stating that journal growth lowers JIF. So, the fact that the journals JIF went up means that the citations outpaced the negative impacts of growth.
  • Historically, the citation rate to articles in JGR Space Physics are rather constant with time, so that a ~10 year old paper has ~29 citations, on average. This is just how our research community likes to cite papers and it would take a massive cultural shift to alter this trend.
  • JGR Space Physics historically has a “cited half life” of at or above 10 years, which means that a 10-year-old paper with X citations will, on average, end up with roughly 2X as its eventual total citation count.
  • Nearly all papers in JGR Space Physics receive at least one citation, which is not the case for nearly half of the papers in the vast Web of Science database.

In summary, I think that the journal is doing very well. Thanks for continuing to support it. Finally, while the post was mostly about the new JIF, I’d like to leave it where I started, on the positive future outlook of DORA, in which we put the JIF in proper perspective according to its strengths and weaknesses as a journal metric, and especially stop using it to assess individual research articles or investigators.

The Moldwin Paper on Citations

Mark Moldwin and I recently published a Commentary on, well, hopefully the title says it all: High-citation papers in space physics: Examination of gender, country, and paper characteristics. He obtained the article information for every paper published in JGR Space Physics in the year 2012, including the citation count as of June 2016 for each paper, and then classified the papers according to, you guessed it, gender, country, and paper characteristics. There were 705 papers in the journal that year, so this task took quite a while to complete, plus we took some time discussing which parameters to even classify for later use. We then analyzed these results to see which qualities about the paper had a statistically significant connection to citation count. A fairly recent year was deliberately chosen to investigate the factors related to citations early in a paper’s lifespan, a time interval of relevance to the calculation of the Journal Impact Factor. As of today, it is still “in press,” so just the accepted version is online, but the paper is Open Access so it is free to read the full text.

MoldwinPaper_header

            Here are the major findings. These qualities of the paper are correlated with more citations in the first few years after publication:

  • More coauthors
  • More institutions in the author affiliations
  • More countries in the author affiliations
  • More references in the paper
  • A colon in the title

These qualities of the paper had no significant correlation with citations:

  • Gender of the first author
  • Number of words in the title
  • Acronyms in the title
  • Geophysical region names in the title

Keep in mind that the standard deviations are wide, so these findings are not necessarily true when comparing any two papers from the “high” and “low” classes. Welch’s t-test statistic uses the standard deviation of the mean, which is a much smaller number than standard deviation (the spread for any one data point in the set), Any individual paper, regardless of its characteristics, could have a high or low citation count a few years after publication. That is, we did not find a “magic parameter” that clearly identifies what will make a paper get many citations, nor one that easily picks out the low-citation papers.

Furthermore, the underlying distribution of values is not Gaussian – but any subset we considered, there is a long, positive tail creating a non-negligible skew to the histogram – yet the probabilities for significance that we used are based on a normal distribution for the two populations. This is why we used a 99% “highly significant” threshold to determine those qualities that are connected to citations.

So, take all of these findings with a grain of salt. Nevertheless, we think the results are interesting for the space physics community to know. The main conclusions that more authors, institutions, countries, and references increases eventual citations are not particularly surprising, but this is the first time it’s been quantified for papers in the field of space physics.

Two results are surprising to us. The first is that there is not a statistical difference in the citation of papers based on the gender of the first author. Other studies have found such bias in other fields, including in other closely related natural sciences, like astronomy. Unlike those studies of other fields, we did not find a statistically significant difference in citations to JGR Space Physics papers based on that parameter.

We did not expect to find any “title parameters” to be connected with citations and most were not. We were rather amused, however, to find that a colon in the title is linked to higher citations. About 20% of the papers that year had a colon in the title. That’s over 100 papers so this is a decently large sample size. We have guesses but, really, we have no good explanation for this. For those wondering, yes, this finding did indeed influence the title of our paper.

In summary, our advice to potential authors of manuscripts for JGR Space Physics is this: collaborate with others and cite the literature. It’s not a guarantee that your paper will receive above-average citations but, based on our analysis, it might help. Happy writing!