DORA and the JIF

No, this isn’t a young Latina’s adventure story about mantequilla de maní (or crema de cacahuate, or one of the other translations for peanut butter). DORA is the Declaration on Research Assessment and is a call to action to put less reliance on the Journal Impact Factor (JIF). This is timely because the new JIFs for 2017 were just released this week.

DORA - logo@2x_withblack

            I particularly like the “general recommendation” of DORA:

  • Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

It is a simple yet powerful message – the JIF (or any other journal metric) is not a measure of the quality and impact of any individual paper in that journal.

For 2017, the JIF for JGR Space Physics is 2.752, which is ever-so-slightly higher than last year’s JIF. A few notes about it:

  • Remember that the JIF is calculated only from citations in one year to papers published in the previous two years. It is an average of a highly skewed, non-Guassian, positive-definite distribution of a very small subset of full journal content.
  • I like to only quote the JIF to two significant figures, so this year’s value is 2.8, which, due to rounding, appears as a small improvement over last year’s 2.7 value.
  • There are other journal metrics out there and I haven’t yet seen these values for 2017.
  • The JIF for Space Weather is 2.9, the first time that that journal’s JIF is bigger than the JIF for JGR Space Physics. Way to go, Space Weather!
  • AGU’s few-year-old journal Earth and Space Science received its first JIF this year, coming in at 3.2. Awesome job, ESS!
  • The journal had significant growth in terms of papers published from 2015 to 2016, up by 42, which is more than 5%. The Scholarly Kitchen just had a post a couple of weeks ago stating that journal growth lowers JIF. So, the fact that the journals JIF went up means that the citations outpaced the negative impacts of growth.
  • Historically, the citation rate to articles in JGR Space Physics are rather constant with time, so that a ~10 year old paper has ~29 citations, on average. This is just how our research community likes to cite papers and it would take a massive cultural shift to alter this trend.
  • JGR Space Physics historically has a “cited half life” of at or above 10 years, which means that a 10-year-old paper with X citations will, on average, end up with roughly 2X as its eventual total citation count.
  • Nearly all papers in JGR Space Physics receive at least one citation, which is not the case for nearly half of the papers in the vast Web of Science database.

In summary, I think that the journal is doing very well. Thanks for continuing to support it. Finally, while the post was mostly about the new JIF, I’d like to leave it where I started, on the positive future outlook of DORA, in which we put the JIF in proper perspective according to its strengths and weaknesses as a journal metric, and especially stop using it to assess individual research articles or investigators.


The Moldwin Paper on Citations

Mark Moldwin and I recently published a Commentary on, well, hopefully the title says it all: High-citation papers in space physics: Examination of gender, country, and paper characteristics. He obtained the article information for every paper published in JGR Space Physics in the year 2012, including the citation count as of June 2016 for each paper, and then classified the papers according to, you guessed it, gender, country, and paper characteristics. There were 705 papers in the journal that year, so this task took quite a while to complete, plus we took some time discussing which parameters to even classify for later use. We then analyzed these results to see which qualities about the paper had a statistically significant connection to citation count. A fairly recent year was deliberately chosen to investigate the factors related to citations early in a paper’s lifespan, a time interval of relevance to the calculation of the Journal Impact Factor. As of today, it is still “in press,” so just the accepted version is online, but the paper is Open Access so it is free to read the full text.


            Here are the major findings. These qualities of the paper are correlated with more citations in the first few years after publication:

  • More coauthors
  • More institutions in the author affiliations
  • More countries in the author affiliations
  • More references in the paper
  • A colon in the title

These qualities of the paper had no significant correlation with citations:

  • Gender of the first author
  • Number of words in the title
  • Acronyms in the title
  • Geophysical region names in the title

Keep in mind that the standard deviations are wide, so these findings are not necessarily true when comparing any two papers from the “high” and “low” classes. Welch’s t-test statistic uses the standard deviation of the mean, which is a much smaller number than standard deviation (the spread for any one data point in the set), Any individual paper, regardless of its characteristics, could have a high or low citation count a few years after publication. That is, we did not find a “magic parameter” that clearly identifies what will make a paper get many citations, nor one that easily picks out the low-citation papers.

Furthermore, the underlying distribution of values is not Gaussian – but any subset we considered, there is a long, positive tail creating a non-negligible skew to the histogram – yet the probabilities for significance that we used are based on a normal distribution for the two populations. This is why we used a 99% “highly significant” threshold to determine those qualities that are connected to citations.

So, take all of these findings with a grain of salt. Nevertheless, we think the results are interesting for the space physics community to know. The main conclusions that more authors, institutions, countries, and references increases eventual citations are not particularly surprising, but this is the first time it’s been quantified for papers in the field of space physics.

Two results are surprising to us. The first is that there is not a statistical difference in the citation of papers based on the gender of the first author. Other studies have found such bias in other fields, including in other closely related natural sciences, like astronomy. Unlike those studies of other fields, we did not find a statistically significant difference in citations to JGR Space Physics papers based on that parameter.

We did not expect to find any “title parameters” to be connected with citations and most were not. We were rather amused, however, to find that a colon in the title is linked to higher citations. About 20% of the papers that year had a colon in the title. That’s over 100 papers so this is a decently large sample size. We have guesses but, really, we have no good explanation for this. For those wondering, yes, this finding did indeed influence the title of our paper.

In summary, our advice to potential authors of manuscripts for JGR Space Physics is this: collaborate with others and cite the literature. It’s not a guarantee that your paper will receive above-average citations but, based on our analysis, it might help. Happy writing!

Top-10 Papers of 2007

Here is one more post in this set of three with my lists of top papers from past years of JGR Space Physics. Here is the list for 2007; so, ~10 year-old papers. As a final reminder, these citation counts listed below were taken from Web of Science on December 30, 2017.

The list of Top-10 Most Cited Papers published in 2007 in JGR Space Physics:

  1. Summers, D., et al., Timescales for radiation belt electron acceleration and loss due to resonant wave-particle interactions: 2. Evaluation for VLF chorus, ELF hiss, and electromagnetic ion cyclotron waves, 310 citations
  2. Zhang, J., et al., Solar and interplanetary sources of major geomagnetic storms (Dst <= -100 nT) during 1996-2005, 267 citations (note: has a correction)
  3. Newell, P. T., et al., A nearly universal solar wind-magnetosphere coupling function inferred from 10 magnetospheric state variables, 232 citations
  4. Li, W., et al., Dynamic evolution of energetic outer zone electrons due to wave-particle interactions during storms, 214 citations
  5. Summers, D., Timescales for radiation belt electron acceleration and loss due to resonant wave-particle interactions: 1. Theory, 183 citations
  6. Lei, J., Comparison of COSMIC ionospheric measurements with ground-based observations and model predictions: Preliminary results, 167 citations
  7. Vadas, S. L., Horizontal and vertical propagation and dissipation of gravity waves in the thermosphere from lower atmospheric and thermospheric sources, 146 citations
  8. Meredith, N. P., et al., Slot region electron loss timescales due to plasmaspheric hiss and lightning-generated whistlers, 128 citations
  9. Omura, Y., et al., Relativistic turning acceleration of resonant electrons by coherent whistler mode waves in a dipole magnetic field, 118 citations
  10. Fejer, B. G., et al., Equatorial ionospheric electric fields during the November 2004 magnetic storm, 117 citations

For this crew, let’s suggest Totally Awesome stickers:


These are, like last time, from Zazzle.

Again, all of these papers have an average of over 10 citations per year. That’s high for any annual count of any paper in our field, but to sustain it for 10 years, that’s truly phenomenal. These papers, and their authors, deserve special acknowledgment. Congratulations on writing such highly-cited papers!

An interesting thing to point out is the one-two punch of the Summers et al. papers, both making it into this top-10 list. Paper #1 derives the formulas for the timescale analysis of several different combinations of plasma wave and energetic electron characteristics and then Paper #2 applies the specific wave properties for several magnetospheric plasma waves of particular relevance to the radiation belts. Danny and crew had a very good year!

Counting the research topics, I see one solar-heliospheric paper, one techniques paper (the “Paper #1: theory” one just discussed), one solar wind-magnetosphere coupling function study, three ionosphere-thermosphere papers, and 4 on radiation belt results. No planetary space environment papers made the top-10 list, but all of the other major disciplines within the journal scope are there.

Top-10 Papers of 2012

I’d like to continue with my lists of top papers from past years of JGR Space Physics. Here is the list for 2012. Again, the citation information used to generate this list was taken from Web of Science on December 30, 2017.

The list of Top-10 Most Cited Papers published in 2012 in JGR Space Physics:

  1. Meredith et al., Global model of lower band and upper band chorus from multiple satellite observations, 95 citations
  2. Usanova et al., THEMIS observations of electromagnetic ion cyclotron wave occurrence: Dependence on AE, SYMH, and solar wind dynamic pressure, 89 citations
  3. Min et al., Global distribution of EMIC waves derived from THEMIS observations, 89 observations
  4. Gjerloev, J. W., The SuperMAG data processing technique, 86 citations
  5. Schrijver, C. J., et al., Estimating the frequency of extremely energetic solar events, based on solar, stellar, lunar, and terrestrial records, 68 citations
  6. Jin, H., et al., Response of migrating tides to the stratospheric sudden warming in 2009 and their effects on the ionosphere studied by a whole atmosphere-ionosphere model GAIA with COSMIC and TIMED/SABER observations, 66 citations
  7. Jia, X., et al., Magnetospheric configuration and dynamics of Saturn’s magnetosphere: A global MHD simulation, 62 citations
  8. Dwyer, J. R., The relativistic feedback discharge model of terrestrial gamma ray flashes, 61 citations
  9. Fu, H. S., et al., Pitch angle distribution of suprathermal electrons behind dipolarization fronts: A statistics overview, 56 citations
  10. Park, J., et al., Effect of sudden stratospheric warming on lunar tidal modulation of the equatorial electrojet, 54 citations

You should go get yourself a Super! sticker:




Remember, all of these papers are only ~5 years old, so these citation numbers indicate a healthy rate of more than 10 per year since publication. That’s very high for our field.

I see one planetary study, one heliospheric, one techniques paper, three ionosphere-thermosphere related, and 4 magnetospheric physics focused papers in the list. So, the studies are distributed across all major themes of the journal’s scope.

That the top 3 are about plasma waves in the magnetosphere is probably not a coincidence; with the launch and commissioning of the Van Allen Probes in late 2012, this topic has become a central focus of the community since then. So, a stating-the-obvious take-away point from these examples: when there will be a flood of papers on a particular topic from a new spaceflight mission, it is perhaps useful to get a pre-mission paper published just before the prime mission phase.

Top-10 Papers of 2015

It’s been suggested to me that I should occasionally use this space to list the “top papers” in JGR Space Physics. I did this once but that was a while ago. As 2017 came to a close (on December 30, to be specific), I surfed to Web Of Science and downloaded the citation information with the “publication name” search term “Journal of Geophysical Research Space Physics.” I did this for papers published in a few selected years: 2015, 2012, and 2007; so, 2, 5, and 10 year-old papers. The 2015 papers will be skewed a bit due to the proportionately large age difference from January 2015 to December 2015, but this is a year included in the Journal Impact Factor, so I thought I’d include it here. Also, not all of the 2017 citations to papers are included in WoS yet, especially from papers published late in the year. Still, these citation values are fairly complete and can provide insight into top papers in these years.

Yeah, this is how I spend my Saturday evenings. Don’t worry about me, though, it didn’t take that long.

I’ll spend a few posts here in January analyzing these citation reports. I won’t go into too much detail, as I know that there is a detailed manuscript on this topic in works. Top 10 lists are good to share, though, as are some basic stats on citations for these specific years.

For this first post, here is the list of Top-10 Most Cited Papers published in 2015 in JGR Space Physics:

  1. Kurth et al, Electron densities inferred from plasma wave spectra obtained by the Waves instrument on Van Allen Probes, 79 citations
  2. Livadiotis, Introduction to the special section on Origins and Properties of Kappa Distributions: Statistical Background and Properties of Kappa Distributions in Space Plasmas, 53 citations
  3. Astafyeva et al., Ionospheric response to the 2015 St. Patrick’s Day storm: A global multi-instrument overview, 50 citations
  4. Saikin et al., The occurrence and wave properties of H+-, He+-, and O+-band EMIC waves observed by the Van Allen Probes, 43 citations
  5. Jaynes et al., Source and seed populations for relativistic electrons: Their roles in radiation belt changes, 39 citations
  6. Li et al., Statistical properties of plasmaspheric hiss derived from Van Allen Probes data and their effects on radiation belt electron dynamics, 35 citations
  7. Saur et al., The search for a subsurface ocean in Ganymede with Hubble Space Telescope observations of its auroral ovals, 33 citations
  8. Engebretson et al., Van Allen probes, NOAA, GOES and ground observations of an intense EMIC wave event extending over 12 h in magnetic local time, 32 citations
  9. Li et al., Upper limit on the inner radiation belt MeV electron intensity, 31 citations
  10. Ni et al., Resonant scattering of outer zone relativistic electrons by multiband EMIC waves and resultant electron loss time scales, 29 citations

These authors all get a gold star for writing a highly-cited paper:


If you need more gold stars, you can buy them for yourself here, where I got the image.

I am not sure if there are any lessons to learn from this list, but it is fun to share it and commend these authors on a job well done. Here are a couple of other tidbits about the list.

The truly surprising one on this list, at least to me, is #2: the special section preface.  Over 50 citations to a preface in just under 3 years is, well, amazing. If you have a look at it, though, then you will quickly realize that it is a tutorial on the topic of Kappa distributions in space plasmas, with 82 references to papers published in a wide range of years, from 1862 to 2014. It’s really a topical review.

Also, 6 of the 10 are about the Earth’s radiation belts or plasma waves relevant to this particle population. This is not surprising given that, in 2015, NASA’s Van Allen Probes mission was just finishing its prime mission phase, with a full scan of local time of data available for analysis. The continued success of this mission’s data set for scientific discovery has propelled radiation belt papers to the top of this list. The top-cited paper (Kurth et al) is not included in this count of 6 but is related to the topic, being a study of the thermal plasma density in Earth’s inner magnetosphere from this same mission. Because the thermal plasma density is a critical controlling factor for plasma waves and wave-particle interactions, it should probably be added to the count, making it 7 of 10. It was a good year for radiation belt papers.