One of my favorite posts so far on If We Assume was "The Pace of NSF Funded Research", in which I showed that NSF-funded astronomy grants produce papers for up to 15 years! I made that figure while on an airplane with my friend Eric (who does cool stuff like this!) so that's fun too.The data for that project came from the brilliant people at Harvard's CFA Library, who gathered every Astro paper published since 1995 that referenced a NSF AST grant. When they updated this database to include the budget amount for each grant, and were kind enough to notify me, I knew it was time to do a follow-up post!
The question that immediately jumped to my mind:
How much does a typical Astronomy paper cost taxpayers?
I want to acknowledge this kind of analysis could be seen as inflammatory, insulting, or misleading. Please consider it in the lighthearted spirit it was intended.
1. A Typical AST Grant Costs $249k
Here I'm just showing a simple histogram (with log $ bins). Almost all grants are a few hundred-thousand dollars. The typical (median) is $249k, which for reference would pay for about 4 years of support for a graduate student at the UW, including overhead, tuition, salary, publication/page charges, 2 new computers, 4 domestic conferences, and a couple international conference trips.
2. Typical Grant size has started to drop recently
The orange line traces the median grant size each year. Our new tradition in America is to evidently not pass federal budgets. I'm not going to claim this is the cause of the drop in median grant allocation, but it's interesting that the last time a budget seems to have been passed in this country is 2009... My belief is that the NSF has tried to keep scientists from leaving the field, so giving out smaller grants means more people can still pay their rent.
3. A typical paper costs about $20k
According to some very simple (read: bad) math, take the # of papers produced divided by the budget of the grant and you get some kind of "cost per paper". This assumes that papers are the only real product of research, which is not entirely true. Conspicuously, this is about on par with a year's stipend for a graduate student (not including overhead and tuition, which about doubles that cost). I don't know if people will think this is too high or low (what is the going market price for a paper?) but the more I consider it the better a deal it seems!
Here is an obtuse way of looking at this. Orange lines track the cost per paper versus grant size for fixed numbers of papers. Kind of silly
4. Paper costs are remarkably stable since 1995
There is a slight steady increase, but generally this is quite flat. The steep rise in the past 5 years is due to grants not yet reaching their full measure (see first post about grant productivity)
5. Small grants are more "efficient"
Maybe this goes without saying, and maybe this is the stupidest result of this entire analysis, but the best "bang for your buck" is in small grants... especially if they're reasonably productive! Naturally this kind of metric rewards people who cite every grant they've ever worked on in every paper, but is that a bad thing?
Below I show the "papers per dollar", literally inverting the metric from before (# of papers produced / grant amount). Once again we assume that papers are all that matters. In red I've highlighted the "most efficient grant", that which produced the most numbers of papers for the least number of dollars. (note this may be supplanted as newer grants continue to rack up papers)
By the power vested in me by the internet, I pronounce Detailed Modeling of Radiation Transport in Supernovae (1998) the most efficient AST grant since 1995, with 56 papers citing the grant and a meager $50267 awarded. Congratulations to Dr Peter Hauschildt.
6. Bigger $ grants don't necessarily yield more citations
If the number of papers is related to the "productivity" of a grant, the number of citations probes the "impact" of a grant. Interestingly, there does not appear to be much correlation between expensive grants and more "impactful" science. Take from that what you will.
I am also pleased to announce the winners for highest "impact per dollar" (literally # of citations for the grant / cost of grant). Below in blue I have marked the winner, Submillimeter Studies of the Cosmological Star Formation and AGN Histories (2000) with 3157 citations and only $37159! Well done, Dr Lennox Cowie. A slim $11.77 per citation! Notable runner up in this category is again Detailed Modeling of Radiation Transport in Supernovae (1998) in red, with 3571 citations.
Lastly: Citations versus Papers
I also realized that this database provided an interesting testbed to consider how papers gather citations. Generally this is a topic of great debate and interest, especially for young researchers. Below I've plotted the # of total citations a grant receives versus the total # of papers it produced. Of course this should show some correlation.
Also shown for reference is the "1:1 line" representing 1 citation for every paper (a baseline for impact?), the "20:1 line" indicating 20 citations for every paper (reasonably good I'd say!), and something I've dubbed the "Line of Self-Citation". This curious line was calculated like so: if every subsequent paper you publish contains a citation for every previous paper you've published. I guess this would be better called the "Line of Cumulative Self-Citation".
Finally: I'm not sure what to really make of this last figure, but I don't think I've ever seen anything quite like it. Have you? I'd love to hear your thoughts/feedback!
