The Cost of Astrophysics


One of my favorite posts so far on If We Assume was "The Pace of NSF Funded Research", in which I showed that NSF-funded astronomy grants produce papers for up to 15 years! I made that figure while on an airplane with my friend Eric (who does cool stuff like this!) so that's fun too.

The data for that project came from the brilliant people at Harvard's CFA Library, who gathered every Astro paper published since 1995 that referenced a NSF AST grant. When they updated this database to include the budget amount for each grant, and were kind enough to notify me, I knew it was time to do a follow-up post!

The question that immediately jumped to my mind: 
How much does a typical Astronomy paper cost taxpayers?

Caveat Lector
I want to acknowledge this kind of analysis could be seen as inflammatory, insulting, or misleading. Please consider it in the lighthearted spirit it was intended.

1. A Typical AST Grant Costs $249k

Here I'm just showing a simple histogram (with log $ bins). Almost all grants are a few hundred-thousand dollars. The typical (median) is $249k, which for reference would pay for about 4 years of support for a graduate student at the UW, including overhead, tuition, salary, publication/page charges, 2 new computers, 4 domestic conferences, and a couple international conference trips.

2. Typical Grant size has started to drop recently

The orange line traces the median grant size each year. Our new tradition in America is to evidently not pass federal budgets. I'm not going to claim this is the cause of the drop in median grant allocation, but it's interesting that the last time a budget seems to have been passed in this country is 2009... My belief is that the NSF has tried to keep scientists from leaving the field, so giving out smaller grants means more people can still pay their rent.

3. A typical paper costs about $20k

According to some very simple (read: bad) math, take the # of papers produced divided by the budget of the grant and you get some kind of "cost per paper". This assumes that papers are the only real product of research, which is not entirely true. Conspicuously, this is about on par with a year's stipend for a graduate student (not including overhead and tuition, which about doubles that cost). I don't know if people will think this is too high or low (what is the going market price for a paper?) but the more I consider it the better a deal it seems! 

Here is an obtuse way of looking at this. Orange lines track the cost per paper versus grant size for fixed numbers of papers. Kind of silly

4. Paper costs are remarkably stable since 1995

There is a slight steady increase, but generally this is quite flat. The steep rise in the past 5 years is due to grants not yet reaching their full measure (see first post about grant productivity)

5. Small grants are more "efficient"

Maybe this goes without saying, and maybe this is the stupidest result of this entire analysis, but the best "bang for your buck" is in small grants... especially if they're reasonably productive! Naturally this kind of metric rewards people who cite every grant they've ever worked on in every paper, but is that a bad thing?

Below I show the "papers per dollar", literally inverting the metric from before (# of papers produced / grant amount). Once again we assume that papers are all that matters. In red I've highlighted the "most efficient grant", that which produced the most numbers of papers for the least number of dollars. (note this may be supplanted as newer grants continue to rack up papers)

By the power vested in me by the internet, I pronounce Detailed Modeling of Radiation Transport in Supernovae (1998) the most efficient AST grant since 1995, with 56 papers citing the grant and a meager $50267 awarded. Congratulations to Dr Peter Hauschildt.

6. Bigger $ grants don't necessarily yield more citations

If the number of papers is related to the "productivity" of a grant, the number of citations probes the "impact" of a grant. Interestingly, there does not appear to be much correlation between expensive grants and more "impactful" science. Take from that what you will.

I am also pleased to announce the winners for highest "impact per dollar" (literally # of citations for the grant / cost of grant). Below in blue I have marked the winner, Submillimeter Studies of the Cosmological Star Formation and AGN Histories (2000) with 3157 citations and only $37159! Well done, Dr Lennox Cowie. A slim $11.77 per citation! Notable runner up in this category is again Detailed Modeling of Radiation Transport in Supernovae (1998) in red, with 3571 citations.

Lastly: Citations versus Papers

I also realized that this database provided an interesting testbed to consider how papers gather citations. Generally this is a topic of great debate and interest, especially for young researchers. Below I've plotted the # of total citations a grant receives versus the total # of papers it produced. Of course this should show some correlation.

Also shown for reference is the "1:1 line" representing 1 citation for every paper (a baseline for impact?),  the "20:1 line" indicating 20 citations for every paper (reasonably good I'd say!), and something I've dubbed the "Line of Self-Citation". This curious line was calculated like so: if every subsequent paper you publish contains a citation for every previous paper you've published. I guess this would be better called the "Line of Cumulative Self-Citation". 
Obviously citation behavior never literally follows this Line of Self-Citation; imagine how horribly boring a paper with 100 different self-citations would be. Also - I'm not sure if this database has intentionally removed self-citations (sometimes done). What I find curious is that this Line of Self-Citation does a reasonable job of at least going through the data.

Finally: I'm not sure what to really make of this last figure, but I don't think I've ever seen anything quite like it. Have you? I'd love to hear your thoughts/feedback!

The Reddit Effect - II

1 comment:
Today I'll share a couple observations about web traffic. Take from it what you will.

Below are two charts/tables, directly taken from my "dashboard". The first lists my Top 10 Articles, ranked by total numbers of pageviews. This shows a smooth exponential-ish distribution, not too heavy on any single article.

(apropos: I really like the built-in stats tools with Blogger!)

The second chart lists the Top 10 Traffic Sources for this site. The traffic from Reddit is more than double that of all other sources combinedWOW! 

This isn't to say that Reddit is the best place to advertise your work. It can go largely unnoticed if you don't participate in the Reddit community, and getting traction within any social news aggregator is often a subtle game. However, your potential exposure can be much higher than places like Facebook.

These stats also don't account for external exposure. For example, I'd wager more than 865 people read Huffington Post's coverage of my Starbucks post.

My intuition is that I need to diversify my readership sources some, that Reddit doesn't necessarily create a stable base (for a host of reasons). But I'm making this whole blog thing up as I'm going, just trying to do my best, so who's to say what's "best"?

April Fools - UFOs and the Humorous New Frontiers of Science

No comments:
"And now for something completely different..."
Today I have posted my first April Fools arXiv paper: Detection Rates of Unidentified Moving Objects in Next Generation Time Domain Surveys. It semi-seriously explores the possibility for LSST to place real limits on the visitation rate of UFOs to our world. This is an idea I'd been kicking around for a few years - it's silly, but not altogether absurd. I'd love to know what you think!

Like many astronomers, I read the astronomy section of the arXiv (astro-ph) daily over coffee. It is a repository where researchers post manuscripts for rapid (and free) dissemination and archival.

Link of interest: When to post to arXiv? (via AstroBetter)

I became aware of April Fools paper on the arXiv a few years ago, which range from silly inside-jokes between friends to the more subtle. My favorite is when you only realize the paper is a joke after you've started reading it! These are in short supply, but every year one or two come along.

More seriously, I love that the arXiv provides a reasonably legitimate forum to publish things that are more complex than a blog post, but perhaps less rigorous than a paper. Especially given how expensive it is to publish (page charges are routinely more than $125/page for authors), the arXiv gives scientists a valuable alternative.

There is value in the absurd. 
Especially in astronomy, we must entertain the totally bizarre and fringe (at least to a point). In this age where astrophysics is becoming truly hard, less funded, and driven by massive collaborations, I have heard it said that astronomers risk becoming less creative. If you only do science that's a "sure thing", if you're not willing to speculate a little, if you haven't the guts to try something new or engage in a bit of academic creativity, then our majestic enterprise will surely fail.

So perhaps April Fools can also be a day where we shamelessly trot out some fun ideas, some semi serious or even speculative notions. We could create a Journal of Speculative Astrophysics specifically for ideas whose time may not have come just yet, one edition per annum (Fritz Zwicky could be the editor in perpetuity).

Or maybe I'll be unemployed on Tuesday! Either way, I want to believe...

A short list of other past April Fools papers...
If you know of any other real gems, drop them in the comments below or shoot me a line!!