The Lego Wall

This has nothing to do with data visualization, but I'm counting it as a hack project.

My wife and I have had an unused fireplace in our downstairs den. We were looking for a colorful and fun way to cover it so our pets wouldn't climb in. This is our solution. It is brilliant, which is how you can tell Sarah (wife) came up with it!

Check the rest of the album out here...

Lego Wall

Happy Holidays!

2016 Election Issues, by the Numbers

You might have read or heard somewhere that in 2016 the US will once again hold a presidential election. For my international readers, a US election is a lot like a game of poker being played by circus animals on live TV: there may be a "winner", but typically we all lose.

There was much media chatter after the Democratic candidate debates (e.g. on NPR) saying how they focused more on "the issues" than the Republicans. This might be empirically true, given how much smaller a field the Democrats are currently competing in. It got me thinking: what are these "issues", and what does each candidate actually say about them?

I'm of the opinion that a good metric to judge a presidential candidate by is their website. They have total control of the content, layout, and aesthetic, and most are well funded (or, well enough to hire a startup web developer at least). You should be able to see what the candidate really thinks of their own image and platform, and their stance on "the issues". It's amazing how terrible some of these websites are...

Words Words Words

Still, websites offer a ready source of information on the distilled message each candidate has prepared on "the issues".  So to gather this data, I went to each of the candidate's websites, found whatever page (or usually pages) they had listed as "issues" or similar, and wholesale copied all the text. This totaled over 128k words, 74k from the Democrats and 54k form the Republicans. Here are two word clouds (generated via wordle.net) showing all of this text for the Democrats and Republicans:

These simple word frequency clouds show fascinating contrasts and similarities. The top words for the Dem's include candidate names, while for the GOP it's "President" and "tax". Over representation of social issues is clear for the Dems (e.g. "health", "care", "women") while the GOP has a strong emphasis on "Security". The word "government" is much more prominent for the GOP than the Dem's.

When I dig down in to the small/medium words, I'm struck by how similar they really are. The main bipartisan "issues" of the day appear to include "security", "jobs", and "health", but that probably doesn't surprise anyone.

Comparing Numbers

While Martin O'Malley may not have had a ton to say in the democratic debates, he certainly has a lot to say online. We get it, O'Malley, you probably were a NaNoWriMo winner three years running too.

On the other end of the spectrum (both in politics and verbiage) is Lindsay Graham , whose website squeaked out a dismal ~2200 words... about the length of two Op Ed pieces in the NYT or WSJ.

I don't know what the optimal number of words is, but I find it fascinating how few some of the GOP candidates have put up, and how much more the Democrats did. Speculating, the paucity of words from GOP candidates could be due to a believed lack of interest among potential voters in long-form content. Typically the GOP websites had fewer topic headings in their "Issues" pages (a notable exception is Rand Paul's page).


One of the best and worst types of analysis you can do with written language is study it's complexity. A very simple version of this, which I've featured repeatedly on this site in the past, is the Flesch-Kincaid Reading Level. This metric attempts to compute how complex language is by measuring the # Words per Sentence, and the # Syllables per Word.

There's good reason to think this is not a fair measure of intellectual complexity, but still there have been curious trends found with it in the past. You typically see a slew of articles about the Readability or Reading Level of State of the Union speeches each year. Also, the government requires some documents to pass certain Reading Level tests to promote more accessibility to the laws of the land.

With appropriate caveats in mind, here is the readability metric for each candidate's "Issues" page(s):

Trump's "Issues" page ranks lowest in terms of reading grade level. Bush isn't far behind him. Interestingly, they had the most number of total words among the GOP sources.

Rubio tops it off by a wide margin. Some say the ideal sentence should be 15-20 words. This randomly selected sentence has 42 words, and certainly contributes to the complexity of his text:

"The horrific mass shootings should prompt us to ask what causes people to commit these acts — like what can be done to improve the way we treat serious mental illness — rather than seize on the weapons they used." [source]

The Dem's, however, are all over the map compared to the GOP, but really are quite close in actual score. I suppose I'm not surprised O'Malley is the champ here too. Given there's only 3 candidates, and no strong outliers, it's not clear what we can say overall about the Dem's readability scores.

Pick your Favorite

For fun, here's the word cloud for each of the candidate's "Issues" pages. (pro tip: make a drinking game with these, maybe for the next debate?) I'll leave these without comment, except to say I'm sure you can interpret them any way you please.

View post on imgur.com

There's a lot you can learn from a bag of words, and a lot of neat games you can play. I've put the data and snippet of code freely available on GitHub. A neat project would be, say, to create a probabilistic sentence generator based on text from each candidate... It would also be awesome to see what we might learn from archived webpages of successful candidates from elections past. Does text-heavy win? Or are people excited about the short-and-sweet message?

I can't say I'm excited about this coming election season, but at least it will give material for Saturday Night Live to work with... and that's something we can/should all appreciate.

Awesome visualization: Kepler Orrery IV

Today my good friend, hopeless Cubs fan, and supremely talented astro PhD student, Ethan Kruse released a new version of the Kepler Orrery animation.

For those not familiar with the term, an "orrery" is a mechanical model of the solar system. These types of animations have become popular in the era of exoplanets to demonstrate the incredible variety of planetary systems we have discovered. Many look very strange compared to our own, but some like Kepler 452 look tantalizingly similar.

Check out Ethan's work below, or download the animated gif version here.


Edit: Like an open source champ, Ethan has even provided the source code on GitHub! Awesome!

How I Cite

A while back this question was posed on Twitter:

Hey astro-twitter, I'm a grad student writing a paper and I need your help with citation stuff! (@astrobetter seems like a good taggable)

— Tom Rice (@tomr_stargazer) June 8, 2015

I offered to "quickly write something up", and in true academic fashion... it took 3 months!

Quick Guide: how I manage citations in LaTeX for writing papers and my thesis

Note: This guide does not tell you which papers to cite... All I can advice is you should spend a long time reading the literature, especially from before you were born (applies to all age groups)

When I turned it in, my PhD thesis totaled 221 pages front-to-back. Like most astronomers I wrote my thesis in LaTeX. Apparently I could have written it in Word, but I cannot fathom how people write long documents in that program given how frequently it crashes on my Mac. Also I have no idea how it would have handled my citations.

My thesis was the culmination of many years of work, and included text from 3 papers I had previously published in various journals - all using LaTeX. One chapter contains material for a soon-to-be-published paper as well. All told, my thesis included citations to 236 unique sources!

Dealing with so many citations was (nearly) trivial because I had invested early on in managing all my references!

Writing the thesis was still a laborious job, of course.

You could simply type all your citations according to the proper style, and format your bibliography manually. What if you forget one? What if it's a preprint that needs updating? I also assume overworked journal editors would hate you for doing this. Instead, use the \cite command, and follow this guide!

tl;dr

Get bibtex citations for every paper, put them in BibDesk to make a nice "library". Every paper or manuscript, just point your bibliography to this one file. The end.


Here's what my workflow looks like:

0. Install the TexShop bundle

This includes TexShop (a very nice text editor for TeX) and BibDesk. It also includes a handy tool called LaTeXiT, which you can use to easily typeset pretty equations to copy/paste as an image in to presentations!

Other software to organize your library is available. I have no experience with and cannot comment on them, except to say: TexShop is free and stable.

Building Your Library

1. Find papers you like

Every paper I see that I might want to cite later I keep a reference to. 99% of the papers I'm reading are indexed in the SAO/NASA ADS. Occasionally I find papers or articles that aren't, but that's ok. At time of writing I have 883 papers store in my library that I can cite.

2. Get the BibTex reference for each paper

This is the magic of the whole system. BibTex is a bit of tex-nology (hah!) that is from the mid 80's, that provides a standard format for citations in LaTeX

If you're using the classic ADS, here is where you click to find the Bibtex entry for every paper:

If you're using the new ADS, here is where you click to find the Bibtex entry for every paper:

Easy!

This is what a BibTex entry from ADS might look like:

Say, that's pretty readable! Almost every academic journal website will also provide BibTex citations for their articles. So does Zenodo. Everybody wins with BibTex!

3. Put the BibTex reference in to BibDesk

The first time you do this, open BibDesk and create a new library (I call mine "references.bib"). Every subsequent time you do this, open that same library file

Copy and Paste the entire BibTex entry directly in to your library in BibDesk. Be sure to save your library. 

Now the paper has been added to your library!

Probably you want to change the default citation key string. BibTex entries from ADS will have obnoxious default strings like "1988PASP..100.1134B", which in your LaTeX paper you'd have to cite as:

\cite{1988PASP..100.1134B}

Yuck. Instead, double click on the paper entry and edit this field in BibDesk. I'd probably change "1988PASP..100.1134B" to be "bessell1988", making the citation in my paper even easier to read:

\cite{bessell1988}

3b. Add the PDF of the Paper for Later Reading

This is the beauty of using BibDesk as the GUI manager for your library. Each paper you save can also have supplemental things attached to it, like a PDF file! Hopefully someday (soon) we'll all be able to have internet access 24/7 anywhere, and that journals will make web-based articles that we actually want to read. That day is not yet here, and PDF is often the only useful format for a paper.

The manual way:
- Add citation to BibDesk (see #3 above)
- Download the PDF
- With the article selected in BibDesk, drag/drop the PDF in to BibDesk.

The auto-magical way:
Jonathan Sick has an awesome widget that will add the BibTex entry AND the paper to your library. Awesome. I use this a ton, and should buy Sick another beer next time I see him.

Now when you want to read the paper itself, just search your library for the paper (by author, year, etc) and then double-click on the PDF. So helpful!

What your BibDesk might start to look like...

3c. Add Bonus Stuff to the Entry!

Just like you can store the PDF attached to a paper, you can store basically anything. If a useful paper makes a long table available as an ASCII file, I will save the file and drop it in BibDesk. This is especially helpful if you've taken the time to digitize or transcribe something from an old paper!

4. Organize Your Library

Honestly, the GUI and organization is kind of clunky in BibDesk. I had to move and resize lots of panels to get it to look nice. There are a TON of features I don't know how to use, from citation formatting to connecting to web-based libraries.

One nice feature: You can create "Static" groups (basically lists), perhaps by topic or author, and drag papers in to them from your main library. I have groups for major separate research ares that I'm reading about. This is very helpful when you start to forget that one obscure paper in some topic you meant to reference....

You can have BibDesk autogenerate a nicer cite key, so that you just hit Command+k and it will reformat it for you. Very handy! Here's my settings to enable my preferred cite key style:

The Format String is: %a1%Y%u0

5. Use the Library

Now in every paper or thesis you write, point to the same file, forever, the end.

When I'm writing a paper, I usually have to include these lines in the top of the file:
\bibliographystyle{apj}
\usepackage{natbib}

This example requires the journal-specific file "apj.bst" to be available. You can either have it in the paper working directory, or someplace TexShop can always see it.

If your bibliography style doesn't know the standard AAS journal abbreviations used by ADS and others, you might need to include near the top of your document a reference to aas_macros.sty:
\usepackage{aas_macros}

When I want to reference a paper from the library I just use the citation key as needed. For example:
\citep{davenport2014b}

This is the last 2 lines of every paper I write:
\bibliography{/Users/james/Dropbox/references.bib}
\end{document}

6. Typeset Your Paper

If you're using TexShop, this is the typical sequence to typeset:
LaTeX (shortcut is CMD + t)
BibTex (shortcut is CMD + b)
LaTeX (shortcut is CMD + t)
LaTeX (shortcut is CMD + t)

The underlying BibTex magic will create the needed .bbl file locally that holds all the references you've cited. If you mistype a key, or try to cite a paper you haven't downloaded, TexShop will warn you in the second typeset step. LaTeX will organize and format the bibliography as needed.

7. Submit Your Paper

To ensure things work smoothly, usually when I submit a paper I put the references directly in to the manuscript file. This makes 1 less file to upload and keep updated, but there are downsides too...

Just delete the line pointing to your "references.bib" file from Step 5, and copy the contents of "your_paper.bbl" in its place. When typesetting now, don't include the BibTex step.

Probably you don't need to do this usually, but if an automated LaTeX system is messing your references up it can help.

Conclusion

By using this system for many years I was able to copy/paste my published papers in to my thesis without worrying about my references. Whenever I start writing a new paper it's trivial find papers to ensure the bibliography will work.

What did I miss? Want to advocate for a different tool or approach? Am I totally wrong? Tweet @ me or leave a comment below!

8. BONUS: Keeping it synced between multiple computers!

I keep my master references.bib file in my Dropbox folder, which makes it easy to always point at the same file on multiple computers... simple enough.

But I also keep the Papers library (e.g. papers, datafiles, etc - see 3c above) in sync! Make sure you point it at Dropbox folder as well: