the art & science of exploring data

In our work teaching people how to communicate effectively with data, we often find ourselves making a distinction between the exploration of data and the explanation of data. We do this both to draw attention to and make sure people clearly recognize the difference. These two phases require a totally different mindset and toolset. In one case, you are asking questions; whereas in the other, you are likely providing or directing people toward answers. They require different methods, perhaps different tools, and distinct visualizations. On the topic of visuals, those made for the exploratory vs. explanatory part of the process definitely require a different approach, particularly in their design. While complex (and even ugly!) can be just fine when analyzing data behind the scenes, we should prioritize efficiency, visual hierarchy, and generally appealing aesthetics when creating graphs and other visuals to explain something to an audience.

One dimension the exploratory and explanatory phases have in common, however, is that to do them well requires an art in addition to the science. I wish apprenticeship was still commonplace, because it would be a phenomenal way to learn exploratory data analysis. A new college grad could sit with a seasoned professional, watch as they work, and discuss why they do (or don’t) do what they do (or don’t). It’s one thing to know statistics. It’s quite another to work with a massive set of data and determine apt questions to ask, where to dig, when to dig further, when to stop, when to aggregate into summary metrics, when to disaggregate and look at every data point, whether outliers are interesting or should be omitted, and so much more. This is the art.

And, while apprenticeship isn’t typical, it doesn’t mean there aren’t opportunities to learn from people who’ve been analyzing data for the entirety of their careers. I’d like to share a couple of these with you.

I recently had a great conversation with Nathan Yau (perhaps best known as the person behind Flowing Data) about his freshly released 2nd edition of Visualize This (which I recommend unhesitatingly). As we talk, Nathan shares some of his tips for digging into data—including a great analogy paralleling the queries you pose to your data to the questions you might ask a prospective partner during the dating process. Have a listen to our discussion:

While Nathan’s work tends to focus more on data of personal interest, where frequently, the interesting bits are in the outliers, working with business clients often requires exploring data and then curating a cohesive story with it at a different level. You have an opportunity to watch the latter, via a recording of our recent mini-workshop. Data storyteller Mike Cisneros from our team will talked attendees through the process of exploring data and then subsequently explaining it in the construct of a specific business scenario. You can watch the mini-workshop recording in its entirety on YouTube now.

I hope these resources are helpful the next time you find yourself needing to explore data!