EPISODE 1
the art of feedback

TIMESTAMPS

01:56 | Feedback
04:30 | Response from the Economist
07:14 | Twitter debate
09:51 | Guideline
15:15 | UX Planet Article
17:02 | Reader feedback in blog post
20:36 | Summary
21:23 | Q&A
22:08 | When should I create a graph?
24:17 | How to apply principles to dashboard design?
26:49 | How can I get my organization to think more analytically?
30:50 | Can you recommend some data viz 101 resources?
32:19 | Outro
32:38 | Quick tip

TRANSCRIPT

Welcome to storytelling with data the podcast where listeners around the world learn to be better storytellers and presenters with bestselling author, speaker, and workshop guru Cole Nussbaumer Knaflic we’ll cover a wide range of topics that will help you effectively show and tell your data stories so get ready to separate yourself from the mess of 3D exploding pie charts and deliver knockout presentations and with that here’s Cole.

Intro: Hi, I’m Cole and I focus on telling stories with data. Today you are listening to my very first podcast. I’m guessing if you are here you might be familiar with some of my other work. Perhaps you follow the blog or have read my book. The podcast is going to be a way for me to really put a voice behind the data and concepts for communicating with data.

Focus: The focus of our podcast today is going to be on giving feedback. Through this you’ll get a sense of some of the things I have been up to recently on the blog, discussions I have had at workshops and behind the scenes. We’ll also tackle reader Q&A where we'll talk about when and why we should use graphs in the first place, considerations for dashboards and reporting and I’ll also outline some data viz 101 resources. When it comes to this podcast, I am really trying this out. I want to see what sort of interest level there is for this medium and the type of content I want to talk about and I'll definitely be iterating over time. I welcome your feedback on that front and really any sort of feedback here will be useful everything from “this is great, keep doing it” to “hey, that was awkward maybe rethink that next time or hear some content that I'd love to see in a future episode. All types of feedback would be great. And you can provide that by sending feedback to feedback@storytellingwithdata.com.

Feedback [1:56]

Speaking of feedback, giving feedback is something that's come up for me in a lot of different ways lately I think there is immense value in being able to both give and receive feedback when it comes to refining our data visualization and the way that we communicate with data broadly and so today I want to talk about some considerations when it comes to giving feedback on data visualization. I’ll do this through a few different examples and scenarios I have encountered lately.

First off, I want to talk about the Economist hurricane challenge that this was one that I ran on the blog back in September and actually early on in September I found myself riding on the subway in New York City. I was there for a couple weeks doing some work. As I’m riding the subway, I'm scrolling through my feed on Feedly and I come across one of The Economist daily charts and it was titled “Hurricanes in America have become less frequent.” And I spent some time staring at this graph realizing I had some gripes with it but not really the time to do anything about it to fully critique it or do a makeover at that point so I thought I’ll turn it over to you, my readers and have you share your views and makeovers.

Now let me back up and explain a bit about what the graph looked like So this was a stacked bar chart the y-axis, the total showed the number of hurricanes that had hit the US and it showed this over time which was the x-axis component and then each bar was split up by the category of hurricane, 1-5. And the response I received to this was much more than I had anticipated. So I receive 60 makeovers! With readers raising a ton of what they believed were issues with the original graph and I thought it was interesting to see the tone of many of these note that came to me. The Economist did this wrong or they made that bad decisions and some people were pretty brutal which is actually incredibly easy to be when you're at arms length like this easy to forget that there is a person or a team who made the various design choices for a variety of reasons and I actually didn’t think about this so directly at the time but was reminded of it very clearly when The Economist emailed me. 

Response from the Economist [4:30]

So Alex Selby-Boothroyd who is head of data journalism at The Economist wrote me.

“Dear Cole, thank you for organising your fascinating “how you would visualize hurricanes” challenge. Here in the Economists’ data journalism department, we have poured over every entry with the same degree of interest and healthy debate that our original chart seems to have provoked. As with all data visualizations we had to make some compromises. The incomplete final decade was, as many people noted, unsatisfactory. Instead of leaving it to our readers to correct for the shorter time period in hindsight we could have used 5 your groupings like Robert did or 15 year clusters like Todd did to maintain consistent spacing on the x-axis. We could of course have improved other elements as well. With more time we could have incorporated every 2017 hurricane up to and including hurricane Irma while accounting for the fact that this year's season is not yet over. As Sharon noted we could have considered hurricane damage too or instead looked at accumulated cyclone energy. That said this data cover all Atlantic storms those that just made landfall in the United States as our article pegged “Irma hitting Florida” the previous day as a description of what has occurred in the past we thought that the headline and subheadline on the story “Hurricanes in America have become less frequent but the most damaging storms now appear to be slightly more common” was accurate and we felt that the trend lines helped to show this, but it's interesting to note that every visualizer who use trendlines to reach a similar conclusion there is another who loudly dismissed them as misleading or is equally vocal about something else we had done wrong. It was heartening to be part of a data visualisation community that has such a strong voice. All the best. Do let me know if we can be involved in any future challenges.”

So I thought it was awesome to see this note and the attention from The Economist data journalism department and it's a pretty amazing balanced response when you consider that they could have come from a very defensive position given that we have these 60 makeovers of different people outlining different issues or potential issues with that and yet the way I read it at least is a really balanced response and I also thought it was great the way that Alex highlighted the constraints. Right? The fact that they made compromises and trade offs for a variety of reasons. [6:57] And that's one of the things that when remake data visualizations we have no visibility into those constraints that the person originally visualizing the data faced.

Twitter debate [7:14]

And I was reminded of this same idea recently. I was following a heated Twitter exchange between a few folks Alberto Cairo, Elijah Meeks and Stephen Redmond. By the way one of the things I think is just fascinating about Twitter so Alberto is in Miami, Stephen is in Dublin, Elijah is in California, right. I am here following from California, so just the way that applications like Twitter allow these conversations to happen in a way that otherwise would never be possible but so they are having this debate. And Alberto links to this article at one point [7:49] In the article is one back from 2015 by Fernando Viegas and Martin Wattenberg. The article is called “Design and redesign in data visualization” and in this article they talk about this idea of popular criticism and how the data visualization field has become ripe for drawing this popular criticism in a way that you really can't do for other mediums, right? If you think of a book or a movie, someone can critique those things that they can’t actually remake them from start to finish if you are remaking a book, maybe you’ll rewrite a sentences, but there is now way you are going to go in and rewrite the book. Whereas in data visualization this actually happens. If you can get to the underlying dataset, anybody can remake a graph. And so in many ways this is good. It’s a way that people can learn but it can also be problematic and that’s what the article was really focuses on and the ways that this can be problematic so one of the things they highlight is how hindsight can be 20:20 and to illustrate this they look at [8:56] Edward Tufte’s criticism and remake of data from the Challenger space shuttle disaster so it's easy knowing what the important things were in that disaster to now go back and make data visualizations that make that brutally clear where is at the time that wouldn't have been possible in the same way because that information simply wasn't known at that point. Now, another way that popular criticism can be problematic is in the removal of context this really reminds me of the hurricane that makeover and that feedback that we heard from Alex that those who were doing the makeovers and providing feedback they just have no visibility in the constraints that those visualizing the data faced [9:47] and Fernando and Martin write “design is compromise” they also provide some 

Guideline [9:51] simple guidelines for data visualization critiques

1. maintain rigor

2. respect the designer

3. respect the critic

and I think tone and framing become so important when we're giving feedback on data visualization so that this is coming from a place of helping or friendly debate and not criticism in the negative aspects of that word in any case excellent article I’ll make sure that we link to it and I considered it a must read for anyone who is creating or critiquing data visualization

And I actually had a recent request for critiquing a data visualization. This came from a follower who has been working at a company where he says, historically, we've had this really data intensive presentation we have had done by our marketing team. They shared the presentation with me in it it had a lot of donut charts a lot of color. And he says, I recently hired someone to specialize in data visualization and I bought him your book. So now this new data visualization guru has revamped the ongoing version of the presentation. So stripped out the clutter made some other changes based on lessons from the book and he says we just delivered the presentation for the first time and we were met with mixed feedback. But we want to make sure that what we are doing is grounded in best practices and not just based on someone's personal preference and so he asked me to weigh in on this. And so, in this scenario it would have been really easy to jump to something like here is all the reasons that you are right as I see you are applying concepts from my book and here are all the reasons they were wrong with their donuts and their color. But I actually don't believe that's necessarily the case. And I definitely don’t think that is what would have been most helpful for anyone in this scenario. So, in this case, I brought it back to audience. If your audience doesn't like your approach, no amount of telling them why they should like it or why they're wrong for not liking it is going to help. Rather, for me, this means we're missing something that our audience needs. So going back to, I mean the different things we think about when we're crafting visualization and when we are following best practices and I actually wrote what this looks like from my perspective in a blog post back in September, “my guiding principles” and outlined the different things that I'm thinking about when I am creating and critiquing data visualization and the last guiding principle the final one is that audience trumps all else. I think to be a successful when it comes to visualizing data in a business setting we really need to be thinking about it audience first and foremost and if there's something that they are confused by or resist then we need to revisit the design and revisit the way that we are doing it and think about what our audience’s needs are and how those needs are being met because I think when we are able to do that, we lower the chances of having that sort of resistance. 

Now, when it comes to giving more specific data visualization feedback, I encounter this scenario in the workshops I do regularly so when I go in to do a half-day or full-day workshop with a team, I’ll typically solicit examples from the team ahead of time. After we go through the core lessons then we look at some of the groups’ specific work and we discuss how we can apply the lessons that have been covered to the folk specific work their work and colleagues work now as you can perhaps imagine this can be a very sensitive topic, right? This is someone's work but now a room full of people are critiquing and this is in the absence of those constraints as we've talked about and without actually having to do any work, right? They aren’t even remaking the examples, they are just outlining some gripes and maybe making some sketches. Now, the way I frame this up is not an exercise in ripping anything apart, but rather thinking about how we can discuss and how we can apply the lessons that we've covered to make the selected examples even stronger. I tell people, you now have discerning new eyes when it comes to looking at graphs and this tends to be both a blessing and a curse, but I challenge folks to use those new eyes that they have for graphs not only to critique because as we've seen, that easy to do when we're looking at other people's work, but also critiquing and applying those same new eyes to their own work. I think by being harsh critics of our own data visualizations we really put ourselves in a place where we can continue to refine them and ultimately make them successful. Now, that said, during this critique and especially receiving feedback can be hard—we get attached to our work.

UX Planet Article [15:15]

There was a recent article from UX planet on Medium called “The Subtle Art” that differentiates good designers from great designers. In the article, they list 5 differences between good and great designers. The first one that they list really really resonated with me. It said “for great designers, a critique is just a critique, not an insult.” And this is one of those things that is really easy to say and incredibly hard to put into practice. We get attached to our work and as we do we lose the ability to objectively see, right? When I have put together a data visualization I know where I want my audience to look I know the things that I want them to connect I know what I want them to walk away with the challenge though is making those that tacit stuff in my head visually clear to my audience so that they look at my graph they see it in the way that I want them to. Now, one of the ways that I get this sort of feedback is from readers, where I'll post something on the blog and then I have people who volunteer makeovers of my makeover or provide feedback and now in some cases this can be challenging, right? Because, as we have talked about, because those doing makeovers they don’t have visibility into the constraints that we face or for me the specific lesson maybe that I was trying to highlight through a particular makeover, but this is also an incredibly helpful part of the process for me as well as the person providing the critique, because it forces us when we critique to articulate our logic in a way that not only makes sense to us, but that will also makes sense to someone else, which can help us think through things in new ways.

Reader feedback in blog post [17:02]

This actually happened recently, this reader feedback. After a blog post that I had posted a couple weeks ago the blog post was titled “A tale about opportunity” and in the post I was looking at an example from the pharmaceutical industry where the original graph was a 100% stacked bar chart. The focus was the bar on the left was the number of patients who have been diagnosed with a particular illness and the bar on the right represented the patients taking this drugs for the particular illness and then the illness itself can be divided into different severities mild, moderate, severe. This was a case where we can actually switch the way that we were categorizing and instead of making the primary categorization those who were diagnosed vs those who were taking and then the categorization within that the severity of the disease, by actually breaking it out by the severity of the disease, first and foremost, and then within each of those stages looking at the number of people taking the drug out of the total diagnosed. This meant that we could flip it around and really highlight the opportunity. So doing it this way allowed us to see not only who's taking the drug, but who else could be taking a drug. To visualize this opportunity, I showed empty space that was outlined. So, imagine a horizontal stacked bar chart where the bottom series is dark blue this represents the number of patients currently taking the drug. Then stacked on top of that is a white series that is outlined in that same dark blue which represents the portion who been diagnosed with a disease but aren’t currently taking the drug—so those who could be taking the drug. Then, in the final version, I had some annotation on there. Listed out some percents in that same dark blue and the percents were meant to describe the white portion of the bar that was outlined in dark blue but it was probably 20 minutes after I posted the blog posted that I had a couple of reader comments saying “hey, that's confusing because when I read it I think about it I can figure out that I meant to tie those percents to the white portions, but when I first looked at it I want to tie the blue numbers to the blue portions of the bar.” This is one of those things that once somebody points it out, yes of course! That was a silly thing to do on my part. Of course it’s natural people are going to want to connect those things that look similar. And I didn’t even see it though when I was doing the graph because I knew where I wanted my audience to connect those pieces. And so, in this case, I used that reader feedback to actually go back and refine my final visual and added another one at the end that showed the opportunity outlined in black and change the percentages that were meant to describe opportunity to be in bold black text. This was a case where I was able to use some reader feedback to make the example even more effective. And now for me it's rarely about this thing better than that when we offer a critique or provide feedback, but rather more about optimizing tradeoffs because, as we have discussed, pretty much every decision we make when we are designing data visualizations in the slides and pages that contain them these decisions involve tradeoffs. So for me, it's about understanding what those tradeoffs are and making smart decisions in light of those things.

Summary [20:36]

So, in summary, giving feedback is a hugely important process of helping everyone be better at visualizing data. I do believe that when we provide feedback, we should be respectful and consider tone and framing, because, as we've discussed, we are critiquing someone else's work and that person likely face constraints that we have absolutely no visibility into. But I think also, as designers of data visualization, we need to have thick enough skin to recognize critique and take it as such take it as critique not as an insult. I’d love to hear your thoughts on this topic; please comment on the blog post so we can continue the discussion there.

Q&A [21:23]

Let’s shift gears now, though, and spend some time on reader questions. I announced this podcast a few weeks ago, where I talked about the fact that my kids as a ton of questions and that I have come to recognize that over time this is a really important part of their learning process. I was recognizing simultaneously that I receive a ton of questions from readers, those who follow my work as well so I thought a great segment in the podcast would be taking some time to answer some of those questions. I want to answer some of the questions I have received to date. Also if you have a related question that you would like to pose you can send that to askcole@storytellingwithdata.com.

Question 1 [22:08]

Matt says, my question is very basic, essentially when should I even be creating a chart as opposed to just relaying my big idea through text? I work in digital advertising and when I'm delivering insights to clients, it's usually text and one or a couple of sentences. I can understand when a chart is useful if you're dealing with a large number of variables and you just want to focus on one thing, but for these relatively simple insights would you even recommend making a chart? It is fascinated to learn the art of beautiful and useful chart. I just want to make sure they continue being useful to the audience.

Answer 1 [22:42]

This is a great question and I love the thoughtfulness behind it. I think anytime we think about showing data we should ask ourselves the question of: do we need to show this data? Just because we have that data doesn't mean we need to show it. And I think for me the sort of things I want to consider when it comes to whether I should be showing data one is just being aware of the value of visuals for making data accessible to helping someone understands or see or remember something. Images are powerful a graph being one sort of in the ability to prompt recall so if there's something in the shape of the data or there is something about showing it that is going to help you reinforce the message you want to make I think I can be immense value in pairing the visual—the graph—with the verbal—the story—so when I do that not only can my audience remember what they heard, but they can remember what they saw. I think another important component is certainly your audience. Is your audience going to need data to trust you, to be convinced? If so, then are you are going to want to think about how to weave that data directly into the story you want to tell. Ultimately, use graphs when they can help someone see or understand or remember and when they're going to be useful for your audience.

Question 2 [24:17]

Brian says, my question for you is how can I apply your principles and methods when creating and designing dashboard? I work for a large hospital in Boston and lately most of requests I get are to create dashboards the issue is that users consuming them is that they want everything on it I'm struggling with this because I know from experience and following your principles that a lot of the information they want is unnecessary please help.

Answer 2 [24:41]

I tend to draw one important distinction, which is between exploratory, which is what you do to understand the data when you look at it this way and that way to find the interesting insights and explanatory which is once you have found the interesting insights and you want to communicate those to someone else and for me dashboards and any sort of regular reporting, these fall more into the exploratory space where dashboards can be super useful at putting a lot of data into a small space where I can scan through and I can look to see where are things in line with my expectations or were where are they not in line with my expectations where might there be interesting things that I can further dig into to get at the story now one thing to keep in mind is that just because the dashboard is how you find the interesting takeaway or story does not necessarily and I would say many cases does not mean that the dashboard is going to be the best way to then tell that story to your audience so I think the dashboard can be incredibly useful for getting at that story and then when you have something specific you need your audience to know or an action you want them to take then then you actually want to take that data out of the dashboard and do a lot of things that I talked about in the book and on my blog to make that story look both visually and verbally clear now when it comes to dashboards and designing effective dashboards,

Big book of dashboards recommendation [26:07]

one resource I recommended is a book that came out earlier this year is The Big Book of Dashboards this is a collection of about 30 different dashboards across various Industries so it can be nice just to flip through and see ideas for how people have visualise different data and then the actual content is the author's commentary about what works well and what they might have done differently had they been designing the dashboards and goes through some alternate visuals there so it's a nice thing to flips through to get ideas and some great content to help develop an understanding of best practices. 

Question 3 [26:49]

Narule says, my question would be how to get people in the organization to think more analytically and not generate reports for the sake of generating reports I’m in the IT team doing data analysis which can be a bit tricky since the business teams are use to the IT teams doing a quick fix when something is broken but not some much process improvement.

Answer 3 [27:08]

So this actually makes me think back to my time at Google and what I think differentiated part of the analytics team that I worked on there is that we saw ourselves not only as analysts but also as coach and consultant to the folks who were we working with to provided data and insight and action. So I think one of the challenges for anyone who works data is that you show data and some one else asks for more data and then they ask for more data and more data and more data. And you can get this death by data and I think this sometimes comes about because of a false hope on the requestors side that more data is going to answer the question for them or tell them how to act when really at the end of the data there is a person always who's making that decision but one question that I found myself often asking that would be useful and you can think about what the right framing is depending on your role and the person with whom you are speaking help me understand how this new data is going to help you make a different decision or a better decision and you can frame that with the tone of help me understand so that I can better meet your needs. But often times is you can get the people requesting to articulate that and really think through their logic for asking for more data can sometimes talk themselves out of the need for my data which can be a useful way to cull this death by data that we've talked about there actually was as sort of related example at a workshop recently where we were discussing and there is actually an aha! moments of understanding that are you could see happen during the discussion and it was where the team recognised that they have historically seen themselves and seen their role as being primarily to inform and the discussion was about how they can move past that both themselves but then also the perception in the organisation as not simply being there just to inform but there to actually influence and drive action and I think there is really important pivot that happens as analyst when we think of our role in that way not simply to inform but to influence and drive action and help people both understand things better but understand them in the context of now how they can make changes or how that reinforces what's been happening from a business standpoint and I think for me this is a place for Analytics often stops short where it's easy to show data and outline some findings but the challenge and the risk in doing so is our audiences are faced by ton of data only so when we give them more data it's a very easy reaction for them to say “oh, that’s interesting” and move on to the next thing whereas if we take it to the next step and say not only here audience is the data, but here is how you should act based on the data, but here is a menu of potential actions you could take based on the data it gives our audience something to react to. That often starts a conversation. Even if they disagree and that’s a conversation that gets missed if we stop by simply showing the data. So I think always as an analyst think through not only what do I want my audience to know, but what do I want them to do with that and how do I make that clear and so it's a combination of being coach consultant helping people understand when and why they need more data but then also helping them think through the actions and really think through providing data is not simply to inform it's to help someone understand something better so that they can make a smarter decision or take a better action

Question 4 [30:50]

Alice from Melbourne <insert proper pronunciation> says I want to ask a very simple question if someone like me needs 101 material on data viz what resources do you recommend? Are there any books you consider data viz gospel?

Answer [31:01]

So I was scanning my bookshelf, trying to figure out how to answer this question and I think one that I recommend is Dona Wong’s the Wall Street Journal Guide to Information Graphics. It’s short, it's accessible, very easy to navigate and covers the basics so different types of charts but then also primers on how to do the math correctly things like how do you calculate percent change. So this would be a good one for anyone just getting started and wanting some basic guidance. Another book one of my early influences was Stephen Few’s Show Me The Numbers, so this is one that is a little more textbook-like, covers graphs and tables but also goes beyond that and get into visual perception and introduces the idea of helping your audience know where you want them to look in what you show. When it comes to data viz gospel, certainly Edward Tufte’s The Visual Display of Quantitative Information would fall into this category this is a beautiful and inspiring book perhaps less pragmatic than some of the others that I've mentioned. I will also mention my book, storytelling with data, which combines data visualisation best practices with storytelling for getting information across effectively in a business setting.

Outro [32:19]

Alright, so I’ve received any more questions then I can cover in a single session. If you asked one that wasn't answered here, please listen for the future podcasts and if you have questions you can email those to askcole@storytellingwithdata.com for potential future inclusion.

Quick tip [32:38]

I want to end with a quick tip here this was one that came up in a workshop a couple of weeks ago and when it was stated I said “Oh my goodness, I can’t believe I never knew that”, which is if you are working in PowerPoint and you save your file as .PPS, it will open the file directly into presenter mode for your audience to whom you are sending it, so try that out if that’s something you desire.

And with that I will say a very big thanks for tuning in you can get much more related content by following the blog at storytellingwithdata.com you’ll get regular tips and tricks from me by following on Twitter and Instagram @storywithdata you can like us on Facebook and do all that great stuff, so thanks very much for joining and I hope you'll tune in again soon.