Test Pattern
I feel a little cruel starting with this one but hey, I’m just warming up. May as well make it easy on myself.
Here’s an article from the Sydney Morning Herald, my local broadsheet newspaper, from 6th July 2005 entitled Too much TV a learning turn-off.
And here’s the graph that accompanied the article.
The moment I saw it my alarm bells chimed distractingly. It’s like it was put together intentionally as an example of Tufte’s chartjunk.
Shiny 3D bars. Meaningless decoration. No clear relationship between what’s being measured and what’s shown. You see, what’s being measured is:
- Educational test results in three subjects
against - Four conditions:
- having a TV in a bedroom
- not having a TV in a bedroom
- having a home computer
- not having a home computer.
So the obvious thing to do is to put the test scores up the Y axis (OK great, the graph does this), and the four conditions along the X axis; but no, instead we have the three subjects along the X axis, as if this was a comparison between them… and then to facilitate the real comparison they’ve had to break it into two graphs, and…
This all seems kind of petty, I suppose. Sure, it’s a weird choice of abscissa, and the 3D bars aren’t strictly necessary, and the stock photo of a mouse on a desk behind everything is a little distracting, but it’s not like the data is actually being misrepresented, is it? After all, here’s the relevant article text:
The research, published in the Archives of Pediatrics & Adolescent Medicine, found that those with a television in their rooms scored 53 in maths, 48 in reading and 51 for languages in a widely used educational test. Marks for those without a set were 63, 55 and 60 respectively.
So the point is that having a TV in a kid’s bedroom will lower their test scores, and the graph shows… Oh. This is where we discover that it’s not a false alarm. The graph shows the opposite. The purple bars, representing Bedroom TV are showing high test scores, and the gold bars representing No Bedroom TV are showing low scores. And this isn’t a one-off - the key has been reversed on the other graph as well. While the article says:
The American study found that computer use actually improved children’s test scores.
the graph shows the opposite. It looks like the chartjunk all over this monstrosity was so distracting that no-one noticed that it was reversed. Not the designer, not the sub-editors, no-one.
This is in a paper of record; they would no more suffer a spelling error than they would put out their own eyes. Yet a graph that violates every basic principle of infographics and ends up showing the opposite of what it purports to show passes without comment. If you’re going to inflict this kind of damage on your data, better to have no graphs at all.
<deep breath>
Okay - let’s see what happens when we try again.