I’ve been hearing much lately about stories, narratives, analytics, data, and “big data”.  I have no need to call out exactly who or which pieces of writing. You know who you are. My aim here is not to criticize, oppose, or take sides. It’s to take a brief critical look at what’s being discussed.

Much of the discussion strikes me as one tribe (I’ll call them non-quants) pleading that stories and narratives are important too!  All of which is an understandable reaction to how the other tribe (I’ll call them quants) have seemingly gained a favored position and perceived superiority at divining the “truth” because they are evidence based!  Because data! I’m actually a member of both tribes and find the posturing of stories and narratives as alternative to quantitative analysis disheartening.

The most encouraging blog piece I’ve read recently comes from Michael Feldstein.  In his lengthy (and excellent) post called Analytics Literacy is a Major Limiter of Edtech Growth.  Please do read it.   He argues for the dissolving this false juxtaposition between “stories” and “data”.

…some of these arguments position analytics in opposition to narratives. That part is not right. Analytics are narratives. They are stories that we tell, or that machines tell, in order to make meaning out of data points. The problem is that most of us aren’t especially literate in this kind of narrative and don’t know how to critique it well.

I wholeheartedly agree.  Feldstein is (correctly) arguing that data points are nothing without stories.  The meaning we take from the data is itself nothing but a story we weave using the data points as we might use punctuation or particular words.  In essence, quantitative analysis is itself a story.

This really isn’t news or at least it shouldn’t be.  I remember how powerful McCloskey’s Rhetoric of Economics was for me when I read it decades ago.  McCloskey powerfully made the point that no matter how much we wrapped an idea in data, mathematical formalism, or econometric analysis, everything we said in economics was just a metaphor or a story we imposed on the data. Alan Grossman long ago pointed out that even that high temple of data-driven evidence, Science(tm), it’s still just rhetoric and it’s still just stories.

Yes, the meaning we attach to a set of data is itself a story.  So stories are not alternatives to data. Data is a story.  But it’s not just the obvious story we tell with the data. There’s a story unstated underneath the data the we use. Our choice of particular data variables constitutes a story itself. We (or at least the data collector) have in mind a story and narrative of what’s important before they collect the data.  They don’t collect data about the context that they don’t see as important or relevant (or easy enough to collect), so they assume a story about that uncollected contextual data holds no meaning.  There’s a story underneath the story we told with the data.

But it keeps getting deeper. Much like the philosophical turtles, it’s stories all the way down. That measure of the data you’re using. The one you think is just basic stats or math, something like the average (properly called arithmetic mean), or the variance, or correlation, or whatever.  It has a story too.  Let’s take that arithmetic mean (average) and each observation’s difference from the average. We think of that average as “the norm” – but that’s just a story invented by a couple of different statisticians in the 19th century.

I can’t really do justice here to the story of how that story of what the average or norm is.  I strongly urge you to read The End of Average by Todd Rose.  It’s fully accessible to members of both tribes, quants and non-quants.  You’ll never use your quantitative data the same way again. Todd Quinn writing in the Elearning magazine of the ACM had the same kind of dramatic reaction as I had.

I’ve finished reading Todd Rose’s The End of Average, and I have to say it was transformative in ways that few books are. I read a fair bit, and sometimes what I read adds some nuance to my thinking, and other times I think the books could stand to extend their own nuances. Few books fundamentally make me “think different,” but The End of Average was one that did, and I believe it has important implications for learning and business.

Rose’s point is pretty simple: All our efforts to try to categorize people on a dimension like GPA or SAT or IQ are, essentially, nonsensical.

But going another level down, as Rose explains in End of Average, there are assumptions beneath the calculation and use of ordinary stats like the average or the variance.  Let’s face it, “assumptions” is another way of staying “believed a story to be so true that it didn’t need to be stated”.  In the case of the average and the calculation of differences from “the norm”, that assumed story has to do with the ergodic properties of what’s being examined.  So what’s “ergodic  properties”? Well here’s Wikipedia’s attempt to explain ergodicity. It’s not very accessible to non-quants (or even most quants!).  Again, I would refer you  to Rose’s book for a beginning glimpse of what ergodicity means. I can’t explain it here, but the essence is that mathematically, statistically the vast majority of the stories being told with quantitative analytics are complete nonsense. Garbage. Invalid. Wishful alchemy.

It’s stories all the way down.  At first this might seem discouraging. But it’s not. I’m calling for not just analytics literacy but a critical analytics.  We need to investigate and become aware of not only the stories we tell using data, but also the assumed stories we slide under the table by choosing particular measures and statistical techniques without thinking about them. We wouldn’t let the semantics of narratives escape critical examination. Why should we let analytics?