Data for the people

I spent the last few years of my life building tools to help fairly technical people analyze data. The experience has given me a certain amount of perspective over what the landscape of data looks like. Let’s get the first thing out of the way, I’m not impressed with what we have achieved: while our ability to crunch data is better than ever, and many problems previously intractable have become daily routine thanks to our advancements in distributed systems design, the way those systems interact with the user still leaves a lot to be desired.
The business intelligence world is in my opinion the biggest disappointment of all, we had been promised “intelligence” and we got “systems and tools to produce charts from a collection of databases after jumping through hoops of variable size”. Business intelligence was supposed to make our businesses run better, to increase the amount of knowledge we extract from the information we have available… all we got is charts. Don’t get me wrong, data visualization is extremely important and the ability to produce visualizations is hugely valuable, but it’s not an end in itself.
I’m wondering if it’s possible to build data tools that communicate more humanly, more understandably and with the fluency of an artificial colleague. Most of the times when someone fires up Tableau, it might take a few hours and some knowledge of how data works to coax the tool into producing the results we already know we want. It’s been done, it’s been working for a while and now all BI tools do more or less the same thing: sure, the UI changes somewhat, the way data is accessed can be more or less powerful, computations can be faster or slower… at the end of it all though, differences are not that big.
What I’m looking for is a system that allows a sales manager to ask a simple question: “how are we tracking against our quarterly goals and is there anyone in my team that needs help?”. Descriptive, predictive and prescriptive BI are all nice, but I’m still looking for humanistic business intelligence.