Mar 14

Written by:
3/14/2014 

Ignaz Semmelweis changed the world by showing that if doctors washed their hands there was a much smaller chance of transmitting infections. This discovery, like many others was based on data.

People that did calculations with data during the life of Ignaz were called computers.

Over time, the meaning of computer has obviously changed as machines have taken over the mundane calculating, but people are still using their powers of questions and curiosity to discover answers to problems.

Machines are challenged when it comes to making inferences – humans are more comfortable with the semantic inferences involved in relating data to decisions.

Think about how people use spreadsheets – many that have “grown up” with spreadsheets often look at formulas to make sure that they are correct – but they may stop at that point. Even if the formulas are working, this doesn’t provide insight into stories that the information may be telling. In this case we may be relying too much on the machine for insight. We need to take it to the next level.

Great users of data combine facts, questions and feelings inside them to decide which way to go with the data. Structure with improvisation is present in music and should be present in analytics. Jazz artists use a magical combination of improvisation and structure to make great music. Many great thinkers have similar skills. The coming together of facts and intuition and feeling and hunches help discover the unexpected – and it’s often the unexpected that can enable big changes – the discovery of penicillin, the light bulb, flying.

People that have great insights are good at shifting their perspectives. Different views of data can provide insights based on perspectives and new tools allow you to shift perspectives quickly to get new insights and ideas.

Disparate information, connected on the fly, can help you create questions and provide you with results that you might not be able to get looking at a single subject.

Consider the following: A table contains information on how long someone is logged onto the system. We join this to a table that has counts of records added. We then join to a table that has data cleanup errors.

Very quickly we have moved from a simple metric on how long someone was logged into the system to how much work was done within that time and how accurately it was done. This can give us a whole new level of insight and we now have a discovery tool to use to help improve our business processes. Does our data entry staff need to slow down to be more accurate? What is their volume per hour?

We do our best thinking about data when we can feel, shift, relate and forage though our data easily. Do we have the tools and the data structures in place to be able to do this?

Chasing unpredictable data down an alley can lead to new ideas – from improvisational thinking – and a base of structure. Is there a way to put a round peg in a square hole – the solution can be surprising – we have to deal with ad-hoc on-the-fly issues like this on a regular basis. It’s a regular part of our standard approach to problem solving.

As you formulate your strategy for analytics, visualizations and tools for our current age, you’ll need to think carefully about relationships between your structured reporting, guided discovery, free-form discovery and free-form improvisation. This will require new skills for analysts, end user communities, executives and our governance bodies. It’s not hard to make the move from classical to jazz but it requires some work and new ways of thinking.

We need make great data available to people that make decisions based on questions that have been answered, will be answered and questions that haven’t been thought of. Our job is to provide holistic and effective tools for discovery – that allow art and science to coalesce.

Reflections From The DRIVE Conference
/Data /Reporting /Information /Visualization /Exchange