Sep 6

Written by:
9/6/2012 

I’ve often talked about how the complexity of our fundraising databases has changed over the years. We used to just collect names and addresses. We then added prospect management, cell phones and emails. We now have online behavior from our online communities and social media. A lot of our data is unstructured – such as what is typically contained in notes fields.

Do we consider fundraising data “big data”? I believe it’s a critical inflexion point for us, especially as historically disparate systems have tended to become more integrated through reporting environments such as data warehouses. Our data seems to be growing more rapidly than ever and we need to take advantage and capitalize on some of the same techniques that “very big data” organizations have been employing.

To me, the term big data is really a reflection of our environment. The “cradle to grave” relationship with donors has to integrate a complex data model that essentially emulates the random nature of human existence and relationships. Not an easy task to collect and manage. Not an easy task to analyze and develop metrics.

How do you need to think about big data and how do you get a return? Return on Big Data = Value of Data/Cost of Data. This is obviously a gross simplification; you’ll need to leverage projects and create your own value propositions.

Definition: Explosive growth in volume, variety and velocity of data. The ratio of external data to internal data is rapidly changing. Over the last decade or two, we’ve gone from (depending on which survey you’ve read) a ration from 80% internal data to 20-40% internal data. Rich guys in hoodies have added a lot to the velocity of data.

There’s an incredible amount of real-time data and it’s being generated so fast that we don’t even have time to scan it and derive meaning from it. Social data is “big interactive data” and interaction is sometimes replacing transaction. How do you load a Twitter feed into your data warehouse and what meta-data do you assign to the tweet? How to you record important tweets in your fundraising systems?

The cloud has created a SAAS (Software as a Service) sprawl. This is increasing the complexity for managing since not all of our data is encapsulated in our organization. Example: linking to online mapping applications to show geographic disbursements of transactions.

Big data is the confluence of big transactions, big interactions and big data processing. We have more donor data than ever before such as online and offline within our own systems, Contact reports done by fundraisers – offline. What our donors did when they logged into our web site – online. Most of us have experienced significant online growth but many of our systems have not caught up to managing the data that we’re collecting if we’re even collecting it.

Organizations that can integrate “big data” will maintain or improve their competitive advantage. For example if we create an online app that passes information tokens, and we’re able to de-dupe and load this data into our transactional system, we now have an advantage in targeting that one of our competing charities may not. Most of us have very fragmented apps that don’t integrate well. Example – are you effectively monitoring the number of duplicates created in your transactional system by automatic downloads of online gifts so you can change the business processes to fix this?

Stay tuned for part 2.