Some of the Main Steps in Leveraging Big Data Access Access needs to be delegated throughout the organization and maintained. If you have an email marketing provider, do the right people have access to be able to download the click-through results and load into your CRM system? Your SAAS (Software As A Service) applications need to be integrated so that data is pulled automatically and loaded. There are some appliances and software that can help with this. Your ETL (extract, transform, load) tool should be robust enough to utilize SAAS processes directly so you don’t waste time on intermediate steps that take expensive staff time. Standards should be developed. Example: If you’re working with multiple telemarketing vendors, you should insist that the data returned from all of your vendors should adhere to your formats and definitions. This makes it a lot easier for you to integrate and manage the data you’re receiving. Discover You need to simplify the data discovery as you get more and more data sources. The key is to have structured entity relationships. Discovery should include a pre-load and profiling analysis before you load into your system. It’s not fun to do one-off data migrations and discovery is key to understanding what you’re getting and to design a process that’s systematic and repeatable….

I’ve often talked about how the complexity of our fundraising databases has changed over the years. We used to just collect names and addresses. We then added prospect management, cell phones and emails. We now have online behavior from our online communities and social media. A lot of our data is unstructured – such as what is typically contained in notes fields. Do we consider fundraising data “big data”? I believe it’s a critical inflexion point for us, especially as historically disparate systems have tended to become more integrated through reporting environments such as data warehouses. Our data seems to be growing more rapidly than ever and we need to take advantage and capitalize on some of the same techniques that “very big data” organizations have been employing. To me, the term big data is really a reflection of our environment. The “cradle to grave” relationship with donors has to integrate a complex data model that essentially emulates the random nature of human existence and relationships. Not an easy task to collect and manage. Not an easy task to analyze and develop metrics….