What are some strategies for effective data collection?
Start with cataloging all of your current data. This is a daunting task, but easier now that there are platforms specifically designed to help you create one. The data catalog needs to be across your entire organization, not just in a departmental silo. You will be amazed, and at times shocked, by where data resides and who has access to it.
With the data catalog as the foundation, you can begin a deeper analysis of the quality of your data as well as identifying missing data. This is never a fun exercise as you will surely find serious quality issues and critical data completely absent. Rather than fear what you will find, recognize the first step in data governance is accepting the inherent imperfections of data. What is important is to then establish processes to increase the quality of your data and minimize instances of missing data.
With the catalog and data analysis completed, you can focus on data provenance, also known as data lineage. This provides a clear picture of the sources of data, enabling you to accurately trace issues and errors back to their source and, if a breach occurs, where it happened and what data was impacted. Whether it is an internal source such as your gift processing team or an external data provider, knowing where, when, and who your data came from can no longer be ignored if your goal is to have an ethical, effective, and efficient data collection operation.
Given the speed at which data is created, collected, and analyzed, it is also worth considering an anomaly detection system. How you monitor for anomalies will depend on the frequency and importance of the data being monitored. You want to find a balance between being alerted to anomalies in time to take action, and not having so many alerts you start to ignore them. To help make the case for anomaly detection, take a data point such as the amount of a donation and trace it through your reports, dashboards, analytics, and predictive models dependent on the data to show, for example, the negative impact of a $1,000 gift being entered as $100,000 or meetings with campaign prospects not being recorded. Anomaly detection was something we did a lot of in my wealth screening company, Prospect Information Network (P!N). What we found was data providers during an update would not always send everything we were expecting. Because of the large volume of data, it would have been easy to miss, for instance, one county's real estate records, and only through anomaly detection were we able to find missing data and fix it!