When it comes to data integrations for nonprofits, many organizations struggle to come up with effective and efficient processes. Nonprofits often use multiple systems involving a lot of staff time and manual work. With this kind of approach, it’s difficult to successfully sync even part of their data, much less all of it.
The Challenge
You probably track your fundraising through System A but email your supporters through System B. You dream of a way to pull all of the data together for easier reporting, but even if you do find a possible solution, will it really help, or will it just make your data management even more complicated?
The National Trust for Historic Places knew this struggle well, but when they connected with the Abila netFORUM team, they saw an opportunity to link their online communications and advocacy platform, Luminate, with their membership and fundraising database on netFORUM.
They knew they wanted to use netFORUM as their primary donor database, using Luminate as their online communications database, but how could they connect the two platforms?
That’s where we came in.
Cornershop Creative undertook with enthusiasm what we knew would be a challenging data sync project and far exceeded expectations. Whether your project is a small design refresh or a major technical undertaking, I can’t recommend Cornershop Creative enough. – Alison Hinchman, National Trust for Historic Places
The Process
Data integrations for nonprofits are often technically possible, but frequently involve a lot of customized attention. The Trust contracted with Cornershop to build a data sync between Luminate and netFORUM. This was a very long process, but here’s a summary of our approach:
- Requirements. First, we worked with netFORUM and the Trust to create a Functional Roadmap, outlining the project goals, user stories, and technical recommendations for the sync. Constituent Use Cases define how and when actions should be applied to records.
- Data Mapping. We tracked every single field from each platform in a Master Data Map. For each type of data, we listed all available fields and how those fields should match across platforms.
- Order of Operations. Next, we then used the Data Map to identify how the data should sync or update. For example, when a constituent record is updated, what should happen to it when that record is synced to netFORUM? For each field, we needed to know which platform had the authoritative data. Then, we need to know how or when corresponding record data should change or stay the same.
- Development and Testing. Once we established how the sync should work, we built and tested scripts to actually do the sync. We first tackled individual scripts by data type, testing constituent records, then donation records, email history, and interactions.
- Load Testing. Once we confirmed that our sync worked as expected, we tested the scripts against progressively larger amounts of data, reviewing any errors on particular types of data and confirming that the sync could handle higher volumes of data between the two platforms.
- User Interface. Finally, we documented the sync and built a simple user interface so that the Trust’s team can see when the sync was last run, identify and resolve data issues, and monitor results.
The Results
By tackling this very complex project through smaller, more specific tasks, we developed an easy-to-maintain, reliable data sync.
Thinking about taking on your own complex data project? These tips will ensure your project smoothly:
- All necessary team members need to have access to the same decisions and documentation. For complicated projects, this is especially important, since you may need to refer to past history to address new issues.
- Identify unknowns and stay flexible. Things didn’t always go perfectly smoothly: Sometimes we discovered that a data field was formatted differently than we realized, or we realized a test process needed to be rethought due to system limitations. We could adapt and adjust to new discoveries as we worked, because we were up-front about our assumptions and certainties from the very beginning.
- Ask questions, even if you worry about sounding stupid. The simplest questions led to in-depth conversations about how, for example, a constituent key, a constituent ID, and a member ID might not actually all be the same thing!
And of course, if you could use a hand with developing your own ideal data sync, please contact us!