What do we do with our existing data? This is a critical question that you may need to address if you are considering a data migration project.
Data Migration may be required in your organisation for a range of reasons – see our post on the Pros and Cons of ETO Data Migrations for more information on what data migration is all about.
Data migrations are challenging processes that require upfront planning. It is therefore important to review different options and alternatives for your data migration during the planning process. There are a number of factors that can impact your decision to migrate data in and out of your ETO database, including:
- Time: The physical duration to complete a data migration from start to finish.
- Cost: The total monetary investment required to complete a data migration from start to finish.
- Effort: The emotional and intellectual stress placed on your internal team when conducting a data migration. Stress can impact productivity, which impacts time available for other tasks.
This post will describe five options to consider when planning for a data migration. Each data migration alternative below is graded on a scale of high, medium, and low for time, cost, and effort to help you make an informed decision.
1. Build new and disable old
If you are transitioning from an older ETO Legacy system to ETO Touchpoints or you are completely changing the data structure of your existing ETO database, building a new data structure with new fields and data collection points while disabling the old data structure is a viable option. This process ensures that your historical data is still in your ETO database even though you are transitioning to a new data structure for data entry and reporting.
The disadvantage of this data migration option is “split reporting”, in that you are maintaining data in two different data structures so you will need one set of reports for the historical data structure (that has been disabled) and another set of reports for the new data structure. Split reporting is dependent on the date of the data migration (i.e. the go-live date for the new data structure) and may only be for a short period of time. Having proficient ETO report writers can help overcome this challenge.
Instead of migrating data, you have the option to archive historical data into an accessible format like Excel, Access, or another not for profit database software.
When you archive historical data, you can start fresh with a new ETO software data system whilst knowing that your old data is available if it is ever needed.
This is a good option if you don’t need frequent access to historical data and the cost to migrate the historical data is high. This method of data migration is under-utilized, but extremely effective.
3. Migrate all data
This data migration option is the most appealing given that all data, historical and new, is formatted the same way and is consistent and clean. However, it takes a lot of time, money and effort to complete.
Migrating all data requires a significant amount of mapping, cleanup, exporting, and importing. It also has the largest opportunity for errors because migrating all data requires that every piece of historical data be perfectly mapped to the new data structure. Given this, in most cases, it is not recommended.
4. Migrate some data
Migrating some data is a better alternative than migrating all data. To determine which data to migrate, which to archive, and which data has no value and should be purged, your organisation will need to conduct a strategic assessment of the value of your historical data by asking the following questions:
- What data is relevant for reporting?
- What data is relevant for end-user research?
- How will the data that is migrated be used in the database?
5. Purge and move on
The least popular data migration option is to purge historical data and moving on from it, as most organisations choose to archive data before purging it forever.
However, there may be cases where historical data has no value to the organisation, such as, if there have been dramatic changes in program design, outcomes, goals, or a program or service is no longer offered. Purging data is therefore an option if you don’t need the data anymore. Purging may also be viable if archiving the data violates privacy policies or data compliance and regulations.
Most data migration plans use a combination of the above methods to ensure data quality and continuity are maintained. It’s important that you engage in adequate planning and weigh up the pros and cons of each data migration option before making any decisions. Good planning will ensure a successful data migration and minimize the time, cost, and effort required to migrate your data.