Organizations often need to develop tailored data migration strategies and use specialized software to complete the data migration process successfully. They also need to decide on which data migration approach is most suited to their needs.
SEE: Data migration testing checklist: Through pre- and post-migration (TechRepublic Premium)
In this guide, we’ll cover the basics of what a data migration entails, but we’ll also dive deeper into the different types of data migration and when you might want to use each one.
- What is data migration?
- Data migration types by system format
- Main types of data migration strategies
- Data migration best practices
What is data migration?
Data migration is the process of transferring data from one location to another. This could be a transfer between databases, storage systems, applications, or a variety of other formats and systems. The data migration process usually includes multiple steps to make data migration-ready, including data preparation, extraction and transformation.
SEE: An introduction to data migration (TechRepublic)
The goals of data migration are to ensure data is accurately and completely migrated, minimize data downtime and minimize migration costs. Common data migration scenarios include website consolidation, legacy system upgrades or replacements, the adoption of a cloud-based system, infrastructure maintenance or consolidation of information systems.
Data migration types by system format
Although many data migration best practices and strategies will stay the same regardless of the data format or system type you’re working with, it’s important to understand that certain steps will need to be added or reworked depending on the type of data you’re moving as well as the source and target systems that are involved.
Database or schema migration
A database or schema migration happens when a database schema is adjusted to a prior or new version of the database to make migration more seamless. Because many companies work in legacy database and file system formats, data transformation steps are often an important part of this type of migration.
This type of project involves moving datasets from one storage system or format to another. Today, this often involves moving data from tape or a traditional hard disk drive to a higher-capacity hard disk drive or the cloud.
Data center migration
A data center migration involves moving your entire data center to a new physical location or a new non-physical system, like the cloud. Because of the scale of this project, extensive data mapping and preparation is necessary to successfully migrate.
Cloud migration occurs when organizations move from legacy on-premises systems to the cloud or when they transfer from one cloud provider to another. Applications, databases and a variety of other business assets will all need to be moved in this kind of migration. Due to its complexity, most people rely on a third-party vendor or service provider to assist with cloud migration.
This type of migration may involve moving application(s) from one environment to another, but it can also involve moving datasets from one application to another application. This type of migration often occurs in parallel with cloud or data center migrations, but it can also happen when you’re switching from one vendor to another for a project management application, for example.
Business process migration
Especially during mergers and acquisitions, as well as other major business transformations, business process migration is used to make sure all knowledge is shared with the target system and acquiring company. This type of migration, depending on the industry and region, may involve a particular emphasis on data governance and security measures.
Main types of data migration strategies
Choosing the right data migration strategy can have a significant impact on the success of the migration, ensuring a smooth transition and no severe delays. The two basic data migration strategies are a big bang data migration and a trickle data migration.
Big bang data migration approach
The big bang approach involves transferring all data, from the source to the target, in one operation. This makes big bang data migrations less complex, less costly and less time-consuming than trickle data migrations. Some organizations can complete a big bang data migration over a holiday or weekend when they are not using the application(s) that are involved.
SEE: Data migration vs data integration: What’s the difference? (TechRepublic)
It is worth noting that during a big bang data migration, there is significant downtime, as the systems that use the data will be down and unavailable until the migration is complete. The downtime could be more for organizations that are moving vast amounts of data.
In addition, the limited throughput of networks and APIs can further delay the data migration process. As the complexity and volume of data continue to increase, the big bang data migration approach could become more challenging to implement.
- Takes less time
- Less complex
- Less costly
- Requires data downtime
- Higher risk of expensive failure
The big bang data migration approach is best suited for small businesses or data migration projects that involve small amounts of data. This approach is not ideal for the migration of mission-critical data that must be available 24/7.
Trickle data migration approach
The trickle data migration approach is a type of iterative or phased migration. It uses agile techniques to complete the data transfer.
The entire process is divided into smaller sub-migrations chunks, each with its own timeline, goals, scope and quality checks. One of the primary goals of trickle data migration is to ensure there is zero downtime, making this strategy ideal for organizations that need access to data 24/7. The source and target systems run in parallel as the data is migrated in small increments.
The drawbacks of the trickle data migration approach are that it takes longer to complete the migration process and significant resources need to be allocated to the project to keep two parallel systems running simultaneously. In addition, data engineers must ensure the data is synchronized in real-time on both systems.
SEE: Hiring kit: Data engineer (TechRepublic Premium)
A common approach is to have the source system running until the end of the migration, with users only switching to the target system once the entire migration is successful. However, data engineers need to be mindful that any updates or changes to the source system must be reflected in the target system.
- Zero downtime
- Less prone to unexpected failures
- More expensive
- More time-consuming
- Needs extra resources to keep two systems running
Medium-sized and larger organizations might prefer this data migration approach as there is no data downtime. Larger organizations might also have the resources and technical expertise required to run two systems simultaneously.
Data migration best practices
Back up data
The purpose of data backup is to create a copy of data that can be recovered in the event of data failure. It is best to profile all source data before writing mapping scripts.
Create a dedicated team for data migration
Allocating or hiring data migration specialists will ensure the project is completed smoothly, If there are issues, a well-trained and highly-qualified team should have the capacity, skills and experience to handle them.
Complete continuous testing
Data engineers must test data migration through all of its phases, including the planning, design and maintenance stages.
Don’t be quick to switch off the old platform
Sometimes, the first attempt to complete data migration is unsuccessful, requiring a rollback and another attempt. It is best to wait until the target migration is completed and tested before you completely move away from old systems and applications.
Read next: Top cloud and application migration tools (TechRepublic)
Subscribe to the Data Insider Newsletter
Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more. Delivered Mondays and Thursdays