When an organization is planning to move data from one system to another, planning the data migration is an important aspect of the project to ensure that the right data ends up in the right place in the new system.
When the data isn’t in the right places, it does appear to a business user like the entire system is broken. Defining data migration requirements is a key aspect of successful business analysis.
In this article, we’ll look at the business analyst role on a data migration project and how to plan the data migration so that potential data mapping issues are discovered before they derail the project.
(By the way, if you are looking to learn more about data modeling, be sure to check out our Free Data Modeling Training.)
Data Migration: The Business Analyst’s Role
System migration projects involve migrating functionality supported by one system to a new system. In today’s technology climate, often organizations are migrating legacy technology systems, such as those that are proprietary and custom to a single business, to commercial off-the-shelf systems that are supported by third-party vendors. For example, an organization could replace their homegrown lead and customer management system with Salesforce.com or SAP.
Another scenario is when an organization is moving from one third-party system to another. For example, at Bridging the Gap, we’ve changed email marketing software, shopping cart software, and online course deliver software. All of these required some sort of data migration work.
A business analyst can take on many different roles in a data migration.
- At a minimum, the business analyst should be involved in clarifying the business requirements for the data migration, and helping answer such questions as what types of data will be migrated from one system to another. (For example, when we moved shopping cart systems, we migrated our products but not the order history.)
- The BA’s role can greatly expand to include analyzing all aspects of moving data from one system to another, without actually doing the technical work of migrating the data itself.
Data Migration Projects Bring Up Lots of Questions
While the actual loading of the data is a technical process, there are important business decisions to be made, such as:
- Do we bring over all data or recent data?
- For any two fields, do they actually mean the same thing? Or, is there logic in either system that will impact how we want the data brought over, based on our understanding of the business process and functional system flow.
- Do all fields have a home in the target database? And vice versa?
- And so on and so on.
Without a business analyst proactively planning the migration, it’s not uncommon to have a client-side database developer provide a recent dump of data and have the vendor-side database developer sort through how to import it into the new tool. This approach can lead to data issues surfacing late in development, potentially throwing the project off course.
Let’s take a look at the data modeling techniques a business analyst can use to clarify these types of requirements.
Data Migration Technique #1: Model Data Flow with a System Context Diagram
In the early stages of the project, while you are still clarifying scope, start by clarifying the high-level data flow. What systems are impacted? How is data flowing or migrating from one system to another?
A System Context Diagram can be useful here to show how the current systems support the business process, or how data flows through those systems. Here’s a video with more details on a System Context Diagram.
Data Migration Technique #2: Get Clear on Business Concepts with a Glossary and Entity Relationship Diagram
Another early analysis step is to understand the the business perspective on the information model.
- A Glossary is a useful technique to identify and clarify business terms and concepts.
- An Entity Relationship Diagram will show the critical relationships between business concepts.
When you move data from one system to another, it’s incredibly common for there to be shifts in the information model, or how the business manages information.
For example, if your current system has a way to associate one customer with multiple sub-accounts and the future system does not, this issue likely impacts a wide range of business processes and even how your stakeholders will think about the requirements for the new system.
I consider ERD’s a critical BA skill set in our current data-oriented world, even if you don’t want to code and don’t know SQL. Here’s an ERD tutorial so you can learn more about this foundational skill set.
(Yes, it is totally possible for a BA to create an ERD, even if you don’t know SQL and don’t consider yourself technical. In fact, this is a great technique to help facilitate better communication with technical stakeholders.)
When I create an ERD, I like to start with the business’s perspective and model how information is organized conceptually by the business. Then, I work with the development or database design team to present this understanding, explore how to reconcile it with the existing database design, and discuss potential adjustments and customizations.
This understanding is essential as I evaluate how to transition business processes and even consider functional system enhancements.
Data Migration Technique #3: Get Specific with Data Dictionaries and Data Maps
The next step is to get really specific and identify how each individual field or attribution from the original system will get loaded into the new system.
- A Data Dictionary for each database will identify key fields and business rules, such as how long the field can be and whether the value is chosen from a list or typed in free form.
- A Data Map identifies how each field from the source database maps to a field in the target database, and any translation rules or data clean-up that’s needed for a successful transition.
It’s not uncommon to map a free form field into a multi-select field or a field with a longer character count to a shorter one. Instead of waiting for these issues to surface during testing, upfront analysis and business involvement can streamline the entire data migration project.
It’s really tempting to leave this work to the database developers – and it’s very likely they will need to augment and update your data maps in order to do a complete data import. As the business analyst, you can bring the business perspective to this analysis, since you’ll understand not just what data is in the field, but how that information is used as part of the business process. That’s key information that not everyone has access to.
Expertise from both the business and technology teams, as well as both the source and the target system, is needed as part of the mapping, so typically there is a lot of collaboration. Here are some of the issues you can expect to work through:
- For mapped fields, you’ll be looking at whether they actually mean the same thing in both systems or whether there is logic in either system that will impact how the data should be migrated over to the COTS product.
- You’ll also want to be sure that all of the important data has a home in the target database, or that they are comfortable letting go of that data or creating a separate back-up.
- You’ll want to evaluate the data source has all the data needed to populate the new system in the format that’s required. Otherwise, a data clean-up project may be required prior to the data migration.
- As you work through these decisions, the team will need to decide where any clean-up and translation rules get implemented, whether in advance of the migration or as part of the data migration process.
- Finally, you’ll have decisions to make about what data to bring over. It’s not uncommon to archive older data and only bring over recent or active data, as this helps expedite the data migration process.
As you can see, there is quite a bit to think about for a data migration project, and we’ve only dug into the details specific to the data migration. A data mapping specification will help you identify the potential issues before the migration happens, creating a smoother transition to the new system.
>>Learn More About Data Modeling (Free Training)
Learn the essential Data Modeling Techniques (even if you don’t know how to code) with this free training.
