Managing the Risk of Data Migration by Addressing the 5 Myths.
Data migration is often one of the biggest risks in any major IT transformation—whether you’re implementing an ERP system, building a data warehouse or lake, or setting up a reporting framework. Anyone who’s been through the process knows that data cleanliness is at the heart of this challenge.
But why is data migration so risky?
- Data has a history: What you see today may not reflect how the data was originally created or what it truly represents.
- Data has an appearance: Just because it looks clean doesn’t mean it is. Surface-level presentation can mask deeper issues.
- Data has a language: Often, data is encoded with internal company knowledge or assumptions that make it hard to interpret correctly.
- Data has hygiene: Some data might be messy, inconsistent, or just plain unreliable.
- Data may not be digitized: Even if your data is in a digital format, it may not be in a usable or structured form for migration.
At Tsunami Tsolutions, we specialize in managing these risks with the right toolsets and the expertise to guide you through the complexities of data migration. With our proven track record, we help you ensure that your data is ready to move, clean, and compliant—no matter how complex the project.
Tsunami has developed a SME-infused process we call W.A.V.E.S. where experts in Data Migration along with Subject Matter Experts who understand and have real-world experience working with Aviation Data:
- Wash and load the data from various source systems
- Analyze the data being migrated
- Verify rules or logic conditions for the data being migrated
- Enhance the data through scripts, translation tables, or reports to the customer
- Store the data in tables ready to be Migrated to the final source once approved
Data Myth #1
“Our data is clean.”
Since data has history, an appearance, it may have its own language and its own hygiene, it is easy to believe what you have NOW works for your compliance needs, your planning needs, and your reporting requirements. It is easy to look at the latest version of that data and feel confident that it is complete and accurate.
In our experience, data recorded repeatedly (compliance data in particular) evolves as the business grows and overcomes challenges. Once overcome, it is natural to forget all the ways that the data WAS recorded in the past. This matters in Data Migration.
Data may appear clean in the reports your organization uses because there may be complex processes in place to gather, clean and present the data to the rest of the organization.
Within the data you store, you may be encoding company information into the Name, Description, Model, or other fields, while this is a particularly useful tactic to keep information organized for the company, it can be difficult to parse out the meaning of the encoded data.
Key Takeaways:
- Data evolves over time, especially compliance and business data, which can introduce hidden complexities.
- Even if data seems “clean,” there may be encoding or past iterations that make it difficult to fully understand.
- W.A.V.E.S. helps ensure a deep understanding of your data’s history, so it can be properly migrated and leveraged for future use.
- Don’t assume your data is clean—take a closer look to ensure it’s truly ready for what’s next.
Data Myth #2
“This is the only data we use.”
It is a common trip-hazard we find in Data Migration that there are often a couple sources of truth that are combined to create a report that everyone believes is the source of truth. When you peel back the report and look at the data behind it, it can be difficult, and sometimes impossible, to recreate the same report when looking at the datasets independently. In some cases, there is a person that maintains a Rosetta Stone independent of the regular data that they reference to manually connect the correct data from one dataset with another dataset. This means that depending on who you ask, you may end up with 2 or more sources of truth, or you may realize there is a hidden source of truth.
Because we know data has history, it is important to be able to rank sources of truth based on the period you are considering the question under. This is especially crucial when looking into something like Compliance History for ADs or SBs.
The way the data appears in the report/dashboard/summary view may not reflect its natural state in the dataset, you need experts familiar with Aviation Organizations PLUS Databases and Data to help expedite the Data Migration efforts.
Key Takeaways:
- Data is rarely as simple as it seems—there are often multiple sources of truth, and it’s important to understand how they connect.
- In complex environments like aviation, context and history are essential to understanding what data is truly relevant.
- W.A.V.E.S. helps ensure that all sources of truth are identified, understood, and accounted for during the data migration process.
- By leveraging the right mix of data expertise and industry-specific knowledge, you can successfully migrate complex data without losing its integrity.
Data Myth #3
“CoNsIs10ncy_is4L0SERS “
You got that right? ALL CAPS, lowercase, mixing characters…everyone has experienced something like this. There are ways in which parts of an organization are taking steps to improve their data that are not harmonized across the organization. Are you SPEC2000 compliant with your part numbers? If so, you may replace all the O (oh) characters with a 0 (zero). Then if the part requirement calls out for SCOTCHBRITE to clean…you find an un-joined error because SC0TCHBRITE ≠ SCOTCHBRITE.
Similarly, maybe your tech pubs team uses the (m) — dash instead of the (n) – dash. 32—11—01—400—801 ≠ 32–11–01–400–801.
Now imagine that once you find this…you suddenly start to question everything you see…did you notice the trailing space in the heading…that is a character that represents data. Are your current fields character limited, and departments have come up with a unique way of concatenating the data to fit, or trim leading or trailing data?
Is the system you are migrating to able to represent the same data lengths and types? Will you need to consistently convert or concatenate fields to support the new system?
As you migrate data into a brand-new system, you need it to be clean and consistent.
Key Takeaways:
- Inconsistent data formats and encoding (like mixed characters or dashes) are common pitfalls in data migration.
- Small discrepancies can lead to big problems when migrating data to a new system.
- W.A.V.E.S. helps ensure your data is clean, consistent, and ready for migration with a focus on identifying and resolving inconsistencies.
- Data consistency is crucial to avoid complications and ensure a smooth migration process.
Data Myth #4
“The way we do things today is the way things have always been done.”
Remember back when your data source was young, it was so simple, just a few columns to load all your data into…then it got older. Remember that time you had to change the structure and add that column so you could also track that other thing…yea, that was messy. And then remember how you realized you needed to re-purpose one of the fields, it really should have been a number, but you just made it a free-text field so you could add that other piece of data…
Sound familiar? Well, according to the data we encounter, your answer should be: Yes!! Data becomes contextual at some point. The data migration strategy and tools need to be adaptable so they can handle: ‘this data means what it says UNLESS this other factor is present, and then it means something else entirely’, or: ‘if it’s before this date, it could mean this, or if it’s after it could mean that…unless it was during the time of ‘he-who-shall-not-be-named’, and then it’s just a crap-shoot…’
Many times, in the data world this situation emerges when there is little-to-no documentation from primary data source systems on changes/enhancements/modifications made to the system over time. Business processes change and that can result in new fields, integrations or streams of data being introduced into decision support processes, tools, and databases.
Key Takeaways:
- Data evolves and often becomes more complex over time.
- Without proper documentation, it’s easy for data to become inconsistent or confusing.
- W.A.V.E.S. is a specialized toolset that combines technology with expert industry knowledge to handle complex data migration.
- The right approach helps ensure that evolving data maintains its integrity and value.
Data Myth #5
“We can clean up the data through attrition.”
While there may be legitimate cases to ‘fill in the blanks’ after migrating to a new system, most decisions to allow attrition to clean up the data is an attempt to avoid the pain today so you can experience it later. This comes at a cost, and the reality is; when you are migrating to a new system, it is the right time to clean up the data. Chasing bad data points through a brand-new system is difficult and demoralizing. It creates roadblocks to adoption, causes doubt and distrust, and can have compliance implications.
It has been rightly questioned: “Why do we always have time to do it right the second time, but can’t seem to find the time to do it right the first time…?”
But where should you look? What are you looking for? What data is MISSING? All the answers rely on a robust analytic approach to the data. Sometimes you (gasp!) must make up missing data using things like fleet leader hours and cycles. Sometimes you may need to pick a default date of manufacture, or task initialization date. Sometimes you need a fleet campaign to validate the Serial Number and Position for components. This can be burdensome and may drive corrections in your old system(s), so you get up-to-date data in your data migration. It may require interim processes to go beside your current processes to update the position mapping tables that support the data migration exercises. All these things need executive support to be successful and effective. If you do it right, you may avoid having to do it twice!
Key Takeaways:
- Waiting to clean data through attrition is a risky approach that can lead to long-term complications, such as compliance issues and roadblocks to system adoption.
- Migrating to a new system is the right time to clean and address data inconsistencies.
- W.A.V.E.S. provides a structured, SME-infused approach that ensures data is clean, complete, and consistent before migration.
- A successful migration relies on addressing data issues upfront, with executive support and robust processes in place.
Summary: Tsunami Tsolutions is your partner to help de-risk Data Migration.
Tsunami Tsolutions has tools, technology and a team of aviation subject matter experts that are ready to work with you to help keep your ERP Implementation, or IT Project on time and in budget.
To learn more about Tsunami Tsolutions, click here or contact us.
Over decades in aviation, Tsunami Tsolutions has seen airlines, particularly smaller ones with fewer configuration engineers and mechanics, struggle to confidently verify that aircraft are configured with the right allowable parts.
Learn the the vulnerabilities A&D manufacturers face from their own aerospace and defense (ERP) software.
ERP and MRP were designed for repetitive manufacturing. How are IFS and Tsunami Tsolutions delivering shipbuilding ERP?
New partnership leverages expertise of both companies to field FireSwarm’s integrated response platform
Managing the Risk of Data Migration by Addressing the 5 Myths.
Data migration is often one of the biggest risks in any major IT transformation—whether you’re implementing an ERP system, building a data warehouse or lake, or setting up a reporting framework. Anyone who’s been through the process knows that data cleanliness is at the heart of this challenge.
But why is data migration so risky?
- Data has a history: What you see today may not reflect how the data was originally created or what it truly represents.
- Data has an appearance: Just because it looks clean doesn’t mean it is. Surface-level presentation can mask deeper issues.
- Data has a language: Often, data is encoded with internal company knowledge or assumptions that make it hard to interpret correctly.
- Data has hygiene: Some data might be messy, inconsistent, or just plain unreliable.
- Data may not be digitized: Even if your data is in a digital format, it may not be in a usable or structured form for migration.
At Tsunami Tsolutions, we specialize in managing these risks with the right toolsets and the expertise to guide you through the complexities of data migration. With our proven track record, we help you ensure that your data is ready to move, clean, and compliant—no matter how complex the project.
Tsunami has developed a SME-infused process we call W.A.V.E.S. where experts in Data Migration along with Subject Matter Experts who understand and have real-world experience working with Aviation Data:
- Wash and load the data from various source systems
- Analyze the data being migrated
- Verify rules or logic conditions for the data being migrated
- Enhance the data through scripts, translation tables, or reports to the customer
- Store the data in tables ready to be Migrated to the final source once approved
Data Myth #1
“Our data is clean.”
Since data has history, an appearance, it may have its own language and its own hygiene, it is easy to believe what you have NOW works for your compliance needs, your planning needs, and your reporting requirements. It is easy to look at the latest version of that data and feel confident that it is complete and accurate.
In our experience, data recorded repeatedly (compliance data in particular) evolves as the business grows and overcomes challenges. Once overcome, it is natural to forget all the ways that the data WAS recorded in the past. This matters in Data Migration.
Data may appear clean in the reports your organization uses because there may be complex processes in place to gather, clean and present the data to the rest of the organization.
Within the data you store, you may be encoding company information into the Name, Description, Model, or other fields, while this is a particularly useful tactic to keep information organized for the company, it can be difficult to parse out the meaning of the encoded data.
Key Takeaways:
- Data evolves over time, especially compliance and business data, which can introduce hidden complexities.
- Even if data seems “clean,” there may be encoding or past iterations that make it difficult to fully understand.
- W.A.V.E.S. helps ensure a deep understanding of your data’s history, so it can be properly migrated and leveraged for future use.
- Don’t assume your data is clean—take a closer look to ensure it’s truly ready for what’s next.
Data Myth #2
“This is the only data we use.”
It is a common trip-hazard we find in Data Migration that there are often a couple sources of truth that are combined to create a report that everyone believes is the source of truth. When you peel back the report and look at the data behind it, it can be difficult, and sometimes impossible, to recreate the same report when looking at the datasets independently. In some cases, there is a person that maintains a Rosetta Stone independent of the regular data that they reference to manually connect the correct data from one dataset with another dataset. This means that depending on who you ask, you may end up with 2 or more sources of truth, or you may realize there is a hidden source of truth.
Because we know data has history, it is important to be able to rank sources of truth based on the period you are considering the question under. This is especially crucial when looking into something like Compliance History for ADs or SBs.
The way the data appears in the report/dashboard/summary view may not reflect its natural state in the dataset, you need experts familiar with Aviation Organizations PLUS Databases and Data to help expedite the Data Migration efforts.
Key Takeaways:
- Data is rarely as simple as it seems—there are often multiple sources of truth, and it’s important to understand how they connect.
- In complex environments like aviation, context and history are essential to understanding what data is truly relevant.
- W.A.V.E.S. helps ensure that all sources of truth are identified, understood, and accounted for during the data migration process.
- By leveraging the right mix of data expertise and industry-specific knowledge, you can successfully migrate complex data without losing its integrity.
Data Myth #3
“CoNsIs10ncy_is4L0SERS “
You got that right? ALL CAPS, lowercase, mixing characters…everyone has experienced something like this. There are ways in which parts of an organization are taking steps to improve their data that are not harmonized across the organization. Are you SPEC2000 compliant with your part numbers? If so, you may replace all the O (oh) characters with a 0 (zero). Then if the part requirement calls out for SCOTCHBRITE to clean…you find an un-joined error because SC0TCHBRITE ≠ SCOTCHBRITE.
Similarly, maybe your tech pubs team uses the (m) — dash instead of the (n) – dash. 32—11—01—400—801 ≠ 32–11–01–400–801.
Now imagine that once you find this…you suddenly start to question everything you see…did you notice the trailing space in the heading…that is a character that represents data. Are your current fields character limited, and departments have come up with a unique way of concatenating the data to fit, or trim leading or trailing data?
Is the system you are migrating to able to represent the same data lengths and types? Will you need to consistently convert or concatenate fields to support the new system?
As you migrate data into a brand-new system, you need it to be clean and consistent.
Key Takeaways:
- Inconsistent data formats and encoding (like mixed characters or dashes) are common pitfalls in data migration.
- Small discrepancies can lead to big problems when migrating data to a new system.
- W.A.V.E.S. helps ensure your data is clean, consistent, and ready for migration with a focus on identifying and resolving inconsistencies.
- Data consistency is crucial to avoid complications and ensure a smooth migration process.
Data Myth #4
“The way we do things today is the way things have always been done.”
Remember back when your data source was young, it was so simple, just a few columns to load all your data into…then it got older. Remember that time you had to change the structure and add that column so you could also track that other thing…yea, that was messy. And then remember how you realized you needed to re-purpose one of the fields, it really should have been a number, but you just made it a free-text field so you could add that other piece of data…
Sound familiar? Well, according to the data we encounter, your answer should be: Yes!! Data becomes contextual at some point. The data migration strategy and tools need to be adaptable so they can handle: ‘this data means what it says UNLESS this other factor is present, and then it means something else entirely’, or: ‘if it’s before this date, it could mean this, or if it’s after it could mean that…unless it was during the time of ‘he-who-shall-not-be-named’, and then it’s just a crap-shoot…’
Many times, in the data world this situation emerges when there is little-to-no documentation from primary data source systems on changes/enhancements/modifications made to the system over time. Business processes change and that can result in new fields, integrations or streams of data being introduced into decision support processes, tools, and databases.
Key Takeaways:
- Data evolves and often becomes more complex over time.
- Without proper documentation, it’s easy for data to become inconsistent or confusing.
- W.A.V.E.S. is a specialized toolset that combines technology with expert industry knowledge to handle complex data migration.
- The right approach helps ensure that evolving data maintains its integrity and value.
Data Myth #5
“We can clean up the data through attrition.”
While there may be legitimate cases to ‘fill in the blanks’ after migrating to a new system, most decisions to allow attrition to clean up the data is an attempt to avoid the pain today so you can experience it later. This comes at a cost, and the reality is; when you are migrating to a new system, it is the right time to clean up the data. Chasing bad data points through a brand-new system is difficult and demoralizing. It creates roadblocks to adoption, causes doubt and distrust, and can have compliance implications.
It has been rightly questioned: “Why do we always have time to do it right the second time, but can’t seem to find the time to do it right the first time…?”
But where should you look? What are you looking for? What data is MISSING? All the answers rely on a robust analytic approach to the data. Sometimes you (gasp!) must make up missing data using things like fleet leader hours and cycles. Sometimes you may need to pick a default date of manufacture, or task initialization date. Sometimes you need a fleet campaign to validate the Serial Number and Position for components. This can be burdensome and may drive corrections in your old system(s), so you get up-to-date data in your data migration. It may require interim processes to go beside your current processes to update the position mapping tables that support the data migration exercises. All these things need executive support to be successful and effective. If you do it right, you may avoid having to do it twice!
Key Takeaways:
- Waiting to clean data through attrition is a risky approach that can lead to long-term complications, such as compliance issues and roadblocks to system adoption.
- Migrating to a new system is the right time to clean and address data inconsistencies.
- W.A.V.E.S. provides a structured, SME-infused approach that ensures data is clean, complete, and consistent before migration.
- A successful migration relies on addressing data issues upfront, with executive support and robust processes in place.
Summary: Tsunami Tsolutions is your partner to help de-risk Data Migration.
Tsunami Tsolutions has tools, technology and a team of aviation subject matter experts that are ready to work with you to help keep your ERP Implementation, or IT Project on time and in budget.
To learn more about Tsunami Tsolutions, click here or contact us.
Over decades in aviation, Tsunami Tsolutions has seen airlines, particularly smaller ones with fewer configuration engineers and mechanics, struggle to confidently verify that aircraft are configured with the right allowable parts.
Learn the the vulnerabilities A&D manufacturers face from their own aerospace and defense (ERP) software.
ERP and MRP were designed for repetitive manufacturing. How are IFS and Tsunami Tsolutions delivering shipbuilding ERP?
New partnership leverages expertise of both companies to field FireSwarm’s integrated response platform
© 2021 TSUNAMI TSOLUTIONS
© 2021 TSUNAMI TSOLUTIONS