Earlier than making architectural choices, it’s price revisiting the broader migration technique. In our earlier publish, we launched Databricks Skilled Companies’ strategy to complicated knowledge warehouse migrations, highlighting the significance of early choices round technique and design. These foundational selections instantly affect the goal platform’s implementation and structure.
We additionally launched two sequencing methods: ETL-first and BI-first. The BI-first strategy delivers fast worth by modernizing the consumption layer, whereas the ETL-first strategy focuses on upstream pipelines. Every has its place, relying on priorities.
On this publish, we discover one of the crucial essential design selections: selecting between a Elevate-and-Shift or Modernization strategy. We clarify what every strategy entails, when to make use of it, and easy methods to merge them right into a hybrid strategy for long-term success on Databricks.
From technique to migration strategy: choosing the right path
After you’ve aligned on the broader migration technique—ETL-first or BI-first—the following main resolution is easy methods to construction the migration. Do you replicate what exists, or reimagine it for the long run?
This architectural resolution usually comes down to 2 core approaches:
- Elevate-and-Shift: Transfer workloads as-is to speed up the migration
- Modernization: Redesign the platform to unlock long-term effectivity and scale
The best strategy depends upon your targets, constraints, and timeline. Beneath, we break down the tradeoffs of every and embrace a hybrid mannequin that many organizations use to mix the perfect of each.
Elevate and shift migration
Elevate-and-Shift entails shifting your current knowledge fashions and codebase to the brand new platform with minimal modifications. You don’t introduce new use instances, and the structure stays unchanged.
This strategy is interesting as a result of it’s simpler to scope, plan, and automate. Instruments like profilers and code analyzers assist measure workload patterns, complexity, and price, making it simpler to judge and execute.
Key advantages embrace:
- Predictable timelines
- Automated tooling (e.g., code converters, reconciliation validators)
- Sooner migration when dealing with deadlines or expiring licenses
For instance, code converters can routinely deal with as much as 80% of scripts. Since performance stays the identical, validation and operating queries on each methods and evaluating outputs are simpler.
On Databricks, Elevate-and-Shift will get you off legacy platforms rapidly whereas unlocking quick efficiency positive factors utilizing options like z-ordering and liquid clustering. After you’ve got migrated, your group can start incrementally modernizing the platform.
Modernize the migration sample
Modernizing, in distinction to Elevate-and-Shift, means constructing a brand new knowledge platform in your goal system with out being constrained by your legacy structure. The main target shifts from merely migrating current belongings to reimagining use instances and designing for future wants. As an alternative of mapping outdated optimizations, you implement finest practices and the well-architected pillars of the lakehouse.
On an open lakehouse, this entails refactoring code and re-architecting knowledge constructions to fulfill your group’s present and future scalability, efficiency, value, and functionality necessities, free from legacy limitations.
Tooling stays helpful, however extra for discovery and planning:
- Profilers and code analyzers assist stock what you should migrate
- Code converters and reconciliation instruments play a minimal position, since this isn’t a direct code translation
This strategy is right when you’ve got versatile timelines and an outdated or overly complicated legacy system, typically with hundreds of tables and scripts. Whereas beginning contemporary can really feel gradual and overwhelming, the long-term advantages are substantial: simplified structure, higher efficiency, and lowered upkeep overhead.
That mentioned, migrating hundreds of scripts typically means sustaining their upkeep complexity. If that appears daunting, contemplate partnering with Databricks Skilled Companies or licensed migration specialists to assist information the planning and design part and guarantee a smoother path.
A hybrid strategy: carry and shift, after which modernize
One other strategy is a hybrid migration technique that balances velocity with long-term worth. You’d start with the Elevate-and-Shift strategy to eliminating their legacy platform as rapidly as attainable, particularly when dealing with pressing constraints like expiring licenses. Automation and repeatable tooling assist speed up this preliminary part and scale back danger throughout execution.
You possibly can transfer into the modernization part after you migrate your workloads to Databricks.
Within the hybrid strategy, you:
- Combine new and trendy knowledge sources
- Implement an information product technique
- Allow superior analytics, AI, and new use instances that drive enterprise choices
This part typically requires architectural updates however permits you to evolve progressively. With a hybrid technique, you don’t need to modernize every little thing on day one—you construct on a steady basis whereas aligning with future necessities.
In case you’re pursuing this strategy, Databricks Skilled Companies and licensed companions will help information your roadmap, making certain a clean transition and a future-ready structure.
Our viewpoint
Choosing a migration strategy will not be a one-size-fits-all. The commonest strategy is a hybrid migration:
- Create a migration manufacturing facility that leverages automation instruments.
- Elevate-and-shift nearly all of the codebase.
- Allow out-of-the-box optimizations, resembling z-ordering and liquid clustering, to begin your modernization effort.
Databricks can act as your major knowledge warehouse. For instance, you possibly can migrate saved procedures to notebooks and use SQL Scripting for scalability and AI integration with out leaving the consolation of SQL. Migrating Transact-SQL to another cloud knowledge warehouse requires an identical effort to migrating that Transact-SQL to a pocket book with Python code wrapped round your SQL performance. The good thing about utilizing a pocket book is that you simply additionally get flexibility and an ideal improvement expertise.
What to do subsequent
Able to modernize your knowledge warehouse? Obtain our eBook, “Remodeling Legacy Information Warehouses: A Strategic Migration Blueprint,” for detailed methods and finest practices that guarantee a low-risk transition to the Databricks Information Intelligence Platform.
In case you are a DBA, try this weblog on Prime 10 Suggestions for DBAs Migrating to Databricks.
