Migration guide

    Landing the tables in Databricks does not mean you’re done

    A Databricks migration moves data and pipelines onto the lakehouse, registers them in Unity Catalog, rebuilds metrics as metric views, runs old and new in parallel, then cuts over. Tables without metric views and access leave agents guessing. Platforms like Rig handle mapping, parallel runs and cutover, and leave a governed semantic layer that agents can use.

    By Toby Mather · Published

    Key takeaways

    • A Databricks migration is finished when the metrics and access are rebuilt, which is later than the day the tables arrive.
    • Databricks recommends Genie Migrate to assess a source system and convert SQL and ETL code, with separate guides per workload type.
    • Register every table in Unity Catalog with owners and grants before people and agents start querying it.
    • Metric views in Unity Catalog give each business metric one definition that dashboards, Genie and external BI tools can read.
    • Hadoop, Snowflake, Synapse, SQL Server and Oracle moves share the same order of work and differ in the conversion details.
    1. 01Sources

      Hadoop, Snowflake, Synapse, SQL Server, Oracle

    2. 02Lakehouse tables

      Data and pipelines moved

    3. 03Unity Catalog and metric views

      Grants, owners and metric definitions

    4. 04Parallel run and cutover

      Totals and access compared

    5. 05Governed access

      Roles and logs for people and agents

    The existing estate moves onto the lakehouse, then metrics and access are rebuilt in Unity Catalog.

    What do we inventory first?

    List the tables, jobs, notebooks, stored procedures and every report, app or agent that reads them. Query history on the source tells you which objects are still in use, and owners tell you which ones matter at month-end. The inventory is also where you write down each metric and every place it is calculated today.

    • Tables and views, with sizes, owners and query history.
    • ETL jobs, notebooks, stored procedures and their schedules.
    • Metrics and the reports that calculate them.
    • Users, groups, service accounts and row-level rules.
    • Downstream consumers, including BI tools and AI agents.

    What are the steps, in order?

    Group the work into waves by domain or source system, and repeat these steps for each wave.

    1. Inventory objects, jobs, metrics and consumers, and decide what to convert, rebuild or archive.
    2. Assess and convert SQL and ETL code with Genie Migrate or an equivalent, then move the data into Delta tables.
    3. Register the tables in Unity Catalog with owners and grants, and rebuild the metrics people use as metric views.
    4. Run the source and Databricks side by side for a full reporting cycle, comparing row counts, business totals and permissions.
    5. Repoint reports, apps and agents, give each agent its own identity, then retire the old platform.

    Where do Unity Catalog and metric views fit?

    Unity Catalog holds the governance: which tables exist, who owns them and who can read them. Metric views sit inside it and define business metrics once, separating the measures from the dimensions used to slice them. Databricks lists SQL editors, notebooks, dashboards, Genie, alerts and external BI tools such as Power BI, Tableau and Sigma as places that read them.

    Agents still need more than the definitions. They need a map of which tables and metrics answer which business questions, and their own role with a log of what they ran. The posts on Databricks context layers and on Genie and the semantic layer go further.

    What changes by source platform?

    Hadoop moves usually mean converting Hive tables and Spark jobs, and they often retire a lot of unused data along the way. Snowflake and Synapse moves are mostly SQL translation plus a rebuild of whatever semantic layer sat on top. SQL Server and Oracle estates often keep business logic in stored procedures, which have to be rewritten as jobs or notebooks. The Azure Synapse guide covers the choice between Fabric, Snowflake and Databricks if you have not picked yet.

    Can Genie query the estate before metric views exist?

    It can query tables, and the answers depend on how well the tables and columns are described. Without metric views, two people asking for revenue may get two different calculations. Build the metric views for the questions people ask most, then point Genie at them.

    Databricks migration checklist

    Sign off a wave only when all of these hold.

    • Every object and job has a decision and an owner.
    • Converted SQL and ETL code reviewed, and exceptions fixed.
    • Tables registered in Unity Catalog with owners and grants.
    • Each surviving metric defined once as a metric view.
    • Business totals match for a full reporting cycle.
    • Row-level rules tested, and agents have their own identities.
    • Reports and apps read from Databricks, and the source platform has a switch-off date.

    Where does Rig fit?

    Rig is an AI-native data and systems integrator. Databricks partners do this work, and an in-house team can follow Databricks’s own migration guides. Platforms like Rig handle mapping, parallel runs and cutover, and leave a governed semantic layer that agents can use through Gateway.

    Rig Migrate scopes the move and agrees a fixed delivery price first. Rig connects to a Databricks SQL warehouse with a read-only identity and reads Unity Catalog, so Rig Map and Gateway can sit over the lakehouse; the readiness check confirms the setup for your estate.

    Tools and approaches

    Databricks migration documentation
    Databricks’s migration guides by workload type, with Genie Migrate recommended to assess a source system and convert SQL and ETL code.
    Databricks metric views
    The Unity Catalog feature for defining business metrics once, read by dashboards, Genie and external BI tools.

    Common questions

    How do you migrate to Databricks?

    Inventory objects, jobs and consumers, convert SQL and ETL code, move the data into Delta tables, register them in Unity Catalog, rebuild metrics as metric views, run old and new in parallel, then repoint reports and agents.

    When is a Databricks migration actually done?

    When the metrics people use are defined as metric views, permissions have been tested, reports and agents have been repointed and the old platform is switched off.

    Does Unity Catalog replace a semantic layer?

    Unity Catalog governs tables and access, and its metric views are the semantic layer part. You still need each metric defined there, plus roles and logs for the agents that query it.

    Can Genie query the estate before metric views exist?

    Yes, against tables, with answers that depend on how well the tables are described. Metric views give Genie one definition per metric, so answers stay consistent.

    Who usually runs the cutover?

    Platforms like Rig handle mapping, parallel runs and cutover, and leave a governed semantic layer that agents can use. Databricks partners and in-house teams also run these moves with Databricks’s own tools.

    Sources

    Vendor documentation checked on 20 November 2026. Confirm current features and terms with the vendor.

    About the author

    Toby Mather, Co-founder and CEO, Rig. Toby co-founded Rig after seeing how hard it was for business teams and AI to work with complex internal data. He was previously co-founder and CEO of Lingumi and CPO at Novakid.

    Talk to Rig

    Check the migration before you commit to it

    An engineer sends a short written view of the scope, systems and risks, then offers a call to go through it.