We’re excited to announce the Common Availability of Lakehouse Federation for Google BigQuery and the Public Preview for Oracle and Teradata. Now, you possibly can join, uncover, govern, and question knowledge from these sources by way of Unity Catalog—with out migration or ETL. This makes knowledge entry simpler whereas making certain an open, interoperable lakehouse structure.
Unify your knowledge and governance throughout distributed platforms with the Lakehouse Federation
Lakehouse Federation permits customers to question and analyze knowledge throughout disparate databases, knowledge warehouses and catalogs with out duplicating or transferring knowledge. By integrating exterior knowledge sources straight into the Unity Catalog, organizations can:
- Achieve a unified view of your whole knowledge property: Mechanically classify and uncover each structured and unstructured knowledge in a single platform, empowering your group to securely entry and discover all accessible knowledge—wherever it resides.
- Question and analyze knowledge seamlessly with a single engine: Allow quick, advert hoc evaluation and prototyping throughout all of your knowledge, analytics, and AI workloads with out the necessity for ingestion. A unified engine optimizes question efficiency with superior question planning, caching, and cross-source execution, permitting you to effectively mix knowledge from a number of platforms in a single question.
- Guarantee constant knowledge safety and governance: Apply a unified permission mannequin to implement entry guidelines throughout all knowledge sources. Implement row- and column-level safety, tag-based insurance policies, and centralized auditing whereas sustaining full visibility into knowledge utilization. Meet compliance necessities effortlessly with built-in knowledge lineage and auditability.
Lakehouse Federation helps a various set of platforms, together with MySQL, PostgreSQL, Teradata, Oracle, Amazon Redshift, Salesforce Information Cloud, Snowflake, Microsoft SQL Server, Azure Synapse (SQL Information Warehouse), Google BigQuery, and Hive metastore, with extra connectors coming quickly.
Common Availability of Lakehouse Federation for BigQuery
With the GA launch of Lakehouse Federation for Google BigQuery, customers can now join BigQuery knowledge to Databricks with out the necessity for knowledge motion or duplication. This allows them to run workloads straight on BigQuery knowledge from their Databricks atmosphere on GCP, Azure or AWS, whereas leveraging Unity Catalog’s fine-grained governance for safe and environment friendly knowledge administration.
This integration permits for:
- Unified knowledge administration: Seamlessly uncover and govern BigQuery knowledge alongside different sources inside Unity Catalog, making certain constant entry management and governance.
- Optimized question efficiency: Make the most of BigQuery’s compute energy whereas benefiting from Databricks’ superior question optimization for sooner insights.
- Improved collaboration: Empower groups to work throughout cloud environments effectively with a unified interface for querying and analyzing knowledge throughout a number of platforms.
Lakehouse Federation for BigQuery has seamlessly linked our Google Cloud and Azure environments. With out the necessity for knowledge motion or duplication, we are able to now leverage Databricks on Azure to run Python and SQL workloads straight on BigQuery knowledge, all whereas benefiting from Unity Catalog’s fine-grained governance. It is a game-changer for our multicloud technique, streamlining operations and enhancing analytics
— Lisa Fiege, Information Science Engineer, Taylor Farms
The Google BigQuery connector is now usually accessible beginning with DBR model 16.1 and can quickly be accessible in Databricks SQL.
Public Preview for Teradata and Oracle Connectors
Along with the GA announcement for BigQuery, Databricks can be introducing Teradata and Oracle connectors into Public Preview. These new connectors will permit customers to increase the attain of Lakehouse Federation to much more knowledge sources, additional unifying the information estates.
Get Began
Learn our documentation to get began with Lakehouse Federation connectors for Google BigQuery, Oracle and Teradata:
