Lakeflow Designer is a part of Lakeflow, which is able to now be usually obtainable. Lakeflow has three modules — Lakeflow Join, Lakeflow Declarative Pipelines, and Lakeflow Jobs. Designer is built-in inside Declarative Pipelines.
United in objective, divided in method
Lakeflow Designer, regardless of being similar to rival Snowflake’s Openflow, differs in its method, analysts say.
“Lakeflow and OpenFlow replicate two philosophies: Databricks integrates knowledge engineering right into a Spark-native, open orchestration cloth, whereas Snowflake’s OpenFlow provides declarative workflow management with deep Snowflake-native semantics. One favors flexibility and openness; the opposite favors consolidation and ease,” Ni stated.
Each the choices additionally differ in maturity, ISG’s Aslett stated. Whereas Snowflake’s OpenFlow is comparatively new, Lakeflow has matured in performance over time, with Designer being its newest instrument. “The Join capabilities had been acquired together with Arcion in 2023. Declarative pipelines performance is the evolution of DLT (Delta Reside Tables), and Jobs is the evolution of Databricks Workflows,” Aslett added.
Individually, Databricks additionally launched one other pro-code built-in improvement atmosphere (IDE) for knowledge engineers, which unifies the info engineer’s full pipeline lifecycle — code, DAGs, pattern knowledge, and debugging — multi functional built-in workspace.
Releasing Lakeflow Designer and the brand new IDE collectively is a strategic play, in keeping with Ni, because the lakehouse supplier is focusing on each ends of the pipeline maturity curve — low-code to maneuver quick and the complete IDE to scale and keep pipelines.
