abric First Enhancing Databrick for Seamless Self - Service BI Solutions 13 /0 3 /2025 Nice to meet you! Analytics Architect at element 61 Main Expertise in Data Warehousing & Data Engineering Helping organizations in how to become Data - Driven Lakehouse and Fabric enthousiast Over 2 years of experience with Modern Data Platforms using Databricks & Microsoft Fabric When I'm not immersed into data, you can always find me planning my next travel adventure or enjoying some sports Tom Claessens Analytics Architect tom claessens @element61.be Phone: +32 4 71 12 67 39 01 02 03 04 Agenda Introduction Overview & Symbiosis of Databricks & Microsoft Fabric What is Self - Service BI with the 4 - tier system From Theory to Practice enhance Databricks with Fabric for seamless Self - Service BI Solutions Introduction Databricks vs. Fabric - The Ultimate Choice Should we reconsider our platform design choices? Which platform is best for our new analytics projects? Is it time to migrate our Databricks solution to Fabric? The introduction of Fabric in 2023 sparked numerous questions Combining Forces: Databricks and Fabric Together Can these tools complement each other? Are they compatible ? Can we leverage Fabric in our existing Databricks solutions ? But today, lets ask different questions Choose between Databricks and Fabric? Combine the best of both worlds in your analytics solution! Let's combine the best of both worlds and see how we can empower Self - Service BI with Databricks and Fabric Overview & Symbiosis Of Databricks and Microsoft Fabric Databricks – Your Powerhouse for Data Analytics Unified & Open analytics platform for building, deploying, sharing, and maintaining enterprise - grade data, analytics, and AI solutions at scale Delta Lake for data management and governance on top of your datalake Collaborative workspaces where data scientists, data engineer, streaming engineers and analysts can work in union Spark as your unified engine that offers scalable computing Build on the lakehouse architecture What is Databricks about? Fabric serves as a full end - to - end analytics and data platform One Unified Solution Data Factory Data Engineering Data Science Data Warehouse Real - Time Intelligence Power BI Databases OneLake Unified Product Experience | Compute and storage | Governance and security | Business model The SaaS - ification of a data platform • Integration into a Power BI - alike platform: o Data Science o Data Warehousing o Real - Time Intelligence o Industry Solutions o Data Factory o Power BI o Databases (new since Microsoft Ignite!) • Different workloads on the same data layer > OneLake • Simplified platform infrastructure setu p Both serve distinct yet complementary roles Both products provide similar functionalities and are moving towards convergence Databricks adopts a code - first approach , while Fabric incorporates a low - code strategy Both platforms have a (semi - ) SaaS component and include ingestion and orchestration features. Fabric offers a self - service BI platform , which includes Power BI and additional workflows. Databricks functions as a cloud - agnostic, unified analytics platform built entirely on Apache Spark and is a mature and proven data platform Both serve distinct yet complementary roles Consideration Fabric Databricks Ingestion & Integration • No code/low - code data ingestion with seamless integration with a multitude of sources • Mainly using Data Factory • More code oriented & requires a more diverse skill set of data professionals. Data Transformation • Low - code with Dataflow Gen 2 • Lakehouse for Spark - based transformations • Warehouses for SQL - based Transformation • PySpark or Spark SQL transformations in Notebooks & Delta Live Tables Model serving & Management • Create and deploy models using notebooks and Synapse ML • Fabric supports a variety of LLM’s, like OpenAI & Ollama • Create and deploy models using ML Flow and deploy endpoints with Model Serving • Open Source LLM’s available within the platform Real Time Analytics • Increasing number of capabilities including Eventhouse or use KQL Queryset • Delta Live Tables for also data warehouse processing & Auto Loader for streams Business Intelligence and Reporting Insights • Connection possible with Import & Direct Query & Direct Lake for optimized performance • No proven Self - Service BI posbilities but with connections possible with Import & Direct Query with cluster or SQL warehouse Databricks offers advanced analytics and machine learning capabilities, while Microsoft Fabric ensures seamless data integration and orchestration So lets combine the best of both worlds, but how? Both are designed to work on Delta How can the symbiosis be formed? • The true enabler is the separation between storage and compute • Zero - copy principle • Fabric engines are designed to work with delta Fabric & Databricks – A deep integration Create a mirrored database from Azure Databricks or ADLS Shortcut • Among the different sources, Mirrored Azure Databricks Catalog is special, as it combines shortcuts and mirroring technology • The Unity Catalog metadata are mirrored but shortcuts are created to your ADLS • It automatically syncs future catalog changes for the selected schema • Tables can be added as shortcut to your Fabric lakehouse Databricks can access your OneLake as if it is a ADLS Gen2 Storage Account Better together trought OneLake Microsoft Fabric Databricks All Fabric engines have been redesigned and optimized for Delta Lake as their storage format for tabular data Databricks can directly read/write data from any Fabric item stored in OneLake throught external link With Mirrored databases , your Databricks Unity Catalog metadata data will be mirrored & shortcuts are created to your ADLS. Existing non - Unity Catalog managed data in ADLS can be a shortcut into Onelake Enabler of Self - Service BI on top of your Databricks Solution What is Self - Service BI With a 4 - tier system What is Self - Service BI all about? An approach to data analysis that enables non - technical users to access, visualize, and analyze data independently , without relying on IT or data specialists , ideally within a data governance framework! What is Self - Service BI all about? A conceptual framework Enterprise BI Self - Service BI Self - Service BI Self - Service BI Data Layer Reporting Layer What is Self - Service BI all about? A conceptual framework ... Embedded in a technical reality Lakehouse Bronze Self - Service Data Source Self - Service Standalone Reports Self - Service Reports Data Layer Reporting Layer Lakehouse Silver Lakehouse Gold Enterprise Data Model Self - Service Composite Model Self - Service Standalone Model Standard Reports What is Self - Service BI all about? A conceptual framework ... Embedded in a technical reality ... Which holds a lot of potential Lakehouse Bronze Self - Service Data Source Self - Service Standalone Reports Self - Service Reports Data Layer Reporting Layer Lakehouse Silver Lakehouse Gold Enterprise Data Model Self - Service Composite Model Self - Service Standalone Model Standard Reports