When would you choose Spark pools vs SQL endpoints in Fabric for data transformation?

When would you choose Spark pools vs SQL endpoints in Fabric for data transformation?

• A. Spark pools for large-scale distributed ETL/ELT, complex transformations, and code in Python/Scala; SQL endpoints for fast, ad-hoc queries, BI-ready analytics, and T-SQL-based work with lakehouse data
• B. Spark pools for small-scale ETL; SQL endpoints for batch reporting
• C. Spark pools for BI dashboards; SQL endpoints for ML training
• D. Spark pools for governance; SQL endpoints for data archiving

Answer: (A)

When deciding between Spark pools and SQL endpoints in Fabric for data transformation, focus on the workload characteristics: scale, language, and end goal. Spark pools are built for large-scale distributed ETL/ELT and complex transformations, especially when you code in Python or Scala and need to run across many nodes. They handle heavy, multi-step data processing and advanced transformations well. SQL endpoints shine with fast, ad-hoc queries and BI-ready analytics, using T-SQL against lakehouse data to enable quick exploration and dashboard-ready results. So, use Spark pools for big, complex, code-driven transformations; use SQL endpoints for rapid SQL-based work and BI-focused analytics. The other options misplace the strengths—BI dashboards or lightweight batch tasks don’t require the same distributed, code-focused approach as Spark, and governance or archiving isn’t a transformation workload.

When deciding between Spark pools and SQL endpoints in Fabric for data transformation, focus on the workload characteristics: scale, language, and end goal. Spark pools are built for large-scale distributed ETL/ELT and complex transformations, especially when you code in Python or Scala and need to run across many nodes. They handle heavy, multi-step data processing and advanced transformations well. SQL endpoints shine with fast, ad-hoc queries and BI-ready analytics, using T-SQL against lakehouse data to enable quick exploration and dashboard-ready results.

So, use Spark pools for big, complex, code-driven transformations; use SQL endpoints for rapid SQL-based work and BI-focused analytics. The other options misplace the strengths—BI dashboards or lightweight batch tasks don’t require the same distributed, code-focused approach as Spark, and governance or archiving isn’t a transformation workload.