L11-Use an Apache Spark notebook in a pipeline

Harini Mallawaarachchi

Dec 20, 20231 min read

In this exercise, we're going to create an Azure Synapse Analytics pipeline that includes an activity to run an Apache Spark notebook.

Before you start

You'll need an Azure subscription in which you have administrative-level access.

Review the Azure Synapse Pipelines rticle in the Azure Synapse Analytics documentation.

DP-203-Labs-22 You can use notebooks in Azure Databricks to perform data engineering tasks, such as processing data files and loading...

DP-203-Labs-21 SQL is an industry-standard language for querying and manipulating data. Many data analysts perform data analytics by...

DP-203-Labs-20 Delta Lake is an open source project to build a transactional data storage layer for Spark on top of a data lake. Delta...