How To Use Switch Statement in Azure Data Factory Pipeline

This article explains how to use switch statement in ADF pipeline instead of multiple if-else activities.

As a Data Engineer, I have had many use cases wherein I needed to use multiple if-else conditions as part of my data pipeline. What I found was that ADF doesn’t support nested if-else statements as well as I’d have imagined/assumed. You can either use many if-else conditions or use one switch activity. Here, I will give you a practical example that uses switch activity.

Use Case:
Multiple datasets called azure, aws and gcp are present in my azure storage container. Each dataset goes into its respective table. The data pipeline needs to read the datasets simultaneously and based on their names, decide which dataset goes into which table.

Requirement:
ADF pipeline should be able to implement a switch activity; for example if dataset name is azure then do a certain activity otherwise do another activity.

Step 1: Get All Datasets Information From Storage Account

• In ADF pipeline, drag the ‘Get Metadata’ activity to the canvas. This activity will tell a user which datasets are present inside a container of our storage account.
• Create a new dataset for ADLSG2 and select csv from the options. Provide the path of the folder where the datasets reside. In my example, I put my datasets in a folder called temp.