Azure Data Factory — Rename Files Dynamically During Copy

This article explains how to rename multiple files with ADF during copy from source to sink.

In this article, we will rename multiple files while copying them between different containers in our Azure storage account.

Image credit — https://www.shareicon.net/documents-papers-interface-files-paper-document-file-documents-set-676794

Use Case:
To be able to rename multiple .csv files using Azure Data Factory while copying it between different storage account containers.

Requirement:
In some cases, a Data Engineer would come across a requirement where the files before moving into a storage account container would need to be renamed according to a given nomenclature. The ADF should be able to do the same.

👉 In one container, I have kept files with names — Azure File 1.csv and Azure File 2.csv in a container called — original file. I need to copy these files into a new container — renamed file with new names Azure_File_1.csv and Azure_File_2.csv.

Step 1: Create a Customized Dataset For Source and Sink

As we are working with csv files, let’s create a new dataset in the following way —

Azure Data Lake Storage Gen2 > DelimitedText > Linked Service to Connect to ADLSG2. In my storage account, I have kept the two csv files in the container whose path will be: dev > original file >…