Azure Logic Apps — How To Copy Files From SharePoint to ADLSG2

This article explains how to use Logic Apps to copy files from SharePoint to ADLSG2.

Use Case:
Multiple csv files need to be copied from SharePoint list to ADLSG2 container.

Requirement:
As a Data engineer, I need to build a data pipeline that should read files from a SharePoint list and copy those files into ADLSG2 under today’s date.

Step 1: Create a New Logic App With a Trigger

Every Logic app that needs to be built requires a trigger. For our use case, I will use a Schedule trigger as shown below.

(Logic app schedule trigger)

Step 2: List All Files From SharePoint

After setting up the recurrence trigger, search ‘List folder’ activity. At this stage, if you do not see a SharePoint icon next to the search results, scroll a bit further down until you reach the activity that looks like below.

(Logic App List SharePoint files)

In this activity, provide the site address which is the https URL of the SharePoint. Use the file identifier section to navigate to the folder on SharePoint that contains files which need to be moved to ADLSG2.

At this stage, if we run the pipeline, the output of List folder shows all the files present under the folder path Shared Documents/General/Data File.

Step 3: Loop Through All SharePoint Files

Type and search ‘control’. From the search results, click For each activity.

(Logic apps for loop)

Here, we will loop through all the child items that were generated by the List folder activity in Step 2. Click in the empty textbox and select Body.

Step 4: Access File Content Through The Loop

Click ‘Add an action’. Under choose an operation, search for Get file content. Scroll through the search results until you reach the SharePoint icon as shown below.

(Logic apps get SharePoint file content)

Enter the same SharePoint https URL in Site Address as was entered in Step 2.

Under File Identifier, click inside the textbox and pick Id as shown below.

The purpose of this activity is that as the for loop goes through the output of the List Folder activity, Get File Content will access the content of each element being referenced in the loop through its unique file identifier.

Step 5: Copy File in ADLSG2

Click Add an action and search for ‘Create blob’ activity. From the search results, select Create blob (V2).

(Logic apps copy file to blob storage)

Enter the Storage Account name where you wish to copy the files to.

Under folder path, use the picker to navigate through the folder list until you arrive at the folder where you want to copy the files to. I’m choosing dev/inbound/test3 to be the folder path where the files will be copied.

Under Blob name, click the textbox and choose Display Name from the Dynamic content popup.

Under Blob content, click the textbox and choose File Content from the Dynamic content popup.

👉If you want to copy the files under current date folder, use the below script that should be pasted in Folder path section.

dev/inbound/test3/formatDateTime(addDays(utcNow()),’yyyy-MM-dd’)

The above script will now the copy files in the folder — dev/inbound/test3/2022–01–08

✔️And done. That’s all you need to build a pipeline to copy files from SharePoint to ADLSG2.