question

AzeemK-2483 avatar image
0 Votes"
AzeemK-2483 asked ShaikMaheer-MSFT commented

Azure Function apps Function to rename file in Azure Data lake storage Gen 2

I have few text files under Azure Data lake Storage Gen 2 raw container, I want to rename these file like strip some characters ProductIDs.20210623.075739.txt to ProductIDs.txt out of file name , also replace some characters inside the file using python or power shell , this really straight forward on prem file share, is there any good documentation on how to do that by using function in Azure Function App or any other simpler way from ADF pipeline??

azure-data-factoryazure-functions
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @AzeemK-2483 ,

Thank you for posting your query in Microsoft Q&A Platform.

You can achieve your requirement either using ADF or using Azure Function App.

Requirement 1: Rename file name on ADLS Gen2

  1. Using Azure Data Factory Pipeline
    - Make API call to ADLS Gen2 using web activity. This API of ADLS Gen2 helps you to rename file name.
    - Have PowerShell code to rename file on ADLS Gen2 and execute that using Custom Activity. Click here for documentation of PowerShell library

  2. Using Azure Functions
    - Have Python or C# or PowerShell code to make REST API call to ADLS Gen2 to rename file.


Requirement 2: Replace some characters inside file

  1. Using Azure Data Factory Pipeline
    - You need to consider data flows inside ADF to perform this transformation. replace() function will help to achieve this.
    - You can consider REST API call to ADLS Gen2 as well, to update file.

  2. Using Azure Functions
    - You can write your own code in Python or C# or PowerShell to make REST API call to ADLS Gen2 to update file.

Click here for Azure data lake Gen2 REST APIs documentation.
Click here to Azure data lake Gen2 PowerShell library documentation.

Hope this will help. Please let us know if any further queries. Thank you.


Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


@ShaikMaheer-MSFT

below is my power shell script , I have created Azure function app and a Blobtrigger Function as well , how do I run this power shell script in Azure function , I am new to Azure function app , see powershell code and function screen shot below

  ------//rename files in ADLS Gen2 Inbound Container 
  Get-ChildItem 'Inbound/BOB/2021/09/*.txt' | rename-item -NewName {$_.name.substring(0,$_.BaseName.length-16) + $_.Extension -replace "_"," "}
  Get-ChildItem 'Inbound/BOB/2021/09/*.txt' | rename-item -newname { [string]($_.name).substring(8) }
  -------------------------///Change double Pipe delimiter to single inside the files
  $Content.replace('||','|')|Set-Content Inbound/BOB/2021/09/*.txt
  $Content.replace('||','|')|Set-Content Inbound/BOB/2021/09/*.txt![128543-functionapp1.jpg][1]


[1]: /answers/storage/attachments/128543-functionapp1.jpg

0 Votes 0 ·
functionapp1.jpg (413.8 KiB)

Hi @AzeemK-2483 ,

Thank you for follow-up query. Please consider Accept Answer if above helped you.

Regarding follow-up query, The Blob storage trigger starts a function when a new or updated blob is detected. Click here, to know more about Blob trigger functions. You test by adding blob to your storage.

0 Votes 0 ·

Hi @AzeemK-2483 ,

Following up to see is above answer helps you? If yes, Please consider accepting answer by clicking on 130210-acceptansw.jpgbutton. Please let us know if any further queries. Thank you.

0 Votes 0 ·
acceptansw.jpg (2.3 KiB)
Show more comments