question

NavinDondapati avatar image
0 Votes"
NavinDondapati asked HimanshuSinha-MSFT commented

Blob Storage Auto cool tier while loading

Hi Guys,

We have Files on-premise which will be moved to cloud ADSL2; we are using azure data factory to load all the files from source to sink; But we were wondering whether we have any options in ADF, which automatically makes all the files loaded in an folder to cool tier by default?

Else we need to create an Data Management life cycle management rule to make the folder files as cool? As i see there is no option like auto in alerts, we need to mention 0 days for last modified or accessed.

Do we have any easy options, else we need to write powershell to do that for us after each ADF load to sink?

Regards,
Navin

azure-data-factoryazure-blob-storage
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@NavinDondapati did my suggestion help? If is solved your issue, please mark as accepted answer. Otherwise let me know how I may better assist.

0 Votes 0 ·
MartinJaffer-MSFT avatar image
1 Vote"
MartinJaffer-MSFT answered

Hello @NavinDondapati and welcome back to Microsoft Q&A.

I haven't seen any options in Data Factory for specifying the storage tier during copy. I suspect it defaults to whatever the account storage tier is set to. I never looked closely at this before, so it is worth a test to see whether changing the inferred property affects blobs written by ADF. Actually, there is an easier way. I'll open up storage explorer and inspect the metadata of a few blobs....

.... Yes, looking at existing outputs, it says "Access tier: Hot (inferred)" . So changing the default should affect this.

120942-image.png

So while making this change should make all files loaded cool by default, it would not be limited to ADF. Other applications writing without specifying which tier, would also default to cool. This may be something you desire, or maybe not.


image.png (127.0 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

NavinDondapati avatar image
0 Votes"
NavinDondapati answered HimanshuSinha-MSFT commented

Nope, What are looking for is, each file not the entire storage account, we know we can do complete storage to Cool Tier, but we want for each file that upload to specific folder within Storage account.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

You mean, you have like this @NavinderKaur-6513 ?

 MyContainer
 MyContainer/ColdFolder
 MyContainer/ColdFolder/ColdFile
 MyContainer/HotFolder
 MyContainer/HotFolder/HotFile

I'll let the product group know about your desire for hot/cold tier to be an option in copy activity.

I think for now you will have to make a lifecycle rule, and add filter to only apply to the ColdFolder.

Something like

 {
     "enabled": true,
     "name": "MakeCool",
     "type": "Lifecycle",
     "definition": {
         "actions": {
             "baseBlob": {
                 "tierToCool": {
                     "daysAfterModificationGreaterThan": 1
                 }
             }
         },
         "filters": {
             "prefixMatch": [
                 "MyContainer/ColdFolder"
             ]
         }
     }
 }



0 Votes 0 ·

Hello @NavinderKaur-6513 ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu

0 Votes 0 ·