question

VarunNair-2546 avatar image
0 Votes"
VarunNair-2546 asked KranthiPakala-MSFT commented

Creating Items in Datalake creates a additional 0kb file with folder name or filename

We have created two Gen 2 storage account. Creating an item in one data lake container creates a 0 KB file with the same name as that of the file or folder. Attached the screen shot of the container items with the additional file that is created and below is the code that creates the directory and items in it.

Note:- The same code in another storage account in the same subscription does not create this additional file.

80453-image.png



    public static async Task StoreTobigatewaygen2(string path, string filename, byte[] barray)
     {
         try
         {                
             var fullpath = $"tagdata-full/";
             string directoryname = $"{fullpath}{path}";
             StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(_accountName, _accountKey);
             string dfsUri = "https://" + _accountName + ".blob.core.windows.net";
             DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClient(new Uri(dfsUri), sharedKeyCredential);
             DataLakeFileSystemClient fileSystemClient = dataLakeServiceClient.GetFileSystemClient(_fileSystemName);
             DataLakeFileClient fileClient = fileSystemClient.GetFileClient(directoryname + @"/" + filename);
             if (!fileClient.Exists(default) || fileClient.GetProperties().Value.CreatedOn < FirstRun.DT)
             {
                 fileClient = await fileSystemClient.CreateFileAsync(directoryname + @"/"+ filename);
                 offset = 0;
             }
             else
             {
                 offset = fileClient.GetProperties().Value.ContentLength;
             }                
             using (MemoryStream memorystream = new MemoryStream(barray))
             {
                 fileClient.Append(memorystream, offset: offset);
                 fileClient.Flush(position: offset + barray.Length);
             }
             return;
         }
         catch (Exception ex)
         {
             Console.WriteLine(ex.Message);
         }
     }
azure-data-lake-storage
image.png (77 B)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Are you creating an item manually in storage account? maybe something wrong the logic if you are using any application which is generating the items

0 Votes 0 ·

I have updated the question with the code I used to create the item in Data lake. This happens in one Data lake container and not in another under the same subscription and same resource group. I see small difference in the ARM templates of both the containers and in Replication property of the containers.

0 Votes 0 ·

Hi @varunnair-2546,

Sorry for the delay in response. Have a you got a chance to try creating a similar storage account to that of the one which is having extra 0 kb file issue and see if the issue exists in the new storage as well. This test is to narrow down the root cause analysis, if it is related to Storage account configuration or issue with the code.

Could you please try this test and let us know how it goes.

Thanks

0 Votes 0 ·
Show more comments

1 Answer

VarunNair-2546 avatar image
0 Votes"
VarunNair-2546 answered KranthiPakala-MSFT commented

Hi

I have got a reply from Microsoft support team. The 0 KB file is created because the datalake is not Gen2 storage account as the hierarchical namespace was not enabled. So this storage account will use a flat structure causing a zero kb file to occur.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @varunnair-2546 , Thanks for sharing the details here. Please feel free to accept your answer as it would help other to find this helpful.

0 Votes 0 ·