question

tn-57-gs-7096 avatar image
0 Votes"
tn-57-gs-7096 asked MohamedRT-4123 commented

[Azure-Table-Storage-Access-RestAPI-via-PowerShell] - Invoke-RestMethod : Bytes to be written to the stream exceed the Content-Length bytes size specified

Target: Fetch a list of installed applications from multiple computers to Azure Table Storage using Invoke-RestMethod.

I am trying to collect information from the computer and put it in a batch lets say 100 rows and then insert it into the table.

Prepared the script based on the https://docs.microsoft.com/en-us/rest/api/storageservices/performing-entity-group-transactions this article.

Error: "Invoke-RestMethod: Bytes to be written to the stream exceed the Content-Length bytes size specified"

Here is the code below

 $tableEndpoint = 'https://****.table.core.windows.net/$batch'
    
    
 $tableName = 'InstalledApplications'
 $StorageAccount = "********"
    
 $URI = $tableEndpoint + $tableName + $SAS
    
 $cot =@()
    
    
 $array1 = Get-ItemProperty HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\* | Select-Object DisplayName, DisplayVersion, Publisher, InstallDate -unique
    
 foreach($arr in $array1){
    
    
 $RequestBody = ConvertTo-Json -InputObject @{
         "TagetName"= $arr.DisplayName;
         "Message"= $arr.DisplayVersion;
         "ComputerName"= $ENV:ComputerName;
         "Username"= $ENV:Username;
         "BatchName" = "_a1e9d677-b28b-435e-a89e-87e6a768a431";
         "Content-Type" = "multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977"
         "PartitionKey"= "$ENV:ComputerName";
         "RowKey"= $arr.DisplayName
         "TableName" = "InstalledApplication"
     }
     $cot += $RequestBody
 }
    
 $batching = $cot[1..10]
    
 $RequestHeaders = @{
     "x-ms-date"=(Get-Date -Format r);
     "x-ms-version"="2009-09-19";
     "Accept-Charset"="UTF-8";
     "DataServiceVersion"="3.0;NetFx";
     "MaxDataServiceVersion"="3.0;NetFx";
     "Content-Type" = "multipart/mixed; boundary=batch_a1e9d677-b28b-435e-a89e-87e6a768a431"
     "Content-Length" =  "1323"
     "Authorization"  = "SharedKeyLite " + $StorageAccount + ":" + $SAS
 }
    
 Invoke-RestMethod -Method POST -Uri $tableEndpoint -Headers $RequestHeaders -Body $cot -ContentType "multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977"


Please let me know why content length exceeds although the returned rows are no more than 60.

azure-storage-accounts
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

the reason why I am attempting to use batch insert operation because data is collected across multiple computers, therefor using the below code will unnecessarily end up piling transactions upon every entity is updated. I have attached a text file contains the previous version of this script, as you will see in the script, Invoke-RestMethod lives inside the loop, so each data in the array performs an insert operation.

please assist me here.


0 Votes 0 ·
shivapatpi-MSFT avatar image
0 Votes"
shivapatpi-MSFT answered

Hello @SujithkumarGSuriyamurthy-7096 ,
Thanks for your query ! I just tried your previous version of the script and was able to insert the required data to the azure table storage.
I was able to repro your issue on your previous version by explicitly mentioning the "Content-Length" to some x value in the headers.

  • What is the reason you want to explicitly mention the content length ?

  • Did you try removing that property ?

  • If the Content-Length is not hardcoded , how are you calculating the content Length dynamically?

Basically , content length implicitly also includes the type of encoding which we use . For example , if the actual data is 5 bytes - then over all content-length can be more than 5 bytes depending upon the encoding used. So while the data is being pushed using REST API when it goes through transit - it implicitly uses some sort of encoding - you might want to think over about that.

ASCIIEncoding encoding = new ASCIIEncoding ();
byte[] samplebyte= encoding.GetBytes (sampledata);

For example: For the below string , if you try to get a normal length - it will just output 1 . But if you try to get the length through some encoding in terms of bytes those will return different outputs (2 & 1)

var s = "é"
var s = "e";

Kindly let us know if you need additional help.



5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

tn-57-gs-7096 avatar image
0 Votes"
tn-57-gs-7096 answered tn-57-gs-7096 edited

Thanks for your response to my query!

the previous version works well for me still the problem is when you run the PS command to get a list of applications it returns 90 rows, what happens is, it calls the API 90 times since the invoke-rest method resides inside the for loop. let's suppose if I use the same script on 8000 computers then you can imagine what will be the transactional cost at the end. correct me if I am wrong here, as per my understanding 1 transaction = $0.0004.

This is the reason why I am trying to use batch insert operation.

Initially, I was encoding the data and copy it into a variable, and pass the same variable in the header where it says "content-length = $variable. length". this way was also giving me the same error.

As per your suggestion, today, I tried removing the "content-length" still no luck.

therefore, please assist me if there any misconstruction in the above script.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

tn-57-gs-7096 avatar image
0 Votes"
tn-57-gs-7096 answered MohamedRT-4123 commented


As per this article https://docs.microsoft.com/en-us/rest/api/storageservices/performing-entity-group-transactions#table-service-support-for-odata-batch-requests,

I tried to insert a batch with two entities using postman but I get an error "one of the request inputs is not valid". please check out the body below I used for testing.

 --batch_f351702c-c8c8-48c6-af2c-91b809c651c4  
 Content-Type: multipart/mixed; boundary=changeset_8a28b620-b4bb-458c-a177-0959fb14c977  
      
 --changeset_8a28b620-b4bb-458c-a177-0959fb14c977  
 Content-Type: application/http  
 Content-Transfer-Encoding: binary  
      
 POST https://*****.table.core.windows.net/TableName HTTP/1.1
 Content-Type: application/json  
 Accept: application/json;odata=minimalmetadata  
 Prefer: return-no-content  
 DataServiceVersion: 3.0;  
      
 {"PartitionKey":"Channel_19", "RowKey":"1", "Rating":9, "Text":".NET..."}  
 --changeset_8a28b620-b4bb-458c-a177-0959fb14c977  
 Content-Type: application/http  
 Content-Transfer-Encoding: binary  
      
 POST https://*****.table.core.windows.net/TableName HTTP/1.1
 Content-Type: application/json  
 Accept: application/json;odata=minimalmetadata  
 Prefer: return-no-content  
 DataServiceVersion: 3.0;  
      
 {"PartitionKey":"Channel_19", "RowKey":"2", "Rating":9, "Text":"Azure..."}  
 --changeset_8a28b620-b4bb-458c-a177-0959fb14c977  
 Content-Type: application/http  
 Content-Transfer-Encoding: binary  
    
 --changeset_8a28b620-b4bb-458c-a177-0959fb14c977--  
 --batch_f351702c-c8c8-48c6-af2c-91b809c651c4  

please assist me with the above JSON body construct.

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I am also sharing the screen shot taken while trying to hit the table endpoint

93153-microsoft-share-postman-req.png


0 Votes 0 ·

did you find out why?

0 Votes 0 ·