question

MayconBrito-3220 avatar image
1 Vote"
MayconBrito-3220 asked MayankBargali-MSFT edited

[Azure Function App] SocketException listing S3 bucket

I'm deploying a .NET 3.1 console application to Azure Functions using HTTPTriggers.
I needed to do some customizations and change some stuff like my DI context, Singleton Request, AWS configure using environment vars and etc.

I could adapt 99% of my app, but I'm having issues making requests from my application to my AWS S3 Bucket using Amazon.S3.Model.ListObjectsV2Response as follows:

 public async Task<IEnumerable<string>> ListFileNames()
 {
     var request = new ListObjectsV2Request
     {
         BucketName = awsConfig.S3BucketName,
     };
        
     IEnumerable<string> listFiles = Enumerable.Empty<string>();
     var response = new ListObjectsV2Response();
        
     do
     {
         response = await s3Client.ListObjectsV2Async(request);
         listFiles = listFiles.Concat(response.S3Objects
             .Where(x => !x.Key.Equals(awsConfig.S3BucketFilePath) && x.Size > 2000)
             .Select(x => x.Key.Replace(awsConfig.S3BucketFilePath + "/", "")));
         request.ContinuationToken = response.NextContinuationToken;
     } while (response.IsTruncated);

     existingAttachmentFiles = listFiles.ToArray();
     logger.Information("Existem [{0}] arquivos anexos no bucket",
         listFiles.Count());

     return listFiles;
 }

The AWS client is configured and perform other minor operations before that one.
Besides, the application runs seamlessly using Azure Function locally or as a Console app, but the SocketException below throws every time using Azure Function App.

This is the exception from Application Insights:

Microsoft.Azure.WebJobs.Host.FunctionInvocationException:

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d_ 20.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 328)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<TryExecuteAsync>d
15.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 94)
Inner exception System.AggregateException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw:
at MyApp.DataImoveis.HttpTriggerDataImoveis.Run (DataImoveis.AzureFunctionApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=nullDataImoveis.AzureFunctionApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: D:\Cloud\dev\myapp\data-imoveis\src\DataImoveis.AzureFunctionApp\HttpTriggerDataImoveis.csDataImoveis.AzureFunctionApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: 54)
at lambda_method (Anonymously Hosted DynamicMethods Assembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)
at Microsoft.Azure.WebJobs.Host.Executors.MethodInvokerWithReturnValue`2.InvokeAsync (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\MethodInvokerWithReturnValue.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 20)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2+<InvokeAsync>d
10.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 52)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<InvokeWithTimeoutAsync>d
27.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 555)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithWatchersAsync>d
26.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 503)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+<ExecuteWithLoggingAsync>d
20.MoveNext (Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35Microsoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.csMicrosoft.Azure.WebJobs.Host, Version=3.0.27.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35: 281)
Inner exception System.Net.Http.HttpRequestException handled at MyApp.DataImoveis.HttpTriggerDataImoveis.Run:
at DataImoveis.CommandLine.Program+<RunOptions>d
1.MoveNext (DataImoveis.CommandLine, Version=1.0.0.0, Culture=neutral, PublicKeyToken=nullDataImoveis.CommandLine, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: D:\Cloud\dev\myu\data-imoveis\src\DataImoveis.CommandLine\Program.csDataImoveis.CommandLine, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: 57)
Inner exception System.Net.Sockets.SocketException handled at DataImoveis.CommandLine.Program+<RunOptions>d
1.MoveNext:
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Net.Http.ConnectHelper+<ConnectAsync>d
_1.MoveNext (System.Net.Http, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)


azure-functions
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

MayankBargali-MSFT avatar image
1 Vote"
MayankBargali-MSFT answered MayankBargali-MSFT edited

Hi @MayconBrito-3220

Welcome to Microsoft Q&A! Thanks for posting the question.

It looks like the port exhaustion issue for your function app. Also, it would be more helpful to troubleshoot if you can follow this to provide the details along with the location you deployed your function app and the time frame (in UTC) of the issue.
I will also suggest you to review the code as it looks like your code might be using a new socket for each call to your service.

Update:
I could see the port exhaustion issue and the suggestion would remain the same as mentioned previously. Please review your code/external calls and see if you can reuse the same connection object so the ports are not exhausted in the function app.


· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MayankBargali-MSFT!
Thank you so much for your Answer!

Yep, I'm listing some objects from the S3 Bucket, and for that reason (using the same request ListObjectsV2Response object) I need to list through the response for every ContinuationToken.

I've updated my question with the function that is throwing this exception.

My execution info:
Id: 3eb14ba0-f5f5-4330-8a24-873b53a4cb83
UTC execution: 2021-05-07T14:42:52.487Z
Region: Brazil South



0 Votes 0 ·