question

klinki avatar image
0 Votes"
klinki asked YihuiSun-MSFT answered

ASP.NET 4.8 WebAPI - streaming large file upload fails with HTTP 2

Hello,

I have problems with streaming large file upload in ASP.NET 4.8 Web API project.

I'm using custom WebHostBufferPolicySelector to disable buffering of input stream (as described here: https://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/) and I'm sending file in HTTP POST request with multipart/form-data. I read data in controller using HttpContext.Current.Request.Files["key"].InputStream. I'm using it with FileStream to write uploaded data to file on server. Basically I use InputStream.CopyToAsync method with 80kB buffer size (this seems to be the default value) to copy data.

This seems to work for HTTP 1.1 but as soon I switch to HTTP 2, I start getting following exceptions:

 System.Web.HttpException (0x80004005): An error occurred while communicating with the remote host. The error code is 0x800703E3. --->     System.Runtime.InteropServices.COMException (0x800703E3): The I/O operation has been aborted because of either a thread exit or an application request. (Exception from HRESULT: 0x800703E3)
 at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
 at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)
 at System.Web.Hosting.IIS7WorkerRequest.ReadEntityBody(Byte[] buffer, Int32 size)
 at System.Web.HttpRequest.GetEntireRawContent()
 at System.Web.HttpRequest.GetMultipartContent()
 at System.Web.HttpRequest.FillInFilesCollection()
 at System.Web.HttpRequest.EnsureFiles()
 at System.Web.HttpRequest.get_Files()
 at Lic.Web.Controllers.FileUploadController.<UploadFileAsync>d__6.MoveNext() in C:\projects\emclient\license-manager\Lic.Web\Controllers\FileUploadController.cs:line 93

sometimes I get this exception after just few first megabytes of files are transfered.

Is this the right approach for large file upload? Basically I want to avoid loading whole file into memory, I'd like to use streams to process it with small buffer to avoid memory congestion problems.


Here is sample code:

     public class FileUploadChunkResBindingModel
     { 
         public string Status { get; set; }
         public string FileName { get; set; }
         public string Guid { get; set; }
     }
        
     [RoutePrefix("api/FileUpload")]
     public class FileUploadController : ApiController
     {
         private ILogger<FileUploadController > _logger;
    
         [HttpPost]
         [Route("UploadFile")]
         [Authorize(Roles = "WriteAdmin")]
         public async Task<FileUploadChunkResBindingModel> UploadFileAsync(CancellationToken ct)
         {
             var req = HttpContext.Current.Request;
    
             if (req.Files.Count != 1)
             {
                 throw new HttpResponseException(new HttpResponseMessage
                 {
                     StatusCode = HttpStatusCode.BadRequest,
                     Content = new StringContent(string.Format("Request must contain 1 file, but contains {0}", req.Files.Count)),
                 });
             }
    
             var guid = req.Form["guid"];
             var file = req.Files[req.Files.Keys.Get(0)];
             var ret = await UploadFileAsync(guid, file, ct);
    
             return ret;
         }
    
         public async Task<FileUploadChunkResBindingModel> UploadFileAsync(string guid, HttpPostedFile file, CancellationToken ct = default)
         {
             var path = Path.Combine("~", "FileUpload", guid);
             var uploadDir = HostingEnvironment.MapPath(path);
    
             Directory.CreateDirectory(uploadDir);
    
             try
             {
                 var filePath = Path.Combine(uploadDir, file.FileName);
    
                 using (var fileReadStream = file.InputStream)
                 using (var fileWriteStream = File.OpenWrite(filePath))
                 {
                     var bufferSize = 81920; // 80kB should be default size based on documentation
                     await fileReadStream.CopyToAsync(fileWriteStream, bufferSize, ct);
                 }
    
                 return new FileUploadChunkResBindingModel
                 {
                     Status = "Done",
                     Guid = guid,
                     FileName = file.FileName
                 };
             }
             catch (Exception exc)
             {
                 _logger.LogError(exc, "File upload failed with exception.");
    
                 Directory.Delete(uploadDir, true);
    
                 return new FileUploadChunkResBindingModel
                 {
                     Status = "Cancelled"
                 };
             }
         }
     }

This is code for WebHostBufferPolicySelector:

    public class NoBufferPolicySelector : WebHostBufferPolicySelector
     {
         public override bool UseBufferedInputStream(object hostContext)
         {
             if (hostContext is HttpContextBase context)
             {
                 if (context.Request.CurrentExecutionFilePath.EndsWith("UploadFile"))
                 {
                     // Do not use buffered input stream for file uploads - this reduces memory footprint
                     return false;
                 }
             }
    
             return true;
         }
     }

And it is registered in Global.asax.cs

 public class Global : HttpApplication
 {
         void Application_Start(object sender, EventArgs e)
         {
             GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicySelector());
         }
 }

(it's little bit simplified code example, I hope I didn't remove any important part for this issue).




dotnet-aspnet-webapi
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

YihuiSun-MSFT avatar image
0 Votes"
YihuiSun-MSFT answered

Hi @klinki,

1.

sometimes I get this exception

This usually happens when the website takes a while to respond or if the client has closed/disconnected from your site before finishing processing the request.
2.

Is this the right approach for large file upload?

You can upload large files using MultipartFormDataStreamProvider.You can click this link to view the code.

If the answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Best Regards,
YihuiSun


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.