HttpRequestException: Request headers must contain only ASCII characters with Asp.Net Core 3.1 on Ubuntu

Question

I have an Asp.Net Core 3.1 Web app with Angular and individual authentication. I'm developing it on Ubuntu 18.04. The database is SQL Server Express. When sending registration confirmation email with SendGrid API, I get an exception "HttpRequestException: Request headers must contain only ASCII characters". I do not see any non-ASCII characters in the header.

I had exactly the same error on Windows 10. I updated Workloads and Individual Components in my Visual Studio and that solved it. I'm not sure how to solve it on Ubuntu. I installed the 3.1 SDK according to these instructions

This is the method:

    public Task SendEmailAsync(string email, string subject, string message)  
    {  
        string apiKey = _config.GetValue<string>("ExternalProviders:SendGrid:ApiKey");  
        return Execute(apiKey, subject, message, email);  
    }  

 public Task Execute(string apiKey, string subject, string message, string email)  
{  
    string sender = _config.GetValue<string>("ExternalProviders:SendGrid:SenderEmail");  
    string senderUser = _config.GetValue<string>("ExternalProviders:SendGrid:SenderName");  
    var msg = new SendGridMessage()  
    {  
        From = new EmailAddress(sender, senderUser),  
        Subject = subject,  
        PlainTextContent = message,  
        HtmlContent = message  
    };  
    msg.AddTo(new EmailAddress(email));  
    var client = new SendGridClient(apiKey);  
    return client.SendEmailAsync(msg);  
}  

This is the error message:

System.Net.Http.HttpConnection.WriteStringAsync(string s)
System.Net.Http.HttpConnection.WriteHeadersAsync(HttpHeaders headers, string cookiesFromContainer)
System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, bool doRequestAuth, CancellationToken cancellationToken)
System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, bool doRequestAuth, CancellationToken cancellationToken)
System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
SendGrid.Helpers.Reliability.RetryDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task<HttpResponseMessage> sendTask, HttpRequestMessage request, CancellationTokenSource cts, bool disposeCts)
SendGrid.BaseClient.MakeRequest(HttpRequestMessage request, CancellationToken cancellationToken)
SendGrid.BaseClient.RequestAsync(Method method, string requestBody, string queryParams, string urlPath, CancellationToken cancellationToken)
SendGrid.BaseClient.SendEmailAsync(SendGridMessage msg, CancellationToken cancellationToken)
NoblyAppsSite.Areas.Identity.Pages.Account.RegisterModel.OnPostAsync(string returnUrl) in Register.cshtml.cs

                    await _emailSender.SendEmailAsync(Input.Email, "Confirm your email",  

Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.ExecutorFactory+GenericTaskHandlerMethod.Convert<T>(object taskAsObject)
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.ExecutorFactory+GenericTaskHandlerMethod.Execute(object receiver, object[] arguments)
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker.InvokeHandlerMethodAsync()
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker.InvokeNextPageFilterAsync()
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker.Rethrow(PageHandlerExecutedContext context)
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.RazorPages.Infrastructure.PageActionInvoker.InvokeInnerFilterAsync()
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted)
Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
IdentityServer4.Hosting.IdentityServerMiddleware.Invoke(HttpContext context, IEndpointRouter router, IUserSession session, IEventService events)
IdentityServer4.Hosting.MutualTlsTokenEndpointMiddleware.Invoke(HttpContext context, IAuthenticationSchemeProvider schemes)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Here is the header:

Variable Value
:authority localhost:5001
:method POST
:path /Identity/Account/Register?returnUrl=%2Fauthentication%2Flogin
:scheme https
Accept text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
Accept-Encoding gzip, deflate, br
Accept-Language en-US,en;q=0.5
Cache-Control max-age=0
Content-Length 280
Content-Type application/x-www-form-urlencoded
Cookie .AspNetCore.Antiforgery.9dU1lrY_fgE=CfDJ8Ma-TG6aJSVPqeaMKCGLfGFJbfQKQm0lFVjSI2UwnYxEquMO8_ii1znT8IGVhFJYOdsQCKReeRKvLZxiaGwOO8gkYGhV-xYhlDw707Kf8Zj235AD3kbPE18X2wrCYP2uUd26cpjYZHYAMcLEow7KdPY
Host localhost:5001
Origin https://localhost:5001
Referer https://localhost:5001/Identity/Account/Register?returnUrl=/authentication/login
TE trailers
Upgrade-Insecure-Requests 1
User-Agent Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0

How can I get rid of this error?

Answer 1

Hi, @Tom Bara

By default (and RFC) SMTP headers are US-ASCII. In order to send non-ASCII characters, you need to use Unicode escape sequences.
You could try to encode header anyway by the methods below.

Here are methods to EncodeNonAsciiCharacters and DecodeEncodedNonAsciiCharacters.

static string EncodeNonAsciiCharacters( string value ) {  
        StringBuilder sb = new StringBuilder();  
        foreach( char c in value ) {  
            if( c > 127 ) {  
                // This character is too big for ASCII  
                string encodedValue = "\\u" + ((int) c).ToString( "x4" );  
                sb.Append( encodedValue );  
            }  
            else {  
                sb.Append( c );  
            }  
        }  
        return sb.ToString();  
    }  
  
    static string DecodeEncodedNonAsciiCharacters( string value ) {  
        return Regex.Replace(  
            value,  
            @"\\u(?<Value>[a-zA-Z0-9]{4})",  
            m => {  
                return ((char) int.Parse( m.Groups["Value"].Value, NumberStyles.HexNumber )).ToString();  
            } );  
    }