c# HttpClient 400 Bad Request when using https

Question

I'm trying to get a web request working using HttpClient. It works when using http, but does not work with https. The URL I'm testing with works in a browser with both http and https, however, when using the code below it works only with http. Do I need to add another confuration to make HttpClient work with https?

Note: I have used the proxy below in a Python app and it works for both http and https so the problem is not the proxy.

public static async System.Threading.Tasks.Task Main(string[] args)
{
    var url = "https://www.google.com";
    using (System.Net.Http.HttpClientHandler handler = new System.Net.Http.HttpClientHandler()
    {
        Proxy = new System.Net.WebProxy(@"http://73.206.190.189:80"),
        UseProxy = true,
    })
    {
        using (System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient(handler))
        {
            hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html)");
            string data = "";
            using (System.Net.Http.HttpResponseMessage response = await hc.GetAsync(url))
            {
                data = await response.Content.ReadAsStringAsync();
                System.Console.WriteLine(data);
            }
            System.Console.WriteLine(data);
        }
    }
}

Thank You

Answer 1

@moondaddy

A browser can automatically switch between a site or sites that is using HTTP or HTTPS, which is built-in functionality in the browsers.

Python I don't know about.

HttpClient works with HTTP or HTTPS.

I would assume the client machine running a program hosted by a Windows O/S trying to access an HTTPS site has a HTTPS certificate installed to access an HTTPS site. That's how it usually works.