question

Schoch-0646 avatar image
0 Votes"
Schoch-0646 asked GaryNebbett commented

WinRM error code 400

Hello

Since a short time I have the problem, that I can't connect to other computers in my network with WinRM, but only if I use the FQDN.
When I use only the computer name without the DNS suffix then it works.
The problem occurred on Windows 10 and also Windows Server 2019.

This is the error message if I use the FQDN:
Enter-PSSession: Processing data for a remote command failed with the following error message: The WinRM client received an HTTP bad request status (400), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic.

I have also a strange behavior if I open for example the Hyper-V Snap-In and try to connect to another computer with Hyper-V. Then I see this message:
The object contained a value in property "Started" that is not valid.

Does anyone have any idea what causes this strange behavior?
DNS resolution is working fine, without FQDN and with.

Thanks for help.



windows-server-powershellwindows-10-network
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GaryNebbett avatar image
0 Votes"
GaryNebbett answered

Hello @Schoch-0646,

Is it possible that your connections use a proxy server if a FQDN is used and connect directly if a plain (local) name is used?

Gary

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

Have you recently done anything with certificates? Is it possible that the hostname has been used on a certificate(s) instead of the fqdn?

Have you checked the "trust" list in the WinRM configurations on the clients (if you're not using Kerberos for authentication)?

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Schoch-0646 avatar image
0 Votes"
Schoch-0646 answered

I have checked the certificates, but they use all the fqdn and I don't know anything about recently certificate changes.

Interestingly, I have some VMs that don't have this problem. From these VM's I can connect to any other VM's in my domain without problem with the fqdn.
These VM's are all in the same OU. So I tried to move a problematic VM to the OU where WinRM is working and I updated the group policies. But it doesn't helped.

For me that would indicate there is a problem with the WinRM client and not the WinRM server, because I can connect on the working VM's to all other VM's with the fqdn. But with the problem VM's no connection to any other VM with the fqdn is working.

I checked also the WinRM client settings, but they are the same on the working and the problem VM's:
Client
NetworkDelayms = 5000
URLPrefix = wsman
AllowUnencrypted = false [Source="GPO"]
Auth
Basic = false [Source="GPO"]
Digest = false [Source="GPO"]
Kerberos = true
Negotiate = true
Certificate = true
CredSSP = false
DefaultPorts
HTTP = 5985
HTTPS = 5986
TrustedHosts

Any idea if there is something else which I can check? Thanks.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

Does this quick test work?

 Test-NetConnection <FQDN> -port 5985

I suppose it's possible that the size of the token is too large if the user is a member of a large number of security groups. I haven't encountered that problem in many years, but I remember having that problem. This describes a way to circumvent that problem: winrm-client-http-bad-request-status-400

How about checking DNS? Does your server have both an IPv4 and IPv6 address? Does a reverse lookup on the addresses return the correct name? Do all your DNS servers return the same IP addresses in a forward lookup for the FQDN?

How about (from the client) running "winrm id -remote:<FQDN>" and/or "winrm id -remote<FQDN> -auth:none"?

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Schoch-0646 avatar image
0 Votes"
Schoch-0646 answered

Test-Connection succeeds yes. Because WinRM returns an error 400 I supposed this.

The size of the token could not be the problem, because I have only a small deployment with one rootdomain and one DC as DNS server.

I use only IPv4 in my network, but link local IPv6 is enabled. The problem occurs within the same subnet and also across subnets. The reverse lookup is working also.
I have only one DNS server, so the DNS answers are always the same.

Here you can see my console output:

PS C:\> Resolve-DnsName rdgateway.blabla.ch

Name Type TTL Section IPAddress


rdgateway.blabla.ch A 1200 Answer 10.137.0.44

PS C:\> Resolve-DnsName 10.137.0.44

Name Type TTL Section NameHost


44.0.137.10.in-addr.arpa PTR 1200 Answer rdgateway.blabla.ch

PS C:\> winrm id -remote:rdgateway.blabla.ch
IdentifyResponse
ProtocolVersion = http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd
ProductVendor = Microsoft Corporation
ProductVersion = OS: 10.0.17763 SP: 0.0 Stack: 3.0
SecurityProfiles
SecurityProfileName = http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos

PS C:\> winrm id -remote:rdgateway.blabla.ch -auth:none
WSManFault
Message = The WinRM client cannot process the request. The destination computer (rdgateway.blabla.ch:5985) returned an 'access denied' error. Specify one of the authentication mechanisms supported by the server. If Kerberos mechanism is used, verify that the client computer and the destination computer are joined to a domain. Possible authentication mechanisms reported by server: Negotiate Kerberos

Error number: -2147024891 0x80070005
Access is denied.
PS C:\> Enter-PSSession rdgateway.blabla.ch
Enter-PSSession: Processing data for a remote command failed with the following error message: The WinRM client received an HTTP bad request status (400), but the remote service did not include any other information about the cause of the failure. For more information, see the about_Remote_Troubleshooting Help topic.
PS C:\> Enter-PSSession rdgateway
[rdgateway]: PS C:\Users\testuser\Documents> exit
PS C:\>

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

Does using the IP address (e.g., 10.137.0.44 in your example) instead of the FQDN work?

Can you check the WinRM "TrustedHosts " on the three machines (the one that fails to connect, the one that successfully connects, and the target machine)?

 Get-Item WSMan:\localhost\Client\TrustedHosts

Do the certificates you checked have a thumbprint that matches the one WinRM is using on each of the machines (particularly the one from which the failure happens)? You can get the thumbprint from WinRM like this (run as administrator):

 get-item WSMan:\localhost\Service\Cert*

Does Get-WMIObject work from the problem machine? Just try something innocuous , like this:

 Get-WmiObject -Computer <fqdn> -Class Win32_logicaldisk



5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Schoch-0646 avatar image
0 Votes"
Schoch-0646 answered Schoch-0646 edited

Does using the IP address (e.g., 10.137.0.44 in your example) instead of the FQDN work?
No, I get this error message:
Enter-PSSession: Connecting to remote server 10.137.0.44 failed with the following error message : The WinRM client cannot process the request. Default authentication may be used with an IP address under the following conditions: the transport is HTTPS or the destination is in the TrustedHosts list, and explicit credentials are provided. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. For more information on how to set TrustedHosts run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

Can you check the WinRM "TrustedHosts " on the three machines (the one that fails to connect, the one that successfully connects, and the target machine)?
The failing machine:
PS C:\> Get-Item WSMan:\localhost\Client\TrustedHosts

WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Client

Type Name SourceOfValue Value
System.String TrustedHosts

The successful machine:
PS C:\Users\seph.CTB> Get-Item WSMan:\localhost\Client\TrustedHosts

WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Client

Type Name SourceOfValue Value
System.String TrustedHosts

The target machine:
PS C:\Users\seph> Get-Item WSMan:\localhost\Client\TrustedHosts

WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Client

Type Name SourceOfValue Value
System.String TrustedHosts

The TrustedHosts list is empty on all three machines.


Do the certificates you checked have a thumbprint that matches the one WinRM is using on each of the machines (particularly the one from which the failure happens)? You can get the thumbprint from WinRM like this (run as administrator):
On all three machines I get an empty value after I ran the cmd as admin in PowerShell:
PS C:\> get-item WSMan:\localhost\Service\Cert*

WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Service

Type Name SourceOfValue Value
System.String CertificateThumbprint


Does Get-WMIObject work from the problem machine? Just try something innocuous.
This command works on the problem machine yes:
PS C:\Windows\system32> Get-WmiObject -Computer rdgateway.blabla.ch -Class Win32_logicaldisk

DeviceID : C:
DriveType : 3
ProviderName :
FreeSpace : 153842049024
Size : 171152764928
VolumeName :


When I run the bellow command on the target machine rdgateway.blabla.ch then I get this:
PS C:\Windows\system32> Winrm enumerate winrm/config/listener
Listener
Address = *
Transport = HTTP
Port = 5985
Hostname
Enabled = true
URLPrefix = wsman
CertificateThumbprint
ListeningOn = 10.137.0.44, 127.0.0.1, ::1

On all my machines I can see "Transport = HTTP". That means HTTP is used for all communication via WinRM right?
Then the certificates can not be the problem I guess.


When I run this on the problem and the working machine I get the exact same results:
PS C:\> winrm get winrm/config/client
Client
NetworkDelayms = 5000
URLPrefix = wsman
AllowUnencrypted = false [Source="GPO"]
Auth
Basic = false [Source="GPO"]
Digest = false [Source="GPO"]
Kerberos = true
Negotiate = true
Certificate = true
CredSSP = false
DefaultPorts
HTTP = 5985
HTTPS = 5986
TrustedHosts

PS C:\>

I can see "AllowUnencrypted = false [Source="GPO"]". Is this possible? Can HTTP work with this setting?


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

RichMatheisen-8856 avatar image
0 Votes"
RichMatheisen-8856 answered

I'm gonna take a flier, here. Let's run this on the failing machine. Do you find the machine's FQDN in any of the "Subject" or "Alternative Subject Name" properties?

 Set-Location Cert:\LocalMachine\My
 #Get the installed certificates
 Get-ChildItem | 
     Select-Object -Property Subject, 
                             FriendlyName, 
                             Thumbprint, 
                             @{name='Subject Alternative Name';expression={($_.Extensions | Where-Object {$_.Oid.FriendlyName -eq "Subject Alternative Name"}).format($true)}} |
     Export-Csv c:\junk\certs.csv
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Schoch-0646 avatar image
0 Votes"
Schoch-0646 answered

Yes I can find a certificate with the fqdn of the failing machine. The machine name of the failing machine is "srvbuild":

Subject : CN=Windows Admin Center
FriendlyName : Windows Admin Center
Thumbprint : CE6527DE45E3976C3A28FC0C1DA9DF157A1A1323
Subject Alternative Name : DNS Name=srvbuild
DNS Name=localhost


Subject : CN=localhost
FriendlyName : IIS Express Development Certificate
Thumbprint : A89C27F72DA467559929DF3262717649B147AA08
Subject Alternative Name : DNS Name=localhost


Subject : CN=srvbuild.blabla.ch
FriendlyName :
Thumbprint : 8C2C145E20E1619C0039AEE2EF8BBA63B6BDFCD1
Subject Alternative Name : DNS Name=srvbuild.blabla.ch


I did something more, I copied the certificate 8C2C145E20E1619C0039AEE2EF8BBA63B6BDFCD1 to the target machine rdgateway.blabla.ch (10.137.0.44) and checked if the certificate is validated as valid. The result is true.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GaryNebbett avatar image
0 Votes"
GaryNebbett answered

Hello @Schoch-0646,

One thing that you could try is to trace the Microsoft-Windows-WebIO ETW provider when trying to use winrm. That might give some insight into the cause of the problem.

There are lots of ways of controlling ETW, including the "logman" command and various PowerShell cmdlets (e.g. Add-EtwTraceProvider).

Let us know if you need any help with the trace or its analysis.

Grüsse aus Basel.

Gary

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.