question

DongleBerry-4652 avatar image
0 Votes"
DongleBerry-4652 asked ·

SCCM CB issues with client on Citrix VPN Gateway

Hi all,
I am having an issue with SCCM clients on the Citrix VPN Gateway. They are not receiving policies or new applications\updates. This is only on the Citrix Gateway. I can connect a client and ping\telenet to all MP’s and DP’s. However they will not pull down any new policy changes.

Subnets are in the correct boundary group.

I have contacted the networks team and they have confirmed that all the same firewall rules are in place on the VPN subnets that are on the existing on premise subnets.

I am getting the below errors in the CcmMessaging.log


<![LOG[Post to http://Xxxxxxx/ccm_system_windowsauth/request failed with 0x87d00231.]LOG]!><time="10:14:42.638-60" date="10-23-2020" component="CcmMessaging" context="" type="2" thread="58996" file="messagequeueproc_outgoing.cpp:452">
<![LOG[Client is not on internet]LOG]!><time="10:14:43.607-60" date="10-23-2020" component="CcmMessaging" context="" type="0" thread="58996" file="requestresponse.cpp:228">
<![LOG[Client is not set to use any webproxy.]LOG]!><time="10:14:43.609-60" date="10-23-2020" component="CcmMessaging" context="" type="0" thread="58996" file="requestresponse.cpp:246">
<![LOG[ccmhttp: Host=Xxxxxxx, Path=/ccm_system/request, Port=80, Protocol=http, CcmTokenAuth=0, Flags=0x4201, Options=0x4c0]LOG]!><time="10:14:43.609-60" date="10-23-2020" component="CcmMessaging" context="" type="0" thread="58996" file="utils.cpp:160">
<![LOG[Created connection on port 80]LOG]!><time="10:14:43.611-60" date="10-23-2020" component="CcmMessaging" context="" type="0" thread="58996" file="requestresponse.cpp:401">
<![LOG[Failed in WinHttpReceiveResponse API, ErrorCode = 0x2f78]LOG]!><time="10:14:43.790-60" date="10-23-2020" component="CcmMessaging" context="" type="2" thread="58996" file="requestresponse.cpp:774">
<![LOG[[CCMHTTP] ERROR: URL=http://Xxxxxxx/ccm_system/request, Port=80, Options=1216, Code=12152, Text=ERROR_WINHTTP_INVALID_SERVER_RESPONSE]LOG]!><time="10:14:43.790-60" date="10-23-2020" component="CcmMessaging" context="" type="1" thread="58996" file="ccmhttperror.cpp:306">
<![LOG[[CCMHTTP] ERROR INFO: StatusCode=<unknown> StatusText=]LOG]!><time="10:14:43.790-60" date="10-23-2020" component="CcmMessaging" context="" type="1" thread="58996" file="ccmhttperror.cpp:317">
<![LOG[Raising event:
instance of CCM_CcmHttp_Status
{
ClientID = "GUID:95c232d8-bf09-4a65-8816-125d568a037e";
DateTime = "20201023091443.792000+000";
HostName = "Xxxxxxx";
HRESULT = "0x80072f78";
ProcessID = 92776;
StatusCode = 0;
ThreadID = 58996;
};
]LOG]!><time="10:14:43.792-60" date="10-23-2020" component="CcmMessaging" context="" type="1" thread="58996" file="Event.cpp:840">
<![LOG[Successfully submitted event to the Status Agent.]LOG]!><time="10:14:43.794-60" date="10-23-2020" component="CcmMessaging" context="" type="0" thread="58996" file="Event.cpp:862">
<![LOG[Successfully queued event on HTTP/HTTPS failure for server 'Xxxxxxx'.]LOG]!><time="10:14:43.794-60" date="10-23-2020" component="CcmMessaging" context="" type="1" thread="58996" file="ccmhttperror.cpp:374">
<![LOG[Post to http://Xxxxxxx/ccm_system/request failed with 0x87d00231.]LOG]!><time="10:14:43.796-60" date="10-23-2020" component="CcmMessaging" context="" type="2" thread="58996" file="messagequeueproc_outgoing.cpp:452">



I have spoken with the Citrix team and they have informed me that the VPN traffic is all tunneled through the VPN as intranet traffic.

Any ideas? Is it something to do with how SCCM is interpreting the traffic? Internet or Intranet.

I’m not sure what the issue is.

Regards
Kevin

mem-cm-generalmem-cm-application
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

TravIs-9015 avatar image
0 Votes"
TravIs-9015 answered ·

Roma, can you confirm that what you said works with new version ADC VPN and SCCM?

If so then it's most likely due to one of these restrictions in the strict profile as those were enhancements due to some new vulnerabilities, which means SCCM is non-compliant with new industry standard vulnerability rules and an issue should be filed with MS.

"Mark HTTP Header with Extra White Space as Invalid"
"Mark RFC7230 Non-Compliant Transaction as Invalid"

Can someone please confirm?

· 1 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi Travls,

We made the following changes on the Citrix end;
1. Created a custom profile based on strict profile
2. Untick "Mark Header with extra white space as invalid and apply.

There was also a problem with the Citrix client 13.0.67.39, it wasn't showing the VPN IP address correctly, so SCCM could not tell what boundary should be in and could not download software after receiving the policies. We had to upgrade to version 13.0.67.42 to resolve it.

Regards
Kevin

0 Votes 0 ·
Jason-MSFT avatar image
0 Votes"
Jason-MSFT answered ·

0x2f78 = "The server returned an invalid or unrecognized response"

Something external to ConfigMgr is interfering or tampering with the client's traffic or the server's response. There is no way, from ConfigMgr's perspective to know which or to determine the source of this. You need to trace the network traffic and identify the source of this.

Is it something to do with how SCCM is interpreting the traffic? Internet or Intranet.

No. This simply dictates which MP the client will use. It doesn't change the nature of the traffic or the traffic itself. The log above clearly shows the client is "not internet". Additionally, it shows exactly which MP it is attempting to connect to so you can easily validate whether this is correct per your expectations and configuration.

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

DongleBerry-4652 avatar image
0 Votes"
DongleBerry-4652 answered ·

Thanks Jason,

I am able to connect to the url "http://Xxxxxxx/ccm_system/request" via IE and I can telnet to that server on port 80 via the machine on the VPN. How else can I identify what is interferring with the traffic?

The MP's are correct and I have force the client to use other MP's, they all have the same issue.

Kevin

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Jason-MSFT avatar image
0 Votes"
Jason-MSFT answered ·

Being able to make an initial connection only proves the path is correct. You may need to examine the traffic using a network sniffer or involve your network team to do the same and validate what other devices may have the ability to interact with the traffic in this path. This is completely specific to your environment so there's no way for anyone outside of your org to guide you directly.

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Ariff-2679 avatar image
0 Votes"
Ariff-2679 answered ·

We have the exact same issue after upgrading to Citrix Gateway version 13. We have a support call logged with Citrix but no solution as of yet other than trying a roll-back to version 12.

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

TravIs-9015 avatar image
0 Votes"
TravIs-9015 answered ·

Can each of you verify what Build of ADC 13.0 was upgraded to when it stopped?

Can anyone also verify if it worked in a prior build of 13.0?

Can anyone verify if it continues to work if the Client Gateway Plugin is NOT upgraded when ADC is upgraded?

Can Microsoft confirm if an added client-ip header in the HTTP request would cause issues with SCCM such as in the image:
38030-image.png



image.png (289.3 KiB)
· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Ariff-2679 avatar image
2 Votes"
Ariff-2679 answered ·

Can each of you verify what Build of ADC 13.0 was upgraded to when it stopped?
13.0.64.35

Can anyone also verify if it worked in a prior build of 13.0?
It worked on 12.1.56.22.
IIS Logs on the SCCM management point has no CCM_POST /ccm_system/request - 80 - ... ccmhttp - 200 0 0 319 123 entries for VPN networks when on version 13

Can anyone verify if it continues to work if the Client Gateway Plugin is NOT upgraded when ADC is upgraded?
Still does not work when old plugin used.

If you roll-back to 12.1.56.22 it works again.




· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

LeighSatchell-4264 avatar image
0 Votes"
LeighSatchell-4264 answered ·

Thanks Ariff! Switched back to ADC v12.1-60.16 (latest 12.1) and works again with SCCM. Had no joy with v13.0.67.39 and your line above was gold.
Need to get Citrix on this. SSLVPN no use to us without SCCM working to deploy apps and patches.

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ROMA-8862 avatar image
0 Votes"
ROMA-8862 answered ·

hi
in Citrix Gateway VPN , unbind HTTP profile restrict and bind the Default HTTP profile , because after update automatically Netscaler bind the HTTP strict profile

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Ariff-2679 avatar image
0 Votes"
Ariff-2679 answered ·

Thanks Roma, I can confirm enabling HTTP default profile resolves issue on version 13.

· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.