question

heetsink-3975 avatar image
1 Vote"
heetsink-3975 asked KennyHalskovAaberg-7610 commented

Windows Server 2019 RDS - Start & Search does not work, desktops don't load

server 2019 1809 2020-12 updates installed OK, all the way back to March when it was stood up.

Its a RDS server, and I originally configured clients VHDX files to be on a storage server. Very reliable for many months.

First came the complaints about slow log ins, a blank screen for several minutes when people signed in.

then the complaints about the Start menu and Search not working. the left mouse button would not open the Start menu, and users could not enter text in the Search field.

users couldn't figure out how to get to network shares, so I had to create shortcuts on the public desktop.

then the reliability issues. the server would accept connections but the desktop would never render. just a dark blank screen first for several minutes, then then the GUI would never render until the server was powered off and on.

the event viewer start filling up DCOM errors in the system log every time a user signed in. Always about Cortona.

log Name: System

Source: Microsoft-Windows-DistributedCOM

Date: 12/18/2020 2:23:22 PM

Event ID: 10001

Task Category: None

Level: Error

Keywords: Classic

User: domain\user

Computer: server.domain

Unable to start a DCOM Server: Microsoft.Windows.Cortana_1.11.6.17763_neutral_neutral_cw5n1h2txyewy!CortanaUI as Unavailable/Unavailable. The error:

"0"

Happened while starting this command:

"C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe" -ServerName:CortanaUI.AppXa50dqqa5gqv4a428c9y1jjw7m3btvepj.mca

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">;

<System>

 <Provider Name="Microsoft-Windows-DistributedCOM" Guid="{1B562E86-B7AA-4131-BADC-B6F3A001407E}" EventSourceName="DCOM" />

 <EventID Qualifiers="0">10001</EventID>

 <Version>0</Version>

 <Level>2</Level>

 <Task>0</Task>

 <Opcode>0</Opcode>

 <Keywords>0x8080000000000000</Keywords>

 <TimeCreated SystemTime="2020-12-18T20:23:22.975245900Z" />

 <EventRecordID>622171</EventRecordID>

 <Correlation />

 <Execution ProcessID="2128" ThreadID="50464" />

 <Channel>System</Channel>

 <Computer>RDS.andresmedical.local</Computer>

 <Security UserID="S-1-5-21-1177238915-1897051121-725345543-7972" />

</System>

<EventData>

 <Data Name="param1">"C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe" -ServerName:CortanaUI.AppXa50dqqa5gqv4a428c9y1jjw7m3btvepj.mca</Data>

 <Data Name="param2">0</Data>

 <Data Name="param3">Microsoft.Windows.Cortana_1.11.6.17763_neutral_neutral_cw5n1h2txyewy!CortanaUI</Data>

 <Data Name="param4">Unavailable</Data>

 <Data Name="param5">Unavailable</Data>

</EventData>

</Event>

SFC /scannow found nothing
DISM /Online /Cleanup-Image /CheckHealth
DISM /Online /Cleanup-Image /ScanHealth
DISM /Online /Cleanup-Image /RestoreHealth with the source switch said it found issues and fixed them but...

a few days later the server was stuck with blank screens again!

I reset profiles to be stored on the C drive and that was not a fix. Start worked briefly. a few days later the server was stuck with blank screens again!

what started as an annoyance mutated into a big disruption. Seen many posts talking about the broken start but not much success.

if I navigate to C:\Windows\SystemApps\Microsoft.Windows.Cortana_cw5n1h2txyewy\SearchUI.exe and run as admin,

Unable to start a DCOM Server: Microsoft.Windows.Cortana_1.11.6.17763_neutral_neutral_cw5n1h2txyewy!CortanaUI.AppXfbn8w4s0jbk3tjevpcn9kaxerc6rby8k.mca as Unavailable/Unavailable. The error:

"0"
Happened while starting this command:
"C:\Windows\system32\backgroundTaskHost.exe" -ServerName:CortanaUI.AppXy7vb4pc2dr3kc93kfc509b1d0arkfb2x.mca
will I have to rebuild from scratch or can this server be saved?

remote-desktop-serviceswindows-server-2019
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I can confirm this issue.

We are experiencing the same problems on Windows 2019 servers (same setup). Howerver the problem began surfacing in January. At first we had only one customer server with this kind of trouble. Now another developed the same fault.
It seems to only affect RDS Servers.

Reinstalling Cortana usually fixes the start menu problems. It also reactivates the notification center. This fix however does not last for long. The bigger issue is a failure of AAD.BrokerPlugin which renders Outlook inoperable.

Whenever shell experience host, cortana crash, search and aad broker go down with it. aad broker cannot be fixed by reinstalling. Reboots are useless. The only fix for us was to restore snapshots. However the problems will resurface after 5 days or even 1 day.

The servers have been running for months or years without any problem of this kind. A solution to this problem is urgent since our servers are being rendered inoperable

0 Votes 0 ·

Hi @BjrnBraun-5155

Please take a look at this post to reset the ADD.BrokerPlugin



Andrew

0 Votes 0 ·

Well that didn't last long

Errors appeared 2 hours later. However I restored a previous snapshot of this particular RDS Server and checked the registry
Turns out its an old 2016 bug.

Open regedit
Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\FirewallRules

FirewallRules contained about 50k + entries. Deleted all entries containing {} and left standard entries untouched.
Took a very long time to delete.

There are scripts out there to automate this.

Didn't think this problem still exists, especially since firewall was disabled.

1 Vote 1 ·

Oh man,

that did it. Well I did not follow the instructions and just purged Teams from a server. The problem occurred again after 6 days. This morning at first login the user also opened teams. Cortana, Shellexperiencehost, aadbroker and searchui were all inoperable. After purging Teams I ran:


 Get-AppXPackage -AllUsers | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

And it fixed everything. When users logged in again, Outlook and everything else worked fine. No DCOM Errors in Eventlog.

What I don't get. Why does this seem to happen after 3-7 days runtime.
According to Teams it was last updated in Dec. 18th 2020. We startet having problems with it in Jan 2021

We also have Office Updates disabled on a few systems. Maybe it could also be a problem but for now removing teams is fine by me

Thanks alot!

0 Votes 0 ·
ElevenYu-MSFT avatar image
0 Votes"
ElevenYu-MSFT answered ElevenYu-MSFT edited

Hi,

Please first check the general troubleshooting steps for Event 10001.
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc774391(v=ws.10)?redirectedfrom=MSDN

And have you tried to reinstall Cortona?

Run PowerShell command as an administrator:
Get-AppXPackage -Name Microsoft.Windows.Cortana | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

Thanks,
Eleven


If the Answer is helpful, please click "Accept Answer" and upvote it. Thanks.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

heetsink-3975 avatar image
0 Votes"
heetsink-3975 answered heetsink-3975 edited

Unable to reinstall Cortana, see below
Description:
Unable to start a DCOM Server: Microsoft.Windows.Cortana_1.11.6.17763_neutral_neutral_cw5n1h2txyewy!CortanaUI.AppXfbn8w4s0jbk3tjevpcn9kaxerc6rby8k.mca as Unavailable/Unavailable. The error:
"0"
Happened while starting this command:
"C:\Windows\system32\backgroundTaskHost.exe" -ServerName:CortanaUI.AppXy7vb4pc2dr3kc93kfc509b1d0arkfb2x.mca
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">;
<System>
<Provider Name="Microsoft-Windows-DistributedCOM" Guid="{1B562E86-B7AA-4131-BADC-B6F3A001407E}" EventSourceName="DCOM" />
<EventID Qualifiers="0">10001</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8080000000000000</Keywords>
<TimeCreated SystemTime="2020-12-18T20:55:37.198494600Z" />
<EventRecordID>622336</EventRecordID>
<Correlation />
<Execution ProcessID="2128" ThreadID="65020" />
<Channel>System</Channel>

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PaulAllan-0163 avatar image
0 Votes"
PaulAllan-0163 answered

We are also running a 2019 RDS Farm and we have had this exact same issue with the Start Menu not working this week. Initially it was on one host, and now it is happening on two hosts! We ran the command on one host and it has fixed it. We have not ran it on the second host yet, as we had to stop it taking connections as it was causing so many issues for users.

Get-AppXPackage -AllUsers | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

Does MS know why this is happening? Has a Jan update caused this issue? It is very worrying since most of our staff are currently WFH and are relying on the RDS servers to be able to work effectively.

Many Thanks
Paul.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

PrithishPoojari-8660 avatar image
0 Votes"
PrithishPoojari-8660 answered

We are having similar issue with our 2019 RDS farm as well, seems like it just slowly showing up on rest of our servers in the farm.
When I ran the below command it fixed the issue on couple of our RDS server but seems to popped up back after like 3 to 4 days ,

Get-AppXPackage -AllUsers | Foreach {Add-AppxPackage -DisableDevelopmentMode -Register "$($_.InstallLocation)\AppXManifest.xml"}

Can't really say what the root cause is, hope someone have an answer that is a permanent fix, hope this is not one of those Microsoft bug .

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

BjrnBraun-5155 avatar image
0 Votes"
BjrnBraun-5155 answered

After removing 70k+ Firewall entries within the registry all our 2019 RDS Servers ran smoothly for about 2 weeks.
One has shown the same problems again today.

At first we notice it because of clients complaining about outlook not being able to connect to o365.
They do not notice the start menu not working cause we have blocked almost everything with gpo. Anyhow eventlog showed once more that shellexperience, cortana and microsoft.aad.broker would not function.

until now no other rds server have shown any sign or this problem. So I hope this was just a "bad day".

Alternatively I found

https://social.technet.microsoft.com/Forums/ie/en-US/c212e6d7-160d-4a54-bfc6-92bd720fc424/windows-2016-rds-start-menu-not-working-getappxpackage-returns-nothing?forum=winserverTS

which gives hints about removing some notification keys. However when I did just that the RDS became unusable. Also I only had about 100 entries or less. Nothing compared to 70k.

The only solution now is to restore a snapshot. I restored a 4 day old one so we will see if this will happen again in about 4 days from now.

Our RDS Servers are now very unreliable and there seems to be no way to solve it. For us it started in Jan and the runtime of each rds does not seem to matter. Some are 8 months old, some 1 1/2 years. Within weeks each developed the problem

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

BjrnBraun-5155 avatar image
7 Votes"
BjrnBraun-5155 answered RobJanssen-9725 published

I guess we found a solution or at least the right combination of solutions.

Since this problem kept reappearing even after we cleared out most of the bloated keys. So we looked and oh boy there was another.

Hiding in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\RestrictedServices\AppIso\FirewallRules

Within were about half a million entries... needles to say it took regedit about an hour to open it.

So I removed it via powershell

 Remove-Item "HKLM:\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\RestrictedServices\AppIso\FirewallRules"
    
 New-Item "HKLM:\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\RestrictedServices\AppIso\FirewallRules"

After this, you don't even need to reboot. Just log off all users and login again. Startmenu and everything else will work as it should.

To prevent this from happening

Set DWORD "DeleteUserAppContainersOnLogoff = 1
in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy

Make sure to Cleat out \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\FirewallRules first. With a script or manual.

If you don't clean first, loggin off users will take a very long time. If everyhting is clean then logoff will take slightly longer because it will clean up all user entries by itself hence not bloating the keys again.


· 9
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you so much!! Solved the problem... hope it is definitive solution!!

1 Vote 1 ·

For a month I could not solve this problem, 5 terminal servers worked without a taskbar and a start menu). Thanks for the solution, everything works great!

0 Votes 0 ·

thank you so much i spent weeks in troubleshooting this problem... your solution did the job.

0 Votes 0 ·

I had the same problems on 3 RDS 2019 server. Ran the DevMode command and cleaned the registry keys. But now my users keep losing their default printer/default apps every time they log out of their session. Anyone else ran into that problem?

0 Votes 0 ·

goodman! worked for me

0 Votes 0 ·
Show more comments
RgisSoares-8658 avatar image
0 Votes"
RgisSoares-8658 answered

Also for me, same problem again!

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MrPink-0257 avatar image
0 Votes"
MrPink-0257 answered

We had the same problem on all our 2019 RDP Host servers a few months ago.
They were rebuilt but unfortunately the problem has returned. :'(

Start menu broken
Office activation failed
Event viewer full of :

Unable to start a DCOM Server: Microsoft.Windows.Cortana_1.11.6.17763_neutral_neutral_cw5n1h2txyewy!CortanaUI.AppXfbn8w4s0jbk3tjevpcn9kaxerc6rby8k.mca as Unavailable/Unavailable. The error:
"0"
Happened while starting this command:
"C:\Windows\system32\backgroundTaskHost.exe" -ServerName:CortanaUI.AppXy7vb4pc2dr3kc93kfc509b1d0arkfb2x.mca



and




Unable to start a DCOM Server: Microsoft.AAD.BrokerPlugin_1000.17763.1.0_neutral_neutral_cw5n1h2txyewy!Windows.Security.Authentication.Web.Core.BackgroundGetTokenTask.ClassId.WebAccountProvider as Unavailable/Unavailable. The error:
"0"
Happened while starting this command:
"C:\Windows\system32\BackgroundTaskHost.exe" -ServerName:BackgroundTaskHost.WebAccountProvider

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

ChrisH-0801 avatar image
0 Votes"
ChrisH-0801 answered

Just came here to say that we encountered this problem with our RDS 2019 session hosts this week.

The steps above provided by @BjrnBraun-5155 fixed the issues for us.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KennyHalskovAaberg-7610 avatar image
0 Votes"
KennyHalskovAaberg-7610 answered KimSchousboe-2097 commented

I'm hoping someone reads this "old" post still, since we are still having major issues with the above problems.

I've tried every single solution I can find on web, and tried all the above - and still I get plenty of errors in event viewer, and nothing seems to work.

Only thing that works so far:
- Make a new profile and move everything to the new vhdx (fslogix)
- Or delete everything from appdata\local from the current user.

But the last option is not at all viable, since every personal items from the user is wiped also, but at least is fixes startmenu and search. Copy in the FSLogix from Local to the new vhdx does give back all users items, but also make the startmenu and search unusable again.
I've tried also today to migrate-vhd / moveto-vhd to new vhdx created for the user, but after frx.exe is done, the problem is reappearing unfortunately.

I will gladly take ANY advice at this point, since we are probably making new users every week for our customers. Very frustrating to say the least.

RDSH consisting of Windows 2019 (1809)
FSLogix 2.9.7838.44263

When following is logged in eventviewer (repeating it self within 2 sec interval, filling up the event viewer), we know we need to contact the user because something is wrong with the vhdx -
The access history in hive \Device\HarddiskVolume22\Profile\AppData\Local\Packages\Microsoft.Windows.Cortana_cw5n1h2txyewy\Settings\settings.dat was cleared while updating x keys and creating x changed pages.

· 10
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hej Kenny!

Have you tried to delete the Firewall Rules like @BjrnBraun-5155?

0 Votes 0 ·

Hej Kim.

Yes, I have deleted every firewall rules mentioned in this post (and other posts), but with no luck.

When the users profile is broken, it sometimes causing other users who connect to the same host as the broken user, rendering the users screen just point black. After around 10-20 minutes or so, the icons and the desktop start re-appearing.

0 Votes 0 ·
KimSchousboe-2097 avatar image KimSchousboe-2097 KennyHalskovAaberg-7610 ·

139795-cleanfw2.txt



Here is the attached script to delete the rules from regedit.

0 Votes 0 ·
cleanfw2.txt (2.8 KiB)
Show more comments