Having run out of ideas, and given up Binging a solution for my intermittent connectivity problems around Windows 8, IE 10, and Outlook, it was time to stop playing nicely. Time instead for a day with my head in the server cabinet, a handful of network cables, some sticky labels, and decisive action.
The problems documented over the past few weeks (see All I Want For Christmas) were still intermittently annoying, as well as being annoyingly intermittent. I was totally unable to track down any DNS problems, despite many hours experimenting with different forwarders, root entries, test scripts, clearing caches, and more.
I'd logged all dropped packets in ISA server for a day, but there were none related to the problem from the machines under test. Though ISA's performance monitor was consistently reporting an average dropped packet rate of 0.3 per second and I wasted half an hour tracing these back to my wireless access point. Even though all of its fancy USB, printer connection, and media sharing features are turned off it still insists on sending out a network discovery packet every three seconds. Highly annoying.
Then I read more on the ISA Server blog sites about how using the proxy client changes the behaviour of machines connected to ISA. Of course, I haven't actually installed the proxy client directly since XP days. I just assumed that some clever mechanism in Windows Vista, 7, and 8 used the Gateway/Router setting specified in DHCP to find the proxy server and set themselves up for it automatically.
What I read suggested that ISA itself is doing DNS lookups in response to requests from clients, whereas a ping or nslookup on the client uses the network DNS server or does its own DNS lookup. So trying to track faults with nslookup after a connection failure was a waste of time. By now I was rapidly tiring of trying to be a network administrator, and I didn't bother following this up any further to see if it really is the case.
All of which prompted the decision to perform some radical surgery in the server cabinet, and get rid of ISA Server altogether. It's a Hyper-V VM, so it won't reduce the physical server count - but it will simplify administration and backup tasks and, hopefully, resolve my connectivity problems. I replicated all the ISA outbound rules in the firewall of my load-balancing router, which sits between the ISA server and the two ISP modems. A day monitoring the router logs and fine-tuning rules suggested this would work fine.
Reconfiguring the network was deceptively easy. Simply power off the ISA VM, change the IP address of the router to the same as ISA (the address already specified in the DHCP scope options), and run some network penetration tests. Instantly everything seems to be faster, web page loads are snappier, and no sign of smoke or loud bangs. And if it all goes wrong, or turns out to be a mistake, I still have the ISA Server VM so I can easily revert to the previous configuration.
But will my simple load-balancing router be able to cope with all that extra firewalling and packet shifting load once I start hammering the network with my usual working-week vigour? It's only an old LinkSys RV042 with a 100 Mbs Ethernet port. Do I need to upgrade to something like the new RV320 instead? I guess I'll soon find out.
And, of course, the question now is what will I do next if my intermittently annoyingly connectivity problem is still annoyingly intermittent...?