For the last couple of days I have been battling with a network mystery where by user traffic to MS TFS (Team Foundation Server) over the ISA 2006 VPN SITE-TO-SITE was denying access to TFS. Users on the internal network or using the VPN client were unaffected. Nothing had changed on the servers or clients.
The Network Topology
TFS Client -> Site2 ISA 2006 SP1 -. IPSEC SITE-TO-SITE –Site1 ISA 2006 -> TFS Server
The error being raised in VS.Net was Team Explorer could not connect to the Team Foundation Server xxxxxx
No error on Site 1 ISA
Error on Site 2 ISA was FWX_E_TCP_NOT_SYN_PACKET_DROPPED to the TFS Server
Network monitors would show a request to the TFS Server but no responses, the TFS Client would show several retransmissions
After a lot of work we discovered that the TFS Server was running windows 2003 SP2, when SP2 is installed it will enable TCP/IP Offload (RSS) for improved network performance, however this improvement can cause it own problems.
After running through http://support.microsoft.com/kb/948496/ to disable the TCP/IP Offload, then rebooting the server the traffic to TFS then started to work.
TCP/IP offload has also been removed from the ISA Servers.
If you experience networking connectivity issues to windows servers then disabling TCP/IP is a good place to start.
Thanks to Andreas on the Microsoft ISA Server Premier Support Team, without him we would have never tried KB948496.
The following issues may occur when Windows Server 2003 SNP is turned on:
- When you try to connect to the server by using a VPN connection, you receive the following error message:
Error 800: Unable to establish connection.
- You cannot create a Remote Desktop Protocol (RDP) connection to the server.
- You cannot connect to shares on the server from a computer on the local area network.
- You cannot join a client computer to the domain.
- You cannot connect to the Exchange server from a computer that is running Microsoft Outlook.
- Inactive Outlook connections to the Exchange server may not be cleaned up.
- You experience slow network performance.
- You may experience slow network performance when you communicate with a Windows Vista-based computer.
- You cannot create an outgoing FTP connection from the server.
- The Dynamic Host Configuration Protocol (DHCP) server service crashes.
- You experience slow performance when you log on to the domain.
- Network Address Translation (NAT) clients that are located behind Windows Small Business Server 2003 or Internet Security and Acceleration (ISA) Server experience intermittent connection failures.
- You experience intermittent RPC communications failures.
- The server stops responding.
- The server runs low on nonpaged pool memory