15 Tips for Troubleshooting VPN Connections

You can construct a VPN in a myriad of ways, but constructing your VPN is just your first step. After you construct the VPN, you have to troubleshoot it.

A minimal VPN implementation has a RAS PPTP server connected to the Internet, a client connected to the Internet, and a PPTP connection between the server and the client. As long as ISP service or Internet connectivity is available, clients can connect to your server or LAN from anywhere in the world. However, most VPNs aren't as simple as a connected server and client. More often, the VPN server is on a routed LAN segment, often behind a firewall, and the client connection uses an ISP network, which also employs routers and firewalls.

You can build a PPTP server as a standalone server or as a domain controller in a couple of steps: You install RAS and the PPTP protocol and configure PPTP ports the same way you configure dial-up connections. Windows NT client setup is equally straightforward: You load PPTP and configure the PPTP connection to locate the PPTP server over the Internet. With such a simple setup, you might assume that the VPN connection will function properly the first time. However, administrators spend a fair amount of time troubleshooting before they successfully deploy a new VPN.

Troubleshooting a VPN, like troubleshooting any WAN connectivity problem, is complex because the data travels through many links before it arrives at its destination. For example, data typically flows from the client to an ISP's router, through a firewall, across the ISP's network, maybe across additional ISPs' networks, to the company's router, to a firewall or proxy server, and finally to the destination PPTP server.

When a client connects to an ISP (this connection uses the Point-to-Point Protocol—PPP—portion of the VPN connection), the ISP assigns the client a TCP/IP address, a DNS server address, and a default gateway. When the client initiates a PPTP connection, that action creates a second TCP/IP session (this session is the tunnel portion of the VPN connection), which is embedded inside the first session and provides packet encryption and encapsulation. When the client connects successfully, the VPN server assigns the client a second IP address, a second DNS server address, frequently a WINS server, and another default gateway. At each link in the connection, something can go wrong. Knowing the common configuration and connectivity problems and having a troubleshooting procedure to follow will help you debug your VPN connections.

VPN Server Recommendations
If possible, start with an NT server that has a minimum number of services installed and limit the protocols to TCP/IP and PPTP. You'll save time if you update your server with service packs before you try to debug client connections. NT 4.0 Service Packs 5 (SP5) and SP6a correct numerous problems with PPTP connections, including performance problems related to fragmented packets, dropped connections, and refused connections. I have four more recommendations to help you keep the server configuration simple and straightforward for troubleshooting purposes.

Configuring a multihomed server. If your PPTP server has two network cards, one for the LAN and one for the WAN, leave the gateway field on the LAN adapter blank (don't enter zeros; leave it blank). In the gateway field of the WAN network interface, enter the TCP/IP address that your ISP provides; the gateway address usually points to a router at your ISP. You need the blank gateway so that the server can route network packets to the client. Leaving the LAN gateway blank is standard practice when you configure a server with multiple network adapters. For test purposes, I recommend you manually enter the TCP/IP address and WINS server address for the LAN NIC (instead of using DHCP to assign these values).

Configuring RAS. When you install RAS, configure only as many VPN ports as you truly need to support active client connections. Although each RAS server can support 256 concurrent connections (assuming you have the bandwidth for all this activity), you might need only 40 concurrent connections to support your mobile users. Next, configure the server to assign client addresses from a static address pool, rather than assigning addresses from a DHCP server. If you configure RAS to assign client addresses from a static address pool, clients inherit the DNS and WINS settings from the RAS server. If your RAS server can browse the network, clients should also be able to browse the network with the same settings.

If you prefer DHCP, verify that DHCP scope option 44 (WINS/NetBIOS name server) points to the WINS server and that scope option 6 shows the address of your DNS server. When you don't define these options, you almost guarantee problems with client browsing.

Enabling PPTP filtering. Configuring and testing a VPN server that resides outside your firewall is easier than testing a server inside your firewall because avoiding the firewall removes one link in the test-and-debug chain. If you aren't running your server in a highly secure environment, you can place the server outside the firewall and restrict incoming VPN traffic to PPTP packets only. To enable PPTP filtering, right-click Network Neighborhood, select Properties and Protocols, double-click TCP/IP Protocols, and select the WAN adapter and Advanced. Then, select the Enable PPTP Filtering check box. When you enable PPTP filtering, the server will refuse all non-PPTP requests. I've tested this feature, and it's an effective method for restricting incoming sessions to PPTP-only connections. PPTP filtering has one important side effect: When you enable filtering, LAN clients can't use the RAS server's WAN connection to browse the Internet because filtering disables incoming HTTP and FTP traffic.

If you want the VPN server to restrict incoming packets to PPTP and host an Internet-accessible Web site, you can make a Registry modification that lets other packets through the filtered interface to the local system only. Go to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RASPPTPF\ Parameters Registry key, and add the value entry AllowPacketsForLocalMachine of REG_DWORD data type 1. When you make this modification, you expose the RAS server to the Internet yet restrict incoming connections to the VPN server, so remote clients can't see any other resources on your network.

Using firewall ports. Before you place a VPN server behind a firewall, verify that your firewall software accepts PPTP packets. Sometimes firewall software packages (including some versions of Check Point Software Technologies' FireWall-1) don't accept PPTP connections when you configure the firewall with Network Address Translation (NAT—for information about NAT, see Zubair Ahmad, "Windows 2000's Network Address Translation," February 2000). In this situation, the client's attempt to connect to the RAS server produces the error message Event ID 721 PPP remote peer not responding. When you place a VPN server behind your firewall, be sure to enable IP protocol 47 (Generic Routing Encapsulation—GRE) and TCP port 1723. The connection uses port 1723 for general housekeeping, such as PPTP tunnel creation, maintenance, and termination. Port 47 passes tunneled data between the client and the server (including the GRE protocol), and you also need TCP port 1723 [established] if you're supporting RAS server-to-server VPN connections.

Before you try to connect a VPN client, verify the server's TCP/IP settings on both NICs and make sure your RAS server can perform all typical network operations (e.g., browse the LAN, connect to LAN resources, connect to the Internet, browse the Internet). Then, enable dial-up permission for your test account. You might also want to enable PPP logging for your initial test.

Client Troubleshooting
To operate successfully, a PPTP client must properly maintain two sets of TCP/IP stack settings: one for the ISP and Internet connection and one for the VPN server connection. The client's routing table must also have two entries: one that directs network packets to the ISP for Internet browsing and one that points to the VPN server interface for LAN browsing. When the stack settings are incorrect, clients experience problems. In general, NT clients maintain separate TCP/IP stack settings, but Windows 95 clients typically have trouble with stack settings when the clients have a network card and a modem. After establishing a PPTP connection, the Win9x default gateway might still point to the ISP, which prevents the client from successfully browsing the LAN. Let's take a look at the five most common client connectivity problems.

Client can't connect to the PPTP server. The first problem you might encounter is the client's inability to connect to the PPTP server. You need to check for three possible causes of this problem.

1. Establishing VPN server Internet connectivity. After you configure the client, you need to verify that the VPN server has a connection to the Internet. The easiest way to verify a connection is to use the server's TCP/IP address and ping the server from the client. (If your PPTP server is behind a firewall and the firewall blocks Internet Control Message Protocol—ICMP—ping messages, this technique won't work.) If the ping gives you the message request timed out, something is amiss with the server's Internet connection. If the server responds by address, you can enter the TCP/IP address in the phone number field of the DUN entry to establish the PPTP session. Although it's less friendly than a Fully Qualified Domain Name (FQDN), this technique works fine when you know the server's address.
   
   Remember that a server with a dial-up connection is likely to get a different address each time the server connects to the ISP. To connect by address, you must know the address the ISP has assigned to the server each time the server makes a dial-up connection. More commonly, your RAS server will have a permanent address, which eliminates one small variable in the connection process.
   
   If the server responds by address, ping it by name. If the server doesn't respond by name, one of two situations is likely: The server might not have a registered domain name, or your ISP DNS server might be down or not working properly.
   
   
2. Checking PPTP filtering. With PPTP filtering enabled on the server, you might see the message Error 678: There is no answer, or Error 650: The Remote Access Server is not responding. Disable PPTP filtering on the server (Net Stop RASPPTPF), and see whether you can establish a nonfiltered connection.
   
   If you can connect with filtering disabled, check the server's filter settings. If you disable UDP ports 137 and 138 or TCP port 139, NetBIOS packets can't pass through the network. You also need to enable these ports on all firewalls and routers that are between the client and the server for unicast (point-to-point) traffic.
   
   
3. Filtering the GRE protocol. If the server responds by address and name but you still can't connect, your ISP's routers or internal routers or firewalls might be filtering GRE packets. To establish a PPTP tunnel, the client and server exchange GRE packets, and some ISPs disable external GRE packets because the ISPs use GRE internally to manage routers. Although GRE filtering is uncommon, it will prevent a PPTP connection, so make sure you have IP protocol 47 (GRE) and TCP port 1723 enabled at both ends of the VPN connection. You can identify GRE filtering with the Microsoft Network Monitor or similar network sniffer tool. For more information about monitoring PPTP packets during a VPN connection, see "Related Articles in Previous Issues."

   Previous  [1]  2  Next