IPsec: RFCs and How-To

Maintained by Tina Bird

Last modified: 15-Jan-2005 10:29

Thanks to VPN list members Atish Charan, Pete Davis, Guy Raymakers, Jon Carnes, David Klann, Dave Klein, Patrick Ethier, T.C. Wolsey, Jonas Eriksson, Fearghas McKay and Trevor Paquette for their contributions.

All information is provided as a service to the reader. Your results may vary. If you need further assistance, please contact your technical support organization or request help on the VPN Mailing List.

The information below is classified loosely by the operating system of the VPN gateway (the device controlling access into the private network).

IPsec RFCs

Internet Requests for Comments (RFCs) and drafts:

NOTE: Internet Drafts expire regularly, which means that the links below frequently go dead and it takes me a while to notice (usually when someone on the VPN mailing list complains). All drafts created within the IETF IPsec working group can be found at

http://www.ietf.org/ids.by.wg/ipsec.html

so check there if you're looking for something that's either broken here, or not listed at all.

Internet Security Protocol Working Group

• Security Architecture for Internet Protocol
• A Framework for IP Based Virtual Private Networks
• IP Authentication Header
• Use of HMAC-MD5-96 Within ESP and AH
• Use of HMAC-SHA-96 Within ESP and AH
• ESP DES-CBC Cipher Algorithm with Explicit IV
• IP Encapsulating Security Payload (obsoletes RFC 1827)
• Internet IP Security Domain of Interpretation for ISAKMP
• Internet Security Association and Key Management Protocol
• Internet Key Exchange
• The NULL Encryption Algorithm and Its Use with IPsec
• IP Security Document Roadmap
• The OAKLEY Key Determination Protocol
• IPsec-NAT Compatibility Requirements
• Negotiation of NAT Traversal in IKE
• Extended Authentication within IKE (XAUTH)
• UDP Encapsulation of IPsec Packets

IPsec Remote Access Working Group Working group focussed on the differences between IPsec implemented between gateway systems, or as a drop-in replacement for IPv4 on a local network, and IPsec used as a client-to-server remote access system.

A method for doing opportunistic encryption with IKE

Authentication Configuration Issues Affecting IPsec VPNs Discussion focusses on Cisco's XAUTH implementation, but very useful discussion even for non-Cisco VPN/user authentication issues.

Understanding the IPSec Protocol Suite

Pushing IPsec through NAT

NAT Traversal: Peace Agreement Between NAT and IPsec

Virtual Private Network Consortium

• VPNC interoperability requirements
• VPNC Members' supported features

IPsec Interoperability Tests conducted by the ICSA

• ICSA Labs interoperability requirements
• ICSA Labs certified products
• IPsec Product Consortium Meeting Summaries

IPsec 2001 Interop Demo Herve Schauer's consulting group conducts an IPsec/IKE demo for attendees of the IPsec 2001 conference.

National Institute of Standards and Technology, which includes reference implementations of IPSec and ISAKMP and interoperability test information.

IPsec Web Based Interoperability Tester

Authentication Configuration Issues Affecting IPsec VPN Security

An Overview of Secure Multicast

Secure Multicast -- discusses issues with IPsec key exchange, security association negotiations, and multicast protocols

A Cryptographic Evaluation of IPsec by Ferguson & Schneier. The title is misleading, because they don't really study IPsec crypto -- after all, that's all based on peer reviewed open source algorithms. The authors consider the requirements for deploying IPsec, and conclude that complexity is the biggest barrier to secure IPsec implementation

IPsec Books (recommended by mailing list readers)

Doraswamy, Naganand & Harkins, Dan. IPsec: The New Security Standard for the Internet, Intranets and Virtual Private Networks. Prentice Hall, July 1999. ISBN 0130118982

Frankl, Sheila. Demystifying the IPsec Puzzle. Artech House Publications, April 2001. ISBN 1-58053-079-6 (available as hardcover book or ebook)

Loshin, Peter, compiler. The Big Book of IPsec RFCs. Morgan Kaufman. ASIN 0124558399

Tiller, James. A Technical Guide to IPsec Virtual Private Networks. Auerbach Publications, December 2000. ISBN 0849308763

IPsec How-To

IPsec through commercial firewalls:

Sidewinder, Cisco, PIX or Gauntlet

IPsec on Cisco (IOS & PIX):

Tons and tons of Cisco configuration examples for IPsec, including Cisco-only VPNs, and VPNs between Cisco and a wide variety of other implementations

IPSec between Raptor Firewall (v5.x or 6.x) and Cisco IOS 12.0

Cisco PIX Firewall and VPN Configuration Guide v6.2

VPN Configurations for Cisco IOS

Cisco VPN Top Issues (available to the public): http://www.cisco.com/warp/public/471/top_issues/vpn/vpn_index.shtml

IPsec on FW-1:

VPN Documentation for Checkpoint FireWall-1

Heaps and heaps of FW-1 VPN documentation hosted by the ineffable PhoneBoy

IPSec between Checkpoint FireWall-1 v4.0 and Network Associates' PGPNet v.6.5.1

Using Linux as an IPsec VPN client to FireWall-1

How to configure a FireWall-1 IKE VPN with Nortel Contivity

IPsec on Linux -- FreeS/WAN and others:

FreeS/WAN

Introduction to FreeS/WAN

FreeS/WAN configurations for remote access VPN and NAT situations

FreeS/WAN and PGPNet with x.509 certificates HOWTO

Hardware Acceleration for Linux-based IPsec

How to set up IPsec interoperable for Linux, OpenBSD and PGPNet

Interoperating with FreeS/WAN

Linux VPN Masquerading, which allows you to use IPsec and PPTP from behind a many-to-one address translating firewall.

NIST Cerberus, an IPsec Reference Implementation for Linux

The Linux VPN Masquerade HOWTO document, by John Hardin

IPsec: secure IP over the Internet

IPsec on Macintoshes:

Flying Raccoons: IPsec, OS X Server 10.2 and you

Flying Raccoons: Host to host, coast to coast

Flying Raccoons: Clients? We don't need no stinking clients

Flying Raccoons: Networks that work

VPN Tracker -- an OS X IPsec client

• Interoperability test results and documentation

VaporSec -- a rather unfortunately named graphical interface to OS X IPsec functionality

Checkpoint's VPN-1 client for Mac OS 8 and 9 -- no info on whether this is interoperable with non-FW-1 gateways

Cisco VPN 5000 client -- Mac 7.6-9x

IPsec on OpenBSD & FreeBSD:

OpenBSD Free, open source operating system with integrated cryptographic functions, and IPSec

FreeBSD as a Remote Access Server for Win2k and other IPsec road warriors

ISAKMP and IPsec in the VPN Environment

IPsec Configurations for OpenBSD, FreeS/WAN and PGPFreeware

IPsec clients with OpenBSD gateways

OpenBSD VPN Configuration Mini-FAQ More details on configuring OpenBSD as an IPSec VPN server.

Setting up a basic VPN between two OpenBSD gateways using ISAKMP

IPsec on Solaris:

Configuring IPsec/IKE on Solaris, by Ido Dubrawsky

IPsec on Sonicwall:

SonicWALL VPN Documentation

SonicWALL to FW-1 Interoperability Tech Note

SonicWALL to Raptor Interoperability Tech Note

IPsec on Symantec Enterprise Firewall (Raptor)

How to configure a VPN tunnel between a Raptor or Symantec Enterprise Firewall and SonicWALL VPN

How to configure a site-to-site tunnel between a Symantec Enterprise Firewall or Symantec Enterprise VPN Server and Microsoft ISA Server

How to connect a VPN tunnel between a Symantec Firewall VPN appliance and a Symantec Enterprise Firewall

How to set up site-to-site VPN tunnel between Symantec Enterprise Firewall and Cisco Pix

IPsec on Windows:

Microsoft's Virtual Private Networking: An Overview

An overview of IPsec on Windows 2000

Basic IPsec Troubleshooting in Windows 2000

Step-by-Step Guide to IPsec on Windows 2000

Building a Microsoft VPN: A Comprehensive Collection of Microsoft Resources

Microsoft L2TP/IPsec VPN Client Microsoft back- ported its Win2k/XP support for Layer 2 Tunnelling Protocol and IPsec to Win98, WinME, and WinNT -- download the client and documentation here.

How to turn off Path MTU Discovery on WindowsNT -- required for use with IPsec in some cases.

IPsec Troubleshooting:

VPN Discovery and Fingerprinting Technique

ICSA Labs IPSec Technical Product Configuration Guidelines A distillation of lessons learned during the course of ICSA's interoperability testing, with a useful checklist of things to check when establishing a multi-vendor IPsec environment.

Basic L2TP/IPsec Troubleshooting in Windows Specific techniques are Windows-specific, but the document contains a pretty good methodology for tracking down VPN problems.

Troubleshooting VPN Problems

Flying Raccoons: Networks that don't work Mac OS X focused, but techniques are generally useful in non-Mac environments too.

Miscellaneous Documentation:

Netscreen Concept & Examples: Screen OS Reference Guide: VPNs

IPsec from a Novell BorderManager to a Third-Party VPN Server

Setting up IPsec policy on an IBM AS/400

IPSec

IPSec traffic consists of three components. UDP/500 is used for ISAKMP key negotiations. IP protocol 50 carries Encapsulating Security Payload traffic, and IP protocol 51 carries the Authentication Header. The following instructions provide details on setting up filters on your firewall to allow these protocols to pass. We assume that the IPSec server is hosted on the internal (private) network, and that address redirection is used to transfer publicly-routable Internet traffic to the internal system.

On a Sidewinder:

• Create and enable a generic proxy for the ISAKMP traffic:
  cf udpproxy create burb=external service=isakmp ports=500-500
  cf udpproxy enable burb=internal service=isakmp
  cf udpproxy enable burb=external service=isakmp
  


• Set up access control list entries permitting ISAKMP traffic to be transmitted and received by the server on the internal network:
  cf acl add name=isakmp_int_ext action=allow agent=proxy \
  sourceburb=internal destburb=external service=isakmp
  cf acl add name=isakmp_ext_ext action=allow agent=proxy \
  sourceburb=external destburb=external redir=ipaddr:10.1.1.5 \
  service=isakmp
  
• Configure packet filters to allow AH and ESP traffic to server:
  Modify the file /etc/sidewinder/ipfilter.conf by adding lines which permit both components of IPSec traffic:
  allow(0.0.0.0 0 external 10.1.1.5 32 internal 50 3 10000)
  allow(0.0.0.0 0 external 10.1.1.5 32 internal 51 3 10000)
  
  The 0.0.0.0 octet designates all possible source addresses as allowable for IPSec connections. If your IPSec remote systems have static IPs, you can improve security by changing these rules to limit traffic to just those sources, by creating a netgroup which contains each of them.
  


• Add another set of rules to /etc/sidewinder/ipfilter.conf to manage the redirection of traffic from the external Sidewinder interface to the internal IPSec server:
  translate(ext_addr int_addr 50 3)
  translate(ext_addr int_addr 51 3)
  


• Configure the IP filter to pass the UDP component correctly, using the command line interface:
  cf ipfilter add allow_udp end_port_B=500 burb_A=0 burb_B=1 \
  addr_B=0.0.0.0 bits_B=0 bits_A=10.1.1.5 threshold=10000 \
  beg_port_B=500 beg_port_A=500 direction=2 end_port_A=500 \
  nat_addr=197.17.1.1 nat=1 compatibility_mode=0



• Finally, load and verify the IP filter database:
  ipfilter -l
  ipfilter -v
  

On a Gauntlet:

Packet Filter Rule Editor - Forward ruleset
Filter Rules:
#perm iface prot srcport srcaddr:srcmask dstport dstaddr:dstmask
--------------------------------------------------------------------
#IPSec negotiation outbound

1 permit inside udp 500 srcaddr:srcmask 500 0.0.0.0:0.0.0.0
#IPSec negotiation inbound

2 permit outside udp 500 0.0.0.0:0.0.0.0 500 dstaddr:dstmask
#IP
3 permit inside 50 * srcaddr:srcmask * 0.0.0.0:0.0.0.0
4 permit inside 51 * srcaddr:srcmask * 0.0.0.0:0.0.0.0

#IP inbound
5 permit outside 50 * 0.0.0.0:0.0.0.0 * dstaddr:dstmask
6 permit outside 50 * 0.0.0.0:0.0.0.0 * dstaddr:dstmask

On a Cisco:

ip access-list extended inet_inbound #define access-list
deny ip 10.0.0.0 0.255.255.255 any
deny ip 172.16.0.0 0.15.255.255 any
deny ip 192.168.0.0 0.0.255.255 any # reject RFC1918 addresses
permit udp any host xxx.xxx.xxx.xxx eq 500 #allow IPSec IKE negotiations
permit esp any host xxx.xxx.xxx.xxx #allow IPSec ESP protocol
permit ahp any host xxx.xxx.xxx.xxx #allow IPSec AH protocol

On some IOS versions, you may have to use an alternative syntax:
permit 50 any host xxx.xxx.xxx.xxx #allow IPSec ESP protocol
permit 51 any host xxx.xxx.xxx.xxx #allow IPSec AH protocol

If you have a pre-11.2 IOS that does not support named IP access lists, you need to precede each access-list line with the access-list xxx command (where xxx=101-199), ie:
access-list 101 permit udp any host xxx.xxx.xxx.xxx eq 500
access-list 101 permit 50 any host xxx.xxx.xxx.xxx

Now put the access-list inbound on the interface where the traffic enters the router:
interface Serial0/0
description Internet interface
ip address zzz.zzz.zzz.zzz 255.255.255.252
ip access-group inet_inbound in #this applies access list to interface

On a PIX:

! These statements apply to current PIX versions (4.X(Y) or later).

! Following examples use 10.1.1.20 for the inside, private host,
! and 200.254.254.254 for the public, translated address for the same host.

!IPSec
static 10.1.1.20 200.254.254.254
conduit permit udp any eq 500 host 200.254.254.254
conduit permit 50 host 200.254.254.254 any
conduit permit 51 host 200.254.254.254 any

On a BorderManager:

If the third-party VPN is running another vendor's VPN software, you must create a new SINFO.VPN file and complete the procedure for adding a server to a VPN, as described in Novell BorderManager Enterprise Edition 3.5 Installation and Setup.

Create a new SINFO.VPN file with the following fields:

Major version number-Should always be set to 1.
Minor version number-Should always be set to 5.
Server name-Arbitrary name assigned to the third-party server. You can pick any name for convenience.
Master or slave ID-Should always be set to 1.
Public IP address-Public IP address of the third-party server.
Public IP address mask-Public IP address mask of the third-party server.
Private IP address-Not used. Should always be set to 0.0.0.0.
Private IP address mask-Not used. Should always be set to 0.0.0.0.
Private IP address mask-Not used. Should always be set to 0.0.0.0.
Tunnel IP address-IP address of the VPN tunnel that you want to assign to the third-party server. The address must belong to the same IP network as the local VPN's tunnel address.

Tunnel mask-Should be set to match the mask of your local VPN tunnel.
Public value length-Length of the third-party server's Diffie-Hellman public value, in bytes.
Public value in BER-Third-party server's Diffie-Hellman public value, in BER format. 1024-bit values are supported.
Security capabilities-Decimal equivalent of a 32-bit binary integer you must compute using the following bit values:

• Bit 0-If you are using the export version of the VPN software, set this bit to 1. Otherwise set it to 0.
• Bit 1-If Keyed_MD5 is supported, set this bit to 1. Otherwise, set it to 0.
• Bit 2-If Keyed_SHA1 is supported, set this bit to 1. Otherwise, set it to 0.
• Bit 3-If HMAC_MD5 is supported, set this bit to 1. Otherwise, set it to 0.
• Bit 4-If HAC_SHA1 is supported, set this bit to 1. Otherwise, set it to 0.
• Bit 5 to bit 15-Set these bits to 0.
• Bit 30-Set this bit to 1.

Disabling Path MTU Discovery on a Windows NT system:

• Boot the machine and log in as a user with local administrative rights.
• Open a command window in your usual fashion (by selecting the Start Menu, Run, and typing "command," or by selecting "Command Prompt" from the Programs option under the Start Menu).
• Run the application regedit32, which is located in the WINNT folder.
  If you are not experienced with using the Registry Editor on Windows, please test this fix on a non-production system first!
  The registry controls the behavior of your Windows system. You can break things really badly if you're not careful.
• Select the HKEY_LOCAL_MACHINE window.
• Expand "SYSTEM".
• Expand "CurrentControlSet".
• Expand "Services".
• Expand "tcpip".
• Select "parameters" and make sure it is highlighted.
• From the "Edit" menu, select "Add Value..."
• For the "Value Name," type "EnablePMTUDiscovery" (it's case sensitive).
• On the "Data Type" drop-down menu, select "REG_DWORD" and click OK.
• In the "Data" input area, type "0" (the digit zero).
• It shouldn't matter which radix you choose, but we select Decimal out of paranoia.
• Click OK.
• From the "Registry" menu, select "Exit".
• Reboot the machine.

Cisco IOS VPN configurations

http://www.cisco.com/warp/public/707/16.html provides a great introduction to configuring and troubleshooting Cisco-based IPsec.

If you're a registered CCO user, you can start at http://www.cisco.com/warp/customer/105/IPSECpart1.html. (Registration information is available there.)

Other useful Cisco links
http://www.cisco.com/univercd/cc/td/doc/cisintwk/ito_doc/vpn.htm
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/12cgcr/secur_c/scprt4/index.htm -- Cisco's IPSec documentation.

IPSec LAN-to-LAN with preshared keys (from Pete Davis):

!#The following section contains the parameters required for IKE Negotiation
!
crypto isakmp policy 20
encr 3des
hash md5
authentication pre-share
group 2
lifetime 86000
!
!
#This line of configuration is used to set the pre-shared key with the IKE peer
crypto isakmp key SHAREDKEYHERE address peer.IP.aDdRess
!
!
#This line sets up the encryption and authentication algorithms to use for IPSec
crypto ipsec transform-set set-NaME esp-3des esp-md5-hmac
!
!
#This section of the configuration sets up information required for the IPSec SA
crypto map cryptomapname 10 ipsec-isakmp
set peer pEeR.Ip.AdDreSs
set security-association lifetime seconds 28800
set transform-set set-NaME
set pfs group1
match address 100
!
!
interface FastEthernet0/0
crypto map cryptomapname
!(this required to turn on IKE/IPSec on this interface)
!
access-list 100 permit ip sOurCe.Net WildCardMask Destination Net WildCardMask
!(packets between these networks will have IPSec applied to them)

IPSec with pre-shared keys and NAT (from Cisco's web site, contributed by Guy Raymakers):
!
version 12.0
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname C1605-REMOTE
!
enable password 7 481951588478970D
!
!
!
!
!
ip subnet-zero
no ip domain-lookup
!
isdn switch-type basic-net3
!
!
crypto isakmp policy 1
hash md5
authentication pre-share
crypto isakmp key eds address 197.71.25.58
!
!
crypto ipsec transform-set cm-transformset-1 ah-md5-hmac esp-md5-hmac
!
!
crypto map cm-cryptomap local-address Dialer1
crypto map cm-cryptomap 1 ipsec-isakmp
set peer 197.71.25.58
set transform-set cm-transformset-1
match address 100
!
!
process-max-time 200
!
interface Ethernet0
ip address 199.227.10.1 255.255.255.0
no ip directed-broadcast
!
interface Ethernet1
description connected to LAN
ip address 204.173.190.1 255.255.255.0
no ip directed-broadcast
ip nat inside
!
interface BRI0
description connected to Internet
no ip address
no ip directed-broadcast
ip nat outside
encapsulation ppp
dialer rotary-group 1
isdn switch-type basic-net3
crypto map cm-cryptomap
!
interface Dialer1
description connected to Internet
ip address 198.132.229.216 255.255.255.0
no ip directed-broadcast
ip nat outside
encapsulation ppp
no ip route-cache
no ip split-horizon
no ip mroute-cache
dialer in-band
dialer string 034141850
dialer hold-queue 10
dialer-group 1
no cdp enable
ppp authentication chap pap callin
ppp chap hostname jhsjdhuh
ppp chap password 7 18881777377733245E
ppp pap sent-username jhsjdhuh password 7 18881777377733245F
crypto map cm-cryptomap
!
router rip
version 2
timers basic 5 10 15 30
passive-interface Dialer1
network 199.227.10.0
network 204.173.190.0
distribute-list 10 out Ethernet0
no auto-summary
!
ip nat inside source list 101 interface Dialer1 overload
ip classless
ip route 0.0.0.0 0.0.0.0 Dialer1
no ip http server
!
access-list 10 deny any
access-list 100 permit ip host 194.7.229.216 host 194.7.250.58
access-list 100 permit ip host 194.7.229.216 206.165.25.0 0.0.0.255
access list 100 permit ip 204.173.190.0 0.0.0.255 206.165.25.0 0.0.0.255
access-list 101 deny ip 204.173.190.0 0.0.0.255 206.165.25.0 0.0.0.255
access-list 101 permit ip 204.173.190.0 0.0.0.255 any
dialer-list 1 protocol ip permit
snmp-server engineID local 000000009020000507305B450
snmp-server community public RO
!
line con 0
exec-timeout 0 0
password 7 1041071E0A1E1C0C
login
transport input none
line vty 0 4
password 7 050408082E45400E
login
!
end