IPsec: RFCs and How-To

Maintained by Tina Bird

Last modified: 15-Jan-2005 10:29

Thanks to VPN list members Atish Charan, Pete Davis, Guy Raymakers, Jon Carnes, David Klann, Dave Klein, Patrick Ethier, T.C. Wolsey, Jonas Eriksson, Fearghas McKay and Trevor Paquette for their contributions.

All information is provided as a service to the reader. Your results may vary. If you need further assistance, please contact your technical support organization or request help on the VPN Mailing List.

The information below is classified loosely by the operating system of the VPN gateway (the device controlling access into the private network).

IPsec RFCs

Internet Requests for Comments (RFCs) and drafts:

NOTE: Internet Drafts expire regularly, which means that the links below frequently go dead and it takes me a while to notice (usually when someone on the VPN mailing list complains). All drafts created within the IETF IPsec working group can be found at


so check there if you're looking for something that's either broken here, or not listed at all.

Internet Security Protocol Working Group

IPsec Remote Access Working Group Working group focussed on the differences between IPsec implemented between gateway systems, or as a drop-in replacement for IPv4 on a local network, and IPsec used as a client-to-server remote access system.

A method for doing opportunistic encryption with IKE

Authentication Configuration Issues Affecting IPsec VPNs Discussion focusses on Cisco's XAUTH implementation, but very useful discussion even for non-Cisco VPN/user authentication issues.

Understanding the IPSec Protocol Suite

Pushing IPsec through NAT

NAT Traversal: Peace Agreement Between NAT and IPsec

Virtual Private Network Consortium

IPsec Interoperability Tests conducted by the ICSA

IPsec 2001 Interop Demo Herve Schauer's consulting group conducts an IPsec/IKE demo for attendees of the IPsec 2001 conference.

National Institute of Standards and Technology, which includes reference implementations of IPSec and ISAKMP and interoperability test information.

IPsec Web Based Interoperability Tester

Authentication Configuration Issues Affecting IPsec VPN Security

An Overview of Secure Multicast

Secure Multicast -- discusses issues with IPsec key exchange, security association negotiations, and multicast protocols

A Cryptographic Evaluation of IPsec by Ferguson & Schneier. The title is misleading, because they don't really study IPsec crypto -- after all, that's all based on peer reviewed open source algorithms. The authors consider the requirements for deploying IPsec, and conclude that complexity is the biggest barrier to secure IPsec implementation

IPsec Books (recommended by mailing list readers)

Doraswamy, Naganand & Harkins, Dan. IPsec: The New Security Standard for the Internet, Intranets and Virtual Private Networks. Prentice Hall, July 1999. ISBN 0130118982

Frankl, Sheila. Demystifying the IPsec Puzzle. Artech House Publications, April 2001. ISBN 1-58053-079-6 (available as hardcover book or ebook)

Loshin, Peter, compiler. The Big Book of IPsec RFCs. Morgan Kaufman. ASIN 0124558399

Tiller, James. A Technical Guide to IPsec Virtual Private Networks. Auerbach Publications, December 2000. ISBN 0849308763

IPsec How-To

IPsec through commercial firewalls:

Sidewinder, Cisco, PIX or Gauntlet

IPsec on Cisco (IOS & PIX):

Tons and tons of Cisco configuration examples for IPsec, including Cisco-only VPNs, and VPNs between Cisco and a wide variety of other implementations

IPSec between Raptor Firewall (v5.x or 6.x) and Cisco IOS 12.0

Cisco PIX Firewall and VPN Configuration Guide v6.2

VPN Configurations for Cisco IOS

Cisco VPN Top Issues (available to the public): http://www.cisco.com/warp/public/471/top_issues/vpn/vpn_index.shtml

IPsec on FW-1:

VPN Documentation for Checkpoint FireWall-1

Heaps and heaps of FW-1 VPN documentation hosted by the ineffable PhoneBoy

IPSec between Checkpoint FireWall-1 v4.0 and Network Associates' PGPNet v.6.5.1

Using Linux as an IPsec VPN client to FireWall-1

How to configure a FireWall-1 IKE VPN with Nortel Contivity

IPsec on Linux -- FreeS/WAN and others:


Introduction to FreeS/WAN

FreeS/WAN configurations for remote access VPN and NAT situations

FreeS/WAN and PGPNet with x.509 certificates HOWTO

Hardware Acceleration for Linux-based IPsec

How to set up IPsec interoperable for Linux, OpenBSD and PGPNet

Interoperating with FreeS/WAN

Linux VPN Masquerading, which allows you to use IPsec and PPTP from behind a many-to-one address translating firewall.

NIST Cerberus, an IPsec Reference Implementation for Linux

The Linux VPN Masquerade HOWTO document, by John Hardin

IPsec: secure IP over the Internet

IPsec on Macintoshes:

Flying Raccoons: IPsec, OS X Server 10.2 and you

Flying Raccoons: Host to host, coast to coast

Flying Raccoons: Clients? We don't need no stinking clients

Flying Raccoons: Networks that work

VPN Tracker -- an OS X IPsec client

VaporSec -- a rather unfortunately named graphical interface to OS X IPsec functionality

Checkpoint's VPN-1 client for Mac OS 8 and 9 -- no info on whether this is interoperable with non-FW-1 gateways

Cisco VPN 5000 client -- Mac 7.6-9x

IPsec on OpenBSD & FreeBSD:

OpenBSD Free, open source operating system with integrated cryptographic functions, and IPSec

FreeBSD as a Remote Access Server for Win2k and other IPsec road warriors

ISAKMP and IPsec in the VPN Environment

IPsec Configurations for OpenBSD, FreeS/WAN and PGPFreeware

IPsec clients with OpenBSD gateways

OpenBSD VPN Configuration Mini-FAQ More details on configuring OpenBSD as an IPSec VPN server.

Setting up a basic VPN between two OpenBSD gateways using ISAKMP

IPsec on Solaris:

Configuring IPsec/IKE on Solaris, by Ido Dubrawsky

IPsec on Sonicwall:

SonicWALL VPN Documentation

SonicWALL to FW-1 Interoperability Tech Note

SonicWALL to Raptor Interoperability Tech Note

IPsec on Symantec Enterprise Firewall (Raptor)

How to configure a VPN tunnel between a Raptor or Symantec Enterprise Firewall and SonicWALL VPN

How to configure a site-to-site tunnel between a Symantec Enterprise Firewall or Symantec Enterprise VPN Server and Microsoft ISA Server

How to connect a VPN tunnel between a Symantec Firewall VPN appliance and a Symantec Enterprise Firewall

How to set up site-to-site VPN tunnel between Symantec Enterprise Firewall and Cisco Pix

IPsec on Windows:

Microsoft's Virtual Private Networking: An Overview

An overview of IPsec on Windows 2000

Basic IPsec Troubleshooting in Windows 2000

Step-by-Step Guide to IPsec on Windows 2000

Building a Microsoft VPN: A Comprehensive Collection of Microsoft Resources

Microsoft L2TP/IPsec VPN Client Microsoft back- ported its Win2k/XP support for Layer 2 Tunnelling Protocol and IPsec to Win98, WinME, and WinNT -- download the client and documentation here.

How to turn off Path MTU Discovery on WindowsNT -- required for use with IPsec in some cases.

IPsec Troubleshooting:

VPN Discovery and Fingerprinting Technique

ICSA Labs IPSec Technical Product Configuration Guidelines A distillation of lessons learned during the course of ICSA's interoperability testing, with a useful checklist of things to check when establishing a multi-vendor IPsec environment.

Basic L2TP/IPsec Troubleshooting in Windows Specific techniques are Windows-specific, but the document contains a pretty good methodology for tracking down VPN problems.

Troubleshooting VPN Problems

Flying Raccoons: Networks that don't work Mac OS X focused, but techniques are generally useful in non-Mac environments too.

Miscellaneous Documentation:

Netscreen Concept & Examples: Screen OS Reference Guide: VPNs

IPsec from a Novell BorderManager to a Third-Party VPN Server

Setting up IPsec policy on an IBM AS/400


FW-1 and PGPNet
IPsec between a FireWall-1 and a Cisco router
Raptor Firewall (v5.x or 6.x) and Cisco IOS 12.0

IPSec traffic consists of three components. UDP/500 is used for ISAKMP key negotiations. IP protocol 50 carries Encapsulating Security Payload traffic, and IP protocol 51 carries the Authentication Header. The following instructions provide details on setting up filters on your firewall to allow these protocols to pass. We assume that the IPSec server is hosted on the internal (private) network, and that address redirection is used to transfer publicly-routable Internet traffic to the internal system.

Note that commercial IPsec products may require access to LDAP directory servers, certificate authorities, or proprietary network traffic. These services can usually be configured as generic (or plug) proxies or services. Your vendor should be able to tell you what additional services are required.

Alcatel/TimeStep IPsec equipment requires access to an Entrust certificate authority on TCP/709.

Configure the firewall to allow UDP/500, IP/50 and IP/51 in both directions.

On a Sidewinder:

On a Gauntlet:

Packet Filter Rule Editor - Forward ruleset
Filter Rules:
#perm iface prot srcport srcaddr:srcmask dstport dstaddr:dstmask
#IPSec negotiation outbound

1 permit inside udp 500 srcaddr:srcmask 500
#IPSec negotiation inbound

2 permit outside udp 500 500 dstaddr:dstmask
3 permit inside 50 * srcaddr:srcmask *
4 permit inside 51 * srcaddr:srcmask *

#IP inbound
5 permit outside 50 * * dstaddr:dstmask
6 permit outside 50 * * dstaddr:dstmask

On a Cisco:

ip access-list extended inet_inbound #define access-list
deny ip any
deny ip any
deny ip any # reject RFC1918 addresses
permit udp any host xxx.xxx.xxx.xxx eq 500 #allow IPSec IKE negotiations
permit esp any host xxx.xxx.xxx.xxx #allow IPSec ESP protocol
permit ahp any host xxx.xxx.xxx.xxx #allow IPSec AH protocol

On some IOS versions, you may have to use an alternative syntax:
permit 50 any host xxx.xxx.xxx.xxx #allow IPSec ESP protocol
permit 51 any host xxx.xxx.xxx.xxx #allow IPSec AH protocol

If you have a pre-11.2 IOS that does not support named IP access lists, you need to precede each access-list line with the access-list xxx command (where xxx=101-199), ie:
access-list 101 permit udp any host xxx.xxx.xxx.xxx eq 500
access-list 101 permit 50 any host xxx.xxx.xxx.xxx

Now put the access-list inbound on the interface where the traffic enters the router:
interface Serial0/0
description Internet interface
ip address zzz.zzz.zzz.zzz
ip access-group inet_inbound in #this applies access list to interface

On a PIX:

! These statements apply to current PIX versions (4.X(Y) or later).

! Following examples use for the inside, private host,
! and for the public, translated address for the same host.

conduit permit udp any eq 500 host
conduit permit 50 host any
conduit permit 51 host any

On a BorderManager:

If the third-party VPN is running another vendor's VPN software, you must create a new SINFO.VPN file and complete the procedure for adding a server to a VPN, as described in Novell BorderManager Enterprise Edition 3.5 Installation and Setup.

Create a new SINFO.VPN file with the following fields:

Major version number-Should always be set to 1.
Minor version number-Should always be set to 5.
Server name-Arbitrary name assigned to the third-party server. You can pick any name for convenience.
Master or slave ID-Should always be set to 1.
Public IP address-Public IP address of the third-party server.
Public IP address mask-Public IP address mask of the third-party server.
Private IP address-Not used. Should always be set to
Private IP address mask-Not used. Should always be set to
Private IP address mask-Not used. Should always be set to
Tunnel IP address-IP address of the VPN tunnel that you want to assign to the
third-party server. The address must belong to the same IP network as the local VPN's tunnel address.

Tunnel mask-Should be set to match the mask of your local VPN tunnel.
Public value length-Length of the third-party server's Diffie-Hellman public value, in bytes.
Public value in BER-Third-party server's Diffie-Hellman public value, in BER format. 1024-bit values are supported.
Security capabilities-Decimal equivalent of a 32-bit binary integer you must compute using the following bit values:

Flag to indicate third-party-Should always be set to 1 to indicate that the server is a member of a third-party VPN.
Local VPN member-Name of your local VPN server that is directly connected to the third-party VPN server. This is the only local server to which the third-party VPN server can be connected.

For further information, consult Novell's Knowledge Base -- from the following URL, select Novell BorderManager and search on "VPN": http://support.novell.com/search/kb_index.htm

Disabling Path MTU Discovery on a Windows NT system:

Cisco IOS VPN configurations

http://www.cisco.com/warp/public/707/16.html provides a great introduction to configuring and troubleshooting Cisco-based IPsec.

If you're a registered CCO user, you can start at http://www.cisco.com/warp/customer/105/IPSECpart1.html. (Registration information is available there.)

Other useful Cisco links
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/12cgcr/secur_c/scprt4/index.htm -- Cisco's IPSec documentation.

IPSec LAN-to-LAN with preshared keys (from Pete Davis):

!#The following section contains the parameters required for IKE Negotiation
crypto isakmp policy 20
encr 3des
hash md5
authentication pre-share
group 2
lifetime 86000
#This line of configuration is used to set the pre-shared key with the IKE peer
crypto isakmp key SHAREDKEYHERE address peer.IP.aDdRess
#This line sets up the encryption and authentication algorithms to use for IPSec
crypto ipsec transform-set set-NaME esp-3des esp-md5-hmac
#This section of the configuration sets up information required for the IPSec SA
crypto map cryptomapname 10 ipsec-isakmp

set peer pEeR.Ip.AdDreSs
set security-association lifetime seconds 28800
set transform-set set-NaME
set pfs group1
match address 100
interface FastEthernet0/0
crypto map cryptomapname
!(this required to turn on IKE/IPSec on this interface)
access-list 100 permit ip sOurCe.Net WildCardMask Destination Net WildCardMask
!(packets between these networks will have IPSec applied to them)

IPSec with pre-shared keys and NAT (from Cisco's web site, contributed by Guy Raymakers):
version 12.0
service timestamps debug uptime
service timestamps log uptime
service password-encryption
hostname C1605-REMOTE
enable password 7 481951588478970D
ip subnet-zero
no ip domain-lookup
isdn switch-type basic-net3
crypto isakmp policy 1
hash md5
authentication pre-share

crypto isakmp key eds address
crypto ipsec transform-set cm-transformset-1 ah-md5-hmac esp-md5-hmac
crypto map cm-cryptomap local-address Dialer1
crypto map cm-cryptomap 1 ipsec-isakmp
set peer
set transform-set cm-transformset-1
match address 100
process-max-time 200
interface Ethernet0
ip address
no ip directed-broadcast
interface Ethernet1
description connected to LAN
ip address
no ip directed-broadcast
ip nat inside

interface BRI0
description connected to Internet
no ip address
no ip directed-broadcast
ip nat outside
encapsulation ppp
dialer rotary-group 1
isdn switch-type basic-net3
crypto map cm-cryptomap
interface Dialer1
description connected to Internet
ip address
no ip directed-broadcast
ip nat outside
encapsulation ppp
no ip route-cache
no ip split-horizon
no ip mroute-cache
dialer in-band
dialer string 034141850
dialer hold-queue 10
dialer-group 1

no cdp enable
ppp authentication chap pap callin
ppp chap hostname jhsjdhuh
ppp chap password 7 18881777377733245E
ppp pap sent-username jhsjdhuh password 7 18881777377733245F
crypto map cm-cryptomap
router rip
version 2
timers basic 5 10 15 30
passive-interface Dialer1
distribute-list 10 out Ethernet0
no auto-summary
ip nat inside source list 101 interface Dialer1 overload
ip classless
ip route Dialer1
no ip http server
access-list 10 deny any
access-list 100 permit ip host host
access-list 100 permit ip host
access list 100 permit ip
access-list 101 deny ip
access-list 101 permit ip any
dialer-list 1 protocol ip permit
snmp-server engineID local 000000009020000507305B450
snmp-server community public RO
line con 0
exec-timeout 0 0
password 7 1041071E0A1E1C0C
transport input none
line vty 0 4
password 7 050408082E45400E