SIP, NAT, and Firewalls: Master's Thesis by Fredrik Thernelius
SIP, NAT, and Firewalls: Master's Thesis by Fredrik Thernelius
SIP, NAT, and Firewalls: Master's Thesis by Fredrik Thernelius
Master’s Thesis
By
Fredrik Thernelius
May 2000
DEPARTMENT
OF
TELEINFORMATICS
Email: SIP:
Private: fredrik_thernelius@hotmail .com Private: sip:fredrik_thernelius@hotsip.com
Page 1 of 69
Work: fredrik.thernelius@uab.ericsson.se Work: sip:fredrik.thernelius@uab.ericsson.se
1 Table of contents
1 TABLE OF CONTENTS .................................................................................................................................2
2 ABSTRACT ......................................................................................................................................................4
3 SAMMANFATTNING.....................................................................................................................................4
4 INTRODUCTION ............................................................................................................................................5
4.1 ACKNOWLEDGEMENTS ................................................................................................................................5
4.2 THE GOAL OF THE PROJECT ..........................................................................................................................5
4.3 UNDERSTANDING THE PROBLEMS WITH SIP AND FIREWALLS .....................................................................5
5 EARLIER STUDIES ........................................................................................................................................6
Page 2 of 69
10.3 WHAT LEVEL OF SECURITY WILL THIS DESIGN GIVE?.................................................................................35
11 OTHER SOLUTIONS................................................................................................................................36
11.1 AN INTRODUCTION TO RSIP ......................................................................................................................36
11.2 HOW TO USE SIP WITH RSIP INSTEAD OF NAPT.......................................................................................37
11.3 EVALUATION – THE CHOICE OF USING AN ALG OR RSIP..........................................................................37
12 CONCLUSION AND DISCUSSION ........................................................................................................38
12.1 SIP DEVELOPMENT ....................................................................................................................................38
13 ACRONYMS...............................................................................................................................................39
14 TABLE OF FIGURES................................................................................................................................40
REFERENCES .......................................................................................................................................................41
APPENDIX .............................................................................................................................................................42
A. COMPLETE CALL SETUP OF A TWO PARTY CALL USING SIP...............................................................................42
B. SIP/SDP MESSAGE GRAMMAR.........................................................................................................................44
C. ABNF...............................................................................................................................................................56
Page 3 of 69
2 Abstract
The work presented in this Master’s Thesis is an examination of how the SIP signaling, which occurs when a so called IP
Telephony session is set up, will be able to traverse firewalls. It is necessary to solve the problems/issues that SIP brings
about when the SIP messages traverse firewalls if this protocol ever will gain popularity.
In order to set up those data streams needed for transporting the sound in an IP telephony session the client enters
his IP address and a port number in the SDP part of the SIP message to tell the other party where he should sent
his audio data. Here is where problems occurs with the firewall. It needs to understand and interpret what the SIP
message says to be able to set up rules for allowing traffic to pass through the firewall to these addresses. The
problem is extended by the fact that it is common today to use “private addresses” on the LAN. These addresses
are not allowed to exist on the Internet and thus the firewall software must remove this address and replace it with
an address that is allowed on the Internet. A Network Address Translator (NAT) in the firewall normally does this
together with Application Level Gateways (ALGs).
The work of this Master’s Thesis has been focused around analyzing the above mentioned problems with SIP and
Firewalls and then using this as input designing a prototype of an Application Level Gateway for SIP, which could
be used together with perhaps a Linux firewall.
3 Sammanfattning
Det här examensarbetet är en undersökning av hur den SIP signalering som sker då ett så kallat IP-telefonisamtal ska
sättas upp ska kunna ta sig genom brandväggar. Att lösa de problem som SIP för med sig då det gäller att ta sig förbi
brandväggar är ett måste om detta protokoll någon gång ska kunna få stor spridning.
För att kunna sätta upp de dataströmmar som behövs för att transportera ljudet i ett IP-telefonisamtal sätter klienterna in
sin IP-adress och ett portnummer i SIP meddelandena för att tala om för den andra parten vart hon ska skicka det
audiodata hon sänder. Det är här problem uppstår med brandväggen. Den behöver förstå och tolka det som står i SIP
meddelandet för att kunna sätta upp regler som tillåter att det passerar trafik genom brandväggen till dessa adresser.
Ytterligare problem uppstår vid brandväggen då det idag är vanligt att många LAN använder ”privata adresser”. Dessa får
inte förekomma ute på Internet och därför måste programvaran i brandväggen även kunna filtrera bort de privata
adresserna och ersätta dem med IP-adress och portnummer som kan tillåtas att släppas ut på Internet. En adressöversättare
(NAT) gör normalt detta tillsammans med ”applikations-gateways” i brandväggen.
Tyngdpunkten på detta arbete har legat på att analysera de ovannämnda problemen med SIP och brandväggar och med
utgång från detta sedan implementera en prototyp till den mjukvara, en ”applikation-gateway” för SIP, som skulle kunna
användas tillsammans med t.ex. en Linux-brandvägg.
Page 4 of 69
4 Introduction
This is a Master’s thesis project carried out at the Department of Teleinformatics at the Royal Institute of
Technology in Stockholm. The work corresponds to twenty weeks (≈one semester) of full time studies and should
demonstrate that the student has the ability to solve a problem independently with knowledge gained both from
previous studies at the school and during the project.
4.1 Acknowledgements
I would like to thank my supervisor at the Department of Teleinformatics, Professor Gerald Q. "Chip" Maguire Jr.,
who has put up with me for such a long time. I would also like to thank everyone at Ericsson Utvecklings AB who
have encouraged me to finally finish my Master’s Thesis.
Firewalls are essentially made to prevent the community on the Internet from accessing hosts on the enterprise
LAN, but still letting the employees on the LAN have access to the benefits of the Internet. This is done by putting
rules in the firewall saying what traffic is allowed through from each direction, i.e. from the inside and from the
outside. As can been seen from this, the rules in the firewall need to be applied asymmetrically. The need for
asymmetric rules is seen from the following statement. “Just because you are allowed to surf on the Internet does
not automatically mean that someone one the Internet is allowed to surf on your Intranet.”
Services like HTTP and Telnet work on well-know ports (80 and 23 [RFC1700] respectively) and don’t give
firewall administrators much problem. SIP is another service, which also works on a well-know port, 5060. So
why should the SIP signaling be any harder for the firewalls to handle? In answering this question one has to
remember that SIP is only a way for setting up sessions, i.e. media streams between clients. It is not SIP in it self
that makes it hard (not entirely true!), it is the information that describes the session that SIP helps to set up that is
the hard part. SIP in itself is just a tool for helping users to find each other and to distribute information between
them. The information that is distributed, the session description, is the IP address and the port number of the
telephone application in the caller’s computer. SIP is described in Section 6. The ports that are used for the media
streams are emphermal (dynamic and > 1023). Thus the firewall will not know that an audio stream destined for a
certain address and port should be let inside/outside or not. This is a problem that can only be solved by having
some software analyzing all the SIP messages that pass the firewall on the well-known SIP port and then letting
that software tell the firewall what it should do with packets to/from a certain address, e.g. let it through or deny
and drop it. A few different kinds of firewalls are discussed in Section 7.
To make things more complicated the addresses that are used on the LAN can be from a range, which is not
allowed on the Internet. These so-called “private addresses” are mostly used due to the fact that is it hard to get
official IPv4 addresses, see Section 8. The private addresses can be used by anyone with the understanding that
they are not allowed to let them outside their own domain. By using private addresses it is possible for a company
to only have one global IP address, used by the outside interface of the firewall (or router if no firewall is used)
while still having the possibility to use many thousands of private addresses in the LAN. The problem with SIP is
now extended from having some software analyze the SIP messages and telling the firewall what ports to open to
providing software that removes all of the private addresses in the outgoing SIP messages and replaces it with the
firewall’s own IP address and telling the firewall about its usage. When messages come back the software has to
take out the global address it entered before and put the private address back. How this works is outlined in more
Page 5 of 69
detail in Section 8.4. The exchange of addresses actually limits the security in SIP since end-to-end encryption and
authentication is impossible. Section 9 gives an introduction to SIP security.
By definition in [RFC2663] a program doing the work described above, i.e. analyze messages and exchange
addresses, would be called an Application Level Gateway (ALG) as it must understand the application data, and
for SIP specifically it would be called a SIP ALG. As a part of the work in this Master’s Thesis a prototype SIP
ALG was implemented, see Section 10.
To use an Application Level Gateway is not the only way to get SIP to work through a firewall. Section 11 shows
how a protocol called Real Specific IP (RSIP) can be used instead.
5 Earlier studies
The report begins with a review of what others already have done in this area: IP telephony call control used
together with Firewalls and private addresses. The most important parts of the literature study was thus to look for
and examine documents in this area.
No studies on the subject SIP and firewalls existed at the time of the start (1998) of this Master’s Thesis. However,
there has been at least one study [Intel] in this area on a similar protocol for session initiation and IP telephony,
namely ITU’s H.323. [Intel] outlines in great detail the problem with sending IP addresses and emphermal port
numbers for the media streams in the set up messages. The paper also touches upon the subject of hiding IP
addresses and performing network address translation (NAT) in the gateway between the LAN and the Internet,
which also happened to be one of the objectives with this paper.
As to the subjects of private IP addresses and NAT numerous documents have been written. Many of these
[Hasenstein97 and RFC1631] outline a similar problem as that which this Thesis addresses, in that FTP sends the
emphermal ports for file transfers in its messages.
In February, two papers [SIPNAT and SIPALG] were published. Both of them address the issues discussed in this
paper. These papers have not been considered in this project but they provide some interesting reading.
Now, we can summarize the whole chain of events that will lead to an IP telephony session:
1. Adam, who wants to talk to Bob, sends a Session Initiation message to a server where he knows that Bob
is registered. This server then forwards the message to Bob.
2. The initiation message contains a session description part that in turn contains the information about the
media Adam wishes to use for the session.
3. When Bob receives the message he issues a response message with a similar session description.
4. When Adam receives the response he issues an acknowledgement message to Bob.
5. At this point both parties in the call can send real-time data to each other.
Page 6 of 69
0 15 16 31
V=2 P X CC M PT 16-bit sequence number
2 bits 1bit 1bit 4 bit 1bit 7 bits
The reason for not supporting retransmission in the protocol is that it would most likely take too long (at least one
RTT, which could be several hundred milliseconds) to request that the source resend the lost RTP packet and for
this copy to arrive. A better solution, for the case of audio at least, is to extrapolate sound from previous audio
samples to make up for the lost ones. Another algorithm to solve the problem due to lost packets is to just ignore
the lost data and go on as if nothing has happened. Simply ignoring lost packets works, because the duration of the
audio in one packet is relatively short, ranging from maybe 20 ms to about 60 ms. The loss of sound for that short
period of time will not have a major influence of the quality. It is likely that it is not even noticed at all.
The topic of retransmission is a major reason for not using TCP [RFC793]. TCP, which is a reliable connection
oriented protocol, uses retransmissions as a way to guarantee the delivery of the data handed to the TCP layer from
the application layer. According to [Stevens97], TCP in BSD implementations uses a retransmission timer with a
lower bound of 500ms. The reason that it cannot be lower is that an OS dependent timer with a resolution of this
value is used to initiate the retransmissions. Thus the timer can have values of N*500 ms, where N ≥ 1. A value of
zero for N would cause an immediate retransmission. What resolution and what lower bounds other OS
manufacturers use for their implementations of the TCP retransmission timer is outside the scope of this thesis to
consider.
Instead of TCP, RTP normally uses UDP [RFC768] as the default transmission protocol. UDP does not provide
any reliability features. UDP in turn uses IP, with best effort delivery to encapsulate its data. Any application level
protocol that depends on UDP for transmission and still has the desire to be sure that any data sent is also received
must implement its own retransmission algorithm.
Now we summarize the processing and encapsulation, see Figure 2, of the audio for an IP telephony session before
it is sent from a host:
1. The sound from the microphone will be sampled at certain times. A number of samples are bundled
together by the application to be the data encapsulated in a RTP packet. Typically 20 ms of sound is
encapsulated into one RTP packet.
2. On the transportation level the RTP packet is encapsulated into a UDP datagram.
3. On the network layer the UDP datagram is encapsulated into an IP packet. For reference The UDP and
TCP headers are shown in Figure 3 and Figure 4.
4. The IP packet is encapsulated into an Ethernet (or any other link layer protocol) frame and then the
frame is sent off.
Page 7 of 69
user
data
Application
Appl. user
Header data
IP datagram
Ethernet frame
0 15 16 31
16-bit source port number 16-bit destination port number
8 bytes
16-bit UDP length 16-bit UDP checksum
Page 8 of 69
0 15 16 31
16-bit source port number 16-bit destination port number
0 15 16 31
4 bit 4- bit header 8-bit type of service 16-bit total length (in bytes)
version length (TOS)
RTP has its own control protocol, the Real-Time Control Protocol (RTCP) [RFC1889]. The main purpose for
RTCP is to give feedback on the quality of the delivery of data from the recipients of the data, to those who send
it. The feedback is given in reports sent with RTCP and can for example contain the number of lost packets in the
session and information about delays in the intermediate networks. RTCP is not streamed continuously in the same
way as RTP, instead it is sent periodically with a typical period of a few seconds between the reports.
Any given RTP media stream is always assigned to an even port number while the associated RTCP reports are
sent on the next higher, and thus odd, port number.
1
For multicast sessions, the session description work opposite to unicast sessions. Here the session descriptions
tell what the client is willing to send. Multicast sessions are not considered in this thesis.
Page 9 of 69
you a session description with his information to you, or else you will not be able to send any media data to him. A
typical session description looks like the one in Figure 6.
v=0
o=uabfrth 955720785594 955720785594 IN IP4 134.138.242.7
s=Basic Session
c=IN IP4 134.138.242.7
t=955720785594 0
m=audio 2328 RTP/AVP 8 0 96 98 99 97
a=rtpmap:96 SC6/6000
a=rtpmap:98 SC6/3000
a=rtpmap:99 RT24/2400
a=rtpmap:97 VR15/1500
Session description
v= (protocol version)
o= (owner/creator and session identifier).
s= (session name)
i=* (session information)
u=* (URI of description)
e=* (email address)
p=* (phone number)
c=* (connection information - not required if included in all media)
b=* (bandwidth information)
One or more time descriptions
z=* (time zone adjustments)
k=* (encryption key)
a=* (zero or more session attribute lines)
Zero or more media descriptions
Time description
t= (time the session is active)
r=* (zero or more repeat times)
Media description
m= (media name and transport address)
i=* (media title)
c=* (connection information - optional if included at session-level)
b=* (bandwidth information)
k=* (encryption key)
a=* (zero or more media attribute lines)
* optional item
Some header fields are of special/major importance to this thesis will be discussed a little bit further, as for the rest
the listing above is sufficient.
• the origin field o=<username> <session id> <version> <network type> <address type> <address>
Page 10 of 69
The parameters of the origin field will together form a unique identifier for the current session. The last parameter
contains an address, which could be an IPv4 address expressed either in dotted-decimal form or as a fully qualified
domain name. None of the parameters in the origin field are intended to be used for routing media between clients.
The connection field can exist in both the session description and the media description part of the SDP. Its
purpose is to give the port number given in the media field an address to be associated with. If there is a
connection field in the media description part then the address in this field is prioritized over the address that was
possibly given in a connection field in the session description part.
The purpose of the media field is to let the other party in the session know what kind of media, audio or video, or
maybe something entirely different, the recipient of the SDP should deliver, to what port on the associated
connection address (see above) the media should be delivered to, and in what way the media should be coded. The
example in Figure 6 uses two standard codecs [RFC1890] denoted 8 and 0 in the media field. In the same media
field are four non-standard codecs, denoted 96, 97, 98 and 99, declared. The non-standard codecs are defined in
the following attribute fields, one for each codec number. The following table shows the standardized payload
types [RFC1890].
Table 1. Payload types (PT) for standard audio and video encodings
Page 11 of 69
“The Session Initiation Protocol (SIP) is an application-layer control (signaling) protocol for creating,
modifying and terminating sessions with one or more participants. These sessions include Internet multimedia
conferences, Internet telephone calls and multimedia distribution. Members in a session can communicate via
multicast or via a mesh of unicast relations, or a combination of these. SIP invitations used to create sessions
carry session descriptions which allow participants to agree on a set of compatible media types. SIP supports
user mobility by proxying and redirecting requests to the user's current location. Users can register their current
location. SIP is not tied to any particular conference control protocol. SIP is designed to be independent of the
lower-layer transport protocol and can be extended with additional capabilities.” [RFC2543]
It is outside the scope of this paper to discuss all the features of the protocol, but I will however discuss at least a few basic
ones. See Figure 9 for an example of a SIP message. The different parts of a SIP message will be discussed in the
following Sections.
v=0
o=uabfrth 955720785594 955720785594 IN IP4 134.138.242.7
s=Basic Session
c=IN IP4 134.138.242.7
t=955720785594 0
m=audio 2328 RTP/AVP 8 0 96 98 99 97
a=rtpmap:96 SC6/6000
a=rtpmap:98 SC6/3000
a=rtpmap:99 RT24/2400
a=rtpmap:97 VR15/1500
Page 12 of 69
Client A Client B
(1) INVITE
A wants to talk to B
The phone rings in B’s office
Provisional Response, Ringing
BYE
Client B ends the call session
200, OK I accept
When B accepts the call his user agent sends a message (2) with a response code of 200. Any 2xx response means that the
message was successfully received, understood, and accepted. See Section 6.3.9 for a summary of response codes. In the
response client B adds his codec capabilities and the port numbers where he wants A to send his RTP data to. The final
part (3) of the three-way handshake occurs when A sends an acknowledgement to B. By sending an ACK the caller
confirms that it has received the response from the callee. After the setup procedure is completed the conversation can
begin.
Figure 10 also shows an optional message, a provisional response. The provisional response here is an informational
message that provides feedback to the caller that the phone is ringing on the receiving side.
The SIP server will read the To field in the message, see figures 9, 11 and 12, and initiate a search for the
particular user, who is pointed out by the SIP URL in this field. The user is located via a query to the location
server, which could be an LDAP server [RFC1777] as suggested in [RFC2543]. When the SIP server receives a
location or locations for that user it will react in one of two ways. It will either forward the packet to its
destination, see Figure 11, or it will send a response back to the caller, see Figure 12, which contains the
location(s) of the callee and thus lets the caller contact the callee himself. The server is said to be working in either
proxy- or redirect mode.
Page 13 of 69
ClientA@location1.com SIPserver@location2.com Location Server ClientB@locationX.com
(1) INVITE ClientB@location2.com
Via: ClientA@location1.com (2) Where is ClientB?
(3) ClientB@locationX.com
(5) 200 OK
Via: SIPserver@location2.com
(6) 200 OK
Via: ClientA@location1.com
Via: ClientA@location1.com
(8) ACK
Via: SIPserver@location2.com
(7) ACK
Via: ClientA@location1.com
Via: ClientA@location1.com
Conversation
(5) ACK
Via: ClientA@location1.com
(7) 200 OK
Via: ClientA@location1.com
(8) ACK
Via: ClientA@location1.com
Conversation
Page 14 of 69
As has already been described briefly earlier, the private IP addresses that are used in an outgoing SIP message
must be replaced by the globally routable IP address of the NAT Firewall. This fact makes it is important to
recognize which header fields can contain IP addresses and which ones do not. This subject is described further in
Sections 9 and 10.
In general IP addresses can occur in all the places, which can hold a SIP URL, i.e. a SIP address. SIP URLs can be
found in (1) the Request line, (2) the To field, (3) the From field, (4) the Via field, (5) the Contact filed, (6) the
Record-route field, and (7) the Route field. It is also common that the last part of the Call-ID field contains the IP
address or hostname of the calling host in order to help ensure uniqueness of the session identification.
Examples of SIP URLs can be seen in Figure 9. They are easily recognized, they all start with “sip:”.and the host
part is either a domain name or a numeric network address. There are several examples of URLs of this type in
Figure 9: sip:uabfrth@134.138.228.159 and sip:Fredrik.Thernelius@uab.ericsson.se are two of them.
Location Server: The location server contains the information about user locations.
User Agent: The User Agent (UA) is an application program running at the endpoints of the call, at the users.
The UA consists of two parts, the User Agent Client (UAC) and the User Agent Server (UAS). The
UAC sends SIP requests on behalf of the user and the UAS listen for responses and notify the user
when they arrive.
SIP server: The SIP server is responsible for the users within a domain. It can work in proxy mode or redirect
mode. The redirect mode it relays information to caller about the callees’ location, see Figure 11. In
proxy mode it relays all messages between the caller and the callee, see Figure 12.
Call leg: The 3-tuple of the (1) Call-ID, (2) the From field, and (3) the To field (including any tags) defines a
call leg, i.e. identifies of a session between two SIP clients.
The structure and the syntax of SIP messages are, as many IETF developed text based protocols such as HTTP and
FTP, based upon the definitions made in the “Standard for ARPA Internet Text Messages”[RFC822]. The specific
character set used by SIP is ISO 10646 with UTF-8 encoding [RFC 2044], just as for SDP. [RFC822] bases its
syntax on the augmented Backus-Naur Form (ABNF) [RFC2234]. The syntax of augmented BNF is described in
appendix 16.3.
The SIP messages are always either a request from a client to a server or a response to a request from a server to a
client. With augmented BNF this is written as:
Page 15 of 69
Both the request message and the response message share a common structure, i.e. the generic message structure.
The generic message structure is defined as:
The generic message syntax gives the structure of the entire SIP msg. Please compare this syntax to the sample
SIP message provided in Section 6.3. After the first line, the start-line follows several message headers and after
the empty line in the middle follows the message body, i.e. the session description.
A summary of the different types of headers can be found in Section 6.3.10. All of the message headers follow the
same syntax and structure.
A request, like the one showed in Section 6.3, have the following main building blocks.
Request = Request-Line
*( general-header | request-header | entity-header )
CRLF
[message-body]
A summary and description of the request methods follow in the next section.
Response = Status-Line
*( general-header | response-header | entity-header )
CRLF
[ message-body ]
INVITE
The INVITE method indicates that the user or service is being invited to participate in a session. The
message body contains a description of the session to which the callee is being invited.
ACK
The ACK request confirms that the client has received a final response to an INVITE request.
OPTIONS
The server is being queried as to its capabilities. A server that believes it can contact the user, such as a
user agent where the user is logged in and has been recently active, MAY respond to this request with a
capability set.
Page 16 of 69
BYE
The user agent client uses BYE to indicate to the server that it wishes to release the call. A BYE request is
forwarded like an INVITE request and MAY be issued by either caller or callee.
CANCEL
The CANCEL request cancels a pending request with the same Call-ID, To, From and CSeq (sequence
number only) header field values, but does not affect a completed request. (A request is considered
completed if the server has returned a final status response, i.e. a 2xx response)
REGISTER
A client uses the REGISTER method to register the address listed in the To header field with a SIP server.
Page 17 of 69
General-header Accept X E Cseq N E Record-Route X H
Accept-Encoding X E Date X E Timestamp X E
Accept-Language X E Encryption N E To N E
Call-ID N E Expires X E Via N E
Contact X E From N E
Entity-header Content-Encoding X E Content-Length X E Content-Type X E
Request-header Authorization X E Priority C E Response-Key C E
Contact X E Proxy-Authorization N H Subject C E
Hide N H Proxy-Require N H User-Agent C E
Max-Forwards N E Route X H
Organization C H Require X E
Response-header Allow X E Server C E WWW-Authenticate C E
Proxy-Authenticate N H Unsupported X E
Retry-After C E Warning X E
X = May be encrypted E = End-to-End header field
N = Must not be encrypted H= Hop-by-Hop header field
C = Should be encrypted
Table 4. Summary of SIP headers
7 Internet Firewalls
Firewalls are the main protection mechanism when it comes to keeping unwanted traffic away from ones own
network. The concept can be easily described, see Figure 13 [Cheswick95]. Everything that is inbound (this is
always seen from the internal networks point of view) or outbound will be filtered. Those packets that do not
fulfill the rules set will be filtered out and dropped in order to enforce the security policy set in the filters.
Filter Filter
inbound
Gateway
outbound
In books like [Cheswick95, Chapman95, and Goncalves00] the subject of firewall architecture is described in
great depth. This thesis will not attempt to cover this area other than the basic concepts. The model referred to in
this text is the dual-homed i.e., the firewall has two network interfaces, host model, which also happens to match
the figure shown above.
Page 18 of 69
distinguish the start of the connection from an established connection. Figure 15 shows how a TCP connection is
set up and then torn down after a connection. A three-way handshake is used to set up a TCP connection between
two hosts while four TCP segments are needed to tear down the connection.
A wants to
connect to B
A B
SYN bit set
[Course488] describes that stateful filters use information from the application part in the filtering process. This
implies that stateful filters can recognize application protocols without having to base its decisions on whether a
certain packet is destined for a service using a well-known port or not. Worth noting is that the packet filtering
gateways do not have the ability to make changes to content of the application data of the IP datagrams.
The filtering functions, rules, on packet filtering gateways are applied on a per interface and per direction (in or
out) basis. The rules also depend on the direction of a certain packet, i.e., if it is going in to the gateway or if it is
leaving the gateway. Figure 15 below shows how rules can be written in Cisco IOS and Linux. They are very
similar. In Cisco IOS the rules are bundled in Access Control Lists (ACL) [Cisco]. The tool for writing rules in
Linux is called ipchains [ipchains].
The figure shows an example where hosts on the internal network, 10.1.1.0, with a 24 bit net mask are allowed to
use their web browsers (running an a port number above 1023) to access outside Web servers (running on port 80)
and then download content from them.
Page 19 of 69
OUTSIDE NETWORK Firewall INSIDE NETWORK
10.1.1.0/24
Filter Filter
ACL101 ACL104
eth0 Gateway eth1
ACL103
ACL102
Cisco IOS
ACL Applied to Direction Permit/Deny Protocol source port destination port option
101 eth0 out permit TCP 10.1.1.0 0.0.0.255 gt 1023 any eq 80
102 eth0 in permit TCP any eq 80 10.1.1.0 0.0.0.255 gt 1023 estabished
103 eth1 in permit TCP any eq 80 10.1.1.0 0.0.0.255 gt 1023 established
104 eth1 out permit TCP 10.1.1.0 0.0.0.255 gt 1023 any eq 80
Linux
ipchains –i eth0 -I output -j ACCEPT -p tcp -s 10.1.1.0/255.255.255.0 1024: -d 0.0.0.0/255.255.255.255 80
ipchains –i eth0 -I input -j ACCEPT -p tcp -s 0.0.0.0/0.0.0.0 80 -d 10.1.1.0/255.255.255.0 1024: ! –y
ipchains –i eth1 -I input -j ACCEPT -p tcp -s 0.0.0.0/0.0.0.0 80 -d 10.1.1.0/255.255.255.0 1024: ! –y
ipchains –i eth1 -I output -j ACCEPT -p tcp -s 10.1.1.0/255.255.255.0 1024: -d 0.0.0.0/255.255.255.255 80
Figure 15. Cisco IOS Access Control Lists and Linux ipchains
The example in the figure shows how rules can be written to let hosts on the internal network 10.1.1.0 with a 24
bit net mask send requests to an outside Web server.
The circuit level gateway (often referred to as circuit-level proxy) is a generic tool for relaying TCP connections
from one side of the firewall to the other, see Figure 16. The internal client makes a connection to a TCP port on
the gateway, which then opens a connection to the external server. The gateway does not try to interpret the
content of the application part of the TCP segment, it only relays any information received from one side to the
other. This procedure has been standardized in [RFC1928]. The SOCKS 5 protocol uses the well-known TCP port
1080. To use the SOCKS service the client software needs to be modified, as described in [Cheswick95], or
replaced so that a SOCK-ified client can be used instead.
Internal client
Connection 2
relay Connection 1
Circuit Level
Gateway
Firewall
The addresses that the clients use on the internal network will not be seen in the IP header of datagrams leaving
the network since it will be replaced with the address of the second interface (on the public side). The port number
in the TCP header will therefore also be replaced.
Page 20 of 69
7.3 Application Level Gateways
The application-level gateway (ALG) is the most sophisticated kind of firewall gateway. It works in a similar
fashion as the circuit level gateway but there are a number of important differences (shown later). Application
level gateways are just like the circuit-level gateways often referred to as proxies. On other occasions, in
[RFC2663] for example, an ALG is often referred to a program running in cooperation with a firewall performing
Network Address Translation (NAT), as described in Section 8. The latter definition is the one that is used in this
thesis. Here follows some features of an ALG:
1. The ALG is not generic like the circuit-level gateway, instead it uses special purpose code for each particular
application/service it is supporting. Since it understands the application protocol it is relaying it achieves a
higher security level.
2. The ALG does not have the shortcoming of only supporting TCP as the circuit-level gateway has. It also
supports UDP based protocols, such as TFTP.
3. If the ALG is used together with a NAT (the main concern of this thesis), then the ALG will examine the
application data for occurrences of internal addresses and replace them with the address of the firewalls
external interface. Section 10 describes an implementation of an ALG for SIP.
ALGs with support for protocols like FTP, DNS, and SNMP are common components in NAT firewalls according
to [RFC2663]. We hope soon it will be just as common to also support SIP in this fashion!
8 IP addressing issues
As the number of computers in the world increases, the limitations of the current Internet Protocol, version 4, are
apparent. The topic discussed here is the rapid depletion of available IPv4 addresses. This is a fact, even though it
seems strange when considering that IPv4 supports 32 bit addresses (232 ≈ 4.3 billion) and the numbers of
computers in the word have not even reached (or has it?) a billion yet. The explanation is given in Section 8.1.
Section 8.2 will show that the next version of the Internet Protocol, IPv6, solves this problem by introducing a
larger address space. Section 8.3 introduces a special kind of addresses, called private addresses. The private
address space is available for every one to use in their own network without having to request them from some
authority. The private addresses are very useful, but since many hosts can use the same addresses at the same time,
they are not allowed on the Internet. Section 8.4 explains how a Network Address Translator (NAT) can be used
when hosts using private addresses want to communicate over the Internet. The use of private addresses and NATs
are seen as an intermediate solution of the address depletion until the deployment of IPv6, which is expected to
take several years.
To summarize: Class A has 50% of the number of possible addresses, Class B has a 25 % and Class C has 12.5%.
Class A, B and C will thus have 87.5% together.
Page 21 of 69
7 b its 2 4 b its
C la s s A 0 n e t id h o s t id
1 4 b its 1 6 b its
C la s s B 1 0 n e t id h o s t id
2 1 b its 8 b its
C la s s C 1 1 0 n e t id h o s t id
2 8 b its
C la s s D 1 1 1 0 m u lt ic a s t g r o u p I D
2 7 b its
C la s s E 1 1 1 1 0 ( r e s e r v e d fo r fu tu r e u s e )
C la s s R ange
A 0 .0 .0 .0 to 1 2 7 .2 5 5 .2 5 5 .2 5 5
B 1 2 8 .0 .0 .0 to 1 9 1 .2 5 5 .2 5 5 .2 5 5
C 1 9 2 .0 .0 .0 to 2 4 3 .2 5 5 .2 5 5 .2 5 5
D 2 2 4 .0 .0 .0 to 2 3 9 .2 5 5 .2 5 5 .2 5 5
E 2 4 0 .0 .0 .0 to 2 4 7 .2 5 5 .2 5 5 .2 5 5
Here is an example that shows how a great deal of the address space has been wasted:
Lets say that a fairly large corporation, with about 20 thousand employees and thus with 20 thousand computers
all of a sudden needed to connect the computers to the Internet. They can either request a class B address from
InterNIC and leave 45000 addresses unused or they can request 80 class C networks. As using a lot of class C
networks creates a lot of administrative problems the natural choice would be to go with a Class B network. This
was the usual way to do it in the old days, now days this would never be possible.
An example of what an IPv6 address can look like in text representation is:
789A:0:0:0:3:4567:89AB:CDEF
The ABNF description of IPv6 text representation is shown in Figure 19 below. More information about the
address architecture for IPv6 can be read in [RFC 2373].
Page 22 of 69
IP v6 add re ss = hex pa rt [ ":" I Pv4 ad dre ss ]
IP v4 add re ss = 1*3 DI GIT " ." 1* 3DI GI T " ." 1* 3D IGI T "." 1 *3D IG IT
As can be seen in the ABNF syntax, the IP version 6 has support for the use of IP version 4 addresses and this is
good since the IPv6 and IPv4 are likely to co-exist during a transition period. Figure 20 shows the 40 byte long
IPv6 header.
0 15 16 31
Version Priority 24 bit Flow Label
4 bits 4 bits
IP version 6 contains more features than just an extension of the address space. Authentication and Privacy
capabilities are also included [RFC2401].
Page 23 of 69
The private addresses are banned on the Internet to avoid ambiguity problems. There would be no way to route IP
datagrams on the Internet to the right destination if two or more receiving hosts have the same IP address.
But why use private addresses if they can lead to this kind of inconsistencies? The reason is that the private
addresses solve the big problem mentioned in the beginning of Section 8, i.e. the rapid depletion of the number of
available IP addresses. The private addresses also enable network administrators to be more flexible in their design
of networks, as more operational and administratively convenient addressing schemes can be used.
It is actually possible to use any kind of addresses, private or global, for private networks as long as it does leave
the private network. In addition, packets leaving the private network must not contain those addresses. If your ISP
catches you with sending out addresses not assigned to you they have the option to disconnect you in immediately.
Making a mistake and sending out datagrams with private addresses could be forgivable, but sending out with
someone else’s addresses is an offence that is much worse.
In order to not “leak” addresses, it is necessary for the IP and the UDP/TCP- headers, see Figure 3, 4, and 5, of any
outgoing packet to be changed, as it passes out onto the Internet and then changed back as responses are received
from the Internet and passed on to the private LAN. This is how Network Address Translation (NAT) comes into
the picture.
8.4 NAT
NAT [RFC1631] was originally thought of as a short-term solution until a long-term solution to the problem of the
depletion addresses was developed. The long-term solution was expected to be a proposal for a new Internet
protocol with larger addresses and we now know that the next generation of IP will be IPv6, see Section 8.2 above.
B
A
NAT 172.16.0.10
Internet ROUTER
138.201.148.32
Figure 20 shows the idea of address translation. When host A, who is using the private address 172.16.0.10,
communicates with host B, who is assigned a global address on by his ISP, the IP addresses in the IP header must
be exchanged from 172.16.0.10 to 138.201.148.32 as packets move across the NAT router. After the exchange it
will look to host B as if host A really has 138.201.148.32 and it will thus send any replies to that address. Host B,
could in fact be behind a NAT. There is no way to tell.
It is worth noting that the NAT router does the translation transparently. This means that the NAT router will swap
the IP address in the IP header without any notification to either of the hosts.
For every session between internal and external hosts the NAT router will have to set up an entry in a translation
table. This is done to be able to correctly map packets to the right host on the other side, i.e. for the translation to
work properly. Such a table could look like the one in Figure 23.
Page 24 of 69
NAT Table
Internal IP External IP
172.16.0.10 138.201.148.32
172.16.0.11 138.201.148.151
The Internal IP address is the private IP address of the internal host and the external IP is a public IP address on
the NAT router, which is assigned to map the private address. That is, in the outgoing packets the NAT router will
exchange the address in the “source” part, 172.16.0.10, of the IP header to the corresponding external IP address,
138.201.148.32. In the same way the “destination” part of incoming messages be exchanged and sent out on the
internal interface if it matches any of the external addresses in the table, thus 138.201.148.32 will be exchanged
for 172.16.0.10. Figure 24 shows the procedure.
Source IP: 138.201.148.32 Destination IP: a.b.c.d Source IP: 172.16.0.10 Destination IP: a.b.c.d
NAT
ROUTER
Source IP: a.b.c.d Destination IP: 138.201.148.32 Source IP: a.b.c.d Destination IP: 172.16.0.10
This seems to be very simple, and it is, but there are a few problems that can occur. Lets use SIP to show them.
The first problem occurs if a session is not initiated from the inside, because then there will be no easy way to set
up the translation table. The address to the right internal host must be found somehow. A second problem is that
some application protocols like SIP for instance (FTP is an other), put IP addresses and port numbers inside the
application data in the IP packets. This information must also be processed somehow. In SIP this information tells
the other party of a call where to send the media. To solve these problems, an Application Level Gateway (ALG)
is needed to analyze these messages.
I will not try to define general solutions to these problems for all protocols. I will focus only on SIP. In order to
solve the first problem the internal clients must be registered at the ALG. This will give the ALG the internal
address of the client, whose name was found in the To field. The second problem, which only applies to outbound
messages, is also solved by having the ALG analyzing the message and exchange the necessary parts. See Section
10, for a more detailed discussion on this subject.
2
In Linux this is called masquerading
Page 25 of 69
The mapping between internal IP address and external IP address is set up manually by the network administrator
or perhaps this work be automated by using protocols like DHCP [RFC2131]. By using DHCP the host would be
assigned an IP addresses by the DHCP server at boot time.
By using Static NAT the problem with sessions initiated from the outside with inbound requests can be avoided.
This is because of the one to one mapping in the translation table, i.e. when a new session is started from the
outside the NAT router will always know which internal IP address to use.
NAT ROUTER
B
138.201.148.32 A
138.201.148.33
138.201.148.39
138.201.148.34 172.16.0.10
138.201.148.38
Internet
138.201.148.36
138.201.148.37
138.201.148.35
When a host wants to connect to the outside an external IP address is allocated from pool of addresses managed by
the NAT. For each one allocated it reduces the number available. This can continue until all the IP addresses are
allocated. At which point the next host that tries to send outbound messages will not be able to do so until a
connection is closed down and the used IP address is returned to the pool of available IP addresses.
For TCP it is possible to tell if the packet closes down the connection by looking in the TCP header, both the FIN
bit (orderly release) and RST bit (abortive release) are considered [RFC2663]. UDP creates a problem since it does
not have these features. This means that there is no way to look at only the UDP header and tell if a certain
datagram will release the connection and thus give the back the address to the address pool.
One way to handle the problem of when NAT can reclaim the address is to use timeouts. When a mapping has not
been accessed for a certain period of time the mapping is said to be dead, and the NAT router reclaims the
allocated address. This scheme is even used with TCP connections since connections can be cut without any FIN
message has been sent. [RFC2663] suggests that a TCP session may be terminated if it has not been used for 24
hours. For non-TCP a timeout of a few minutes is suggested.
Figure 26 shows how NAPT works. The source IP and source port of host A are exchanged on outgoing packets,
compare item 1 and item 2 of the figure. When the user on host B replies to the request, he sends 3. The router will
recognize the information contained in the headers, translate the information, see part 4, and send it off to host A.
Page 26 of 69
B
A 172.16.0.10
NAPT
Internet
ROUTER
138.201.148.32
NAPT Table
Source IP: 138.201.148.32 Destination IP: a.b.c.d IP address of external Interface is: 138.201.148.32 1 Source IP: 172.16.0.10 Destination IP: a.b.c.d
Source Port:10000 Destination Port: 12345
2 IP address of internal Interface is: 172.16.0.1 Source Port:15000 Destination Port: 12345
NAPT
Global addresses Private addresses
For more detailed information than the one given below, see [RFC2543].
9.1 Encryption
Encryption can take place either end-to-end between user agents, or hop-by-hop between any two SIP entities.
Hop-by-hop encryption encrypts the whole SIP message and is supposed to work on the transport level or the
network layer. The algorithm used is not specified, but IPsec [RFC2401] is suggested.
When using end-to-end encryption between user agents some basic rules are necessary to follow. They are:
1. All header fields must not be encrypted since they are needed to be understood by intermediate SIP entities,
see Table 4.
2. All header fields that are not encrypted must precede those that are encrypted, see Figure 27.
3. It is not necessary to encrypt any SIP headers, see Figure 28.
4. An encryption header must be inserted to indicate the encryption mechanism
5. The responses to encrypted requests should be encrypted with a key given in the Response-key header field in
the request. If none is given then the answer should be sent unencrypted.
6. The headers that were encrypted in the request should also be encrypted in the response.
3
Encryption enables the possibility to ensure privacy, i.e. no one except the intended receiver will be able to read the message
4
Authentication is the concept of being sure of whom you are talking to.
Page 27 of 69
INVITE sip:watson@boston.bell-telephone.com SIP/2.0
Via: SIP/2.0/UDP 169.130.12.5
To: T. A. Watson <sip:watson@bell-telephone.com>
From: A. Bell <sip:a.g.bell@bell-telephone.com>
Encryption: PGP version=5.0
Content-Length: 224
Call-ID: 187602141351@worcester.bell-telephone.com
CSeq: 488
******************************************************
* Subject: Mr. Watson, come here. *
* Content-Type: application/sdp *
* *
* v=0 *
* o=bell 53655765 2353687637 IN IP4 128.3.4.5 *
* c=IN IP4 135.180.144.94 *
* m=audio 3456 RTP/AVP 0 3 4 5 *
******************************************************
************************************************
* *
* v=0 *
* o=bell 53655765 2353687637 IN IP4 128.3.4.5 *
* c=IN IP4 135.180.144.94 *
* m=audio 3456 RTP/AVP 0 3 4 5 *
************************************************
9.2 Authentication
When using authentication the user digitally signs the message that is about to be sent. The signature extends over
the whole SIP message, from the first line, i.e. the request-line or the status-line, to the message body, but not all
SIP header fields are included in the signature. Those header fields that are not included are those that changes
between hops, such as the Via field for instance. If it were included then the message integrity calculation on the
receiving side would produce an error if any intermediate proxies had entered (which they must do!) their address
in the Via list. Figure 29 below shows how this works. At the receiving side those headers that are not included in
the signature must be removed before the integrity calculation can be performed.
Page 28 of 69
INVITE sip:watson@boston.bell-telephone.com SIP/2.0
Via: SIP/2.0/UDP 169.130.12.5
Authorization: PGP version=5.0, signature=... Bold characters
From: A. Bell <sip:a.g.bell@bell-telephone.com> are unsigned
To: T. A. Watson <sip:watson@bell-telephone.com>
Call-ID: 187602141351@worcester.bell-telephone.com
Subject: Mr. Watson, come here.
Content-Type: application/sdp
Content-Length: ...
v=0
o=bell 53655765 2353687637 IN IP4 128.3.4.5
c=IN IP4 135.180.144.94
m=audio 3456 RTP/AVP 0 3 4 5
v=0
o=bell 53655765 2353687637 IN IP4 128.3.4.5
c=IN IP4 135.180.144.94
m=audio 3456 RTP/AVP 0 3 4 5
Figure 29. Upper - the SIP request that is to be sent. Lower - the signed parts of the message
Messages that are not authenticated may be challenged by any SIP entity downstream5 from the user. The
challenge could either be included in a 401 (Unauthorized) or a 407 (Proxy authorization required) response
message, which may only be used by proxies.
Register INVITE
Register INVITE
Authorization: … Proxy-Authorization: …
200 OK 200 OK
The reason for having this feature is that the information in the Via list contains information about traffic patterns,
which eavesdroppers might find very interesting.
The NAT will give the SIP ALG every IP packet that uses the default SIP port, 5060, in the destination address, no
matter if the packet comes from the private network side or the Internet side. With this scheme a SIP client cannot
5
Downstream is the direction from the client issuing the request to the intended recipient and upstream is the
opposite way.
Page 29 of 69
reach another SIP client unless it uses the default SIP port, this eliminates the possibility of running several SIP
clients on a number of different ports on a host, but it should not limit the possibility of reaching the intended
client entirely.
In the case of incoming SIP INVITE messages from external clients intended for internal clients it is clear that
some kind of location service must be available in the SIP ALG in order to direct the message to the correct host
on the inside. Implementing a simple SIP registrar server together with the SIP ALG solves this problem. The
internal clients would have to register at the registrar server in order for the server to get the clients internal IP
address. Unfortunately this scheme has not been tested in practice, because of the lack of time for the
implementation Instead, the SIP ALG was coded to always use the same IP address and NAT mapping for the
single internal client I had.
For outgoing calls it is not necessary for the internal client to be registered at the server.
The second objective for the SIP ALG is to take care of IP addresses in the SIP message body, i.e. the session
description. SIP can include any type of information in the message body, but for the purposes of this thesis it is
assumed that only SDP is used for conveying the session description. Anything other than SDP in the SIP body
would generate a parse error in the SDP parser and cause the ALG to drop the SIP message. In the SDP the clients
insert the IP address that they want to receive the RTP data streams on in at least one field, the connection-field,
denoted by a “c”. It can also happen that the originator-field, denoted by an “o”, contains this address.
Additionally, the client inserts the port number(s) that the RTP data stream(s) will be directed to in the media-
fields, denoted by an “m”. Both the IP address and the port number(s) must be replaced by the ALG in outbound
SIP messages. Examples of the fields mentioned can be seen in Figure 31 below.
The replacement of the IP address in SDP follows the pattern used for the SIP part of the message. For outbound
SIP messages containing a session description the ALG will have to replace the IP address of the originating host
with that of the firewall, and reversing the procedure for inbound SIP messages, as mentioned earlier. The SDP of
inbound SIP messages will not have to be touched by ALG since the data contained there only applies to the host
which sent the message.
The procedure for replacing the port numbers in the media field is more complicated than just replacing them right
away. The ALG first has to find out what they will be replaced with. This is done by creating a NAT mapping for
the expected incoming data. As the observant reader has noticed, the RTP data is always associated with RTCP at
the next higher port number. This means that not only one, but two mappings have to be created. One for incoming
RTP data and one for incoming RTCP data. Again, not even this is the whole truth. In the last line of Figure 31 the
port number is followed by “/2” and this indicates that the client wishes to use media with two layers, e.g. MPEG-
2 [RFC2343]. Each layer uses a pair of RTP and RTCP ports and this example gives a total of four ports. For this
Page 30 of 69
scheme to work these ports must be consecutive. When the call is made to the NAT to create a mapping for this
media it would have to do a search to find four consecutive ports and return the first of these ports and insert it into
the media field. If more than one media field exists, several of these mappings need to be created.
If NAT is unable to find enough consecutive ports, possibly because of fragmentation of the ports space managed
by NAT or maybe because all ports have been allocated already then a SIP message should be generated telling
the client that the call is not possible to connect at the moment and to try again later. Two error messages that
could be used are 486 – Busy Here and 600 – Busy Everywhere. It does not feel right that the ALG whose purpose
is to replace IP addresses and ports inside the message will also take an active part in the SIP signaling by
generating SIP messages, but no other way has been found to relay this information back to the host sending the
SIP message.
After the ALG has accomplished these two objectives it will also have to manipulate the IP header and UDP
header of the IP packet, which contained the SIP message. For outbound messages the address of the SIP ALG
should be entered as source address and source port should be set to be the default SIP port, 5060. Before the IP
packet can be sent off the new length of the packet needs to be entered and checksums needs to be recalculated.
For incoming messages the destination address and port needs to be altered to be the address and port of the
internal SIP client. The updates of the length and the check sum fields work in the same way as for the outbound
messages.
When the SIP session is terminated, the “holes” in the firewall need to be closed. This is normally done after the
ALG intercepts a BYE message from either of the clients. The implementation done in this Master’s Thesis project
counts on that BYE messages always being sent upon termination of the session. This is not a realistic situation
though. Messages might be lost on the Internet and the SIP clients may be terminated without sending the BYE
message. A draft [SessionTimer], recently submitted to the IETF, proposes a solution to solve the problem with
lost BYE messages.
To begin with, the SIP ALG parses the SIP part of the message to make sure that no illegitimate packets are let
through. If the first part is OK, then the SDP part, if it exists, is parsed for the same reason. If the message does not
pass the both of parsers, then the SIP message is silently dropped. One cannot be forgiving when enforcing a
security policy!
If this is an INVITE for a new session then a lot of state information is collected from the message. It is the
headers that constitute the call leg that are kept for later use. They are needed for finding out if other SIP messages
belong to this call leg or not. The To-, From-, and Call-ID field of this first INVITE message must also be
compared against other active SIP sessions before it can be concluded that it really is an INVITE for a new
session.
During the parsing of the message the fields containing IP addresses were found, if there are any. They are
recorded in a linked list, where each item in the list has a pointer to the first character of the IP address that had
been found. This ALG implementation works in a very crude way when it comes to the replacement of IP
addresses in outbound and inbound messages. What it did for outbound messages was only to find out which items
in the list that had the same IP address as the internal host and then replace that address with the address
configured for the ALG. If the address had a port associated with it, then the default SIP port, 5060, replaces it.
The ALG reversed this replacement as SIP messages came in. This was all very neat and very simple.
Before any enthusiastic reader starts implementing his/her own SIP ALG it must be said that the above solution
does not always work. The reason for this is that the ALG is not context aware when replacing the addresses. It is
important for the ALG to know not only from which side (external/internal) that first initiated the call, but also if
the SIP is a response or a request. An example will show why it is important for the ALG to be context aware.
Page 31 of 69
The example given here is an extension to the example given in [SIP, Section 6.40]. What has been added is the
port number in the received parameter. This port number must be there if the NAT really is a NAPT, otherwise it
will not be able to match the incoming IP packet to any mapping. In the figure below only the header fields
relevant to this example have been outlined.
The scenario here is that Bob wants to reach Adam at his home. On their separate networks they happen to use the
same private addresses (10.0.0.1) for their respective host machine.
INVITE
Bob@work Firewall/NAT Firewall/NAT Adam@home
& SIP Proxy & SIP ALG
10.0.0.1 134.138.229.17 134.138.229.18 10.0.0.1
INVITE
Via: SIP/2.0/UDP 10.0.0.1:5060
INVITE
Via: SIP/2.0/UDP sip.work.com:5060
Via: SIP/2.0/UDP 10.0.0.1:5060 ;received=134.138.229.17:1234
INVITE
Via: SIP/2.0/UDP sip.work.com:5060
Via: SIP/2.0/UDP 10.0.0.1:5060 ;received=134.138.229.17:1234
200
Via: SIP/2.0/UDP sip.work.com:5060
Via: SIP/2.0/UDP 10.0.0.1:5060 ;received=134.138.229.17:1234
What will the
client do?
200
Via: SIP/2.0/UDP sip.work.com:5060
Via: SIP/2.0/UDP 134.138.229.18:5060 ;received=134.138.229.17:1234
200
Via: SIP/2.0/UDP 134.138.229.18
ERROR!!!
When the INVITE from Bob reaches the SIP proxy, the IP packet containing the SIP message has already
traversed the NAT, which has changed the source address of the IP packet to be the globally routable address of
the firewall, 134.138.229.17. The proxy adds it own address in the top-most Via field, as it should. The proxy has
the responsibility to make sure that the address given in the Via header matches the source address in the IP
header, because it needs to be able to route the responses in the correct way. The proxy therefore adds a received
parameter to the Via field received from Bob.
The ALG validates an incoming INVITE, sees that it is an INVITE for a new session, sets up the necessary holes
in the firewall and passes it on to the internal host. In the response generated by the internal user the ALG looks
for the host’s IP address within the SIP message. It will now find the Via field with the address it is supposed to
look for and change this to the address configured to the ALG, in this case 134.138.229.18. At the proxy the
received parameter and the top-most Via field are removed and then the message will be sent to the destination
address that was given in the received parameter. The NAT will get a hold get a hold of the message and forward
it to the origination client, Bob. Bob will now receive a SIP message with a Via field that does not contain his own
address. What happens now is up to Bob’s software. Since the software knows it works as a client and not as a
server that forewords SIP messages it might chose to ignore the Via field and settle for the call leg parameters,
since the To-, From-, and the Call-ID header field match and thus accept the response given.
Page 32 of 69
The solution to this problem with IP address being exchanged when they really should not is to add more context
awareness in the SIP ALG. For the example above it would only have required a rule saying: “if this is an
outbound SIP message and it is a response then IP addresses in the Via fields should not be altered”.
The table below shows how the ALG should behave in different situations. Notice that for several of the header
fields it is clear that the ALG must know more information than just if the message came from the outside or the
inside. It must also know which client who initiated the call and if the message is a request or a response. Notice
also the Via field have the rule discussed above.
(I) The route-header contains the addresses of the SIP equipment that a request will need to pass through. It will not
contain the address of the client sending the request.
(II) Responses do not have this field
(III) Record-route cannot be used in a request where a route header is.
(IV) The internal clients will not add a record-route header to any requests. Only proxies can insert this field. There
will be none on the inside of the ALG
(V) A SIP client will never see a route field. If there was one earlier it is removed by earlier proxies.
(VI) The internal client will not add anything to the record route header, only reverse the order of the entries.
(VII) The route field will not contain any local addresses. The internal client will take the entries of the previous
record-route header, reverse the order, and last it enters the contact-field value, i.e. the address of the external
client6.
6
It could include a route field if there is a hierarchy (at least one) of SIP proxies on the private network. This
thesis does not consider this topology.
Page 33 of 69
This does not solve all the problems that can occur, unfortunately some still remain. For instance, what should the
SIP ALG do with a SIP URL looking like: ”sip:uabfrth@sprite.uab.ericsson.se:5060”? Here we have a
combination of a hostname and a port number. Since this implementation of a SIP ALG does not have the
capabilities to resolve the hostname to an IP address (this feature would of course a must in a commercial version.
One cannot have a policy of not reveling the private addresses if one goes ahead and gives the hostnames away.
This would be a strange inconsistency if it was done like this!) it will not be able to see if the hostname and the IP
address really point to the same logical entity or not. The only possible thing to do in this case is to not do any
changes at all, even if the port number is the right one. Some context awareness can of course result in that the SIP
URL was in a field that did not need to be processed.
The SDP part of the SIP messages raises a few issues concerning the SIP protocol. SIP says that it is OK not to
have a session description in the INVITE and instead include it in the ACK (if one wants to receive anything at
all). SIP also states that if a session description was included in the INVITE it is OK to change it completely by
the time the ACK is sent. It is also allowed to leave the message body empty in the ACK, indicating that the
session description session sent in the INVITE was accepted.
This many degrees of freedom make it hard for the ALG to open the right holes in the firewall and to decide at
what time these holes should be opened to the passing media streams. Given the circumstances is seems like the
solution to the latter is to wait with opening the holes until the ACK has gone through. The solution to the first part
is to introduce the concept of reserved ports. This means that the ALG can allocate ports for the media, see Section
10.1, but leave them in an inactive state (drop everything that is received on them). These ports are now said to be
reserved by the SIP ALG. The reservation is done if the INVITE contained a session description. When the ACK
is received in the ALG and the media description either matches the one in the INVITE or if the message body is
empty, then it is OK to change the state of the reserved ports to active. If new session description is entered in the
ACK then the reserved ports from the INVITE must be dropped and new ones need to be reserved instead. The
newly reserved ports need to be put in active state immediately. If the ACK instead contained a changed session
description compared to the one in the INVITE then those reserved ports that are not being used need to be
scratched and new ports need to reserved and opened for the new media.
So far we have only looked at sessions with just one client at each end, which has been sort of straight forward to
handle in the SIP ALG. Now we will have to face the fact that one single INVITE can result in at lot of responses,
each with its own media description that must be processed in the ALG. It is the concept of “forking proxies”,
mentioned in Section 6.3.6, which we will have to deal with. When an INVITE is received at a forking proxy then
if the callee is registered on several hosts it will forward the INVITE to all the places the recipient is registered on.
Page 34 of 69
Bob@home
INVITE
INVITE
Bob@work INVITE SIP Proxy SIP ALG INVITE Adam@the.beach
Bob@schrink INVITE
Bob@home 200 OK
200 OK 200 OK
200 OK SIP Proxy SIP ALG Adam@the.beach
Bob@work
200 OK
Bob@schrink
Bob@home
ACK
ACK ACK
SIP Proxy SIP ALG Adam@the.beach
Bob@work ACK
Bob@schrink ACK
The SIP ALG must in order to let the media flow from the caller to the callees create NAT mappings for all three
responses. The audio sessions set up will work like this: all three callees will hear what Adam says and Adam will
hear will hear what the callees say, but the callees will not be able hear or talk to each other. I think all participants
in this session will be very confused. It must be noted that this is likely scenario since people want to be reached
wherever they are.
This forking proxy scenario can easily become even more complex since SIP allows for changing the session
description in the ACK. Adam might decide to use a different codec for each of the answering SIP clients. I don’t
know how on earth this scenario would be done in practice, but since SIP allows it, it better be supported. The
amount of state information that has to be kept in the SIP ALG will of course increase with the number of
participants and the number of different media they are using.
Page 35 of 69
In some ways the ALG actually limits the security. The ALG cannot handle encrypted SIP messages and therefore
encryption cannot be allowed. Another thing that the ALG does not support is the concept of signed
(authenticated) messages. The ALG will always wreck the signature by messing with IP addresses and port
numbers within the message.
11 Other solutions
A protocol called Real Specific IP (RSIP), which is now being developed by the NAT working group of the IETF.
RSIP is proposing a solution to the problem with NAT and application protocols that transmits IP addresses and
ports in their messages without the need for an ALG for every protocol.
Give me an address!
Gateway
Figure 37 below, shows that the datagrams that the private host wants to send out to the public realm are
encapsulated into another IP datagram and tunneled across the private address space to the RSIP server, which
then removes the outer IP header and sends out the original datagram put together by the private host. Messages
from the public realm directed to the address allocated by the internal host are treated similarly to the outbound
datagrams. The only difference is that now the RSIP server does the encapsulation and the private host
decapsulates it. The tunnel used may be encrypted, but it does not have to be.
Tunnel
Pub_IP | Pub_TCP | data
Gateway IP | TCP [ Pub_IP | Pub_TCP | data ]
172.16.0.10
B A
Pub_IP | Pub_TCP | data IP | TCP [ Pub_IP | Pub_TCP | data ]
Internet
RSIP Server
The scenario above shows that the need for an application level gateway vanishes when RSIP is used. Since the
need for an ALG that has to be able to read the data part of the datagrams has vanished it is now also possible to
use end-to-end encryption between hosts.
Page 36 of 69
It is clear that RSIP proposes an attractive generic solution for the problems with NAT and the need ALGs, but
from what I have been able to see in [rsip-frame] and [rsip-prot] there has not yet been a solution proposed of how
sessions initiated from the public realm to internal hosts will be supported. This problem has yet to be solved. This
fact does not affect the possibility to receive inbound SIP calls, as is shown below.
When the client wishes to start a SIP session there will be a need for more public ports to be able to receive
inbound RTP streams from the other client. These will also have to be requested from the RSIP server. When the
responding client on the Internet accepts the session and sends a 200 OK, he will say in his media description
which addresses and ports he wants to receive RTP data streams on. The number of ports that have to be used for
sending him media may be greater than, equal to or less than the number of media in the INVITE. Since the ports
requested in the INVITE are opened bi-directionally, the internal client could reuse them when he wants to send to
the external client. But what would happen if the number of media streams is higher in the outbound direction?
Then the internal client would have to request additional ports from the RSIP server in order to be able to send the
last few media streams. Instead one can make the design a little bit easier and decide that the internal client will
asks the RSIP server for the amount of ports specified in the 200 OK response. These last ports are of course
intended to be used when sending to the Internet host.
Sessions started from hosts on the Internet will work just as well as for sessions started from the inside, as will be
shown below. It will be problem to reach the internal client from a host on the Internet as already mentioned in
Section 11.1. So, when the INVITE arrives at the internal SIP client, the client will have to analyze the SDP and
contact the RSIP server to open ports for the outbound RTP streams. However, this request for public ports may be
delayed (and extended by the ports used for inbound RTP data streams) until the call is accepted and the 200 OK
is created and sent. This saves one request to the RSIP server.
Both solutions have good and bad parts and it is impossible to say if one is better than the other. It might even be
possible to combine the both solutions. If a combination is implemented the ALG can then check the content of the
SIP message and make sure that it is OK, but without having to alter anything. Again, this will not work with
encrypted SIP messages.
Page 37 of 69
12 Conclusion and discussion
The goals of this Master’s thesis, outlined in Section 4.1, have been achieved. The first one was to study what was
required by the firewall in order to pass SIP signaling in and out of a private network that uses NAT. This subject
has been discussed in great detail throughout the entire paper. The second goal was to implement an application
layer gateway for SIP. How this was done is described in Section 10.
During the work with this thesis a draft for a new exiting protocol, RSIP, has been proposed to the IETF. This
protocol has attributes, which enables end-to-end encryption and authentication within SIP sessions. How RSIP
and SIP could work together was shown in Section 11.
This report has not in any way studied or proposed any solutions as to how the media streams in a SIP session can
have ensured privacy and authentication. This is an issue that remains to be solved.
The speed of the development shows that it might be too early to implement a commercial release of a SIP ALG
that supports anything other than basic SIP signaling.
Page 38 of 69
13 Acronyms
AAL5 ATM Adaptation Layer 5
ASN.1 Abstract Syntax Notation 1
ALG Application Level Gateway
ATM Asynchronous Transfer Mode
Codec Coder decoder
IP Internet Protocol
IPX Internet Packet Exchange
ISO International Organization for Standardization
ISP Internet Service Provider
DHCP Dynamic Host Configuration Protocol
HTTP Hypertext Transfer Protocol
LAN Local Area Network
LDAP Lightweight Directory Access Protocol
NAT Network Address Translation
NAPT Network Address and Port Translation
PSTN Public Switched Telephony Network
QoS Quality of Service
RFC Request For Comments
RTP Real-Time Protocol
RTCP Real-Time Control Protocol
RSIP Realm-Specific IP addressing
SDP Session Description Protocol
SIP Session Initiation Protocol
RTO Retransmission Time Out
RTT Round Trip Time
TCP Transport Control Protocol
UA User Agent
UAC User Agent Client
UAS User Agent Server
UDP User Datagram Protocol
URL Uniform Resource Locator
UTF Unicode Transfer Format
Page 39 of 69
14 Table of Figures
FIGURE 1. THE RTP HEADER................................................................................................................................7
FIGURE 2. ENCAPSULATION AND LAYER DISTRIBUTION ...........................................................................8
FIGURE 3. THE UDP HEADER ...............................................................................................................................8
FIGURE 4. THE TCP HEADER................................................................................................................................9
FIGURE 5. THE IPV4 HEADER...............................................................................................................................9
FIGURE 6. EXAMPLE OF A SESSION DESCRIPTION ......................................................................................10
FIGURE 7. SDP HEADER FIELDS........................................................................................................................10
FIGURE 8. SIP DESCRIPTION ..............................................................................................................................12
FIGURE 9. EXAMPLE OF A SIP MESSAGE.......................................................................................................12
FIGURE 10. SETTING UP A SIP SESSION ..........................................................................................................13
FIGURE 11. SIP SERVER IN PROXY MODE ......................................................................................................14
FIGURE 12. SIP SERVER IN REDIRECT MODE ................................................................................................14
FIGURE 13. THE PRINCIPLES OF A FIREWALL.............................................................................................18
FIGURE 14. SETUP AND TEAR DOWN OF A TCP CONNECTION ................................................................19
FIGURE 15. CISCO IOS ACCESS CONTROL LISTS AND LINUX IPCHAINS ................................................20
FIGURE 16. CIRCUIT LEVEL GATEWAY ..........................................................................................................20
FIGURE 17. IPV4 ADDRESSING ..........................................................................................................................22
FIGURE 18. IPV4 ADDRESS RANGE...................................................................................................................22
FIGURE 19. ABNF FOR TEXT REPRESENTATION OF IPV6 ADDRESSES....................................................23
FIGURE 20. THE IPV6 HEADER...........................................................................................................................23
FIGURE 21. THE PRIVATE ADDRESS SPACE...................................................................................................23
FIGURE 22. NETWORK ADDRESS TRANSLATION .........................................................................................24
FIGURE 23. NAT TABLE.......................................................................................................................................25
FIGURE 24. ADDRESS TRANSLATION IN PRACTICE ....................................................................................25
FIGURE 25. A NAT ROUTER WITH A POOL OF ADDRESSES........................................................................26
FIGURE 26. NETWORK ADDRESS AND PORT TRANSLATION ....................................................................27
FIGURE 27. ENCRYPTED SIP MESSAGE - WITH ENCRYPTED SIP HEADERS ...........................................28
FIGURE 28. ENCRYPTED SIP MESSAGE - WITH NO ENCRYPTED SIP HEADERS ....................................28
FIGURE 29. UPPER - THE SIP REQUEST THAT IS TO BE SENT. LOWER - THE SIGNED PARTS OF THE
MESSAGE........................................................................................................................................................29
FIGURE 30. AUTHENTICATION..........................................................................................................................29
FIGURE 31. EXAMPLES OF ORIGIN-, CONNECTION-, AND MEDIA-FIELDS .............................................30
FIGURE 32. RECEIVER TAGGED VIA FIELD....................................................................................................32
FIGURE 33. FORKING PROXY - INVITE MESSAGE.........................................................................................35
FIGURE 34. FORKING PROXY - 200 OK MESSAGE .........................................................................................35
FIGURE 35. FORKING PROXY - ACK MESSAGE .............................................................................................35
FIGURE 36. REQUEST TO AN RSIP SERVER FOR A PUBLIC ADDRESS......................................................36
FIGURE 37. TUNNELING WITH RSIP .................................................................................................................36
15 Table of Tables
TABLE 1. PAYLOAD TYPES (PT) FOR STANDARD AUDIO AND VIDEO ENCODINGS ............................11
TABLE 2. SIP RESPONSE CODES........................................................................................................................17
TABLE 3. COMPLETE LIST OF SIP RESPONSE CODES ..................................................................................17
TABLE 4. SUMMARY OF SIP HEADERS............................................................................................................18
TABLE 5. SUMMARY OF ALG ACTIONS ..........................................................................................................33
Page 40 of 69
References
[Chapman95] Building Internet Firewalls. D. Brent Chapman, Elizabeth D. Zwicky. November 1995.
[Cheswick95] Firewalls and Internet Security – Repelling the Wily Hacker. William R. Cheswick, Steven M. Bellowin. 5th
printing, April 1995.
[Cisco] http://www.cisco.com/
[Course488] Learning Tree course 488 – Deploying Internet and Intranet Firewalls. 1999
[DataBeam] A primer on the H.323 Series Standards. ≤1998
http://www.databeam.com/standards/index.html
[Goncalves00] Firewalls, a Complete Guide. Marcus Goncalves. 2000.
[Hasenstein97] Diplomarbeit, IP Network Address Translation. Michael Hasenstein, 1997.
http://www.suse.de/~mha/linux-ip-nat/diplom/nat.html
[Intel] The Problems and Pitfalls of Getting H.323 Safely through Firewalls. 1997
http://support.intel.com/support/videophone/trial21/h323_wpr.htm
[ipchains] Linux IPCHAINS-HOWTO. Paul Russell, ipchains@rustcorp.com v1.0.7, 12 March 1999.
http://www.linux.org/help/ldp/howto/IPCHAINS-HOWTO.html
[Stevens97] TCP/IP Illustrated, volume 1 – The protocols. W. Richard Stevens. 10th printing July 1997.
[Grammar] http://www.cs.columbia.edu/~hgs/sip/SIPgrammar.html
[SIPSite] http://www.cs.columbia.edu/~hgs/sip/
[SIPWhere] http://www.cs.columbia.edu/~hgs/sip/where.html
Page 41 of 69
Appendix
v=0
o=bell 53655765 2353687637 IN IP4 128.3.4.5
s=Mr. Watson, come here.
c=IN IP4 kton.bell-tel.com
m=audio 3456 RTP/AVP 0 3 4 5
Page 42 of 69
To: T. Watson <sip:watson@bell-tel.com> ;tag=37462311
Call-ID: 3298420296@kton.bell-tel.com
CSeq: 1 INVITE
Content-Length: 0
v=0
o=watson 4858949 4858949 IN IP4 192.1.2.3
s=I'm on my way
c=IN IP4 boston.bell-tel.com
m=audio 5004 RTP/AVP 0 3
Watson can only receive PCMU and GSM. Note that Watson's list of
codecs may or may not be a subset of the one offered by Bell, as
each party indicates the data types it is willing to receive. Watson
will send audio data to port 3456 at c.bell-tel.com, Bell will send
to port 5004 at boston.bell-tel.com.
Since the two sides have agreed on the set of media, Bell confirms
the call without enclosing another session description:
Page 43 of 69
B. SIP/SDP Message Grammar
Compiled by Arjun Roychowdhury and Henning Schulzrinne.
Based on RFC 2543,RFC 2327, RFC 2616, RFC 1123, RFC 2234, and RFC 2396, Sip Notes and Clarifications.
Page 44 of 69
global-phone-number = "+" 1*phonedigit [isdn-subaddress] [post-dial]
entity-header = Content-Encoding
| Content-Length
| Content-Type
request-header =
Authorization
| Contact
| Hide
| Max-Forwards
| Organization
| Priority
| Proxy-Authorization
| Proxy-Require
| Route
| Require
| Response-Key
| Subject
| User-Agent
|WWW-Authenticate
Page 45 of 69
response-header = Allow
| Authorization
| Proxy-Authenticate
| Retry-After
| Server
| Unsupported
| Warning
| WWW-Authenticate
Informational
| Success
| Redirection
Status-Code = | Client-Error
| Server-Error
| Global-Failure
| extension-code
extension-code = 3 DIGIT
Reason-Phrase = *<TEXT-UTF8, excluding CR, LF >
Informational = "100" Trying
| "180" ; Ringing
| "181" ; Call Is Being Forwarded
| "182" ; Queued
Page 46 of 69
Success = "200" ; OK
Redirection = "300" ; Multiple Choices
| "301" ; Moved Permanently
| "302" ; Moved Temporarily
| "303" ; See Other
| "305" ; Use Proxy
| "380" ; Alternative Service
Client-Error = "400" ; Bad Request
| "401" ; Unauthorized
| "402" ; Payment Required
| "403" ; Forbidden
| "404" ; Not Found
| "405" ; Method Not Allowed
| "406" ; Not Acceptable
| "407" ; Proxy Authentication Required
| "408" ; Request Timeout
| "409" ; Conflict
| "410" ; Gone
| "411" ; Length Required
| "413" ; Request Entity Too Large
| "414" ; Request-URI Too Large
| "415" ; Unsupported Media Type
| "420" ; Bad Extension
| "480" ; Temporarily not available
; Call Leg/Transaction Does Not
| "481"
Exist
| "482" ; Loop Detected
| "483" ; Too Many Hops
| "484" ; Address Incomplete
| "485" ; Ambiguous
| "486" ; Busy Here
Server-Error = "500" ; Internal Server Error
| "501" ; Not Implemented
| "502" ; Bad Gateway
| "503" ; Service Unavailable
| "504" ; Gateway Time-out
| "505" ; SIP Version not supported
Global-Failure = "600" ; Busy Everywhere
| "603" ; Decline
| "604" ; Does not exist anywhere
| "606" ; Not Acceptable
message-header = field-name ":" [ field-value ] CRLF
field-name = token
field-value = *(field-content | LWS )
Page 47 of 69
<the OCTETs making up the field-value and consisting of either *TEXT-
field-content =
UTF8 or combinations of token, separators, and quoted-string >
"q" "=" qvalue| ("action" "=" "proxy" | "redirect") |( "expires" "=" delta-
contact-params =
seconds | <"> SIP-date <">) | extension-attribute
addr-params = tag-param
tag-param = "tag="UUID
Hide = "Hide" ":" ( "route" | "hop" )
Max-Forwards = "Max-Forwards" ":" 1*DIGIT
Organization = "Organization" ":" *TEXT-UTF8
Priority = "Priority" ":"priority-value
priority-value = "emergency" | "urgent" | "normal" | "non-urgent"
Record-Route = "Record-Route" ":" 1# name-addr
Require = "Require" ":" 1#option-tag
Proxy-Require = "Proxy-Require" ":" 1#option-tag
option-tag = token
Response-Key = "Response-Key" ":" key-scheme 1*SP #key-param
key-scheme = token
key-param = token "=" ( token | quoted-string )
Page 48 of 69
"Retry-After" ":"( SIP-date | delta-seconds ) [ comment ] [ ";" "duration" "="
Retry-After =
delta-seconds ]
( "To" | "t" ) ":" (name-addr | addr-spec ) *( ";" addr-params ) *(";" // Extension params are allowed in
To =
extension-params) From to and Via
( "Via" | "v") ":" 1#(sent-protocol sent-by *( ";" via-params ) [ comment ] // Extension params are allowed in
Via =
) *(";" extension-params) From to and Via
Page 49 of 69
pgp-algorithm = "algorithm" "=" ( "md5" | "sha1" |token )
nonce = "nonce" "=" nonce-value
nonce-value = quoted-string
Authorization = "Authorization" ":" credentials
pgp-response = realm | pgp-version | pgp-signature| signed-by | nonce
pgp-signature = "signature" "="quoted-string
signed-by = "signed-by" "=" <"> URI <">
pgp-eparams = 1# ( pgp-version | pgp-encoding| pgp-key )
pgp-encoding = "encoding" "=" "ascii" | token
pgp-key = "key" "=" quoted-string
OCTET = %x00-ff ; any 8-bit sequence of data
; any US-ASCII character (octets 0 -
CHAR = %x00-7f
127)
"A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" |
upalpha =
"O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z"
"a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p"
lowalpha =
| "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z"
"(" | ")" | "<" | "" | "@" | "," | ";" | ":" | "\" | "<" | "/" | "[" | "]" | "?" | "=" | "{" |
separators =
"}" | SP | HT
Page 50 of 69
qvalue = ( "0" [ "." 0*3 DIGIT ] ) | ( "1" [ "." 0*3("0") ] )
// According to 2453:4.3 - ReqURI is
URI-reference = ([ absoluteURI | relativeURI] [ "#" fragment] ) | (sip-url ) SIPUri (without certain tokens) or
GeneralURI
absoluteURI = scheme":" ( hier_part | opaque_part )
relativeURI = (net_path | abs_path | rel_path ) [ "?" query ]
hier_part = ( net_path | abs_path ) [ "?" query ]
opaque_part = uric_no_slash *uric
uric_no_slash = unreserved | escaped | ";" | "?" | ":" | "@" | "&" | "=" | "+" | "$" | ","
net_path = "//"authority [ abs_path ]
abs_path = "/"path_segments
rel_path = rel_segment [ abs_path ]
rel_segment = 1*(unreserved | escaped | ";" | "@" | "&" | "=" | "+" | "$" | "," )
scheme = alpha *( alpha | digit | "+" | "-" | "." )
authority = server_H | reg_name
reg_name = 1*( unreserved | escaped | "$" | "," | ";" | ":" | "@" | "&" | "=" | "+" )
server_H = [ [ userinfo_H "@" ] hostport ]
userinfo_H = *( unreserved | escaped | ";" | ":" | "&" | "=" | "+" | "$" | "," )
path = [ abs_path | opaque_part ]
path_segments = segment *( "/" segment )
segment = *pchar *( ";" param )
param = *pchar
pchar = unreserved | escaped | ":" | "@" | "&" | "=" | "+" | "$" | ","
query = *uric
fragment = *uric
product = token ["/" product-version]
product-version = token
rfc1123-date = wkday "," SP date1SP time SP "GMT"
date1 = 2DIGIT SP month SP 4DIGIT ; day month year (e.g., 02 Jun 1982)
time = 2 DIGIT ":" 2 DIGIT ":" 2DIGIT ; 00:00:00 - 23:59:59
wkday = "Mon" | "Tue" | "Wed"| "Thu" | "Fri" | "Sat" | "Sun"
"Jan" | "Feb" | "Mar" | "Apr" | "May" | "Jun" | "Jul" | "Aug" | "Sep" | "Oct" |
month =
"Nov" | "Dec"
Page 51 of 69
proto-version
origin-field
session-name-field
information-field
uri-field
email-fields
announcement = phone-fields
connection-field
bandwidth-fields
time-fields
key-field
attribute-fields
media-descriptions
Page 52 of 69
key-data = email-safe | "~" | "
*( media-field
information-field
*(connection-field)
media-descriptions =
bandwidth-fields
key-field
attribute-fields )
att-field = 1*(alpha-numeric)
att-value = byte-string
multicast-address
connection-address =
| addr
repeat-interval = typed-time
Page 53 of 69
typed-time = 1*(DIGIT) [fixed-len-time-unit]
bwtype = 1*(alpha-numeric)
bandwidth = 1*(DIGIT)
DIGIT
| POS-DIGIT DIGIT
decimal-uchar = | ("1" 2*(DIGIT))
| ("2" ("0"|"1"|"2"|"3"|"4") DIGIT)
| ("2" "5" ("0"|"1"|"2"|"3"|"4"|"5"))
Page 54 of 69
Integer = POS-DIGIT *(DIGIT)
POS-DIGIT = "1"|"2"|"3"|"4"|"5"|"6"|"7"|"8"|"9"
alpha-numeric |
"'" | "'" | "-" | "." | "/" | ":" | "?" | """ |
Safe = "#" | "$" | "&" | "*" | ";" | "=" | "@" | "[" |
"]" | "^" | "_" | "`" | "{" | "|" | "}" | "+" |
"~" | "
Space = SP
Page 55 of 69
C. ABNF
Network Working Group D. Crocker, Ed.
Request for Comments: 2234 Internet Mail Consortium
Category: Standards Track P. Overell
Demon Internet Ltd.
November 1997
Copyright Notice
TABLE OF CONTENTS
1. INTRODUCTION .................................................. 2
3. OPERATORS ..................................................... 5
3.1 CONCATENATION RULE1 RULE2 ............................. 5
3.2 ALTERNATIVES RULE1 / RULE2 ................................... 6
3.3 INCREMENTAL ALTERNATIVES RULE1 =/ RULE2 .................... 6
3.4 VALUE RANGE ALTERNATIVES %C##-## ........................... 7
3.5 SEQUENCE GROUP (RULE1 RULE2) ................................. 7
3.6 VARIABLE REPETITION *RULE .................................... 8
3.7 SPECIFIC REPETITION NRULE .................................... 8
3.8 OPTIONAL SEQUENCE [RULE] ..................................... 8
3.9 ; COMMENT .................................................... 8
3.10 OPERATOR PRECEDENCE ......................................... 9
Page 56 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
7. ACKNOWLEDGMENTS ............................................... 12
8. REFERENCES .................................................... 13
9. CONTACT ....................................................... 13
1. INTRODUCTION
The differences between standard BNF and ABNF involve naming rules,
repetition, alternatives, order-independence, and value ranges.
Appendix A (Core) supplies rule definitions and encoding for a core
lexical analyzer of the type common to several Internet
specifications. It is provided as a convenience and is otherwise
separate from the meta language defined in the body of this document,
and separate from its formal status.
2. RULE DEFINITION
The name of a rule is simply the name itself; that is, a sequence of
characters, beginning with an alphabetic character, and followed by
a combination of alphabetics, digits and hyphens (dashes).
Page 57 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
Unlike original BNF, angle brackets ("<", ">") are not required.
However, angle brackets may be used around a rule name whenever their
presence will facilitate discerning the use of a rule name. This is
typically restricted to rule name references in free-form prose, or
to distinguish partial rules that combine into a string not separated
by white space, such as shown in the discussion about repetition,
below.
where <name> is the name of the rule, <elements> is one or more rule
names or terminal specifications and <crlf> is the end-of- line
indicator, carriage return followed by line feed. The equal sign
separates the name from the definition of the rule. The elements
form a sequence of one or more rule names and/or value definitions,
combined according to the various operators, defined in this
document, such as alternative and repetition.
For visual ease, rule definitions are left aligned. When a rule
requires multiple lines, the continuation lines are indented. The
left alignment and indentation are relative to the first lines of the
ABNF rules and need not match the left margin of the document.
b = binary
d = decimal
x = hexadecimal
Page 58 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
Hence:
CR = %d13
CR = %x0D
CRLF = %d13.10
Hence:
rulename = "abc"
and:
rulename = "aBc"
will match "abc", "Abc", "aBc", "abC", "ABc", "aBC", "AbC" and "ABC".
For example:
or
rulename = %d97.98.99
Page 59 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
3. OPERATORS
foo = %x61 ; a
bar = %x62 ; b
Page 60 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
then used variously among higher-level rules. The "core" rules might
be formed into a lexical analyzer or simply be part of the main
ruleset.
foo / bar
oldrule =/ additional-alternatives
ruleset =/ alt3
Page 61 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
DIGIT = %x30-39
is equivalent to:
The sequence group notation is also used within free text to set off
an element sequence from the prose.
Page 62 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
<a>*<b>element
where <a> and <b> are optional decimal values, indicating at least
<a> and at most <b> occurrences of element.
<n>element
is equivalent to
<n>*<n>element
[foo bar]
is equivalent to
*1(foo bar).
3.9 ; Comment
Page 63 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
alternation = concatenation
*(*c-wsp "/" *c-wsp concatenation)
Page 64 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
5. SECURITY CONSIDERATIONS
Page 65 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
6. APPENDIX A - CORE
CHAR = %x01-7F
; any 7-bit US-ASCII character,
excluding NUL
CR = %x0D
; carriage return
CRLF = CR LF
; Internet standard newline
DIGIT = %x30-39
; 0-9
DQUOTE = %x22
; " (Double Quote)
HTAB = %x09
; horizontal tab
LF = %x0A
; linefeed
OCTET = %x00-FF
; 8 bits of data
SP = %x20
Page 66 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
; space
VCHAR = %x21-7E
; visible (printing) characters
WSP = SP / HTAB
; white space
7. ACKNOWLEDGMENTS
The syntax for ABNF was originally specified in RFC 733. Ken L.
Harrenstien, of SRI International, was responsible for re-coding the
BNF into an augmented BNF that makes the representation smaller and
easier to understand.
This recent project began as a simple effort to cull out the portion
of RFC 822 which has been repeatedly cited by non-email specification
writers, namely the description of augmented BNF. Rather than simply
and blindly converting the existing text into a separate document,
the working group chose to give careful consideration to the
deficiencies, as well as benefits, of the existing specification and
related specifications available over the last 15 years and therefore
to pursue enhancement. This turned the project into something rather
more ambitious than first intended. Interestingly the result is not
massively different from that original, although decisions such as
removing the list notation came as a surprise.
Page 67 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
8. REFERENCES
[RFC822] Crocker, D., "Standard for the Format of ARPA Internet Text
Messages", STD 11, RFC 822, August 1982.
9. CONTACT
Page 68 of 69
RFC 2234 ABNF for Syntax Specifications November 1997
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
Page 69 of 69