End-to-End Transport Over Wireless II:

Snoop and explicit loss notification

Introduction to wireless networks design

Lecture 3
PhD. Alexander B. Hilario Tacuri

[Parts adapted from J . Kyle, B. Karp and N. Vaidya]

Today
1. Transmission Control Protocol (TCP)
– Window-based flow control
– Retransmissions and congestion control

2. TCP over Wireless

– TCP Snoop
– Explicit Loss Notification

2
Window-Based Flow Control: Motivation

• Suppose sender sends one packet, awaits ACK, repeats…

• Result: At most one packet sent, per RTT
• e.g., 70 ms RTT, 1500-byte packets à Max t’put: 171 Kbps
3
Idea: Pipeline Transmissions (Fixed Window-Based Flow
Control)

• Dispatch window of segments sequentially,

But: RTT w/o grant
idle time from awaiting ACKs
of new
– Retain packets until ACKed,
windowtrack which
to data are ACKed
arrival at receiver
– Set retransmit timer for
Even better each window
approach, used by TCP: sliding window,
extends
• When expires, as each
resend ACK segments
unACKed returns, so no idle time!
in window
4
Choosing Window Size: The Bandwidth-Delay Product
• Network bottleneck: point of slowest rate along path between sender and
receiver

• What size sender window keeps the pipe full?

• Window too small: can’t fill pipe

• Window too large: unnecessary network load/queuing/loss

5
Increasing utilization with pipelining
Data packet size L bits, bottleneck rate R bits/second
sender receiver
First bit sent, t = 0
Last bit sent, t = L / R

RTT
last bit of 2nd packet,
send ACK
ACK arrives, send
next
packet, t = RTT + L / R
The bandwidth-delay product
Data packet size L bits, bottleneck rate R bits/second
• Keep sending for time RTT = (N-1)L / R
sender receiver
N ⋅L /R
Usender = =1
RTT + L / R
( N −1) L = RTT ⋅ R
RTT
Number of bits Delay × Bandwidth
“in flight” product

Goal: window size = RTT × bottleneck rate

e.g., to achieve bottleneck rate of 1 Mbps, across a
70 ms RTT, need window size:
W = (106 bps × .07 s) = 70 Kbits = 8.75 Kbytes
TCP Packet Header

• TCP header: 20 bytes long

• Checksum covers TCP packet + “pseudo header”
– Length of TCP segment (TCP header + data)

8
TCP Header Details
• Connections inherently bidirectional; all TCP headers carry both data &
ACK sequence numbers

• 32-bit sequence numbers are in units of bytes

• Source and destination port numbers

– Multiplexing of TCP by applications
– UNIX: local ports below 1024 reserved (only root may use)

• Window field: advertisement of number of bytes advertiser willing to

accept
9
TCP: Data Transmission
• Each byte numbered sequentially (modulo 232)

• Sender buffers data in case retransmission required

• Receiver buffers data for in-order reassembly

• Sequence number (seqno) field in TCP header indicates first user

payload byte in packet

10
TCP: Receiver functionality
• Receiver indicates offered window size W explicitly to sender in
window field in TCP header
– Corresponds to available buffer space at receiver

• Receiver sends cumulative ACKs:

– ACK number in TCP header names highest contiguous byte number
received thus far, +1
– one ACK per received packet, or:
• Delayed ACK: receiver batches ACKs, sends one for every pair of data
packets (200 ms max delay)

11
TCP:
702
Sender’s Window
TCP Data Flow and Window Management

2IIHUHG:LQGRZ
Offered window(SND.WND)
(advertised by receiver)

           

6HQWDQG 6HQWDQG1RW %HLQJ6HQW &DQQRW6HQG8QWLO

$FNQRZOHGJHG $FNQRZOHGJHG 8VDEOH:LQGRZ :LQGRZ0RYHV

• Usable window at sender: &ORVHV 6KULQNV 2SHQV

/HIW(GJH 5LJKW(GJH
– Left edge advances as packets sent
(SND.UNA)
SND.NXT
(SND.UNA + SND.WND)
– Right edge advances as receive window updates arrive
Figure 15-9 The TCP sender-side sliding window structure keeps track of which sequence numbers
have already been acknowledged, which are in flight, and which are yet to be sent. The
size of the offered window is controlled by the Window Size field sent by the receiver
in each ACK.
12
Today
1. Transmission Control Protocol (TCP)
– Window-based flow control
– Retransmissions and congestion control

2. TCP over Wireless

– TCP Snoop
– Explicit Loss Notification

13
TCP: Retransmit Timeouts
• Recall: Sender sets timer for each sent packet
– Expected time for ACK to return: RTT
– when ACK returns, timer canceled
– if timer expires before ACK returns, packet resent

• TCP estimates RTT using measurements mi from timed packet/ACK pairs

– RTTi = ((1 − α) x RTTi − 1 + α x mi)

• Original TCP retransmit timeout: RTOi = β × RTTi

– original TCP: β = 2

14
Mean and Variance: Jacobson’s RTT Estimator
• Above link load of 30% at router, β × RTTi will retransmit too early!
– Response to increasing load: waste bandwidth on duplicate packets;
result: congestion collapse!

• Idea [Jacobson 88]: Estimate mean deviation vi, (EWMA of |mi – RTTi|), a
stand-in for variance:
vi = vi-1×(1-γ) + γ×|mi − RTTi|
– Then use retransmission timeout RTOi = RTTi + 4vi

Mean and Variance RTT estimator used by all modern TCPs

15
Self-Clocking Transmission

• Self-clocking transmission: Conservation of Packets

– each ACK returns, one data packet sent
– spacing of returning ACKs: matches spacing of packets in time at slowest link on
path
16
Goals in Congestion Control
1. Achieve high utilization on links; don’t waste capacity!

2. Divide bottleneck link capacity fairly among users

3. Be stable: converge to steady allocation among users

4. Avoid congestion collapse

17
Congestion Collapse
Knee

Throughput (bps)
Congestion
collapse!

Offered load (bps)

18
Congestion Requires Slowing Senders
• Bigger buffers cannot prevent congestion: senders must slow down
• Absence of ACKs implicitly indicates congestion
• TCP sender’s window size determines sending rate

• Recall: Correct window size is bottleneck link bandwidth-delay

product

• How can the sender learn this value?

– Search for it, by adapting window size
– Feedback from network: ACKs return (window OK) or do not return
(window too big)

19
Reaching Equilibrium: Slow Start
• At connection start, sender sets congestion window size, cwnd, to
pktSize (one packet’s worth of bytes), not whole window

• Sender sends up to min(cwnd, W)

– Upon return of each ACK, increase cwnd by pktSize bytes until W
reached
– “Slow” means exponential window increase!

• Takes log2(W / pktSize) RTTs to reach receiver’s advertised window size

W

20
Avoiding Congestion: Multiplicative Decrease
• Recall sender uses window of size min(cwnd, W), where W is receiver’s
advertised window

• Upon timeout for sent packet, sender presumes packet lost to

congestion, and:
1. sets ssthresh = cwnd / 2
2. sets cwnd = pktSize
3. uses slow start to grow cwnd up to ssthresh

• End result: cwnd = cwnd / 2, via slow start

21
Taking Your Fair Share: Additive Increase
• Drops indicate sending more than fair share of bottleneck
• No feedback to indicate using less than fair share

• Solution: Speculatively increase window size as ACKs return

– Additive increase: For each returning ACK,
cwnd = cwnd + (pktSize × pktSize) / cwnd
– Increases cwnd by ≈ pktSize bytes per RTT

Combined algorithm: Additive Increase,

Multiplicative Decrease (AIMD)

22
Refinement: Fast Retransmit (I)
• Sender must wait well over RTT for timer to expire before loss detected

• TCP’s minimum retransmit timeout: 1 second

• Another indicator of loss:

– Suppose sender sends: 1, 2, 3, 4, 5 (...but 2 is lost)

– Receiver receives: 1, 3, 4, 5

– Receiver sends cumulative ACKs: 2, 2, 2, 2

• Loss causes duplicate ACKs

23
Fast Retransmit (II)
• Upon arrival of three duplicate ACKs, A B
sender:

1. sets cwnd = cwnd / 2

time
2. retransmits “missing” packet
3. no slow start

• Not only loss causes dup ACKs

– Packet reordering, too

24
AIMD in Action

25
Modeling Throughput, Loss, and RTT
• How do packet loss rate and RTT affect throughput TCP achieves?

• Assume:
1. Only fast retransmits
2. No timeouts (so no slow starts in steady-state)

26
Evolution of Window Over Time
W

W/2

time

• Average window size: ¾W

• One window of packets is sent per RTT
• Bandwidth:
– ¾W packets per RTT
– (¾W x packet size) / RTT bytes per second
– W depends on loss rate…
27
Window Size Versus Loss
• Assume no delayed ACKs, fixed RTT

• cwnd grows by one packet per RTT

– So it takes W/2 RTTs to go from window size W/2 to window size W; this
period is one cycle

• How many packets sent in total, in a cycle?

– ( ¾W / RTT) x (W/2 x RTT) = 3W2/8

• One loss per cycle (as window reaches W)

– So, the packet loss rate p = 8/3W2
– W = √(8/3p)

28
Throughput, Loss, and RTT Model
• W = √(8/3p) = (4/3) x √(3/2p)

• Recall, bandwidth B = (3W/4 x packet size) / RTT

B = packet size / (RTT x √(2p/3))

• Consequences:
1. Increased loss quickly reduces throughput

2. At same bottleneck, flow with longer RTT achieves less throughput

than flow with shorter RTT!

29
Today
1. Transmission Control Protocol (TCP) primer, cont’d

2. TCP over Wireless

– TCP Snoop
– Explicit Loss Notification

30
Review: TCP on Wireless Links
• TCP interprets any packet loss as a sign of congestion
– TCP sender reduces congestion window

• On wireless links, packet loss can also occur due to random channel errors, or
interference
– Temporary loss not due to congestion
– Reducing window may be too conservative
– Leads to poor throughput

31
Review: Two Broad Approaches
1. Mask wireless losses from TCP sender
– Then TCP sender will not reduce congestion window

– Split Connection Approach

– TCP Snoop

2. Explicitly notify TCP sender about cause of packet loss

32
TCP Snoop: Introduction
• Removes most significant problem of split connection: breaking end-to-end
semantics
– No more split connection
– Single end-to-end connection like regular TCP

• TCP Snoop only modifies the AP

• Basic Idea (Downlink traffic):

– AP “snoops” on TCP traffic to and from the mobile
• Quickly retransmits packets it thinks may be lost over the wireless link

33
Snoop Protocol: High-level View
Per TCP-connection state

application application application

transport transport transport
network network network
link link link
physical physical physical

TCP connection

wireless
34
TCP Snoop: Downlink traffic case

TCP Segments

40 39 38 37

Content Wired Internet 34 Wireless Link 36

AP Mobile
Server
TCP ACKs

• AP buffers downlink TCP segments

– Until it receives corresponding ACK from mobile

• AP snoops on uplink TCP acknowledgements

– Detects downlink wireless TCP segment loss via duplicate ACKs or time-out
35
TCP Snoop Goal: Recover wireless downlink loss
• When AP detects a lost TCP segment:
– Locally, quickly retransmit that segment over the wireless link
– Minimize duplicate ACKs flowing back to server

• Goal: Content server unaware of wireless loss and retransmission

– No reduction in cwnd

36
TCP Snoop: Downlink Example
TCP Segments
35
Snoop Cache (at AP):
TCP segments seen 36
(whose ACKs have
not yet been seen) 37

38

40 39 38 37
Wired Internet 34 Wireless Link 36

TCP ACKs

37
 Downlink traffic operation, at Snoop AP

Downlink TCP segments:

Packet arrives

1. Forward packet
No 2. Reset local rexmit
New pkt?
counter
Sender rexmission
Yes

No 1. Mark as cong. loss

In-sequence?
2. Forward pkt
Congestion loss
Yes

1. Cache packet
2. Forward to
mobile
Common case

Figure 1. Flowchart for snoop_data().

. A new packet in the normal TCP sequence: This is the common case, when a new packet in the normal 38
TCP Snoop: Downlink example

35 39

36

37

38

41 40 39 38

34 36

39
Downlink traffic operation, at Snoop AP

Downlink TCP segments: Uplink TCP ACKs:

Ack arrives
1. Free buffers
2. Update RTT
Yes
New ack? estimate
3. Propagate ack
to sender
No
Common case

No
Discard Dup ack?
Spurious ack
Yes

Yes Retransmit lost

No
Discard First one? packet with high
priority
Later dup acks
for lost packet Next pkt lost

Figure 2. Flowchart for snoop_ack().

40
1. A new ACK: This is the common case (when the connection is fairly error-free and there is little user m
TCP Snoop: Downlink example

35 39

36

37

38

41 40 39 38

34 36

41
TCP Snoop: Downlink example

37 40

38

39

42 41 40 39

36 36

First
dupack

• TCP receiver does not delay duplicate ACKs (dupacks)

42
TCP Snoop: Downlink example

37 40

38 41

39

43 42 41 40

36 36 36

First Second
dupack dupack

43
Downlink traffic operation, at Snoop AP

Downlink TCP segments: Uplink TCP ACKs:

Ack arrives
1. Free buffers
2. Update RTT
Yes
New ack? estimate
3. Propagate ack
to sender
No
Common case

No
Discard Dup ack?
Spurious ack
Yes

Yes Retransmit lost

No
Discard First one? packet with high
priority
Later dup acks
for lost packet Next pkt lost

Figure 2. Flowchart for snoop_ack().

44
1. A new ACK: This is the common case (when the connection is fairly error-free and there is little user m
TCP Snoop: Downlink example

37 40

38 41

39 42

44 43 37 41

36 36 36

First Second Third

dupack dupack dupack

• Dupack triggers retransmission of packet 37 from AP

45
Downlink traffic operation, at Snoop AP

Downlink TCP segments: Uplink TCP ACKs:

Ack arrives
1. Free buffers
2. Update RTT
Yes
New ack? estimate
3. Propagate ack
to sender
No
Common case

No
Discard Dup ack?
Spurious ack
Yes

Yes Retransmit lost

No
Discard First one? packet with high
priority
Later dup acks
for lost packet Next pkt lost

Figure 2. Flowchart for snoop_ack().

46
1. A new ACK: This is the common case (when the connection is fairly error-free and there is little user m
TCP Snoop: Downlink example

37 40 43

38 41

39 42

45 44 42 37

36 36
Discard
nd
2 dupack

36

47
TCP Snoop: Downlink example

37 40 43

38 41 44

39 42

46 45 43 42

36 41
Discard
rd
3 dupack

36 36

48
TCP Snoop: Downlink example

37 40 43

38 41 44

39 42 45

47 46 44 43

41
Discard
th
4 dupack

36 36 36

• TCP sender does not fast retransmit

49
TCP Snoop: Downlink example

42 45

43 46

44

48 47 45 44

41 43

36 36 36

50
Uplink traffic case

TCP Segments

40 39 38 37

Wireless Link 34 Wired Internet 36

Mobile AP Server
TCP ACKs

• Less-common case but becoming more prevalent

• Buffer & retransmit TCP segments at AP? Not likely useful

• Run Snoop agent on the Mobile? Not likely useful

51
Negative ACKs: Recovering uplink loss

TCP Segments

40 39 38 37

Wireless Link 34 36

TCP ACKs

• AP detects wireless uplink loss via missing sequence numbers

• AP immediately sends L2 negative ACK (NACK) to mobile

– Mobile quickly & selectively retransmits data

– Requires modification to AP and mobile’s link layer

52
Snoop TCP: Advantages
• Downlink works without modification to mobile or server

• Preserves end-to-end semantics. Crash does not affect correctness, only

performance.

• After an AP handoff: New AP needn’t Snoop TCP

– Can automatically fall back to regular TCP operation

– No state need be migrated (but if done, can improve performance)

– Note such “state” is called soft state

• Good if available, but correct functionality otherwise

53
Two Broad Approaches
1. Mask wireless losses from TCP sender
– Then TCP sender will not reduce congestion window

– Split Connection Approach

– TCP Snoop

2. Explicitly notify TCP sender about cause of packet loss

54
Explicit Loss Notification (ELN)
• Notify the TCP sender that a wireless link (not congestion) caused a certain
packet loss

• Upon notification, TCP sender retransmits packet, but doesn’t reduce

congestion window

• Many design options:

– Who sends notification? How is notification sent? How is notification
interpreted at sender?

55
ELN for uplink TCP traffic

• AP keeps track of gaps in the TCP packet sequence received from the
mobile sender

Gap, size 1
at seqno 2
TCP Segments

4 3 2 1 4 3 1
Wireless Link TCP ACKs
Mobile Server

56
ELN for uplink TCP traffic
• When AP sees a dupack:
– AP compares dupack seqno with its recorded gaps
• If match: AP sets ELN bit in dupack and forwards it

• When mobile receives dupack with ELN bit set:

– Resends packet, but doesn’t reduce congestion window

Gap, size 1
TCP Segments at seqno 2
4 3 2 1 4 3 1
1 1 1 1
Mobile Server
Dupack with ELN set
57
 Next Class
Lab 1: Build a spectrum analyzer (Holiday)

Next Lecture (05/09)

Link Layer I: Time, Frequency,
and Code Division

58