Case Study on troubleshooting BGP flaps over IPSec using Frame-Freer tracking

Problem Description:

AOL reported random BGP session flaps over IPsec between two M10i’s.

Topology:

IPSec tunnel are shown as red dotted lines over ISP.

Configuration

Plain IPsec configuration with:-

o Default tunnel mtu of 1500

o No BGP TCP MSS set
o DF bit is set as per default
ipsec-vpn {
rule MTC {
term one {
then {
remote-gateway 64.12.123.41;
dynamic {
ike-policy IKE-POLICY;
ipsec-policy IPSEC-POLICY;
}
tunnel-mtu 1417;
}
}
match-direction input;
}
rule DTC {
term one {
then {
remote-gateway 205.188.123.28;
dynamic {
ike-policy IKE-POLICY;
ipsec-policy IPSEC-POLICY;
}
tunnel-mtu 1417;
}
}
match-direction input;
}
rule MDC2-SYG {
term one {
then {
remote-gateway 206.222.230.6;
dynamic {
ike-policy IKE-POLICY;
ipsec-policy IPSEC-POLICY;
}
}
}
match-direction input;
}
ipsec {
proposal IPSEC {
protocol esp;
authentication-algorithm hmac-md5-96;
encryption-algorithm 3des-cbc;
}
policy IPSEC-POLICY {
proposals IPSEC;
}
}
ike {
proposal IKE {
authentication-method pre-shared-keys;
dh-group group1;
authentication-algorithm md5;
encryption-algorithm 3des-cbc;
}
 policy IKE-POLICY {
mode main;
proposals IKE;
}
}
establish-tunnels immediately;
}
}

Troubleshooting

o Enabled Frame-Freer-tracking on services pic

sp10(atr8)# set ser frame-freer-tracking on

Frame freer tracking would basically would help us which function /process of Services is dropping the
packets.

o Print the output of ‘frame-free-tracking’

sp10(atr8)# show ser frame-freer-tracking

File Line Count

-----------------------------------------------------------------------------

../common/applications/svcs/fwnat/fwnat_process.c 227 26

../common/applications/svcs/ipsec/ipsec_data_process.c 459 244

sp10(atr8)# show ser frame-freer-tracking

File Line Count

-----------------------------------------------------------------------------

../common/applications/svcs/fwnat/fwnat_process.c 227 26

../common/applications/svcs/ipsec/ipsec_data_process.c 459 247

As seen above, counter for line number 459 of ipsec_data_process.c gets incremented.
Let’s see what’s there in line 459 of ipsec_data_process.c. JUNOS used is 8.3R3.4

/8.3R3.4/src/juniper/pfe/common/applications/svcs/ipsec/ipsec_data_process.c

457 PREPROCESS_DONE:
458 START_PROCESS_ERROR:
459 sf_free((Frame *)ipf);
460 return(IPSEC_COMPLETE);
461
462 START_PROPCESS_NEED_FRAG:
463 return(IPSEC_NEED_FRAG);
Line 459, calls the pointer function ‘sf_free’. The sf_free would clean up the memory blocks used by the
IPSec packet and drop the packet. The packet is dropped as the result START_PROCESS_ERROR
being called.

From here, we understand the packet is dropped due to START_PROCESS_ERROR function. Now, let’s
see under what circumstances this would be called.

ipsec_data_process.c 230 goto START_PROCESS_ERROR;

261 goto START_PROCESS_ERROR;
355 goto START_PROCESS_ERROR;
366 goto START_PROCESS_ERROR;
374 goto START_PROCESS_ERROR;

Hmm…there are many lines calling START_PROCES_ERROR.

Lets looks for pointers on other stats that would help.

sp10(atr8)# show services ipsec stats total

ESP Counters:

clear_bytes_sent: 533891474709

clear_bytes_recvd: 3006595054821

protected_bytes_sent: 544476412456

protected_bytes_recvd: 3030019310200

packets_sent: 1556401415

packets_recvd: 3837987853

AH Counters:

clear_bytes_sent: 0

clear_bytes_recvd: 0

protected_bytes_sent: 0

protected_bytes_recvd: 0

packets_sent: 0

packets_recvd: 0

ESP Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 200
rplDuplicate: 7

rplZero: 0

AH Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 0

rplDuplicate: 0

rplZero: 0

----------------------------------------------

esp_seq_num: 0

ah_seq_num: 0

auth_failed: 0

replay_errors: 0

ipsec_bad_headers: 0

esp_bad_trailers: 0

decrypt_errors: 0

tunnel_rcv_errors: 0

ipsec_frags: 0

ipsec_frag_errors: 0

ipsec_defrags: 0

ipsec_defrag_errors: 0

auth_sent: 1556401415

auth_successful: 3837987646

number of encrypts: 1556401415

number of decrypts: 3837987646

esp_auth_failed: 0

rejected DF packets from local side: 1278086

rejected DF packets from remote side: 0

Total Drops: 0

sp10(atr8)# show services ipsec stats total

ESP Counters:

clear_bytes_sent: 533891586144

clear_bytes_recvd: 3006598787904

protected_bytes_sent: 544476531184

protected_bytes_recvd: 3030023064544

packets_sent: 1556402438

packets_recvd: 3837991922

AH Counters:

clear_bytes_sent: 0

clear_bytes_recvd: 0

protected_bytes_sent: 0

protected_bytes_recvd: 0

packets_sent: 0

packets_recvd: 0

ESP Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 200

rplDuplicate: 7

rplZero: 0

AH Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 0

rplDuplicate: 0
rplZero: 0

----------------------------------------------

esp_seq_num: 0

ah_seq_num: 0

auth_failed: 0

replay_errors: 0

ipsec_bad_headers: 0

esp_bad_trailers: 0

decrypt_errors: 0

tunnel_rcv_errors: 0

ipsec_frags: 0

ipsec_frag_errors: 0

ipsec_defrags: 0

ipsec_defrag_errors: 0

auth_sent: 1556402438

auth_successful: 3837991715

number of encrypts: 1556402438

number of decrypts: 3837991715

esp_auth_failed: 0

rejected DF packets from local side: 1278086

rejected DF packets from remote side: 0

Total Drops: 0

sp10(atr8)# show ser ipsec stats total

ESP Counters:

clear_bytes_sent: 533894458814

clear_bytes_recvd: 3006667546824

protected_bytes_sent: 544479531288

protected_bytes_recvd: 3030092266408
packets_sent: 1556421387

packets_recvd: 3838067394

AH Counters:

clear_bytes_sent: 0

clear_bytes_recvd: 0

protected_bytes_sent: 0

protected_bytes_recvd: 0

packets_sent: 0

packets_recvd: 0

ESP Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 200

rplDuplicate: 7

rplZero: 0

AH Replay:

rplBeyondWindow: 0

rplOutOfOrder: 0

rplBeforeWindow: 0

rplDuplicate: 0

rplZero: 0

----------------------------------------------

esp_seq_num: 0

ah_seq_num: 0

auth_failed: 0

replay_errors: 0

ipsec_bad_headers: 0

esp_bad_trailers: 0
decrypt_errors: 0

tunnel_rcv_errors: 0

ipsec_frags: 0

ipsec_frag_errors: 0

ipsec_defrags: 0

ipsec_defrag_errors: 0

auth_sent: 1556421387

auth_successful: 3838067187

number of encrypts: 1556421387

number of decrypts: 3838067187

esp_auth_failed: 0

rejected DF packets from local side: 1278111

rejected DF packets from remote side: 0

Total Drops: 0

o From the above two captures, could see ‘rejected DF packets from local side’ counter incremented.

Let’s see what variable ‘rejected DF packets from local side’ corresponds to:-

459 u_int32_t l_pmtu; /* # times we rejected a DF packet from local side */

The variable here is l_pmtu. Let’s look when this counter would be incremented. We need search for
something like l_pmtu ++ ( l_pmtu = l_pmtu +1)

115 /* Handle the MTU exceeded conditions. */

116
117 if (sadb_ptr->mtu < outer_buf_size) {
118 int diff = outer_buf_size - inner_pkt_len;
119 int new_mtu = sadb_ptr->mtu - diff - 2;
120
121 if (can_frag) {
122 /* Fragment the packet as DF bit is not set or cleared. */
123
124 ipf->info_new_mtu = new_mtu;
125 return (IPSEC_NEED_FRAG);
126 } else {
127 /* Send the ICMP error if the DF bit is set. */
128
129 new_mtu &= ~0x7;
 130 ipsec_icmp_error(sadb_ptr, rx_ptr, ICMP_UNREACH,
131 ICMP_ERROR_DFSET, new_mtu);
132 if (ipsec_debug->level >= IPSEC_DEBUG_LEVEL_MEDIUM) {
133 jsyslog (LOG_DEBUG, "Sending ICMP_UNREACH : %d "
134 "NEW MTU : %d\n", outer_buf_size,new_mtu);
135 }
136 stats->l_pmtu++;
137 return (-1);

As seen above, l_pmtu is incremented at line number 136. On reading the code above, this counter is
incremented when the packet size exceed the tunnel mtu and when unable to fragment. The RETURN of
-1 is called. The -1 indicates ERROR condition.

“mtu < outer_buf_size”

mtu -- > Configured tunnel mtu.

outer_buf_size -- > Is the projected packet size after encryption and authentication based on the IPSec
properties defined.

68 #ifdef IPSEC_PROCESS_AUTH
69 int ah_auth_size = esp_crypt_size + ip_ipsec_size +ESP_CHECKSUM_SIZE;
70 int outer_buf_size = ah_auth_size;
71 #else
72 int ah_auth_size = esp_crypt_size + ip_ipsec_size;
73 int outer_buf_size = ah_auth_size;
74 #endif

In IPSec, we take an incoming "clear" packet and place it inside a second packet called the outer IPSec
packet. The inside packet is then encrypted and authenticated as configured. The outer ipsec has
source and destination of the two tunnel end-points and a size of the original inner packet size
(inner_pkt_len) plus the size of the outer IPSec packet. The exact size of the IPSec overhead depends
on exactly what encryptions and authentication is preformed.

With the help l_pmtu counter stats, looks like STAR_PROCESS_ERROR (from frame-freer-tracking )
was called as the result of :

373 if ( (ret = sa_ptr->hwaccel.start_func(sp, (Frame *)ipf)) < 0) {

374 goto START_PROCESS_ERROR;

215 ip_ptr = IP_PACKET(ipf->comm_frame.payload);

216
217 if (ICMP_ERROR_MESSAGE(ICMP_TYPE(ip_ptr))) {
218
219 ret = handle_icmp_error(frame, sadb_ptr);
 220
221 /*
222 * ret > 0 : frame is dispatched in the clear
223 * ret < 0 : frame to be dropped as the dest doesnot match SA selectors
224 * ret = 0 : frame is to be tunneled, follow the normal path
225 */
226
227 if (ret > 0) {
228 goto START_PROCESS_DONE;
229 } else if (ret < 0) {
230 goto START_PROCESS_ERROR;
231 }
232
233 /*
234 * the frame is to be tunneled, so flow through as a normal frame.
235 */
236 }
237 }

If the BGP packet size + ipsec overhead is more than tunnel-mtu, we send drop the BGP packet and
send an ICMP PTMU message requesting for a lower MSS.

How to approach this issue:-

We can approach this problem two ways. The first one is preferable from an IPSec performance
perspective. You cap the size of the inner packet so that when you stuff that inner packet inside the
IPSec packet the overall size isn't more than the tunnel MTU. In this case the IPSec packet ( with the
original packet inside) will travel to the other end of the tunnel where it's received by the IPSec process
running on the remote end, it's decrypted, the original packet is recovered and sent along it's way.

So example: if we set tcp-mss to 1400b, IPSec adds say 64b, the total size of the packet is 1464b which
is lower than the 1500b tunnel and interface mtu, the packet is encapsulated by IPSec set to the other
end, and decapped.

The second option is to RAISE the tunnel MTU. For this example we'll just jump right to the example.
We can raise the tunnel MTU to 1600b. IPSec takes the 1500b packets stuffs it into IPSec packet
adding an addtional 64b. IPSec sends this new IPSec packet to the other end of the tunnel. Ipsec
doesn't have a problem with this packet because the packet size is less than the tunnel mtu. However,
this 1564b packet is too large to go out over a gige link so the gige interface fragments the packet before
sending it out (this is OK because the IPSec packet doesn't have the DF bit set the packet inside the
IPSec packet did but that is a different packet). The fragments arrive at the remote end of the tunnel
were IPSec reassembles the fragments, forming the orginal 1564b packet and decapsulates the original
1500b which is again sent on its way.