BRKSEC-3007 - Advanced Cisco IOS Security Features (2010 Las Vegas)

Troubleshooting Cisco IOS Security Features
BRKSEC3007
Agenda
Troubleshooting Cisco IOS Firewall
Cisco IOS Firewall Overview Cisco IOS Firewall Packet Flow Cisco IOS Firewall Troubleshooting Common Issues and Resolutions Summary
Zone Based Firewall Troubleshooting Example Troubleshooting Cisco IOS Intrusion Prevention System
Cisco IOS IPS Overview Cisco IOS IPS Packet Flow Cisco IOS IPS Troubleshooting Common Issues and Resolutions Summary
Presentation_ID 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public
What is Not Covered

Troubleshooting Firewalls on PIX/ASA and FWSM
BRKSEC-3020: Advanced Firewalls
IPS Appliance Troubleshooting

BRKSEC-3030: Advanced Intrusion Prevention Systems
VPN
BRKSEC-3011: Troubleshooting GET VPN
BRKSEC-3012: Troubleshooting DMVPN NRLSEC-3013: Troubleshooting Remote Access SSL VPN
Presentation_ID
2010 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Cisco IOS Firewall Overview
Presentation_ID
Cisco Public
Zone-Based Policy Firewall Overview

Allows grouping of physical and virtual interfaces into zones Firewall policies are applied to traffic traversing zones Simple to add or remove interfaces and integrate into firewall policy
Supported Features
Stateful inspection Application inspection: IM, POP, IMAP, SMTP/ESMTP, HTTP URL filtering
12.4(6)T
Per-policy parameter
Transparent firewall VRF-aware firewall
Private-DMZ Policy DMZ-Private Policy
DMZ Public-DMZ Policy
Trusted
E0
S0
Internet
Untrusted
Private-Public Policy
Zone-Based Policy Firewall Configuration

class-map type inspect match-any myprotocol match protocol smtp match protocol ftp match protocol http class-map type inspect match-all myclass match access-group 102 match class-map myprotocol policy-map type inspect mypolicy class type inspect myclass inspect zone security private zone security public
Define services inspected by policy
Services with ACL to define permitted/denied hosts (Optional)

Define firewall action for traffic
Setup zones
zone-pair security priv-pub source private destination public service-policy type inspect mypolicy interface Ethernet0 zone-member security private interface Serial0 zone-member security public access-list 102 permit ip 192.168.0.0 0.0.255.255 any
Establish zone-pair & apply the policy Assign interfaces to zones
Presentation_ID
Cisco Public
Cisco IOS Firewall Packet Flow
Presentation_ID
Cisco Public
Understanding the Packet Flow

End-to-end packet path must be identified
Narrow down the issue to the device level
Determine the packet flow based on SRC IP, DST IP, SRC port, DST port, and protocol
Determine the interfaces/zones through which the flow passes Then perform a systematic walk of the packet flow through the device based on feature configured
Source Address:a.b.c.1 Destination Address:d.e.f.1 Source Port: xxxx Destination Port:yyy Protocol: UDP Source Interface: Fa 0/0 Destination Interface: Fa 1/0 Flow is narrowed to 2 interfaces only
IP S: a.b.c.1 D: d.e.f.1 Proto: 17 (udp) UDP -- S: xxxx -- D: yyy
Packet Flow
Packet Flow
PAYLOAD
interface Fa 0/0
Presentation_ID
interface Fa Cisco Public 2/0
interface Fa 1/0
8
General Packet Flow

Inbound ACL Input Int NAT Before Routing Routing NAT After Routing Stateful IPS Output Int
Auth Proxy
Fragment Inspection
Stateless IPS Input Int
Outbound ACL Output Int
Decrypt Packet N IPSec Pkt? Y Inbound Input ACL Encrypt Packet Y IPSec Pkt? N
IOS FW
Stateless IPS Input Int
Cisco IOS Firewall Troubleshooting
Presentation_ID
Cisco Public
10
The problem solving Process

Assess
Whats going on Prioritize
Ask the right questions to better define and clarify the problem
Acquire
What information do we need but we dont have? How to get that information?
Analyze
Understand the flow Whats supposed to happen vs. What actually happened
Act
Test assumptions Deploy changes
11
IOS Firewall Troubleshooting Tools

Syslog
Show commands
Packet capture Debug commands
Presentation_ID
Cisco Public
12
Syslog
Most effective troubleshooting tool available for Zone-Based Policy Firewall Tool for alert and audit trail Tool to help identify packet dropped by the firewall Tool for capturing the debug command output Use of syslog server strongly recommended when deploying firewall solutions
Presentation_ID
Cisco Public
13
SyslogDissection of a Syslog Message

Symptom: An user complains that he is unable to browse to an web server at 172.16.1.100
Cause of the reset EC-SUN[100]# grep "172.16.1.100"
Jul 26 13:58:16 200.1.1.1 2167: Jul 26 18:02:34.907 UTC: %APPFW-4-HTTP_JAVA_APPLET: HTTP Java Applet detected resetting session 172.16.1.100:80 10.1.1.100:3372 on zonepair publicPrivateOut class myClassMap appl-class HttpAic
Name of the Zone-Pair

Presentation_ID
Class-map name
2010 Cisco and/or its affiliates. All rights reserved. Cisco Public
AIC Policy Name

14
CBAC
SyslogCheck for Packet Drops

Configure ip inspect log drop-pkt to help identify packet dropped by the Firewall and drop reason Feature introduced in 12.3(8)T Rate limited at 30 seconds intervals
Router(config)#ip inspect log drop-pkt Router# ... *Mar 25 19:21:27.811: %FW-6-DROP_PKT: Dropping tcp session 1.1.1.20:0 2.1.1.2:0 due to Invalid Header length with ip ident 7205 ... *Mar 25 19:30:23.131: %FW-6-DROP_PKT: Dropping tcp session 1.1.1.20:59807 2.1.1.2:23 due to RST inside current window with ip ident 14992 tcpflags 0x5004 seq.no 7916131 ack 1538156964
15
SyslogCommon Packet Drop Reasons

Invalid Header length The datagram is so small that it could not contain the layer 4 TCP, Universal Computer Protocol (UCP), or Internet Control Message Protocol (ICMP) header Non-initial TCP segment is received without a valid session. The packet contains an invalid TCP sequence number. The packet contains an invalid TCP acknowledgement number. A synchronization packet is seen within the window of an already established TCP connection. The TCP packet received is out of order. A TCP segment is received that should not have been received through the TCP state machine such as a TCP SYN packet being received in the listen state. The TCP responder proposes an illegal window scale option when the initiator does not offer the window scale option A reset (RST) packet is observed within the window of an already established TCP connection.
Segment matching no TCP connection Invalid Seq# Invalid Ack (or no Ack) SYN inside current window Out-Of-Order Segment Stray Segment
Invalid Window scale option RST inside current window
SYN with data or with PSH/URG flags

Presentation_ID
TCP SYN packet is seen with data.

16
Syslog alert and audit-trail

Check the syslog for firewall alerts that may indicate potential hostile events
*Jun 26 04:05:59.803: %FW-4-HOST_TCP_ALERT_ON: Max tcp half-open connections (10) exceeded for host 2.1.1.2 *Jun 26 04:07:04.347: %FW-4-ALERT_ON: getting aggressive, count (101/100) current 1-min rate: 173
*Jun 26 04:07:04.347: %FW-4-ALERT_OFF: calming down, count (99/100) current 1-min rate: 173
Audit-trail for session establishment and tear down

*Jun 26 03:47:36.879: %FW-6-SESS_AUDIT_TRAIL_START: Start tcp session: initiator (1.1.1.2:11081) -- responder (2.1.1.2:23) *Jun 26 03:47:52.843: %FW-6-SESS_AUDIT_TRAIL: Stop tcp session: initiator (1.1.1.2:11081) sent 63 bytes -- responder (2.1.1.2:23) sent 96581 bytes
17
Show Commands
Use to display the configuration, and connections statistics information MOST of the problem can be diagnosed with the Syslog & Show commands Show commands are different for Classic Cisco IOS Firewall and Zone-Based Policy Firewall
Presentation_ID
Cisco Public
18
Show CommandsZone-Based Firewall

To display zone and member interfaces
show zone security [zone-name]
To display zone-pair information

Router#show zone-pair security source private destination public Zone-pair name priv-pub source-Zone private Destination-Zone public service-policy priv-pub-pol
Show policy stats and session

show policy-map type inspect { <policy name> [class <class name>] | zone-pair [<zone-pair name>] [sessions | urlfilter cache] }
Presentation_ID
Cisco Public
19
Show Commands - Zone-Based Firewall

To display the firewall statistics
Router# show policy-map type inspect zone-pair policy exists on zp priv-pub Zone-pair: priv-pub Service-policy inspect : firewall-pmap Class-map: L4-inspect-class (match-any) Match: protocol tcp 1 packets, 24 bytes 30 second rate 0 bps Inspect Packet inspection statistics [process switch:fast switch] tcp packets: [44:0] Session creations since subsystem startup or last reset 1 Current session counts (estab/half-open/terminating) [1:0:0] Maxever session counts (estab/half-open/terminating) [1:1:0] Last session created 00:00:40 Last statistic reset never Last session creation rate 1 Maxever session creation rate 1 Last half-open session total 0 Class-map: class-default (match-any) Match: any Drop 0 packets, 0 bytes
20
Show Commands - Zone-Based Firewall
To display the Firewall sessions

Router# show policy-map type inspect zone-pair sessions policy exists on zp priv-pub Zone-pair: priv-pub Service-policy inspect : firewall-pmap Class-map: L4-inspect-class (match-any) Match: protocol tcp 1 packets, 24 bytes 30 second rate 0 bps Inspect Number of Established Sessions = 1 Established Sessions Session 5346C90 (1.1.1.20:44181)=>(2.1.1.2:23) tcp SIS_OPEN Created 00:09:22, Last heard 00:09:17 Bytes sent (initiator:responder) [46:119] Class-map: class-default (match-any) Match: any Drop 0 packets, 0 bytes
Presentation_ID
Cisco Public
21
How to use packet captures for troubleshooting firewall issues?
Typical problem scenario: Application x failing when going through the firewall Capture Server
Internet
Capture
Client
Inside
Outside
Setup the capture filter for the flow in question Start packet capture on both inside and outside of the firewall
Start the application thats failing

Compare the packet captures to look for packet drops and match that up with the firewall logs
Presentation_ID
Cisco Public
22
Using IOS Embedded Packet Captures

Key configuration steps
Create the capture buffer and capture point Associate the capture point to the buffer Start/stop the capture
Router#monitor capture buffer test-buffer Router#monitor capture buffer test-buffer filter access-list 120 Filter Association succeeded Router# Router#monitor capture point ip cef test-capture serial 2/0 both *Mar 26 20:33:10.896: %BUFCAP-6-CREATE: Capture Point test-capture created. Router#monitor capture point associate test-capture test-buffer Router#monitor capture point start test-capture *Mar 26 20:34:03.108: %BUFCAP-6-ENABLE: Capture Point test-capture enabled. Router# Router#monitor capture point stop test-capture *Mar 26 20:34:21.636: %BUFCAP-6-DISABLE: Capture Point test-capture disabled.
23
Using IOS Embedded Packet Captures
Dump the packet on the router itself
Now we have the packets captured, whats next?

Router# show monitor capture buffer test-buffer dump 15:34:07.228 EST Mar 26 2009 : IPv4 LES CEF : Se2/0 None 05CECE30: 0F000800 45C0002C ....E@., 05CECE40: 6D170000 FE0649DD 02010102 01010114 m...~.I]........ 05CECE50: 0017A353 0FB6B952 3EF1499C 60121020 ..#S.69R>qI.`.. 05CECE60: 917A0000 02040218 00 .z....... . .
Or export it out and analyze it in Ethereal/Wireshark
Router# monitor capture buffer test-buffer export ? ftp: Location to dump buffer http: Location to dump buffer https: Location to dump buffer rcp: Location to dump buffer scp: Location to dump buffer tftp: Location to dump buffer
24
IPSec and Cisco IOS Firewall

Problem Statement:
How IPSec works/interacts with IOS Firewall
Solutions:
IOS Firewall works with IPSec in one of the two ways: IOS Firewall and IPSec enabled on the same router
IOS FW does packet inspection on the decrypted packets for inbound traffic IOS FW does packet inspection before encryption for outbound traffic
IOS Firewall for IPSec pass-through traffic

IOS FW will not inspect encrypted IPSec packets as the protocol number in the IP header is not TCP or UDP ISKMP which is UDP/500 will be inspected Router needs to allow UDP/500 (ISKMP) UDP/4500 (NAT-T), IP 50 (ESP)/ IP 51 (AH) for IPSEC
Presentation_ID
Cisco Public
25
IPSec and Zone-Based-Firewall

Two types of IPSec configuration
Non-VTI based Classic configuration with crypto map applied to an interface

Interface-based IPSec configuration
GRE over IPSec
DMVPN
Static VTI (Virtual Tunnel Interface) EzVPN using Dynamic VTI
Using VPN with Zone-Based Policy Firewall

http://www.cisco.com/en/US/prod/collateral/vpndevc/ps5708/ps5710/ps101 8/prod_white_paper0900aecd8062a909.html
Presentation_ID
Cisco Public
26
Classic IPSec with ZBF

Clients
Server
Zone Private Zone Public
Internet Traffic (TCP/UDP/ICMP) Clients IPSec Tunnel
Web server 192.168.1.0/24
R1 Internet
R2
192.168.2.0/24
Define the zone security policies

Source Zone Destination Zone
Private N/A Allow TCP/UDP/ICMP traffic from the tunnel, and Web traffic to server 192.168.1.10
Public Allow all outbound TCP/UDP/ICMP traffic N/A

27
Private
Public
Presentation_ID
Cisco Public
Classic IPSec with ZBF - Configuration

class-map type inspect match-any all-traffic match protocol tcp match protocol udp match protocol icmp class-map type inspect match-all pub-pri-cmap match class-map all-traffic match access-group name tunneltraffic class-map type inspect match-all inbound-web match protocol http match access-group name web-server ! policy-map type inspect pri-pub-pmap class type inspect all-traffic inspect policy-map type inspect pub-pri-pmap class type inspect pub-pri-cmap inspect class type inspect inbound-web inspect
Presentation_ID 2010 Cisco and/or its affiliates. All rights reserved.
zone security public description Internet facing zone zone security private description Secure private zone zone-pair security pub-pri source public destination private service-policy type inspect pub-pri-pmap zone-pair security pri-pub source private destination public service-policy type inspect pri-pub-pmap ! interface FastEthernet0/0 zone-member security public crypto map test ! interface FastEthernet1/0 zone-member security private ! ip access-list extended tunnel-traffic permit ip 192.168.2.0 0.0.0.255 192.168.1.0 0.0.0.255 ip access-list extended web-server permit ip any host 192.168.1.10
Cisco Public
28
Interface-based IPSec with ZBF

Clients
Server
Zone Private
Zone Public
Internet Traffic (TCP/UDP/ICMP) Clients
Web server
192.168.1.0/24
IPSec Tunnel R1 R2 Internet
Zone VPN
Define the zone security policies

Destination Source Zone Zone
192.168.2.0/24
Private N/A
Allow Web traffic to 192.168.1.10
Public Allow all TCP/UDP/ICMP N/A Deny

Cisco Public
VPN Allow all TCP/UDP/ICMP Deny N/A

29
Private Public VPN

Presentation_ID
Allow All TCP

Interface-based IPSec with ZBF Configuration

class-map type inspect match-any tcptraffic match protocol tcp ! policy-map type inspect pri-pub-pmap class type inspect all-traffic inspect policy-map type inspect pub-pri-pmap class type inspect inbound-web inspect policy-map type inspect pri-vpn-pmap class type inspect all-traffic inspect policy-map type inspect vpn-pri-pmap class type inspect tcp-traffic inspect ! zone security public description Internet facing zone zone security private description Secure private zone zone security vpn description This is the VPN zone zone-pair security pub-pri source public destination private service-policy type inspect pub-pri-pmap zone-pair security pri-pub source private destination public service-policy type inspect pri-pub-pmap zone-pair security vpn-pri source vpn destination private service-policy type inspect vpn-pri-pmap zone-pair security pri-vpn source private destination vpn service-policy type inspect pri-vpn-pmap ! interface Tunnel0 zone-member security vpn tunnel mode ipsec ipv4 tunnel protection ipsec profile test ! interface FastEthernet0/0 zone-member security public ! interface FastEthernet1/0 zone-member security private
Presentation_ID
Cisco Public
30
Common Issues and Resolutions
Presentation_ID
Cisco Public
31
Performance Degrades
Symptom:
After turning on IOS Firewall, the connection is very Slow Valid Packet Drops after a while of turning the Firewall ON
Troubleshooting Steps:
Step1: Check & investigate which process utilizes MAXIMUM CPU
Router# show processes cpu | exclude 0.00
CPU utilization for five seconds: 70%/39%; one minute: 52%; five minutes: 43% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
74 84 120 1388 983836 24468 31823 305327 3070 43 3222 7970 0.08% 38.18% 1.22% 0.04% 37.74% 1.27% 0.04% 37.02% 1.26% 0 EAPFramework 0 IP Input 0 Inspect process
Solution:
IP Input process is expected to be higher than any process If any process > IP Input process, need investigation of that process, may e0 s0 not be related to IOS Firewall If IP Input process is HIGH, it could be related to IOS Firewall
Public Network
Presentation_ID
Cisco Public
32
Performance Degrades (Cont.)

Zone-Based Policy Firewall DoS Protection
Every class-map configured with the "inspect" action in a policy-map carries its own set of DoS protection counters
Counters of the number of "half-open" TCP and UDP connections Total connection rate through the firewall and IPS software
Each class-map's DoS protection is individually configurable with a parameter-map that modifies the DoS protection values The legacy default settings prior to Release 12.4(11)T may interfere with proper network operation if they are not configured for the appropriate level
Presentation_ID
Cisco Public
33
Performance Degrades ZBF

Step2: Define a parameter-map and set the max-incomplete high values to very high values
parameter-map type inspect DoS-param-map max-incomplete high 20000000 one-minute high 100000000 tcp max-incomplete host 100000 block-time 0
Cisco IOS Step3: Apply the parameter-map to every class-map's inspection action Public policy-map type inspect z1-z2-pmap Network Firewall
class type inspect my-cmap inspect DoS-param-map e0 s0
Presentation_ID
Cisco Public
34
Performance Degrades ZBF

Step 4: check the DoS counters with the following command

router#sh policy-map type inspect zone-pair priv-pub < Removed > Maxever session counts (estab/half-open/terminating) [92:46:33] Last session created 00:00:45 Last statistic reset never Last session creation rate 1 Maxever session creation rate 270
Step 5: Tune the DoS settings for every inspect-type class-map contained Public within a policy-map that must have unique DoS protection requirements
Network
http://www.cisco.com/en/US/prod/collateral/vpndevc/ps5708/ps5710/ps1018/pr e0 s0 od_white_paper0900aecd8055e6ac.html
Presentation_ID
Cisco Public
35
HTTP Connection Reset

Symptom:
Unexpected web connection reset while browsing a web site
Step1a: Analyze syslog messages generated by the router Jul 26 13:58:16 200.1.1.1 2167: Jul 26 18:02:34.907 UTC: %APPFW-4HTTP_JAVA_APPLET: HTTP Java Applet detected - resetting session 172.16.1.100:80 10.1.1.100:3372 on zone-pair publicPrivateOut class myClassMap appl-class HttpAic Step1b: Review the configuration with show command.
class-map type inspect http match-any HttpAic match response body java-applet exit policy-map type inspect http HttpAicPolicy class type inspect http HttpAic reset log Exit Reason for the connection reset
Solution:
Remove the reset command under policy map
36
HTTP Connection Reset (Cont.)

2a. Analyze Syslog messages generated by the router Jul 26 15:03:51 200.1.1.1 2768: Jul 26 19:08:08.751 UTC: %APPFW-4-HTTP_CONTENT_LENGTH: Content length (82271) out of range - resetting session 208.254.0.103:80 10.1.1.100:3491 on zone-pair publicPrivateOut class myClassMap appl-class HttpAic 2b. Using show command reveals the Body Length of the web traffic was configured too LOW.
Solution:
Reset the body length for request/response to higher value
class-map type inspect http match-any HttpAic match req-resp body length gt 1000000 exit
37
HTTP Connection Reset (Cont.)

3a. Analyzing Syslog reveals the following messages
Jul 27 13:12:39 200.1.1.1 5448: Sig:12 HTTP URI length exceeded. Received 10.1.1.100:1451 to 216.73.86.52:
3b. Using show command in reviewing configuration may reveal Request URI Length was set Too LOW.
Resolution:
Reset URI Length to 256 as follows
class-map type inspect http match-any HttpAic match request uri length gt 256 exit
Presentation_ID
Cisco Public
38
Zone Based Firewall Troubleshooting Example
Presentation_ID
Cisco Public
39
Zone Based Firewall Desired setup

Zone Outside
Clients
Zone Inside
10.2.3.0/24
Server
R1
10.2.1.0/24
R2 IOS Firewall
R3 .3
Clients
.1
.2 IPsec tunnel .2
.2
Zone DMZ
10.2.4.0/24
.4
http server
R4
40
Zone Based Firewall Example

Desired Policy
R1 Zone Outside Zone Inside
R2
R3
Three Zones
inside zone outside zone dmz zone
R4 Zone DMZ
http server
Traffic policies
TCP and UDP connections from inside to outside
TCP and UDP connections from dmz to outside, http from the outside to the dmz any other required connections from the outside to the inside
Presentation_ID
Cisco Public
41
Zone Outside
Zone Inside R2 R3
Zone Based Firewall

Class Map Configuration
R1
Zone DMZ
class-map type inspect match-any INSIDE match protocol tcp match protocol udp class-map type inspect match-any DMZ match protocol tcp match protocol udp class-map type inspect match-all OUTSIDE match protocol http match access-group name OUT_DMZ match access-group name OUT_IN
http server R4
ip access-list extended OUT_DMZ permit tcp any host 4.4.4.4 eq www

42
Zone Based Firewall

Zone Configuration
R1 Zone Outside R2 Zone Inside R3
Zone DMZ
zone security inside

R4
http server
zone security outside

zone security dmz
Presentation_ID
Cisco Public
43
Zone Based Firewall

Policy Map Configuration policy-map type inspect IN_OUT class type inspect INSIDE inspect class class-default drop
Zone Outside R1 R2
Zone Inside R3
Zone DMZ http server R4
policy-map type inspect OUT_IN class type inspect OUTSIDE inspect class class-default drop
zone security inside zone security outside zone security dmz
44
Zone Outside
Zone Inside R2 R3
Zone Based Firewall

policy-map type inspect DMZ_OUT class type inspect DMZ inspect class class-default drop
policy-map type inspect OUT_DMZ class type inspect OUTSIDE inspect class class-default drop
R1
Policy Map Configuration (continued)

zone security inside zone security outside zone security dmz
45
Zone Outside
Zone Inside R2 R3
Zone Based Firewall

Zone-pair Configuration
R1
zone-pair security IN->OUT source inside destination outside service-policy type inspect IN_OUT zone-pair security OUT->IN source outside destination inside service-policy type inspect OUT_IN
zone-pair security DMZ->OUT source dmz destination outside service-policy type inspect DMZ_OUT
zone-pair security OUT->DMZ source outside destination dmz service-policy type inspect OUT_DMZ
46
Zone Based Firewall

Firewall Interface Configuration
Zone Outside Zone Inside R2 R3
interface Loopback0 ip address 2.2.2.2 255.255.255.255 ! interface Ethernet0/0 ip address 10.2.1.2 255.255.255.0
R1
zone-member security outside

! interface Ethernet1/0 ip address 10.2.3.2 255.255.255.0 zone-member security inside ! interface Ethernet2/0 ip address 10.2.4.2 255.255.255.0 zone-member security dmz
47
Zone Based Firewall

Additional Configuration Enable telnet on all the routers
Line vty 0 15 password hello Login
Zone Outside R1 R2
Zone Inside
R3
Zone DMZ
http server
R4
Enable http server on R4 (DMZ)

R4#conf t Enter configuration commands, one per line. End with CNTL/Z. R4(config)#ip http server
Enable logging on R2 (Zone Based Firewall)

R2#conf t Enter configuration commands, one per line. End with CNTL/Z. R2(config)#ip inspect log drop-pkt
48
Zone Based Firewall

Testing
Telnet from R4 to R1
Telnet from R3 to R1
Telnet from R1 to R3 Telnet from R1 to R4.
Telnet from R1 to R4 on port 80 (http access)

Telnet
R1 R2 R3
Telnet
http server
R4
49
Zone Based Firewall Telnet should work

Telnet from R4 to R1 should work
R2#sh policy-map type inspect zone-pair DMZ->OUT sessions policy exists on zp DMZ->OUT Zone-pair: DMZ->OUT
Service-policy inspect : DMZ_OUT
Class-map: DMZ (match-any) Match: protocol tcp
1 packets, 24 bytes
30 second rate 0 bps .. Inspect Number of Established Sessions = 1
R4#telnet 1.1.1.1 Trying 1.1.1.1 ... Open
Established Sessions
Session 6A62F98 (10.2.4.4:59121)=>(1.1.1.1:23) tcp SIS_OPEN/TCP_ESTAB Created 00:00:05, Last heard 00:00:04 Bytes sent (initiator:responder) [30:69]
User Access Verification
Password:
Cisco Public
50
Zone Based Firewall Telnet blocked

Telnet from R1 to R3 is blocked
R2#sh policy-map type inspect zone-pair OUT->IN sess
policy exists on zp OUT->IN Zone-pair: OUT->IN

Service-policy inspect : OUT_IN
Class-map: OUTSIDE (match-all) Match: protocol http Match: access-group name OUT_IN
Inspect
R1#telnet 3.3.3.3 Trying 3.3.3.3 ...
Class-map: class-default (match-any) Match: any Drop 10 packets, 240 bytes
% Connection timed out; remote host not responding

Presentation_ID
51
Zone Based Firewall http should work

Telnet from R1 to R4 on port 80 (http access) works
R2#sh policy-map type inspect zone-pair OUT->DMZ sessions policy exists on zp OUT->DMZ Zone-pair: OUT->DMZ Service-policy inspect : OUT_DMZ Class-map: OUTSIDE (match-all) Match: protocol http
R4 Zone DMZ http server R1 Zone Outside R2 Zone Inside R3
Match: access-group name OUT_DMZ Inspect
Number of Established Sessions = 1

Established Sessions Session 6A62C48 (10.2.1.1:34095)=>(4.4.4.4:80) http:tcp SIS_OPEN/TCP_ESTAB Created 00:01:29, Last heard 00:00:13 Bytes sent (initiator:responder) [2:0]
Class-map: class-default (match-any)

Match: any Drop 0 packets, 0 byte
R1#telnet 4.4.4.4 80
Trying 4.4.4.4, 80 ... Open
52
Zone Based Firewall Policies Again

Three Zones
inside zone outside zone dmz zone.
R4 Zone DMZ http server Zone Outside R1 R2 Zone Inside R3
Traffic policies
TCP and UDP connections from inside to outside
TCP and UDP connections from dmz to outside, http from the outside to the dmz any other required connections from the outside to the inside
Presentation_ID
Cisco Public
53
Zone Based Firewall IPsec does not work!

Telnet from R1 to R3 (IPsec peers) works
R2#conf t
Zone Outside R1 R2 Zone Inside R3
Enter configuration commands, one per line. End with CNTL/Z.
R2(config)#ip inspect log drop-pkt R2(config)#end
R2#
*Apr 5 23:45:25.723: %SYS-5-CONFIG_I: Configured from console by console R2# *Apr 5 23:47:10.931: %FW-6-DROP_PKT: Dropping udp session 10.2.1.1:500 10.2.3.3:500 on zone-pair OUT->IN class class-default due to DROP action found in policy-map with ip ident 0
R1# *Apr 5 23:46:18.687: %SYS-5CONFIG_I: Configured from console by console R1#ping 10.2.3.3 .. Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.2.3.3, timeout is 2 seconds: ..... Success rate is 0 percent (0/5)
54
R2# *Apr 5 23:48:38.055: %FW-6-LOG_SUMMARY: 3 packets were dropped from 10.2.1.1:500 => 10.2.3.3:500 (target:class)-(OUT->IN:class-default)
Zone Based Firewall Whats missing?

Zone Outside
Clients
Zone Inside
10.2.3.0/24
Server
R1
10.2.1.0/24
R2
.2 ??? .2
R3 .3
Clients
.1
.2
Zone DMZ
10.2.4.0/24
??? Need a policy for the IKE and IPsec traffic

.4
http server
R4
Cisco Public
55
Zone Based Firewall ACL Configuration

Zone Outside Zone Inside R2 .2 .2 Zone DMZ 10.2.4.0/24 .4 R4 http server 10.2.3.0/24 .3 R3
Allow IKE and IPsec
R1
10.2.1.0/24 .1 .2
ip access-list extended OUT_IN permit udp host 10.2.1.1 host 10.2.3.3 eq isakmp permit udp host 10.2.1.1 host 10.2.3.3 eq non500-isakmp permit esp host 10.2.1.1 host 10.2.3.3 ip access-list extended VPN_OUT permit udp host 10.2.3.3 host 10.2.1.1 eq isakmp permit udp host 10.2.3.3 host 10.2.1.1 eq non500-isakmp permit esp host 10.2.3.3 host 10.2.1.1
Presentation_ID
Cisco Public
56
Zone Based Firewall Configuration

Add Class maps and Policy maps for IKE & IPsec
policy-map type inspect IN_OUT
class-map type inspect match-any INSIDE

class type inspect INSIDE
match protocol tcp

inspect
match protocol udp

class type inspect VPN_OUT
class-map type inspect match-all VPN match access-group name OUT_IN class-map type inspect match-any DMZ match protocol tcp match protocol udp class-map type inspect match-all OUTSIDE match protocol http match access-group name OUT_DMZ match access-group name OUT_IN class-map type inspect match-all VPN_OUT match access-group name VPN_OUT
pass
policy-map type inspect OUT_IN

class type inspect OUTSIDE inspect class type inspect VPN pass
Note: Order of inspection.
policy-map type inspect DMZ_OUT

class type inspect DMZ inspect policy-map type inspect OUT_DMZ class type inspect OUTSIDE inspect
Presentation_ID
Cisco Public
57
Zone Based Firewall IPsec should work

Telnet from R1 to R3 (IPsec peers) works now
R2#sh policy-map type inspect zone-pair OUT->IN sess
R1
policy exists on zp OUT->IN Zone-pair: OUT->IN

Service-policy inspect : OUT_IN

Class-map: OUTSIDE (match-all) Match: protocol http Match: access-group name OUT_IN Inspect Class-map: VPN (match-all) Match: access-group name OUT_IN Pass 5 packets, 652 bytes Class-map: class-default (match-any) Match: any Drop 0 packets, 0 bytes
R1#ping 10.2.3.3 Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.3.3, timeout is 2 seconds:

.!!!! Success rate is 80 percent (4/5), roundtrip min/avg/max = 8/10/12 ms
58
Firewall Summary
Presentation_ID
Cisco Public
59
Firewall Summary
ALWAYS TAKE Systematic Approach to troubleshoot IOS Firewall issues Establish base-line traffic profile for your network through IOS Firewall, and set the DoS settings accordingly DO NOT change the default UDP & DNS session timeout value Use syslog and show commands to troubleshoot IOS firewall
Presentation_ID
Cisco Public
60
Troubleshooting Cisco IOS Intrusion Prevention System
Presentation_ID
Cisco Public
61
Cisco IOS IPS Overview
Presentation_ID
Cisco Public
62
OverviewWhat Is Cisco IOS IPS

Previously called IDS before 12.3(8)T, use ip audit CLI Introduced in 12.3(8)T, now refers to Cisco IOS IPS Software based inline intrusion prevention sensor Support Cisco IPS version 5.x signature format starting from 12.4(11)T* Signature based packet scanning, use same set of signatures as the Cisco IPS 4200 sensor platform Dynamic signature update, no need to update IOS image Variety event actions configurable per-signature and per-category Ease of managementCCP, CSM**
* Version 5.x Signature Format Is Not Backward Compatible with Version 4.x Signature Format ** CCP = Cisco Configuration Professional; CSM = Cisco Security Manager
63
Cisco IOS IPSSystem Components

Signature Micro-Engines (SMEs)
A SME defines parameters for signatures in a specific protocol category, e.g. HTTP
Signature Files
Contains signature engine, parameter information such as signature name, signature ID and signature actions etc.
Signature categories*
A signature category contains pre-selected signature sets for a specific vulnerability
SEAP (Signature Event Action Processor)

SEAP allows for advanced event action filtering and overrides on the basis of the Event Risk Rating (ERR) feedback
Event Monitoring
Syslog messages and/or SDEE** alerts for events generated by IOS IPS
* Version 5.x Signature Format Only (i.e. 12.4(11)T or later) ** SDEE = Security Device Event Exchange
64
Signature Categories
IOS IPS with Cisco 5.x/6.x format signatures operate with signature categories Signature category is a group of relevant signatures represented by a meaningful name All signatures are pregrouped into categories An individual signature can belong to more than one category
Router#sh ip ips category ?
adware/spyware attack ddos dos email instant_messaging ios_ips l2/l3/l4_protocol network_services os other_services p2p reconnaissance releases viruses/worms/trojans web_server
Adware/Spyware (more sub-categories) Attack (more sub-categories) DDoS (more sub-categories) DoS (more sub-categories) Email (more sub-categories) Instant Messaging (more sub-categories) IOS IPS (more sub-categories) L2/L3/L4 Protocol (more sub-categories) Network Services (more sub-categories) OS (more sub-categories) Other Services (more sub-categories) P2P (more sub-categories) Reconnaissance (more sub-categories) Releases (more sub-categories) Viruses/Worms/Trojans (more sub-categories) Web Server (more sub-categories)
Presentation_ID
Cisco Public
65
Packet Flow
Presentation_ID
Cisco Public
66
Cisco IOS IPS Packet FlowInbound
Packet Re-injection
Layer 2 decapsulation
Stateless IPS
IPSEC?
Inbound ACL
IPSec decryption
Inbound crypto map ACL
N
Auth Proxy
Inbound ACL
NAT
Forwarding
Presentation_ID
Cisco Public
67
IPSec/IPS Packet FlowOutbound
Forwarding
Stateless IPS
NAT
Fragment Inspection
Outbound ACL
Stateful IPS & Firewall
IPSEC?
Outbound crypto map ACL

IPSec encryption
Layer 2 encapsulation
Forwarding
68
Troubleshooting IPS
Presentation_ID
Cisco Public
69
The Problem Solving Process

Assess
Whats going on Prioritize
Ask the right questions to better define and clarify the problem
Acquire
What information do we need but we dont have? How to get that information?
Analyze
Understand the flow Whats supposed to happen vs. What actually happened
Act
Test assumptions Deploy changes
70
Basic Configuration Example

ip ips config location flash:ips/ retries 1 ip ips notify SDEE ip ips name iosips ip ips signature-category category all retired true category ios_ips advanced retired false
ALWAYS remember first select category all AND retire all signatures
crypto key pubkey-chain rsa named-key realm-cisco.pub signature key-string 30820122 300D0609 2A864886 F70D0101 01050003 82010F00 3082010A 02820101 | snip | F3020301 0001 quit
IOS IPS crypto key
interface GigabitEthernet0/1 ip address 10.1.1.6 255.255.255.0 ip ips iosips in ip virtual-reassembly duplex auto speed auto
enable IOS IPS policy on interface
Presentation_ID
Cisco Public
71
Configure Event Notification Using SDEE

SDEE messages are transported over HTTP/HTTPS You must enable HTTP/HTTPS in order to use SDEE Recommend to set the number of concurrent subscriptions to three when using IME
Router(config)#ip sdee subscriptions ? <1-3> Number of concurrent SDEE subscriptions
IOS IPS log message format:

*Mar 22 03:53:13.827: %IPS-4-SIGNATURE: Sig:5114 Subsig:1 Sev:75 WWW
IIS Unicode Attack [10.1.1.252:4150 -> 192.168.1.249:80] RiskRating:75
*Mar 22 03:53:13.827: %IPS-4-SIGNATURE: Sig:5081 Subsig:0 Sev:100 WWW WinNT cmd.exe Access [10.1.1.252:4150 -> 192.168.1.249:80] RiskRating:100
SDEE = Security Device Event Exchange

72
Common Troubleshooting Steps

1. Check IOS IPS configuration, to confirm policy is applied to the right interface in the right direction
show run
2. Check signatures status, to confirm signatures are compiled show ip ips config show ip ips signatures count 3. Check flows inspected by IOS IPS, to verify IOS IPS is inspecting traffic show ip ips sessions detail 4. Check SDEE alerts / syslog messages, to verify attacks are being detected show ip sdee alerts
show logging
5. Use appropriate debug commands
Presentation_ID
Cisco Public
73
IOS IPS Troubleshooting Commands

Step 1: Check IOS IPS configuration
Router#sh run Building configuration... Configure IPS signature storage location Enable IPS SDEE event notification
-- output skipped -! ip ips config location flash:ips/ retries 1 ip ips notify SDEE ip ips name iosips Configure IOS IPS to use one ! of the pre-defined signature ip ips signature-category categories category all retired true Configure an IOS IPS crypto category ios_ips advanced key which is used to verify the retired false digital signature on the ! signature package crypto key pubkey-chain rsa named-key realm-cisco.pub signature key-string 30820122 300D0609 2A864886 F70D0101 01050003 82010F00 3082010A 02820101 -- output skipped -F3020301 0001 quit ! interface GigabitEthernet0/1 ip address 10.1.1.6 255.255.255.0 ip ips iosips in ip Presentation_ID virtual-reassemblyCisco and/or its affiliates. All rights reserved. 2010
Enable IPS rule on the desired interface and specify the direction the rule will be applied to
Cisco Public
74

Step 2: Check IOS IPS Configuration and Signatures Status
Router#sh ip ips all IPS Signature File Configuration Status Configured Config Locations: flash:ips/ Last signature default load time: 16:42:08 PST Mar 1 2008 Last signature delta load time: 22:59:57 PST Mar 3 2008 Last event action (SEAP) load time: -noneGeneral SEAP Config: Global Deny Timeout: 3600 seconds Global Overrides Status: Enabled Global Filters Status: Enabled IPS Auto Update is not currently configured IPS Syslog and SDEE Notification Status Event notification through syslog is enabled Event notification through SDEE is enabled IPS Signature Status Total Active Signatures: 581 Total Inactive Signatures: 1623 IPS Packet Scanning and Interface Status IPS Rule Configuration IPS name iosips IPS fail closed is disabled IPS deny-action ips-interface is false Fastpath ips is enabled Quick run mode is enabled Interface Configuration Interface GigabitEthernet0/1 Inbound IPS rule is iosips Outgoing IPS rule is not set IPS Category CLI Configuration: Category all: Retire: True Category ios_ips advanced: Retire: False
Presentation_ID
Determine the # of active signatures
Verify the IOS IPS policy is applied to the right interface in the right direction Verify the signature category being used
Cisco Public
75

Step 2: Check Signatures Status
Router#show ip ips signatures count Cisco SDF release version S318.0 Trend SDF release version V0.0
Check signature release version
Signature Micro-Engine: multi-string: Total Signatures 8 multi-string enabled signatures: 8 multi-string retired signatures: 8 - output omitted Signature Micro-Engine: service-msrpc: Total Signatures 27 service-msrpc enabled signatures: 27 service-msrpc retired signatures: 19 service-msrpc compiled signatures: 1 service-msrpc inactive signatures - invalid params: 7
Total Signatures: 2204 Total Enabled Signatures: 873 Total Retired Signatures: 1617 Check Total Compiled Signatures: 580 Total Signatures with invalid parameters: 7 Total Obsoleted Signatures: 11
there are signatures being compiled
Presentation_ID
Cisco Public
76

Step 3: Check Flows Inspected by IOS IPS
Router#show ip ips sessions detail Established Sessions
Src.address/port & dest.address/port
Session 47506A34 (10.1.1.252:3959)=>(192.168.1.249:21) tcp SIS_OPEN Created 00:02:49, Last heard 00:02:44 Bytes sent (initiator:responder) [25:95] sig cand list ID 14272 sig cand list ID 14273
Bytes sent and received
Presentation_ID
Cisco Public
77

Step 4: Check Alert Messages
Verify that the router is seeing IOS IPS related event and alert messages.
Router#sh logging Syslog logging: enabled (12 messages dropped, 7 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
-- output skipped -Log Buffer (4096 bytes): *Mar 22 03:53:13.827: %IPS-4-SIGNATURE: Sig:5114 Subsig:1 Sev:75 WWW IIS Unicode Attack [10.1.1.252:4150 -> 192.168.1.249:80] RiskRating:75 *Mar 22 03:53:13.827: %IPS-4-SIGNATURE: Sig:5081 Subsig:0 Sev:100 WWW WinNT cmd.exe Access [10.1.1.252:4150 -> 192.168.1.249:80] RiskRating:100
Router#sh ip sdee alerts Alert storage: 200 alerts using 75200 bytes of memory SDEE Alerts SigID Sig Name SrcIP:SrcPort DstIP:DstPort or Summary Info 1: 5114:1 WWW IIS Unicode Attack 10.1.1.252:4150 192.168.1.249:80 2: 5081:0 WWW WinNT cmd.exe Access 10.1.1.252:4150 192.168.1.249:80
Presentation_ID
Cisco Public
78
Cisco IOS IPS Debugging Commands

Step 5: Use Debug Commands Enable debugs on specified IOS IPS engines
Router# debug ip ips timers
Router# debug ip ips [object-creation | object-deletion]

Router# debug ip ips function trace Router# debug ip ips detail
L3/L4 debug commands:
Not recommended in production network
Router# debug ip ips [ip | icmp | tcp | udp]
Application-level debug commands:

Router# debug ip ips [tftp | smtp | ftp-cmd | ftp-token]
Enable debug on specified SDEE attributes

Router# debug ip sdee [alerts | details | messages | requests | subscriptions ]
Presentation_ID
Cisco Public
79
Presentation_ID
Cisco Public
80
Common Issues
Misunderstanding of terms used for signature status Memory allocation errors when compiling signatures Total number of signatures that can be compiled
Signature failed to compile

Configuration steps Cisco IOS IPS policy is applied at the wrong direction and/or interface Signature does not fire with matching traffic
81
Misunderstanding of Terms Used for Signature Status

Retire vs. unretire
Enable vs. disable

Compiled vs. loaded Cisco IOS IPS inherited these terms from IPS 4200 series appliance Due to memory constraints, most of the signatures on router are retired by default IOS IPS users need to worry about enable/disable as well as retire/unretire
Presentation_ID
Cisco Public
82
Misunderstanding of Terms Used for Signature Status (Cont.)

Retire vs. Unretire Select/de-select which signatures are being used by IOS IPS to scan traffic Retiring a signature means IOS IPS will NOT compile that signature into memory for scanning
Unretiring a signature instructs IOS IPS to compile the signature into memory and use the signature to scan traffic
You can use IOS command-line interface (CLI) or CCP to retire or unretire individual signatures or a signature category
Presentation_ID
Cisco Public
83

Enable vs. Disable
Enabling a signature means that when triggered by a matching packet (or packet flow), the signature takes the appropriate action associated with it
However, only unretired AND successfully compiled signatures will take the action when they are enabled. In other words, if a signature is retired, even though it is enabled, it will not be compiled (because it is retired) and it will not take the action associated with it
Disabling a signature means that when triggered by a matching packet (or packet flow), the signature DOES NOT take the appropriate action associated with it
In other words, when a signature is disabled, even though it is unretired and successfully compiled, it will not take the action associated with it
You can use IOS command-line interface (CLI) or CCP to enable or disable individual signatures or a signature category Enable/disable is NOT used to select/de-select signatures to be used by IOS IPS
84

Compiled vs. Loaded Loading refers to the process where IOS IPS parse the signature files (XML files in the config location) and fill in the signature database
This happens when signatures are loaded via copy <sig file> idconf or the router reboots with IOS IPS already configured
Compiling refers to the process where the parameter values from unretired signatures are compiled into a regular expression table
This happens when signatures are unretired or when other parameters of signatures belonging to that regular expression table changes Once signatures are compiled, traffic is scanned against the compiled signatures
85
Memory Allocation Errors When Compiling Signatures

The number of signatures that can be compiled depends on the free memory available on the router When router does not have enough memory to compile signatures, memory allocation failure messages are logged Already compiled signatures will still be used to scan traffic. No additional signatures will be compiled for that engine during the compiling process. IOS IPS will proceed with compiling signatures for the next engine
*Mar 18 07:09:36.887: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x400C1024, alignment 0 Pool: Processor Free: 673268 Free: 0 Cause: Memory fragmentation Cause: No Alternate pool Alternate Pool: None
-Process= "Exec", ipl= 0, pid= 3, -Traceback= 0x4164F41C 0x400AEF1C 0x400B4D58 0x400B52C4 0x400C102C 0x400C0820 0x400C23EC 0x400C0484 0x424C1DEC 0x424C2A4C 0x424C2FF0 0x424C31A0 0x430D6ECC 0x430D7864 0x430F0210 0x430FA0E8
*Mar 18 07:09:36.911: %SYS-2-CHUNKEXPANDFAIL: Could not expand chunk pool for regex. No memory available Process= "Chunk Manager", ipl= 3, pid= 1, -Traceback= 0x4164F41C 0x400C06FC *Mar 18 07:09:37.115: %IPS-4-SIGNATURE_COMPILE_FAILURE: service-http 12024:0 - compilation of regular expression failed *Mar 18 07:09:41.535: %IPS-4-SIGNATURE_COMPILE_FAILURE: service-http 5280:0 - compilation of regular expression failed *Mar 18 07:09:44.955: %IPS-4-SIGNATURE_COMPILE_FAILURE: service-http 5284:0 - compilation of regular expression failed *Mar 18 07:09:44.979: %IPS-4-SIGNATURE_COMPILE_FAILURE: service-http 12023:0 - compiles discontinued for this engine
86
Memory Allocation Errors When Compiling SignaturesResolution

The pre-defined IOS IPS Basic and Advanced signature categories contain optimum combination of signatures for all standard memory configurations, providing a good starting point Never unretire the all category For routers with 128MB memory, start with the IOS IPS Basic category For routers with 256MB memory, start with the IOS IPS Advanced category Then customize the signature set by unretiring/retiring few signatures at a time according to your network needs Pay attention to the free memory every time after you unretiring/retiring signatures
Presentation_ID
Cisco Public
87
Total Number of Signatures Can Be Compiled

There is no magic number!
Many factors can have impact:

Available free memory on router Type of signatures being unretired, e.g. signatures in the complex STRING.TCP engine
When router free memory drops below 10% of the total installed memory, then stop unretiring signatures
Presentation_ID
Cisco Public
88
Signature Failed to Compile

There are mainly three reasons that could cause a signature fail to compile
Memory constraint, running out of memory
Signatures are not supported in IOS IPS: META signatures Regular Expression table for a particular engine exceeds 32MB entries
Check the list of supported signatures in IOS IPS at:

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/p s6586/ps6634/prod_white_paper0900aecd8062ac75.html
Retire signatures not supported by IOS IPS and signatures not applicable to your network to save memory
Presentation_ID
Cisco Public
89
Configuration Steps
Follow the steps in the following order for initial Cisco IOS IPS configuration:
Step 1: Download IOS IPS signature package to PC Step 2: Create IOS IPS configuration directory Step 3: Configure IOS IPS crypto key Step 4: Create IOS IPS policy and apply to interface(s)
Remember to FIRST retire the all category
Step 5: Load IOS IPS signature package
Next verify the configuration and signatures are compiled:

show ip ips configuration show ip ips signatures count
Presentation_ID
Cisco Public
90
Configuration Steps (Cont.)

Next you can start to tune the signature set with the following options:
Retire/unretire signatures (i.e. add/remove signatures to/from the compiled list) Enable/disable signatures (i.e. enforce/disregard actions) Change actions associated with signatures
Refer to Getting Started Guide at:

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps653 7/ps6586/ps6634/prod_white_paper0900aecd805c4ea8.ht ml
Presentation_ID
Cisco Public
91
Case A: IOS IPS Policy Is Applied at the Wrong Issue Direction/InterfaceIncorrect Configuration
Protecting Attacks from Inside

Inside Outside
Head Office
Branch Office
Worms FE0/0 FE0/1 Web Clusters Internet Traffic IPSec Tunnel Cisco 28xx Internet Interface FastEthernet0/0 Branch Office PCs/Laptops ip ips ips-policy out Policy applied to the wrong direction Head Office PCs Application Servers
Cisco 18xx
Presentation_ID
Cisco Public
92
IOS IPS Policy Is Applied at the Wrong Direction/InterfaceResolution

Protecting Attacks from Inside
Inside Outside
Case A: Solution
Head Office
Branch Office
Worms FE0/0 FE0/1 Web Clusters Internet Traffic IPSec Tunnel Cisco 28xx Internet Interface FastEthernet0/0 Branch Office PCs/Laptops ip ips ips-policy in Policy applied to the right direction Head Office PCs Application Servers
Cisco 18xx
Presentation_ID
Cisco Public
93
Case B: IOS IPS Policy Is Applied at the Wrong Issue Direction/InterfaceIncorrect Configuration
Protecting Attacks from Outside
attacks
Inside Outside
Head Office
Branch Office
FE0/0 FE0/1 Web Clusters Internet Traffic IPSec Tunnel Cisco 28xx Internet DMZ Interface FastEthernet0/1 Branch Office PCs/Laptops
Presentation_ID
Cisco 18xx
Application Servers
ip ips ips-policy out Policy applied to the wrong direction

Head Office PCs

94
IOS IPS Policy Is Applied at the Wrong Direction/InterfaceResolution

Protecting Attacks from Outside
Case B: Solution
attacks
Inside Outside
Head Office
Branch Office
FE0/0 FE0/1 Web Clusters Internet Traffic IPSec Tunnel Cisco 28xx Internet DMZ Interface FastEthernet0/1 Branch Office PCs/Laptops ip ips ips-policy in Policy applied to the right direction
Cisco 18xx
Application Servers
Head Office PCs

95
Presentation_ID
Signature Does Not Fire with Matching Traffic

Verify IOS IPS is applied in the right direction (inbound/outbound) and on the right interface
Is IOS IPS event notification enabled? i.e. syslog/SDEE

Do you see alarms/alerts showing signature matching? It is essential that we see whether signatures are triggered by the traffic
Use show ip ips signatures statistics | i <sig id> to see signature hits
Run debugs:
debug ip ips <engine name>
debug ip ips detailed

debug ip ips function-trace (if the above two do not show anything)
Presentation_ID
Cisco Public
96
IPS Summary
Presentation_ID
Cisco Public
97
Cisco IOS IPS Enhancements

ENHANCEMENT
1 Lightweight IPS Engines for existing and new signatures optimized for HTTP, SMTP and FTP protocols New Default IOS IPS Category signatures updated frequently by Cisco Signature Team
BENEFIT
Memory efficient traffic scanning for attack signatures consuming up to 40 % less memory on the router. More comprehensive and effective attack coverage by default. Much quicker inclusion of most relevant new threat signatures within the default set (category). Capability to load more signatures simultaneously and provide protection for larger number of threats and vulnerabilities
Chaining of Traffic Scanning (Regular Expression) Tables
Configurable Threshold (Upper Limit) to be dedicated to IPS feature
Avoid large amount of router memory by IPS signature Tables. Prevent IPS feature to consume all the free processing memory available and cause performance and other operational problems
Presentation_ID
Cisco Public
98
IPS Summary
Use the Getting Started Guide as a reference to check that IOS IPS is configured properly.
Always remember to RETIRE ALL signatures first.

ip ips signature-category category all retired true
Recommendation is to use pre-defined IOS IPS Basic or Advanced signature category and tune the signature set based on your network applications
Cisco IOS IPS show Commands and SDEE are the most essential component for troubleshooting
Presentation_ID
Cisco Public
99
Documentation and Links
Presentation_ID
Cisco Public
100
Documentation for Cisco IOS Security

Router Security
www.cisco.com/go/routersecurity
Cisco IOS Security Commands Reference

http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_command _reference_chapter09186a00801a7f84.html#wp1187286
Cisco IOS Firewall

www.cisco.com/go/iosfw
Cisco Zone-based Firewall Design and Application Guide

http://www.cisco.com/en/US/products/sw/secursw/ps1018/products_tech_note 09186a00808bc994.shtml
Cisco IOS IPS

http://www.cisco.com/go/iosips
Cisco Configuration Professional (CCP)

http://www.cisco.com/go/ccp
Presentation_ID
Cisco Public
101
Q&A
Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Cisco Preferred Access points for each session evaluation you complete. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
Dont forget to activate your Cisco Live and Networkers Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.
Cisco Public
Presentation_ID
103
Appendix : Classic IOS Firewall
Presentation_ID
Cisco Public
105
Simple Classic IOS Firewall Configuration

Inside Outside
CBAC
Internet
e0
1. Define the security policy
s0
Deny any connections initiating from outside

Allow only SMTP, ftp, and http connections from inside
2. Convert the security policy into IOS configuration

access-list 101 deny ip any any interface serial0 ip access-group 101 in access-list 102 permit any any eq smtp access-list 102 permit any any eq ftp access-list 102 permit any any eq http ip inspect name foo smtp ip inspect name foo http ip inspect name foo ftp interface ethernet0 ip inspect foo in ip access-group 102 in
ACL to deny inbound connection ACL to allow only SMTP, FTP, and HTTP from inside to outside Inspection for necessary protocols Inspection rule, and ACL both applied as inbound on ethernet 0 interface
106
CBAC
Show CommandsClassic IOS Firewall

To display the firewall policy and sessions
Router# show ip inspect all Session audit trail is disabled Session alert is enabled one-minute (sampling period) thresholds are [400:20000] connections max-incomplete sessions thresholds are [400:20000] max-incomplete tcp connections per host is 100000. Block-time 0 minute tcp synwait-time is 30 sec -- tcp finwait-time is 5 sec tcp idle-time is 3600 sec -- udp idle-time is 30 sec dns-timeout is 5 sec Outgoing inspection rule is TESTING_REALWORD smtp max-data 20000 alert is on audit-trail is off timeout 3600 ftp alert is on audit-trail is off timeout 3600 tcp alert is on audit-trail is off timeout 3600 udp alert is on audit-trail is off timeout 30 Inbound access list is 101 Outgoing access list is not set Established Sessions Session 49AA929C (106.0.0.6:14320)=>(100.0.0.6:53) udp SIS_OPEN Half-open Sessions Session 467479EC (106.0.0.6:20150)=>(100.0.0.3:25) smtp SIS_OPENING
107
CBAC

To display the firewall statistics
Router# show ip inspect statistics
Packet inspection statistics [process switch:fast switch]

tcp packets: [616668:0] http packets: [178912:0] Interfaces configured for inspection 1 Session creations since subsystem startup or last reset 42940 Current session counts (estab/half-open/terminating) [0:0:0] Maxever session counts (estab/half-open/terminating) [98:68:50] Last session created 5d21h Last statistic reset never Last session creation rate 0 Last half-open session total 0
108
CBAC

Displays session related information
Router# show ip inspect session Established Sessions Session 25A3318 (10.0.0.1:20)=>(10.1.0.1:46068) ftp-data SIS_OPEN Session 25A6E1C (10.1.0.1:46065)=>(10.0.0.1:21) ftp SIS_OPEN
Connection states
SIS_OPENING SYN has been received but Three way Hand-shake is not complete SIS_OPEN When Three WAY Hand-Shake is complete SIS_CLOSING FIN is received but the entire closing sequence has not been achieved SIS_CLOSE When FIN and FIN-ACK have been received from both sides
Inside Client Outside Inside Outside
Server
SYN SYN+ACK ACK
Presentation_ID
Client
Server
FIN FIN+ACK ACK 2
1
3
1
2 3
Cisco Public
109

Performance degrades When I turn on IOS Firewall Cisco IOS Firewall dropping valid packets Inspect applied in wrong direction Fragmentation and Cisco IOS Firewall IPSec and Cisco IOS FW issues HTTP connection resets Multi-channel protocol not working (FTP, VoIP)
Presentation_ID
Cisco Public
110
Inspect Applied in Wrong Direction

Symptom: No return traffic is making it through the router, possibly getting dropped by the ACL
access-list 101 deny ip any any interface Serial0 description outside ip access-group 101 in ip inspect name IOSFW tcp ip inspect name IOSFW udp
interface Serial0 description outside ip inspect IOSFW in
Public Network s0
Cisco IOS Firewall

Private Network e0
Internet
ACL 101 Inspect
Inbound inspection and ACL are both applied on the outside interface and return traffic gets dropped by ACL 101
111
Inspect Applied in Wrong Direction

Do a show ip inspect sessions on the router to see if we built anything into the session table, dont see anything Check the direction of the applied interface ACL vs. inspection; both are applied in the same inbound direction
Public Network Internet
Inspect
Private Cisco IOS Network Firewall s0

e0
ACL 101
Resolution: Apply Inspection Outbound on the Internet facing interface (while, ACL is applied Inbound)
112
Fragmentation and Cisco IOS Firewall

Before IOS release 12.3(8)T
Applying fragmentation control in situations where legitimate fragments are likely to arrive out of order, may have an impact on application performance as they are discarded
Router(config)# ip inspect name inspection-name fragment
As of 12.3(8)T release
IOSFW now takes advantage of virtual fragmentation reassembly. VFR provides a mechanism to buffer incoming IP fragments for re-ordering and virtual reassembly. This now enables IOS FW to manage sessions that include fragmented packets. Should be enabled on both public/private interface
Router(config-if)# ip virtual-reassembly
Presentation_ID
Cisco Public
113

Step2a: Check Firewall Statistics

Router# show ip inspect statistics < Removed > Session creations since subsystem startup or last reset 2 Current session counts (estab/half-open/terminating) [4214:16853:566] Maxever session counts (estab/half-open/terminating) [4214:16853:566]
Step2b: Check the DoS settings IOS Cisco

Public Network ip inspect max-incomplete high value (default 500) Firewall
ip ip ip ip
inspect max-incomplete low value (default 400) inspect one-minute e0 high values0 (default 500) inspect one-minute low value (default 400) inspect tcp max-incomplete host value (default 50) [block-time minutes
Presentation_ID
114

Step3: Verify the IOS Firewall Policy to see if the HTTP traffic is inspected
ip ip ip ip ip inspect inspect inspect inspect inspect name name name name name IOSFirewall IOSFirewall IOSFirewall IOSFirewall IOSFirewall http https pop3 smtp dns
Inspect http" adds capability to inspect returned content for java applets hence get substantial performance hit
Solution:
If Java Applet filter is NOT required, turn off http inspection. Otherwise, create Java-list to bypass inspection from the known trusted sites.
ip inspect name IOSFirewall http java-list 20 ip inspect name IOSFirewall smtp ip inspect name IOSFirewall dns access-list 20 permit 10.1.1.0 0.0.0.255
115

Step4: Check to see if the default UDP & DNS-Timeout is reset
If the DNS and UDP timeout is set too high, the router will ended up building too many UDP and DNS unused sessions If UDP & DNS timeout is set too LOW, session may pre-maturely get reset causing creating many more connections than needed
Solution:
Set the UDP timeout to 30 seconds (default) and DNS timeout to 5 Seconds (default) unless otherwise required.
Router(config)#ip inspect dns-timeout 5
Configuring DNS in the firewall policy results in performance degradation bug ID (CSCse35588). This was fixed in 12.4(11)T
Presentation_ID
Cisco Public
116

Solution: Tune the DoS protection parameters
Step1: Be sure your network is not infected with viruses or worms that could lead to erroneously large embryonic connection values Step2: Set the max-incomplete high values to very high values initially, and see if the performance improve, then base-line traffic in your network, and see the value accordingly
ip inspect max-incomplete high 20000000 ip inspect one-minute high 100000000 ip inspect tcp max-incomplete host 100000 block-time 0
Prior 12.4(11)T default DoS settings were set low

http://www.cisco.com/en/US/products/sw/secursw/ps1018/products_white_paper0900aec
12.4(11)T onwards DoS settings are max out by default

117
Multi-Channel Protocol Not Working

Symptoms:
Example1: Can FTP to a server but unable to list the directory (ls) Example2: Can call and receive call, but unable to hear anything
Use show ip inspect session, and check the state of the data connection Analyze Syslog Message
Resolution:
Every multi-channel protocol needs to be inspected
Presentation_ID
Cisco Public
118
Matching Traffic Is Detected but Not Dropped by Default

In version 4.x signature format releases (i.e. prior to 12.4(11)T), pre-built signature files (128/256MB.sdf) with version 5 or earlier versions have signatures with Risk Rating of 95 or higher have a default action to drop packets This default action setting has caused issues with customers To be consistent with the Cisco IPS appliance, starting from version 6 of pre-built signature files (128/256MB.sdf), the default action for signatures in IOS IPS is set to produce-alert 12.4(11)T or later releases (version 5.x signature format) have the default action for signatures in IOS IPS set to produce-alert
Presentation_ID
Cisco Public
119
FW Drops Out-of-Order Packet

FW Drops Out-of-Order Packet Slows Down Network Traffic
After turn on IPS, web traffic response time slows down. Go to the router and find out there are syslog messages dropping out of order packets. *Jan 6 19:08:45.507: %FW-6-DROP_PKT: Dropping tcp pkt10.10.10.2:1090 => 199.200.9.1:443 *Jan 6 19:09:47.303: %FW-6-DROP_PKT: Dropping tcp pkt10.10.10.2:1091 => 199.200.9.1:443 *Jan 6 19:13:38.223: %FW-6-DROP_PKT: Dropping tcp pkt66.102.7.99:80 => 192.168.18.21:1100 debug ip inspect detail shows Out-Of-Order packet
*Jan 6 19:15:28.931: CBAC* sis 84062FEC L4 inspectresult: SKIP packet 83A6F83C (199.200.9.1:443) (192.168.18.21:1118) bytes 174 ErrStr = Out-Of-OrderSegment tcp *Jan 6 19:15:28.931: CBAC* sis 84062FEC pak 83A6FF64SIS_OPEN/ESTAB TCP ACK 842755785 SEQ 2748926608 LEN 0 (10.10.10.2:1118) => (199.200.9.1:443) *Jan 6 19:15:28.931: CBAC* sis 84062FEC pak 83A6F83CSIS_OPEN/ESTAB TCP ACK 2748926608 SEQ 842755785 LEN 1317 (199.200.9.1:443) <= (192.168.18.21:1118) *Jan 6 19:15:28.931: CBAC* sis 84062FEC L4 inspectresult: SKIP packet 83A6F83C (199.200.9.1:443) (192.168.18.21:1118) bytes 1317 ErrStr = RetransmittedSegment tcp *Jan 6 19:15:28.935: CBAC* sis 84062FEC pak 83A6F83CSIS_OPEN/ESTAB TCP PSH ACK 2748926608 SEQ 842758636 LEN 137 (199.200.9.1:443) <=(192.168.18.21:1118) *Jan 6 19:15:28.935: CBAC* sis 84062FEC L4 inspectresult: SKIP packet 83A6F83C (199.200.9.1:443) (192.168.18.21:1118) bytes 137 ErrStr = Out-Of-OrderSegment tcp
Presentation_ID
Cisco Public
120
FW Drops Out-of-Order Packet Resolution

FW Drops Out-of-Order Packet Slows Down Network Traffic
IPS requires packets arrive in order to perform signature scanning, thus drops out-of-order packet; this is one of the reasons for slow response and longer latency in network traffic IOS IPS supports Out-of-Order packet starting from 12.4(9)T2 and later 12.4T releases Not fixed in 12.4 mainline releases Out-of-Order fix also applies to application firewall Out-of-order fix DOES NOT work when IOS IPS interface is included in a Zone-Based FW zone Out-of-order fix works between IOS IPS and Classic IOS FW (ip inspect) If using a release that does not have the fix, workaround is to use ACL to bypass IOS IPS inspection for the traffic flow in question
router(config)#access-list 120 deny ip any host 199.200.9.1 router(config)#access-list 120 deny ip host 199.200.9.1 any router(config)#access-list 120 permit ip any any router(config)#ip ips name myips list 120
In the example, ACL 120 denies traffic and remove the traffic from IPS scanning; the network traffic between the two site do not experience slow response
Presentation_ID
Cisco Public
121
Cisco IOS Firewall Configuration Models

Two Configuration Models
Classic IOS Firewall Interface-based stateful inspection Zone-Based Policy Firewall Zone-based stateful inspection
Firewall policies are configured Firewall Policy = Inspection policy on traffic moving between zones combined with ACL policy Policy correlation is simple, and Policy correlation is difficult therefore easier to troubleshoot More granular inspection policy
Conceptual Difference Between Cisco IOS Classic and Zone-Based Firewalls http://www.cisco.com/en/US/prod/collateral/vpndevc/ps5708/ps5710/ps1018/p rod_white_paper0900aecd806f31f9.html Zone-Based Policy Firewall is supported since 12.4(6)T
122
Zone Based Firewall IPsec Configuration

crypto isakmp policy 1 authentication pre-share
crypto isakmp key p address 0.0.0.0 0.0.0.0 ! crypto ipsec transform-set e esp-des
R1
Zone DMZ
! crypto map blah 1 ipsec-isakmp set peer 10.2.3.3 set transform-set e match address 101 !
R4
http server
interface Ethernet1/0
ip address 10.2.1.1 255.255.255.0 crypto map blah ! access-list 101 permit ip host 10.2.1.1 host 10.2.3.3
123

BRKSEC-3007 - Advanced Cisco IOS Security Features (2010 Las Vegas)

Uploaded by

Copyright:

Available Formats

You might also like

BRKSEC-3007 - Advanced Cisco IOS Security Features (2010 Las Vegas)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKSEC-3007 - Advanced Cisco IOS Security Features (2010 Las Vegas)

Uploaded by

Copyright:

Available Formats

Troubleshooting Cisco IOS Security Features

What is Not Covered

IPS Appliance Troubleshooting

2010 Cisco and/or its affiliates. All rights reserved.

Cisco IOS Firewall Overview

2010 Cisco and/or its affiliates. All rights reserved.

Zone-Based Policy Firewall Overview

Private-DMZ Policy DMZ-Private Policy

DMZ Public-DMZ Policy

Zone-Based Policy Firewall Configuration

Define services inspected by policy

Services with ACL to define permitted/denied hosts (Optional)

Establish zone-pair & apply the policy Assign interfaces to zones

2010 Cisco and/or its affiliates. All rights reserved.

Cisco IOS Firewall Packet Flow

2010 Cisco and/or its affiliates. All rights reserved.

Understanding the Packet Flow

IP S: a.b.c.1 D: d.e.f.1 Proto: 17 (udp) UDP -- S: xxxx -- D: yyy

interface Fa Cisco Public 2/0

General Packet Flow

Stateless IPS Input Int

Outbound ACL Output Int

Stateless IPS Input Int

Cisco IOS Firewall Troubleshooting

2010 Cisco and/or its affiliates. All rights reserved.

The problem solving Process

IOS Firewall Troubleshooting Tools

2010 Cisco and/or its affiliates. All rights reserved.

2010 Cisco and/or its affiliates. All rights reserved.

SyslogDissection of a Syslog Message

Name of the Zone-Pair

AIC Policy Name

SyslogCheck for Packet Drops

SyslogCommon Packet Drop Reasons

Invalid Window scale option RST inside current window

SYN with data or with PSH/URG flags

TCP SYN packet is seen with data.

Syslog alert and audit-trail

Audit-trail for session establishment and tear down

2010 Cisco and/or its affiliates. All rights reserved.

Show CommandsZone-Based Firewall

To display zone-pair information

Show policy stats and session

2010 Cisco and/or its affiliates. All rights reserved.

Show Commands - Zone-Based Firewall

Show Commands - Zone-Based Firewall

To display the Firewall sessions

2010 Cisco and/or its affiliates. All rights reserved.

How to use packet captures for troubleshooting firewall issues?

Start the application thats failing

2010 Cisco and/or its affiliates. All rights reserved.

Using IOS Embedded Packet Captures

Using IOS Embedded Packet Captures

Dump the packet on the router itself

Now we have the packets captured, whats next?

Or export it out and analyze it in Ethereal/Wireshark

IPSec and Cisco IOS Firewall

IOS Firewall for IPSec pass-through traffic

2010 Cisco and/or its affiliates. All rights reserved.