f5 Afm Operations Guide
f5 Afm Operations Guide
f5 Afm Operations Guide
Operations Guide
Our series of operations guides address real-world scenarios and challenges. The content
was written by the engineers who design, build, and support our products, as well as other
F5 professionalssome former customers worked in the field and have firsthand experience.
In this guide youll find recommendations, practices, and troubleshooting tips to keep your F5
products running at peak efficiency and elements that may be incorporated into your own run
books.
Julian Eames
ii
Contents
Acknowledgements 1
Glossary 3
Customization 3
Issue escalation 4
Document conventions 4
Change list 6
Introduction 7
BIG-IP AFM features 7
Packet flow 9
Packet flow in BIG-IP hardware 9
Post-L4 processing 12
Firewall Rules 14
Network firewall 14
IP intelligence 16
Protocol security 18
NAT iRules 40
iii
Denial of Service 42
BIG-IP AFM DoS mitigations 43
Device DoS 53
External Tools 70
BIG-IQ Centralized Management 70
Syslog 74
IPFIX 74
sFlow 75
Troubleshooting 84
BIG-IP AFM network firewall modes 86
Rule actions 91
Policy compilation 91
Logging 92
Statistics 94
IP intelligence 103
F5 certification 107
iv
Review self-help options 108
Copyright 123
Trademarks 123
Patents 123
Notice 124
v
ACKNOWLEDGEMENTS
Acknowledgements
Executive sponsor: Julian Eames, Executive Vice President and Chief Operations Officer
Project team, writers, editors, and testers: James Affeld, Steve Brown, Chris Christian, Michael Earnhart, Brandon Frelich,
Brian Lawrence, Tikka Nagi, Nilesh Mistry, Dawn Moullet, Ryan Nelson, Brian Van Lieu, and Fred Wittenberg
BookSprints facilitator, designer, editor, and support team:Mark Brokering, Henrik van Leeuwen, Katerina Michailidi, Laia
Ros, and Raewyn Whyte. For information about the BookSprints process, see the BookSprints web site. (This link takes you to
an outside resource.)
Content, support, and assistance: Don Martin, Vice President, Global Services Strategic Programs; the Global Services New
Product Introduction Team, Bryan Gomes, Ilana Trager, Andy Franklin, Phillip Esparza, Derek Smithwick, Beth Naczkowski, Joe
Taylor, Mark Kramer, Andrew Pemble, Dave Bowman, Jim Williams, David Katz; and the rest of the Global Services management
team
1
ABOUT THIS GUIDE
Limits of this guide
The goal of this guide is to assist customers with keeping their BIG-IP system healthy, optimized, and performing as designed. It
was written by F5 engineers who assist customers with solving complex problems every day. Some of these engineers were
customers before joining F5. Their unique perspective and hands-on experience has been used to serve the operational and
maintenance guides F5 customers have requested.
This guide describes common information technology procedures and some that are exclusive to BIG-IP systems. There may be
procedures particular to your industry or business that are not identified. While F5 recommends the procedures outlined in this
guide, they are intended to supplement your existing operations requirements and industry standards. F5 suggests that you read
and consider the information provided to find the procedures to suit your implementation, change-management process, and
business-operations requirements. Doing so can result in higher productivity and fewer unscheduled interruptions.
For information on how to help improve future versions of this guide, see Feedback and notifications.
Installed your F5 platform according to its requirements and recommendations. Search the AskF5 Knowledge Base for
platform guide to find a selection of platform guides.
Followed the general environmental guidelines in the hardware platform guide to make sure of proper placement, airflow,
and cooling.
Set recommended operating s for your industry, accounting for predictable changes in load. For assistance, you can
contact F5 Professional Services.
Familiarized yourself with F5 technology concepts and reviewed and applied appropriate recommendations from the F5
BIG-IP TMOS: Operations Guide.
2
ABOUT THIS GUIDE
Customization
There is a wealth of documentation covering these areas in the AskF5 Knowledge Base. The F5 self-help community,
DevCentral, is also a good place to find answers about initial deployment and configuration. You can find additional resources
detailed in Acknowledgments. The following figure shows where this guide can best be applied in the product life cycle.
Glossary
A glossary is not included in this guide. Instead, F5 Glossary and Terms offers an up-to-date and complete listing and
explanation of common industry and F5-specific terms.
Customization
Customization may benefit your implementation. You can get help with customization from a subject matter expert, such as a
professional services consultant from F5 Professional Services.
3
ABOUT THIS GUIDE
Document conventions
Issue escalation
See Optimize the support experience for issue escalation information.
If you have an F5 WebSupport contract, you can open a support case by clicking Open a support case on the AskF5
Knowledge Base home page.
F5 operations guides are updated frequently, and new guides are being written. If you would like to be notified when new content
is available, email opsguide@f5.com and your name will be added to our distribution list for updates and new releases.
Document conventions
To help you easily identify and understand important information, the document in this guide uses the stylistic conventions
described here.
Examples
All examples in this document use only private IP addresses. When you set up the configurations described, use valid IP
addresses suitable to your own network in place of our sample addresses.
Configuration utility
The BIG-IP Configuration utility is the name of the graphic user interface (GUI) of the BIG-IP system and its modules. It is a
browser-based application you can use to install, configure, and monitor your BIG-IP system.
Configuration utility menus, sub-menus, links, and buttons are formatted in bold text. For more information about the
Configuration utility, see Introducing BIG-IP Systems in BIG-IP Systems: Getting Started Guide.
Command-line syntax
We show command line input and output in Courier font.
The corresponding # prompt is not included. For example, the following command shows the configuration of the specified pool
name:
tmsh show /ltm pool my _ pool
4
ABOUT THIS GUIDE
Document conventions
The following table explains additional special conventions used in command line syntax:
Character Description
<> Identifies a user-defined variable parameter. For example,
if the command has <your name>, type in your name but
do not include the brackets.
[] Indicates that syntax inside the brackets is optional.
Indicates that you can type a series of items.
You can run TMSH and issue commands in the following ways:
You can issue a single TMSH command at the BIG-IP system prompt using the following syntax:
tmsh [command] [module . . . module] [component] (options)
You can open the TMSH utility by typing tmsh at the BIG-IP system prompt:
#tmsh
(tmos)#
Once at the (tmos) prompt, you can issue the same command syntax, leaving off tmsh at the beginning.
For the sake of brevity, all TMSH commands provided in this guide appear in the first previous example.
You can use the command line utilities directly on the BIG-IP system console, or you can run commands using a remote shell,
such as the SSH client or a Telnet client. For more information about command line utilities, see Bigpipe Utility Reference Guide
or the Traffic Management Shell (tmsh) Reference Guide.
5
ABOUT THIS GUIDE
Change list
Change list
Date Chapter/Section Change Reason
Packet Flow
July 2016 Updates for BIG-IP 12.1 BIG-IP 12.1 release
Denial of Service
6
INTRODUCTION
BIG-IP AFM features
Introduction
BIG-IP AFM delivers the most effective network-level security for enterprises and service providers. Whether on-premises or in a
software-defined data center, BIG-IP AFM tracks the state of network sessions, maintains application awareness, and mitigates
threats based on more attack details than traditional network firewalls. BIG-IP AFM also protects your organization from
aggressive distributed denial-of-service (DDoS) attacks before they can reach your data center.
The majority of DDoS attacks exploit the transport and network layers. Layer
7 DDoS attacks are a more sophisticated form of DDoS attack which mimic
human behavior as they interact with the user interface at the application
level.
App-centric policy enforcement unifies the application configuration with security parameters for tighter policy
7
INTRODUCTION
BIG-IP AFM features
enforcement.
Intelligent control automatically guards against known bad actors at the earliest traffic flow point. In BIG-IP AFM 12.1
and higher, bad actor treatment is expanded to cover most DoS vectors to help select and disable individual sources of
malicious traffic. Each bad actor is handed off to IP intelligence and dropped for a configurable period of time
Layer-3 and layer-4 attack protection terminates all connections and runs checks to identify and mitigate network-
level threats before they reach the data center.
Centralized management enables efficient deployment and management for a consistent and effective security
posture across an expanding set of firewall devices.
High-volume logging controls log DDoS events, provide controls that prevent log servers from becoming
overwhelmed, and support SNMP, SIP, DNS, and IPFIX collectors.
ScaleN Virtual Clustered Multiprocessing (vCMP) consolidates multiple firewalls onto a single device for more
flexible and isolated allocation of resources.
8
PACKET FLOW
Packet flow in BIG-IP hardware
Packet flow
Unlike a firewall, which filters traffic based on internal versus external interfaces, BIG-IP AFM processes traffic through any
non-management interface using the same ingress to egress packet flow method. This means the packet processing is handled
the same way, regardless of the BIG-IP AFM interface being traversed.
The following figure provides an overview of the packet processing path as it traverses BIG-IP AFM.
For more detailed information on platforms which include the ePVA chips, see AskF5 article: SOL12837: Overview of the ePVA
feature.
Flow lookup
The system has two flow tables:
9
PACKET FLOW
Packet flow in BIG-IP AFM software
When a new packet is received, the BIG-IP system performs flow lookup by querying the hardware flow table.
1. If the BIG-IP platform uses ePVA hardware acceleration and the flow matches the hardware flow table, then the packet is
processed and sent directly to egress.
2. If there is a match on the hardware flow table, the packet is passed on to flow input for post L4 processing, in the direction
of egress.
3. If there is not match to an existing flow, the packet is processed for IP intelligence and L2-L7 DoS protection before being
passed on to TMOS flow lookup.
IP intelligence hardware
The ePVA is also used to process and implement IP intelligence rules to block malicious actors. If DoS sweep protection detects
a bad actor or group of actors, it can set an auto-blacklist. It can also signal the ePVA to drop the offending IP addresses in
hardware on some BIG-IP platforms so that they are not sent to software for further processing.
For more detailed information on hardware-processed attack vectors, see BIG-IP Systems: DoS Protection and Protocol Firewall
Implementations Manual.
Flow lookup
At packet ingress, TMOS checks to see if the packet is associated with an already established flow. During software flow lookup,
BIG-IP AFM tries to match the packet to an entry in the software connection flow table. There are two possible results:
If the packet does not match an existing flow, it is considered a new connection. TMOS then tests the packet against
L2-L4 DoS protection, listener lookup, IP intelligence, and the network firewall contexts.
If the packet does match a software flow connection table entry, TMOS sends it to flow input, bypassing the flow lookup
tests.
10
PACKET FLOW
Packet flow in BIG-IP AFM software
If an incoming packet exceeds the rate limit for that type of packet, it is dropped.
DoS protection device configuration and DoS profiles provide different vector protections; however, applying DoS protection
device configuration is essentially the same as applying a DoS protection profile at the global context.
Auto-blacklisting option
Only theSingle Endpoint Sweepvector supports theauto-blacklistconfiguration option. If auto-blacklist is configured and
packets exceed the rate limit for Single Endpoint Sweep, DoS protection triggers IP Intelligence to block the source whether a
packet is legitimate or part of an attack.
Listener lookup
Listener lookup checks to see if the packets destination is valid.
If there is no listener at the packet destination address, the packet is dropped or rejected, depending on your configuration.
To change the setting to drop unmatched packets using the Configuration utility
2. Under Properties go to Reject Unmatched Packets and clear the Enabled checkbox.
IP intelligence
IP address intelligence blocking can block requests from IP addresses that have questionable reputations.
IP intelligence is applied to packets in the global, route domain, and virtual server contexts. A set of IP intelligence policies can be
configured so that they might allow a packet to pass through at the global context and the route domain context, but drop it in a
virtual server context.
Network firewall
The BIG-IP AFM network firewall provides policy-based access control to and from address and port pairs, inside and outside of
your network.
The Network Firewall uses the same three context settings as IP intelligence: global, route domain, and virtual server. In each
context, the IP intelligence packet handling occurs first, followed by the network firewall handling:
11
PACKET FLOW
Post-L4 processing
IP intelligence route domain context, then network firewall route domain context.
IP intelligence virtual server context, then network firewall virtual server context.
Post-L4 processing
Packets are routed to post-L4 processing. Packets arrive through flow input after passing through hardware processing or
through flow accept after passing through both hardware and software processing.
Flow accept
Once a flow is accepted decisively or simply accepted at the virtual server/self-IP context, it reaches flow accept, where it is
recorded in the flow table and passed to the proxy for higher-level protocol processing.
BIG-IP AFM 12.1 and higher allows you to specify the L7 protocol you would like to use. HTTP traffic allowed through a firewall
typically uses port 80, and malicious traffic often tunnels through this port. The L7 protocol option allows you to specify the port
you would like to use and it does not need to be the customary application port. You can then restrict traffic only suited to that
application. This ensures that holes in the firewall are being used for the intended applications.
For more detailed information on profiles, see DoS Protection and Protocol Firewall Implementations Detecting and Preventing
DNS DoS Attacks and DoS Protection and Protocol Firewall Implementations Detecting SIP DoS Attacks.
Protocol security
After L7 DoS protection profile processing, L7 protocol protection profiles are applied. These are available for HTTP and DNS
profiles.
The protocol security profile for DNS determines which DNS queries are permitted.
The protocol security profile for HTTP performs protocol checks, length checks, checks request types (for example GET and
POST), and file types. The profile can also send a custom response page to any requests that are blocked by the policy.
TMOS proxy
Traffic is passed to the proxy, where normal traffic processing by BIG-IP modules occurs. This may include BIG-IP ASM DoS
vectors to enhance the other layers of DoS protection covered by BIG-IP AFM.
12
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
FIREWALL RULES
Network firewall
Firewall Rules
The BIG-IPnetwork firewall uses rules to specify traffic handling actions. Rules are collected in policies, which are applied at the
global context, to a route domain, to a virtual server, or to a self IP address. Rules for the management port do not require a
policy, but are defined directly in the management port context.
The BIG-IP system itself provides some access control measures: packets that do not match a listener are dropped or rejected.
Additionally, application profiles, such as SIP, add more limits regarding the type of traffic that is permitted.
Protocol Security provides fine-grained controls for the DNS and HTTP protocols.
Network firewall
14
FIREWALL RULES
Network firewall
As shown in the previous figure, packet flow arrives at the BIG-IP network firewall. The network firewall uses a collection of
network ACLs to process it.
BIG-IP AFM applies the global, route domain, virtual server, and self IP contexts to the packets in order, each context looking for
an ACL match.
If no match is found, the Global Default Firewall Action or the Virtual Server & Self IP Default Firewall Action is
triggered.
If the flow matches a configured virtual server but no ACLs match within this context, the Virtual Server & Self IP Default
Firewall Action is triggered.
Rules may include protocol, source address and port, source VLAN, destination address and port, schedule, action, and
logging. BIG-IP AFM can also assign an iRule and a service policy to an individual firewall rule, allowing additional functionality to
be added to the network firewall.
You can apply a service policy, such as an idle timeout, to an ACL match rather than to a listener. Doing this enables you to
customize a policy at a granular level without requiring a large number of configuration objects.
You can use iRules within a firewall rule to allow scripting to be applied when an ACL-match occurs. Used with the FLOW_INIT
event, you can change ACL actions or other early flow items. iRules can also be triggered for higher-level events such as
HTTP_REQUEST.
In order to make all of the firewall objects reusable, BIG-IP AFM includes a number of lists that can be used in the policies.
Port list
The port list groups port numbers and port ranges. It contains a list of numbers that are not specific to any protocol. For
example, a list containing port 53 could be used to both TCP and UDP based DNS rules.
Address list
The address list contains IP addresses of a variety of types, including the following:
Geolocation match
15
FIREWALL RULES
IP intelligence
Rule list
The rule list is a grouping of individual rules.
F5 recommends using these lists for all aspects of rule creation. That is, all rules should be made up of address and port lists and
should only be created within a rule list. This practice simplifies administration in the long-term by allowing groups of components
to be used within rule lists or policies.
For more detailed information on rule lists, see BIG-IP Network Firewall: Policies and Implementations: Firewall Rules and Rule
Lists and BIG-IP Network Firewall: Policies and Implementations: Setting Timers with Service Policies.
IP intelligence
IP Intelligence is a firewall protection, separate from the network firewall, which examines only the source address of a packet. It is
possible to create network firewall rules and policies to block on source and destination address, but using IP intelligence makes
automation easier.
IP reputation subscription
An IP reputation database feed, provided by a third-party security vendor, serves as the first input source for IP intelligence.
F5 offers a built-in subscription that can be added to any existing BIG-IP AFM deployment.
Dynamic whitelist/blacklist
A dynamic whitelist/blacklist feed provides is another IP intelligence input source. It allows BIG-IP AFM to consume a custom
feed of IP addresses to be enforced.
The dynamic whitelist/blacklist feed may come from external sources, including firewall logs, IPS alerts, or other sources of
known bad actors.
The format of the feed must contain an IP address and may contain several optional fields.
For more information see IP Intelligence in Network Firewall in BIG-IP Network Firewall: Policies and Implementations.
There is a limit of 100 bad actors on all platforms other than the VIPRION 2250 blade because this is the maximum that the single
endpoint Sweep can handle. The 2250 blade implements IP intelligence in hardware, so it can drop traffic from bad actors before
DoS Protection sees it. This allows the single endpoint Sweep vector to move on to another set of 100 bad actors on that
platform. On all other platforms, IP intelligence acts after software DoS Protection. In these cases DoS Protection continues to
see traffic from the bad actors.
16
FIREWALL RULES
IP intelligence
5. ClickFinished.
IP intelligence whitelist
IP intelligence uses a comprehensive whitelist feature. F5 recommends including critical infrastructure on it.
You can add whitelist IP addresses to your configuration automatically by setting up feeds and capturing them with a feed list.
4. Configure Feed URLs with an HTTP, HTTPS, or FTP URL, the list type, the blacklist class, and the polling
interval. Specify a username and password, if required to access the feed list.
A feed URL includes the actual URL to the text file, and information about the defaults for that file. Within the feed
file, however, any URL can be configured to be a whitelist or blacklist entry, and assigned to a blacklist class.
6. ClickFinished.
17
FIREWALL RULES
Protocol security
For information on configuring IP intelligence, see Enabling IP Intelligence in BIG-IP Local Traffic Manager: Implementations.
Protocol security
Protocol security allows restrict of application behavior for the DNS and HTTP protocols. Protocol Security profiles are applied to
application profiles which are applied to virtual servers.
A DoS protocol security profile is applied to a Local Traffic DNS profile. It is a prerequisite for using DoS Protection profiles for
DNS.
18
FIREWALL RULES
Protocol security
POST request with Content-Length: 0 A POST should always have non-zero length.
Body in GET or HEAD requests A GET or HEAD request should not have body.
Some HTTP protocol checks are only appropriate for particular web applications which may have clients that exhibit unusual
behavior.
To decide whether or not the HTTP protocol checks are suitable for your application, F5 recommends using the guidelines in the
following table:
High ASCII characters in headers High ASCII may hide injected code which the
application might eventually map to regular ASCII. It is
highly unusual for a client to have a legitimate need to
do this. Unless such a client is used, enable this check.
Host header contains IP address Most web applications (such as web browsers) use
a domain name in the Host header. IP addresses
are valid, but are commonly used by bots. Some
mobile applications use IP addresses. If the expected
application traffic is entirely from normal web
browsers, enable this check.
By default, an HTTP protocol security profile triggers alarms but does not block traffic.
F5 recommends that you enable Block after a period of tuning your policy to avoid false positives.
For example:
/webroot/legit-directory/../../../etc/shadow
Other examples include the use of obscure encodings (multiple encoding, UNICODE, ASCII, and others). The web server
19
FIREWALL RULES
BIG-IP AFM rules
Some legitimate clients may present behavior that appears evasive. F5 recommends reviewing alarms to determine a false
positive rate and enabling blocking if that rate is acceptable.
Request checks
Requests checks options inspect requests. By default, the checks are set to trigger an alarm, not to block traffic.
Length Checks: You can configure settings for URL length, Query String length, Request length, and POST Data length.
F5 recommends alarming on these checks and enabling Blocking if the false positive rate is acceptable.
Methods: You can select HTTP methods to accept (for example GET, HEAD, and POST).
F5 recommends blocking undesired request methods.
Blocking page
The profile can return a blocking page in response to a blocked request. You can use the default response, create a custom
response, redirect to a URL, or use a Simple Object Access Protocol (SOAP) error message.
Tracking can help determine change details, which is useful when a change generates an outage or needs to be justified to an
auditor, or simply as documentation for future operators.
For consistency and ease of use, F5 recommends using a predetermined, standardized format when entering this data.
Rule efficiency
BIG-IP AFM 12.0 introduces performance optimizations for the rules compiler that can produce smaller, faster compiled policies
from previous versions of BIG-IP AFM. Complex rules with multiple lists and ports can now be compiled more efficiently than
previous versions.
20
FIREWALL RULES
BIG-IP AFM rules
Rule expiration
When creating standardized descriptions for firewall rules, consider adding an expiration date in the description section, when
applicable.
Expiration dates can assist during firewall audits and help to avoid leaving a temporary troubleshooting rule in place that was
intended to be used for only a short time period.
Note Rule expiration is not meant to be used as a function of the BIG-IP AFM
expiration scheduling feature. The expiration scheduling feature is a separate
function, available within a firewall rule.
Redundant rule: A rule which has address, user, region, or port information that completely overlaps with another rule
with the same action. Redundant rules should be removed to simplify policy management.
Conflicting rule: A conflicting rule is a special case of a redundant rule, in which address, user, region or port
information overlaps with another rule, but the rules have different actions, and thus conflict.
Note A rule may be identified as conflicting with another rule, even if the
result of applying the two is the same. For example, two rules designed to
accept packets and applied to the same IP address can be identified as in
conflict if one is configured to Accept and the other is configured to Accept
Decisively. Accept and Accept Decisively can lead to different behaviors
based on the context and design of the firewall rule set.
To view redundant and conflicting rules using TMSH at the command line
21
FIREWALL RULES
BIG-IP AFM policies
To reset the stats of an individual global rule using TMSH at the command line
Contexts
With the BIG-IPnetwork firewall, you use a context to configure the level of specificity of a firewall rule or policy. Firewall policies
can be applied at the global, route domain, virtual server/self IP contexts.
Depending on your organizations needs, you may prefer to put all active rules in a single policy applied at the global context or
apply firewall policies for specific virtual servers. The latter allows for application-specific policies to be developed and applied
only where required. When processing policies and rules on a virtual server, only those specific to the application are processed.
Staging
Policies can be enforced or staged within a context. A staged policy logs rule matches and increment statistics but does not take
any enforcement actions.
A staged policy previews a policys effect if enforced. A staged policy can easily be promoted to enforced policy once youve
validated the policys effects.
2. In the Firewall Policy Management section, for Firewall Compilation Mode, select Manual.
3. Click Update.
When Firewall Compilation Mode is set to Manual, changes to a firewall policy cause the policy to enter Pending Rules
Compilation status. This status line displays in the upper left-hand corner in the Configuration utility (see the following figure):
22
FIREWALL RULES
BIG-IP AFM policies
3. Click Compile.
4. Status line changes to Firewall: Pending Rules Deployment (see the following figure):
1. Click Deploy.
23
FIREWALL RULES
BIG-IP AFM policies
If the Network Firewall Options for Firewall Policy Management is configured so that Log Configuration Changes is set
on and either Firewall Compilation Mode or Firewall Deployment Mode is set to Manual, the following entries display as
compilation and deployment progresses:
Compilation Start
Compile Success
Deploy Start
Deploy Success
Warning: Large policy changes that are being compiled and deployed may
put load on logging.
1. Go to Security >> Options : Network Firewall. The Firewall Options screen opens.
2. From the Firewall Compilation Mode list, select the compilation mode for the firewall rule set.
Select Automatic to compile the firewall rule set whenever a change is made to any firewall item that is used
in the firewall rule set.
Select Manual to delay compilation of the firewall rule set, collect all firewall rule changes, and apply the entire
24
FIREWALL RULES
BIG-IP AFM policies
3. From the Firewall Deployment Mode list, select the deployment mode for firewall rule set changes.
Select Automatic to deploy the firewall rule set whenever a change is compiled, either manually or
automatically.
Select Manual to delay deployment of the firewall rule set, collect all compiled firewall rule set changes, and
deploy the entire set of changes manually at another time.
4. From the Log Configuration Changes list specify the logging option for firewall rule set compilation and
deployment configuration changes.
Select Automatic to specify that configuration changes are logged only if Firewall Compilation Mode or
Firewall Deployment Mode is set to Manual.
Select Off to specify that policy configuration changes are not logged.
As part of firewall maintenance, regularly validate expired rules, unused rules, conflicting rules, and rules not ordered correctly.
Each rule is listed as Enabled, Disabled, or Scheduled. In addition, a rule can have one of the states listed
below. View and adjust rules with these states, if necessary.
Redundant The rule is enabled, disabled, or scheduled, and redundant. All the functionality of this rule is
provided by a previous rule or rules. Hover over the State column to see why the rule is considered redundant,
25
FIREWALL RULES
BIG-IP AFM iRules
Conflicting The rule is enabled, disabled, or scheduled, and conflicting. All the match criteria of this rule is
covered by another rule or rules, but this rule has a different action. Hover over the State column to see why
the rule is considered conflicting, and possible solutions.
Conflicting & Redundant The rule is enabled, disabled, or scheduled, and conflicting or redundant with the
actions of more than one other rule.
4. Resolve conflicting or redundant rules by editing, deleting, or disabling them. To edit, delete, or disable a rule,
click the name and complete the required action.
View unused rules to reduce firewall processing and simplify the rules, rule lists, and policies.
The BIG-IP AFM records the ACL hit count and the last hit time in the Configuration utility and TMSH for ease of identifying
unused or infrequently used rules.
If an iRule is defined within a firewall rule, it is called when the firewall rule is processed.
If the iRule is attached to a virtual server, the iRule is called after BIG-IP AFM firewall rules have been processed.
4. Click Finished.
26
FIREWALL RULES
BIG-IP AFM iRules
Command Action
Node Redirects to specified remote host, which could be another virtual server or pool
member.
TCP::[close | respond] Closes a TCP connection or sends the specified data directly to the peer.
FLOW_INIT
BIG-IP AFM iRules can perform the same actions as firewall rules. You can use the FLOW_INIT event in BIG-IP iRules to override
an ACL action, to control bandwidth on client and server flows, and to route to another VIP.
For more information on iRule events see Master List of iRule Events.
Action Description
27
FIREWALL RULES
Rules and policies troubleshooting
Action Description
Allow-final Allows the connection and bypass any further ACL. Allow-final is equivalent to
Accept-Decisively.
If a firewall rule is configured to accept traffic but an iRule rejects the traffic, the event logs show the traffic as Accepted.
For remote HSL use the HSL iRule facilities HSL::open and HSL::send.
The dwbld daemon retrieves feed lists configured and managed by the user. It logs to /var/log/dwbl/dwbld.log. A db key
governs the log level, but for this daemon debug logging does not add much detail to the output.
28
FIREWALL RULES
Rules and policies troubleshooting
To see view the classification of an IP address using TMSH at the command line
The dwbld daemon retrieves feed lists configured and managed by the user. It logs to /var/log/dwbl/dwbld.log. A db
key governs the log level, but for this daemon debug logging does not add much detail to the output.
To see view the classification of an IP address using TMSH at the command line
29
FIREWALL RULES
Rules and policies troubleshooting
Note If a host is listed under several classifications, you need to delete the
host from every classification that has an undesired policy action defined.
Example: 10.10.10.10 is classified under botnets and spam_sources, and
both are dropped by the relevant IP intelligence policy. Deleting it from
botnets will not affect its membership in spam_sources, and its traffic will be
dropped by IP intelligence.
You can also check /var/dwbl/.cache/* to see if the system is healthy. The
cache is refreshed every cycle, which should be every five minutes.
Feed List
If the BIG-IP system connects to the Internet using a forward proxy server, use the following commands to set these system
database variables.
To specify the host name of the proxy server using TMSH at the command line
To specify the port number of the proxy server using TMSH at the command line
To specify the user name to log in to the proxy server using TMSH at the command line
30
FIREWALL RULES
Rules and policies troubleshooting
To specify the password to log in to the proxy server using TMSH at the command line
To check the BIG-IP DNS client configuration using the Configuration utility
31
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
NETWORK ADDRESS TRANSLATION (NAT)
Rules and policies troubleshooting
Outbound NAT
Outbound NAT translates an internal source address to a public address. A NAT can also be used to translate an internal nodes
IP address to an Internet routable IP address.
Inbound NAT
Inbound NAT translates a public destination address to an internal address. When an external client sends traffic to the public IP
address defined in a NAT, BIG-IP translates that destination address to the internal node IP address.
33
NETWORK ADDRESS TRANSLATION (NAT)
Rules and policies troubleshooting
To create NAT to allow translation of one IP address to another using the Configuration utility
4. In the NAT Address field, type the IP address to use as the translation address, that is, the address to which
the origin address will be translated.
5. In the Origin Address field, type the IP address of the node to which the translation will be applied.
34
NETWORK ADDRESS TRANSLATION (NAT)
SNAT
At the end of this task, the NAT appears in the list of NATs on the system.
NAT can leverage IPv6, IPv4 or translate between the two for either the client or server side of the connection.
For more detailed information NATs and SNATs, see NATs and SNATs in BIG-IP TMOS: Routing Administration.
SNAT
NAT, by design, is a one-to-one operation, while Secure Network Address Translation (SNAT) can be used to map many IP
addresses to one IP in order to hide the source IP network(s).
Inbound SNAT
In the most common client-server network configuration, the BIG-IP address translation mechanism ensures that server
responses return to the client through the BIG-IP system.
35
NETWORK ADDRESS TRANSLATION (NAT)
SNAT
The solution is to create a SNAT. The BIG-IP system then translates the client nodes source IP address in the request to the
SNAT address, causing the server node to use that SNAT address as its destination address when sending the response. This,
in turn, forces the response to return to the client node through the BIG-IP system rather than through the server default
gateway.
36
NETWORK ADDRESS TRANSLATION (NAT)
SNAT
Figure 3.4 SNAT used when BIG-IP is not the default route
Outbound SNAT
When an internal server initiates a connection to an external host, a SNAT can translate the internal source IP addresses of one
or more servers within the outgoing connection to a single, publicly routable address. The external destination host can then use
this public address as a destination address when sending the response. In this way, the internal source IP addresses of the
internal nodes remain hidden from the external host.
1. BIG-IP receives a packet from an internal server with an internal IP address and checks to see if that source address is
defined in a SNAT.
2. If the internal IP address is defined as the origin IP in SNAT, BIG-IP changes that source IP address to the translation
address defined in the configured SNAT.
3. BIG-IP then sends the packet, with the SNAT translation address as the source address, to the destination host.
SNAT types
Standard SNAT
A standard SNAT is an object created using the BIG-IP Configuration utility that specifies the mapping of one or more IP
addresses to a translation address. For this type of SNAT, the criteria that the BIG-IP system uses to decide when to apply the
translation address is based strictly on the IP address. If a packet arrives from the IP address configured in the SNAT, then the
37
NETWORK ADDRESS TRANSLATION (NAT)
SNAT
BIG-IP system translates that address to the configured translation address. There are three types of standard SNATs:
A SNAT configured to select an address from the SNAT pool as the translation address.
Intelligent SNAT
Like a standard SNAT, an intelligent SNAT is the mapping of one or more IP addresses to a translation address. This type of SNAT
mapping is implemented within an iRule instead of creating a SNAT object. For this type of SNAT, the criteria that BIG-IP uses to
decide when to apply a translation address is based on the logic of the iRule.
Port exhaustion
Each SNAT address has only 65,535 ports available. This is a limit of the TCP and User Datagram Protocol (UDP) protocols,
which use a 16-bit unsigned integer for source ports.
Port exhaustion or collisions may occur under heavy usage or unusually distributed client traffic patterns. For performance
reasons, the BIG-IP will not search exhaustively for an available source port. Port exhaustion may occur well before all 65,535
ports are used. As a result, connections that cannot be translated due to lack of available ports on a given translation address
may be dropped.
To determine when SNAT port exhaustion is occurring by reviewing the system log files. When port exhaustion occurs, BIG-IP
logs messages to the /var/log/ltm file.
To mitigate port exhaustion, use a SNAT Pool. If already using a SNAT Pool, add more IP addresses to the pool.
Port mapping
When a SNAT is configured on the BIG-IP system (independently or in conjunction with a virtual server), the source address of
each connection is translated to a configured SNAT address, and the source port is mapped to a port currently available for that
38
NETWORK ADDRESS TRANSLATION (NAT)
Rules and policies troubleshooting
SNAT address. By default, the BIG-IP system attempts to preserve the source port, but if the port is already in use on the
selected translation address, the system translates the source port.
For more detailed information on SNAT, see AskF5 article: SOL7336: The SNAT Automap and self IP address selection.
SNAT statistics
To monitor the number of concurrent connections going through the SNAT using the Configuration utility
To monitor the number of concurrent connections going through the SNAT using TMSH at the command
line
39
NETWORK ADDRESS TRANSLATION (NAT)
NAT iRules
NAT iRules
iRules can be used to create intelligent SNAT or to apply NAT to IP addresses that traverse a BIG-IP system.
The following is a sample iRule to apply NAT based on entries in a data group.
when CLIENT _ ACCEPTED {
set ip _ split [split [IP::local _ addr] %]
#log local0. split $ip _ split
set remote _ ip [lindex $ip _ split 0]
#log local0. split remote is $remote _ ip
set snat _ addr [class match -value $remote _ ip equals /All _ Firewalls]
#log local0. IP: $remote _ ip SNAT: $snat _ addr
if { $snat _ addr ne } {
#log local0. IP: $remote _ ip SNAT: $snat _ addr
snat $snat _ addr
}
}
40
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
DENIAL OF SERVICE
NAT iRules
Denial of Service
The BIG-IP AFM system provides mitigation techniques against DoS/DDoS attacks.
Denial of Service (DoS) attacks are attempts to render a machine or network resource unavailable to its intended users. Most
network DoS attacks are carried out at OSI Layers 3, 4, or 7.
The attacks work by overwhelming the server resources by directing traffic at a particular IP address and/or port with an
inordinate amount of either legal traffic or malformed requests that exhaust available resources. For example, targeting of the
number of concurrent layer 4 connections or available bandwidth are just a couple of common DoS attacks. The end result of
these attacks is the service is denied to legitimate users.
Stateful Asymmetric Attacks designed to abuse memory by invoking timeouts of session-state changes.
42
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
Attacks can be launched as a single attack or a combination of any of the listed typed.
On-premises equipment is the second tier of DoS mitigation. While the volume of the attack may not be enough to consume all
available network resources, the attackers strategy is to consume enough resources to disrupt an application or group of
applications. The mitigation strategy here is to provide capacity to absorb the DoS while maintaining targeted service delivery
goals.
BIG-IP AFM detects attacks based on thresholds in packets per second (PPS) or by percentage deviations from the previous
hours baseline observed values. To mitigate attacks detected by these vectors, rate limiting can be applied to devices violating
the thresholds.
In BIG-IP AFM 12.1 and higher, auto-thresholds can be used to make an educated estimate for you if you do not have the time or
ability to research baseline levels.
Attack vectors
The BIG-IP system classifies common types of DoS attacks into attack vectors. There are many different types of attack vectors
which have been designed to exploit different system vulnerabilities. The specific number and type of attack vectors that BIG-IP
AFM protects against depends on the BIG-IP TMOS version being used. New releases of TMOS incorporate additional attack
vectors and configuration options as attack types evolve.
Each DoS attack vector contains three different settings to customize when BIG-IP AFM detects an attack has started, and
when BIG-IP AFM begins mitigating the attack by rate limiting attack packets.
Attack detection
Each BIG-IP AFM attack vector contains two configuration settings that can be used to recognize when a possible attack has
started against the device: Detection Threshold Packet Per Second (PPS) and Detection Threshold Percentage.
Detection Threshold Packets Per Second is used as an early warning indicator that an attack may be occurring. When the
43
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
detection threshold is exceeded, BIG-IP AFM generates a log message of this event to notify you of the condition. This setting is
for attack detection only and is not used to mitigate attacks.
When an attack vector is enabled, BIG-IP AFM tracks statistics on how many packets arrive at the device related to that vector.
Packets are sampled once every second. This configured value is per Traffic Management Microkernel (TMM) instance, not a
system total. Set this value based on your preference and the traffic-handling ability of the application.
For more detailed information on TMM, see Ask F5 article SOL 14358: Overview of Clustered Multiprocessing (11.3.0 and later).
Figure 4.1 Detection Threshold Packets Per Second detects when configured threshold is exceeded
Detection Threshold Percentage is also used as an early warning indicator that an attack may be occurring. When the
detection threshold is exceeded, BIG-IP AFM generates a log message of this event to notify you of the condition. This setting is
for attack detection only and is not used to mitigate attacks.
When an attack vector is enabled, BIG-IP AFM compares the average rate of traffic related to that vector over the last hour to the
average rate of traffic over the last minute.
When the quotient between the average rate of traffic over the last hour and the average rate of traffic over the last minute
exceeds the threshold setting, BIG-IP AFM creates a log message and reports the PPS every second until the traffic recedes
below the threshold.
Note BIG-IP AFM must first collect three hours worth of traffic in order to
create a value for the average rate of traffic over the last hour. In addition the
average rate must be above 100 PPS for this threshold to be triggered.
This configured value is per TMM instance, not a system total. Set this value based on your preference and the traffic handling
ability of the application.
For more information on threshold limits for each TMM see Ask F5 article SOL 15023: The BIG-IP AFM system enforces
configured thresholds and limits for each TMM.
44
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
Figure 4.2 Detection Threshold Percentage compares the average rate of traffic related to that vector over the last hour to the average rate of traffic
over the last minute
Attack mitigation
Each BIG-IP AFM attack vector can be configured to rate-limit traffic based on a specified PPS value.
Internal Rate Limit is used to specify an absolute PPS value which cannot be exceeded for traffic related to the vector. Once
the limit is reached, BIG-IP AFM begins dropping traffic which exceeds it. All packets exceeding the threshold are dropped and
continue to be dropped until the PPS rate falls below the configured rate limit.
The Internal Rate Limit configured value applies to each TMM instance, not to a combined system total. Set this value based
on your preference and the traffic handling ability of the application.
Figure 4.3 Packets exceeding the Internal Rate Limit are dropped. In this example, the Internal Rate Limit is set to 10000
45
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
Attack phases
There are four phases to a BIG-IP AFM DoS attack mitigation:
46
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
The previous figure models BIG-IP AFM detects a rapid increase (fast ramp) in packets using the Default Threshold Percent
to detect attacks and the Default Internal Rate Limit to mitigate. Detection Threshold set to Infinite.
Phase 1 Vector mitigation is enabled and the rate of packets for the vector maintains an average of 500
PPS. No attack present.
Phase 2 Event threshold begins with a flood of packets arriving that match the vector. Attack is detected
once the volume of packets crosses the Detection Threshold Percentage.
Phase 3 BIG-IP AFM logs the packet rate for the detected DoS vector once every second per TMM. The
packet rate continues to increase very quickly (within a few minutes) until it has exceeded the
Default Internal Rate Limit. At this point, BIG-IP AFM begins to rate limit all packets for this
DoS vector above the 10000 PPS threshold. BIG-IP AFM rate limits packets until the rate drops
below the Default Internal Rate Limit. Once the rate drops below the Default Internal
Rate Limit, BIG-IP AFM stops dropping packets, but logging continues every second the PPS
rate for the detected vector.
Phase 4 PPS drops below the Detection Threshold Percentage. BIG-IP AFM stops logging packets
and changes the attack state to none.
47
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
The previous figure models BIG-IP AFM detects a slow increase (slow ramp) in packets.
Phase 2 Over several hours the number of packets steadily increases, which in turn increases the
Average Rate Over Last Hour statistic. BIG-IP AFM observes and updates. Phase 2 begins
when the packet rate crosses the Detection Threshold PPS.
Phase 3 BIG-IP AFM checks the PPS every second and logs the DoS attack vector event messages.
The rate continues to steadily increase over several hours. This increases the Average Rate
Over Last Hour. The slow rise in PPS prevents the Detection Threshold Percentage
increase from triggering. Eventually the packet rate exceeds the default internal rate limit and
BIG-IP AFM begins to rate limit all packets above that level until the packet rate drops below
the rate limit threshold.
Phase 4 When the attack PPS drops below the rate limit threshold, BIG-IP AFM stops rate-limiting
packets. If the attack is still ongoing BIG-IP AFM continues to log event information every
second until the rate drops below the Detection Threshold PPS.
For more information on using BIG-IP AFM to reduce the impact of DoS attacks, see The F5 DDoS Protection Reference
48
DENIAL OF SERVICE
BIG-IP AFM DoS mitigations
Architecture.
Architecture
DoS configuration occurs either at the device level or the virtual server level. Device DoS configuration is designed to detect and
mitigate network layer DoS attacks across all services protected by the BIG-IP AFM.
DoS profiles configured and applied to the virtual server level usually are applied to detect and mitigate attacks to a particular
application or group of application servers.
Both virtual server and device DoS protection statistics are processed and counted. Global DoS configuration is applied first then
virtual server detection and mitigation thresholds. Attacks dropped at the virtual server are still counted against the device
threshold count.
DoS profiles designated to be applied at the virtual server level offer a subset of the overall DoS attack vectors. Some DoS
vectors can only be configured at the device level. Counters can count against both when determining thresholds for detection
and mitigation.
Note DoS Profiles thresholds values are enforced for the device as a whole
not per TMM. For more information see Ask F5 article SOL15023 The BIG-IP
AFM system enforces configured thresholds and limits for each TMM.
DoS profiles for SIP require that a SIP protocol profile be applied on the virtual server. Likewise, DoS profiles for DNS require a
DNS protocol profile and a DNS protocol protection profile be applied on the virtual server.
To reduce the requirements of the operating system (TMOS) processing attacks and maximize system resources, many DoS
vectors are offloaded into programmable FPGA hardware.
For more information on which vectors are processed in hardware see Detecting and Preventing System DoS and DDoS Attacks
in BIG-IP Systems: DoS Protection.
Whitelisting
DoS whitelists provide a mechanism to configure trusted networks, protocols, and VLANs to bypass DoS checks and mitigations.
In BIG-IP TMOS 12.0 and higher, the DoS whitelist is limited to eight (8) entries. They can be based on source and/or destination
of hosts, networks, or VLANs. They include network protocols with or without specific ports, or combinations of these settings.
Use of super nets and larger masks to white list traffic or ingress VLANs is typically more efficient due to the limit on entries.
2. Click Create.
49
DENIAL OF SERVICE
Packet processing (SYN cookie protection)
4. Click Finished.
When the SYN Check Activation Threshold value is reached, the BIG-IP system responds to SYN requests by sending back
50
DENIAL OF SERVICE
Packet processing (SYN cookie protection)
to the client the SYN+ACK response containing an encoded secret. The system then discards the SYN queue entry and waits
for a correctly constructed ACK from the client before establishing an entry in the connection table.
The SYN cookie secret can be calculated in hardware or software depending on the platform. This behavior can be modified by
adjusting the value for the SYN cookie algorithm database key.
For more detailed information on connection SYN Cookies, see AskF5 article: SOL 16500: Overview of the connection.
syncookies.algorithm database key.
When the SYN cookie authentication method is active for a virtual server or self IP address, established connection/packet
handling and high-availability features such as mirroring should perform normally.
When SYN cookie protection is enabled for the protocol profile, the feature operates as follows:
1. A client sends a TCP SYN request to the BIG-IP virtual server or self IP address.
2. The receiving TMM instance determines whether to enable hardware or software SYN cookie protection as follows:
If the platform contains the high speed bus (HSBe2) chip, and hardware SYN cookie protection is enabled in the profile,
the TMM notifies the HSB chip and other TMM instances in the cluster to enable hardware SYN cookie protection. The
HSB chip and receiving TMM instance will then program HSB hardware for hardware SYN cookie generation and
51
DENIAL OF SERVICE
Packet processing (SYN cookie protection)
validation for the virtual server or self IP address, and synchronize the status to all TMM instances in the cluster.
For TCP and FastHTTP profiles, if the platform does not contain the HSB chip, or hardware SYN cookie protection is
disabled in the profile, the TMM notifies other TMM instances in the cluster to enable software SYN cookie protection.
For FastL4 profiles, if the platform does not contain the HSB chip, or hardware SYN cookie protection is disabled, SYN
cookie protection is not available for the virtual server unless the software SYN cookie protection option is specifically
enabled.
3. The BIG-IP system sends the SYN+ACK response back to the client, but discards the SYN queue entry. The BIG-IP
system does not maintain the SYN-RECEIVED state that is normally stored in the connection table for the initiated session.
Because the system does not maintain the SYN-RECEIVED state for the connection, the SYN queue is not exhausted, and
normal TCP communication continues.
4. If the BIG-IP system then receives a subsequent ACK response from the client, the system reconstructs the SYN queue
entry by decoding data in the TCP sequence number.
5. After the BIG-IP system validates the clients ACK, the system adds the session to the connection table and initiates a
connection to the pool member.
For more detailed information on SYN cookie protection see AskF5 article: SOL 14779: Overview of BIG-IP SYN cookie protection
(11.3.x - 12.x) .
To validate SYN Cookie performance in hardware or software issue using TMSH at the command line
52
DENIAL OF SERVICE
Device DoS
Device DoS
Device-level DoS protection allows BIG-IP AFM to detect and automatically mitigate DoS attacks.
Various detection options are available, including a packets per second threshold, rate increase, rate limit, and other parameters
for DoS attack types.
For more detailed information on each attack type, see Detecting and Preventing System DoS and DDoS Attacks chapter in
BIG-IP Systems: DoS Protection and Protocol Firewall Implementations.
Invalid packets (bad header*, fragmentation, IP unknown protocol, land attack, and others.)
Probably invalid packets (TCP RST, TCP SYN-ACK, and TCP Push)
The detection and rate limit properties of these categories and vectors can be set with the following ranges:
The values vary depending on BIG-IP platform and environment, as well as whether the vector type is configured through DoS
device configuration or a DoS profile:
DoS Device Configuration settings should be set at the level required to protect the stability of the BIG-IP system. Packets that
are presumed to be valid should only drop in an emergency since this action affects every virtual server on the BIG-IP system.
DoS Profiles are applied to virtual servers. This configuration places the mitigation with the target, so rate limiting may drop
legitimate traffic, it limits the mitigation effects to the virtual server under DoS attack. It is also likely that the server pool members
associated can tolerate less overall stress than the BIG-IP system.
F5 recommends setting a lower detect and rate limit for DoS profiles to protect pool members.
53
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
Invalid packets
Invalid packets are those which violate protocol specifications. This includes bad header vectors.
While standard TMOS packet handling silently drops invalid packets, it can be more efficient to allow BIG-IP AFM DoS to handle
them earlier to reduce TMOS workload.
DoS also allows you to configure alerts to your preference. You can configure DoS detection and rate limiting depending on
whether you want to be alerted to all invalid packets or only at higher traffic levels.
TCP ACKs are presumably valid because they may be seen when SYN Cookies are active. If they fail a hardware SYN Cookie
check, presumably valid packets are dropped or rejected before they reach DoS protection.
If SYN Cookies are not active, any unsolicited, bare ACKs are dropped by TMOS.
Device configuration
You can configure many BIG-IP AFM DoS vectors using Device Configuration steps. Unless otherwise noted, the following
configuration steps apply to each DoS vector in this section.
3. Click Update.
54
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
For example:
tmsh modify /security dos device-config dos-device-config dos-device-
vector { ip-err-chksum { detection-threshold-pps 1 default-internal-rate-
limit 1 } }
Bad header
The various bad header vectors all check for packets violating their respective protocol specifications.
Because packets matching a bad header vector are dropped later by TMOS, there are no desirable bad header packets. There is
no reason to keep them around. F5 recommends setting Detection and Rate Limiting thresholds to MINIMUM for these vectors.
DNS
Domain Name System (DNS) attacks allow an attacker to attack use malformed packets and protocol errors in an attempt to
cause disruption of name resolution.
Flood
Flood attacks attempt to overwhelm a resource. While flood vector packets are Probably Invalid, because they follow protocol
specification, it is possible that they are valid. If the packets are valid, they match on a flow lookup table and are handled without
DoS protection seeing them.
F5 recommends setting Detection and Rate Limiting to LOW for the following vectors:
TCP SYN-ACK
Some of the flood vectors are presumably valid. These are valid packets, indistinguishable from legitimate connection requests.
55
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
F5 recommends setting Detection and Rate Limiting to HIGH for the following vectors:
TCP ACK (when SYN Cookies are active the ACK will not match a Flow Table Lookup)
UDP
The recommended settings for the remaining flood vectors depend on the environment. F5 recommends setting Detection and
Rate Limiting to HIGH for the remaining vectors in the flood group, pending an assessment of traffic patterns. This assessment
may indicate LOW or MINIMUM settings are appropriate for specific vectors.
Fragmentation
Fragmentation is the process of breaking a single IP datagram into multiple packets of smaller size.
Fragmentation attacks allow an attacker to use this method as an attack vector. The vectors in the Fragmentation section all
represent invalid packets that will be dropped by TMOS. They are not legitimate IP fragments so there is no reason to keep them
around.
The legitimate IP and IPv6 fragment traffic is tracked in the IP Flood and IPv6 Flood vectors. F5 recommends setting Detection
and Rate Limiting thresholds of MINIMUM for these vectors.
Single endpoint
Single endpoint vectors allow for detection and rate limit packets-per-second involving a single device.
The single endpoint sweep vector tracks the 100 source addresses sending the most of the selected packet types. The single
endpoint flood vector tracks the 100 destination addresses receiving the most of the selected packet types. These two vectors
are especially useful because their mitigation efforts are directed only at the 100 involved hosts. The other vectors drop traffic
without regard to its legitimacy.
Bad actor detection was introduced with single endpoint sweep with the auto-blacklist feature. This feature works with IP
Intelligence to classify hosts sending too much traffic as malicious, and drop traffic from them.
1. Go to Security >> DoS Protection : Device Configuration : Single Endpoint : <sweep|flood >.
3. If using Sweep, optionally link the Sweep vector to IP Intelligence and select IP intelligence to use to
56
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
classify identified bad actors, how long they must sustain an attack to be blacklisted, and how long to blacklist
them for.
4. Click Update.
To modify a specific Single Endpoint Vector using TMSH at the command line
Other
The Other vector allows for detection and mitigation against other attack types.
The IP Unknown Protocol and Land Attack vectors are invalid packet types. There is no reason to keep them around. F5
recommends setting Detection and Rate Limiting thresholds of MINIMUM for these vectors.
DoS profiles
DoS Profiles allow a BIG-IP provisioned with BIG-IP AFM the ability to detect and automatically mitigate DoS on an individual
virtual server.
Similar to device DoS, various detection and mitigation thresholds can be specified for DoS attack types to more accurately
detect, track, and rate limit attacks.
For more detailed information on each attack type, see Detecting and Preventing System DoS and DDoS Attacks chapter in
BIG-IP Systems: DoS Protection and Protocol Firewall Implementations.
DoS profiles provide the ability to mitigate attacks at a granular level, contrary to global settings in Device DoS, while also giving
the operator the ability to further tune many DoS attack vector thresholds on the virtual server level.
57
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
Protocol DNS
Protocol SIP
Network
Protocol DNS
Protocol DNS DoS profiles allow attack mitigation of both malformed protocol packets as well as volumetric attacks with valid
packets.
To modify DNS Error Attack Detection within a DoS profile using the Configuration utility
1. Go to Security >> DoS Protection: DoS Profile and select the DoS profile.
4. Specify the specific rate of increase, rate threshold and rate limit for the protection.
5. Click Update.
To modify the protocol DNS from using TMSH at the command line
1. Go to Security >> DoS Protection: DoS Profile and select the DoS profile.
58
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
4. Within DNS Query Attack Detection, click the checkbox next to the query attacks that are being configured
5. Specify the specific rate of increase, rate threshold and rate limit for the protection.
6. Click Update.
To modify the DNS Query Attack Detection using TMSH at the command line
For more detailed information on DNS DoS attacks, see Detecting and Preventing DNS DoS Attacks chapter in BIG-IP Systems:
DoS Protection and Protocol Firewall Implementations.
Protocol SIP
Protocol SIP DoS profiles allow attack mitigation of both malformed protocol specific packets and volumetric attacks with valid
packets. This mechanism can also be useful to detect unusual increases in protocol traffic.
To modify SIP Protocol Error Detection within a DoS profile using the Configuration utility
1. Go to Security >> DoS Protection: DoS Profile and select the DoS profile.
4. Within Protocol Errors Attack Detection, click the checkbox next to enable.
5. Specify the specific rate of increase, rate threshold and rate limit for the protection.
6. Click Update.
59
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
To modify SIP Attack Method Detection within a DoS profile using the Configuration utility
1. Go to Security >> DOS Protection: DOS Profile and select the DOS profile.
4. Within SIP Method Attack Detection click the checkbox next to a method type to enable.
5. Specify the specific rate of increase, rate threshold and rate limit for the method.
6. Click Update.
To modify the SIP Attack Method using TMSH at the command line
Network
Network DoS protection allows mitigation from a number of attack vectors at a VIP level. This includes both individual attacks as
well as behavioral analysis.
For more detailed information on which vectors are accelerated in hardware, see Detecting and Preventing Network DoS Attacks
in BIG-IP Systems: DoS Protection and Protocol Firewall Implementations.
Network DoS protection contains two features: Behavioral Analysis and Attack Types.
Behavioral analysis
Behavioral Analysis allows the DoS profile to inspect and report on sampled data that may indicate an attack.
Make sure to allow time for BIG-IP AFM to sample data from multiple sources. If traffic volume is low and from only one source,
you may see false positives.
60
DENIAL OF SERVICE
BIG-IP AFM DoS vectors
To modify Network DoS Behavioral Analysis within a DoS profile using the Configuration utility
1. Go to Security >> DoS Protection: DoS Profile and select the DoS profile.
5. Click Update.
Attack types
Configure Attack Types to have the DoS profile inspect and protect against layer-3 and layer-4 attack vectors at a specified
threshold and/or percentage of increase.
To modify Network DoS Attack Types within a DoS profile using the Configuration utility
1. Go to Security >> DOS Protection: DoS Profile and select the DoS profile.
7. Click Update.
61
DENIAL OF SERVICE
DoS policy development
To enable a DoS profile on the virtual server using the Configuration Utility
1. Go to the virtual server properties Local Traffic >> Virtual Servers : Virtual Server List.
5. Select Enable and then use the drop-down menu to select a log profile.
6. Click Update.
To add a DoS profile to a virtual server using TMSH at the command line
Depending on the use case, you may require a simple DoS policy or one requiring more extensive development and tuning. F5
recommends that you clearly identify your use case--what your policy must protect--before you create it. Having this information
should make development and enforcement easier.
Tune policy
Maintain policy
Tune policy
False positive violations are identified and policy settings are adjusted to allow legitimate traffic to pass through to the protected
62
DENIAL OF SERVICE
DoS reporting and visibility
applications and infrastructure. This is necessary as some legitimate traffic may not pass the configured policy rules and may
wrongly be identified as an attack.
Maintain policy
The DoS policy allows for adaption to application and infrastructure changes, new security requirements, and activities, based on
the review of logs, reports and statistics of attacks mitigated by the BIG-IP AFM system.
Consider a cloud-based solution such as F5 Silverline: Cloud Based DDoS Protection to augment on-premises DoS
mitigation and especially for volumetric attacks that may exceed on-premises bandwidth.
Start with a policy that allows a higher percentage of traffic nonconformity to allow all legitimate behavior and disallow
malicious requests.
Remember that Global DoS traffic settings are applied across the entire BIG-IP system, meaning all ingress traffic is
counted.
Use DoS profiles associated with individual virtual servers for specific attack remediation on a more granular basis.
Set a lower detection and rate limiting thresholds for attack vectors where there is no level of desirable traffic.
Other reports show transaction outcomes and correlate the impact of system detection with the mitigation of DoS attacks. The
reports and event logs can show whether or not DoS protection is functioning properly or whether tuning is necessary. They can
also help identify and track DoS attacks. Analyzing attack and trend data can provide insight into DoS threats.
63
DENIAL OF SERVICE
DoS reporting and visibility
Note To allow DoS data to populate DoS reports on a virtual server basis,
you must associate a DoS profile with one or more virtual servers.
DoS Overview The DoS Overview displays real-time information about all DoS attacks on the system. The system displays
recent attacks.
Logged Attacks shows a flag for an attack in progress. The log includes the 100 most recent events per protocol for
application and network attacks. Up to 200 attacks may be shown in the charts. If the information desired is not shown, try
increasing the time period selected in the filter.
You can filter your view by attack impact (High Impact, Medium Impact, Low Impact). To focus in on the specific details in
the charts, hover on the chart for the time period of interest. The system displays in a tooltip the details about what was
happening at that time.
To learn more about attacks that have occurred, click the Attack ID number in the Historical & Recent Attacks Log.
The system displays events associated with the attack. If there are more than 100 events, there will be a link to the event log,
which you can click to see more events.
The Overview screen includes information on throughput, RAM and CPU usage. Because the statistics vary from system to
system, it is a good idea to become familiar with typical memory and CPU usage and throughput on your system in association
with recent attacks.
For more detailed information on DoS reporting, see the Help tab in the Configuration utility.
Logging
BIG-IP AFM logs when an attack started, when an attack is mitigated, and when an attack has stopped.
logs show different fields, depending on the feature for which logs are being displayed. The default fields are listed in the
following table:
64
DENIAL OF SERVICE
DoS reporting and visibility
Table 4.1 DoS Logging Fields [format= Table {chapter #}.{table number}]
Field Description
Virtual Server For DoS profile events, the virtual server on which the profile is enabled.
Packets in / sec Packets related to the vector type that BIG-IP AFM has sampled in the last second.
Dropped Packets Packets related to the vector type that BIG-IP AFM has dropped in the last second
For more detailed information on event log messages, see Event Messages and Attack Types in External Monitoring of BIG-IP
Systems: Implementations.
Note Device DoS does not need a logging profile configured to log events.
The system supplied global-network logging profile logs the results to
the configured publisher.
65
DENIAL OF SERVICE
Signaling and intelligence
Assign the logging profile within the virtual server that the DoS Profile is being used.
For more information on configuring a logging profile, see Monitoring and Logging BIG-IP AFM.
To configure DoS logging with a log profile using the Configuration utility
1. Go to Security >> Event Logs: Logging Profile and select logging profile.
3. Under network DoS Protection, use the drop-down menu to select publisher.
4. Click Update.
To enable a logging profile on the virtual server using the Configuration utility
1. Go to the virtual server properties Local Traffic >> Virtual Servers: Virtual Server List.
4. Get to Log Profile, use the drop-down menu to select the log profile.
5. Click Update.
Within BIG-IP AFM, the virtual server DoS profile sweep detection feature can provide intelligence to allow bad actors to be
blocked automatically for a pre-defined period of time based on the detection threshold. This blocking configuration is mapped to
an IP Intelligence category.
66
DENIAL OF SERVICE
Signaling and intelligence
Additionally, blacklists can be created from external intelligence sources. F5 offers a subscription IP-intelligence service for
categorization of discovered bad actors and allows the implementation of custom IP Intelligence policies.
In order to get a feed list for IP Intelligence, the configuration requires an external host to provide via either a website or FTP
server a list of offending IP addresses.
The update interval is configured to specify the timeframe in which BIG-IP AFM will regularly poll the server for updates. If an
update poll fails, BIG-IP AFM will continue to use the last known good list. The feed list is a simple CSV file.
To configure the external feed list using TMSH at the command line
For more detailed information on Signaling and Intelligence, see About IP Address Intelligence in the Network Firewall section of
BIG-IP Network Firewall: Policies and Implementations.
67
DENIAL OF SERVICE
Signaling and intelligence
SOL15368:The BIG-IP AFM system logs network firewall events using the logging profile associated with the network
firewall rule
Protect Against Evolving DDoS Threats: The Case for Hybrid Mitigation
68
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
EXTERNAL TOOLS
BIG-IQ Centralized Management
External Tools
Several external tools can be used to assist with management of one or multiple BIG-IP AFM systems, logging, and transfer of
information. The following are covered in this chapter:
Syslog
sFlow
Management of shared objects (address lists, port lists, rule lists, policies, and schedules).
Firewall audit log used to record every firewall policy change and event.
Deployment of configurations from snapshots and the ability to preview differences between snapshots.
Monitoring of rules.
Reports on security.
This section describes common operational tasks that can be performed when leveraging BIG-IQ for BIG-IP AFM management.
Manage policies
When using BIG-IQ for policy management, BIG-IQ acts as a centralized database and a backup source for BIG-IP AFM firewall
policies. By design, BIG-IP AFM devices are discovered by BIG-IQ and the management authority for firewall policies and shared
70
EXTERNAL TOOLS
BIG-IQ Centralized Management
When using BIG-IQ to manage BIG-IP AFM devices, F5 recommends making all changes to the systems in BIG-IQ. If changes
are made directly to the BIG-IP, they will be lost when BIG-IQ deploys a new change set unless BIG-IQ is made aware of the
changes using one of two of the following methods:
BIG-IQ re-discovers BIG-IP AFM and changes discovered are designated as Use BIG-IP.
Note This practice can affect shared objects used by other BIG-IP AFM
devices such as address lists, ports lists, policies, and rule lists.).
Changes performed locally to the BIG-IP AFM device are replicated in BIG-IQ prior to deploying other changes from
BIG-IQ.
Compare policies
As the central manager, BIG-IQ is has authority over all policy objects.
To manage revisions and changes to policy and shared objects, BIG-IQ keeps snapshots of the entire policy set as it relates to
all firewalls. Because snapshots are kept, no individual firewall backups are maintained. From a policy standpoint, a central
shared data set exists with point in time copies of policy state.
Full BIG-IP system backups can be initiated through the device module in BIG-IQ; however doing this is not required to maintain
policy backups.
When a deployment task executes, a snapshot is automatically created that contains all BIG-IQ data at that specific point in time.
A deployment task can contain multiple device deployments. The policy point is used to compare an existing policy on the
BIG-IP AFM with the working configuration stored in BIG-IQ. Differences in the two are recognized as changes.
You can compare changes on a BIG-IP AFM by creating an evaluation task and selecting Working Config or selecting a
specific snapshot to compare with the firewall. If you choose an earlier snapshot, it is possible to view the differences in policy
changes that have been deployed to the firewall since that point.
For small-to-medium environments with infrequent changes, F5 recommends that you generate a snapshot of the configuration
prior to making modifications to the policy.
In larger environments with frequent changes, you may want to consider creating a snapshot at least once a day. Frequent
snapshots provide a consistent set to compare against older policies, and this makes rollback of policy changes easier if a quick
restore is required after deploying firewall policy changes.
Restore a policy
When you restore a policy using BIG-IQ, you can evaluate policies against historical snapshots so that only change sets are
restored.
71
EXTERNAL TOOLS
BIG-IQ Centralized Management
1. Identify the timestamp of the previous policy deployment by searching the deployment tasks for the firewall name. If the
deployment task isnt available through the BIG-IQ configuration manager, search BIG-IP LTM logs for previous pccd
compile times.
2. Create a new deployment task by selecting the snapshot from the previous deployment. If it is unavailable, select the next
most-recent snapshot.
BIG-IQ provides a change set between the current deployment and the deployment snapshot you selected.
4. Deploy the changes to restore the BIG-IP AFM policy to the earlier configuration.
While the previous steps restore the policy to the BIG-IP AFM, BIG-IQ retains a working configuration of the previously made
changes. To reapply that configuration, BIG-IQ must first re-discover BIG-IP AFM or you have to edit the BIG-IQ policy objects.
Note To restore a BIG-IP AFM policy without using BIG-IQ, you must back
out of the changes youve made to the policy or restore a previous version of
the configuration using a SCF or UCS file.
For more information, see BIG-IQ Centralized Management: Security Managing Audit Logs in BIG-IQ Network Security.
Reporting
BIG-IP AFM creates several reports for displaying DoS device information and firewall rule statistics. You can view these reports
through the Configuration utility for an individual BIG-IP AFM system or through the BIG-IQ Security Reporting interface, where
you can view the results for one or more BIG-IP AFM systems.
Note Bi-directional HTTPS access between the BIG-IQ and BIG-IP AFM is
required to generate reports for centralized viewing in BIG-IQ.
Manage software
BIG-IQ supports automation and scheduling of the collection of BIG-IP backup files, as well as management of TMOS software
images. With the BIG-IQ Device it is possible to stage and deploy TMOS upgrades directly from BIG-IQ, eliminating the need to
log into individual BIG-IP AFM devices to import, install, and reboot devices.
72
EXTERNAL TOOLS
SNMP polling and alerting
BIG-IP AFM supports version 1, 2c, and 3 of SNMP for manager access and trap destinations.
For more detailed information on the initial configuration of SNMP, see BIG-IP TMOS: Concepts SNMP.
You can find events specific to BIG-IP AFM in the F5-BIGIP-LOCAL-MIB.txt file.
Using an SNMP manager, you can collect information for firewall rules, contexts, and rule hits for BIG-IP AFM. You can use the
following SNMP command syntax to pull rule statistics:
snmpwalk -c public <HOSTNAME> ltmFwRuleStat
In addition to firewall rules, you can query BIG-IP AFM DoS statistics and attack information using an SNMP manager. You can
use the following SNMP command syntax to pull DoS statistics:
snmpwalk -c public <HOSTNAME> ltmDosAttackDataStat
Table 5.1: Relevant BIG-IP AFM Notifications to Send to SNMP Trap Receiver
BIGIP_TMM_TMMERR_ The start of a possible DoS attack Determine your response to this type
DOS_ATTACK_START was registered. of DoS attack, if required.
(.1.3.6.1.4.1.3375.2.4.0.133)
73
EXTERNAL TOOLS
IPFIX
Syslog
Syslog is a widely used standard for message logging. Each message is labeled with a facility code and assigned a severity. The
facility code indicates the software type of the application that generated the message. The syslog messages may be directed to
various destinations, tuned by facility and severity, including console, files, remote syslog servers, or relays.
The default information sent to a syslog destination may contain more detailed log information than required for firewall rule event
logging. F5 recommends modifying the output with logging filters to reduce the size of the messages, as well as to make the
messages more easily understood. F5 also recommends controlling access to the logs to a select group of administrators and
operators on a need-to-know basis. Log collectors are designed to prevent or notify, based on log modification attempts.
BIG-IP AFM also contains logging formats for commercial log aggregation and security information and event management
(SIEM) solutions such as Splunk and Arcsight. These log destinations are preconfigured for ease of deployment.
For more detailed information see Monitoring and Logging BIG-IP AFM.
External logging
The BIG-IP system uses a high-speed logging (HSL) mechanism, which allows it to efficiently generate and transmit log
messages to one or more log collectors.
F5 recommends sending logs of system and firewall messages to a remote server for event collection and indexing. Doing so
allows you to view messages without impacting the performance of the system generating them, and in the event of a system
failure, remote logs can help troubleshoot the cause of the failure.
You can configure remote logging destinations for long-term storage of data to use for trend analysis and auditing. Many
third-party logging collectors can display time-series events to help with troubleshooting.
For more detailed information on external logging configurations, see External Monitoring of BIG-IP Systems: Implementation
guide.
IPFIX
IPFIX is a universal standard of export for IP flow information from devices that are used by mediation systems, accounting/billing
systems, and network management systems to facilitate services such as measurement, accounting, and billing.
IPFIX provides a means for standardizing IP flow information to be formatted and transferred to an external collector. The details
of the protocol are defined by RFC 5103, and RFCs 7011 through 7015. The BIG-IP system can be configured to send IPFIX data
to a collector for consumption.
Using IPFIX for traffic analysis provides guidance on traffic patterns and system usage for capacity planning and charge back
based on consumption, troubleshooting network and application issues, identifying volumetric DoS attacks, and evaluating the
effectiveness of security policies.
For more detailed on IPFIX configuration and options, see Logging Network Firewall Events to IPFIX Collectors in BIG-IP Network
Firewall: Policies and Implementations.
74
EXTERNAL TOOLS
Change and configuration management
The IPFIX entities IANA definitions of the supported Information Elements (IEs) within the BIG-IP software release can be found in
the Downloads section of the Welcome screen of the Configuration utility.
sFlow
Sampled flow (sFlow) is an industry standard for packet export. sFlow provides a means for exporting truncated packets,
together with interface counters. The BIG-IP system can be configured to poll internal data sources and send data samples to an
sFlow receiver. The collected data can be used to analyze the traffic that traverses the BIG-IP system.
Using sFlow for traffic analysis can provide guidance on traffic patterns and system usage for capacity planning and charge back
based on consumption, troubleshoot network and application issues, identify volumetric DoS attacks, and evaluate the
effectiveness of security policies.
For more detailed sFlow configuration and options, see Implementations Monitoring BIG-IP System Traffic with sFlow in External
Monitoring of BIG-IP Systems.
In smaller environments, the BIG-IP AFM policy rule description field may be used to track change requests. The description field
is limited to 255 characters.
75
EXTERNAL TOOLS
Change and configuration management
In larger environments, F5 recommends using an external tracking system in combination with in-policy documentation. There
are many third-party external configuration management databases (CMDB) for facilitating and documenting larger use cases.
F5 recommends selecting a system that follows the Information Technology Infrastructure Library (ITIL) or similar methodologies.
76
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
MONITORING AND LOGGING BIG-IP AFM
BIG-IP AFM monitoring
F5 also highly recommends establishing, documenting, and following a log maintenance plan so that any security incidents can
be reviewed during the defined log retention period.
Note This chapter covers only monitoring and logging elements and
processes relevant to BIG-IP AFM. For information about logging the BIG-IP
system or other modules, see x-ref to TMOS.
SNMP
F5 supports the industry-standard SNMP protocol to manage BIG-IP devices on a network. The SNMP agent on the BIG-IP
system must be configured. The primary tasks in configuring the SNMP agent are configuring client access to the SNMP agent,
and controlling access to SNMP data.
The following are some SNMP management information base (MIB) files to look at for alerting and monitoring of your BIG-IP AFM
system:
ltmFw*
ltmFwIpint*
ltmDos*
For example:
snmpwalk -c public localhost F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.global./Common/global-firewall-
rules.rd-fw-rule1../Common/rd-fw-policy.staged = Counter64: 4220
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.global./Common/global-firewall-
78
MONITORING AND LOGGING BIG-IP AFM
BIG-IP AFM logging
rules.allow-udp-53../Common/global-fw-policy.enforced = Counter64: 0
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.global./Common/global-firewall-
rules.port-2002-global../Common/global-fw-policy.enforced =
Counter64: 4220
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.global./Common/global-firewall-
rules.disallow-source-10.12.27.214../Common/global-fw-policy.enforced
= Counter64: 0
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.virtual./Common/http _ vs.rd-
rule1../Common/fw-policy-vs.enforced = Counter64: 0
F5-BIGIP-LOCAL-MIB::ltmFwRuleStatCounter.virtual./Common/http _
vs.http-rule../Common/fw-policy-vs.enforced = Counter64: 0
CPU
RAM
DISK
For more detailed information on this SNMP traps, see SNMP Trap Configuration in BIG-IP Systems: DoS Protection and
Protocol Firewall Implementations.
For more detailed information on SNMP MIBs, see AskF5 article: SOL13322: Overview of BIG-IP MIB files (10.x - 12.x).
To configure local.syslog
The core BIG-IP AFM functions are part of TMM and use a separate logging system. Messages from this logging system can be
forwarded to: local syslog by using the logging destination local-syslog-publisher, the native BIG-IP AFM database. You can
use the logging destination local-db-publisher, a remote logging server that uses syslog, the IPFIX protocol, Splunk, or
Arcsight
79
MONITORING AND LOGGING BIG-IP AFM
BIG-IP AFM logging
Logging guidelines
F5 recommends enabling logging on each drop|reject rule applied to the Network Firewall and IP Intelligence. Configure this
logging for every object that the firewall applies to. Configure an Aggregate Rate Limit in logging profiles used with the
Network Firewall and IP Intelligence.
F5 also highly recommends external logging to ensure optimal performance of your BIG-IP AFM system.
Some regulatory environments require logging all firewall events, including those whose action is Accept.
Profile settings
Logging profiles are used to define how firewall and DoS logs are sent to the log publisher. In logging profiles you can configure or
modify log components such as the fields to send in a log message.
The Network Firewall profile also allows aggregate rate-limiting of log messages. This rate-limiting controls how many messages/
second BIG-IP AFM sends to the destination.
This setting is useful for extremely high throughput firewalls or firewalls being flooded with bad traffic. After the rate limit is
reached, additional messages are not logged. Log message volume is sampled from that point, as well as summary messages
detailing how many log messages have been dropped. Log throttling is an important tool to keep the log destination functional.
For more information on logging profiles, see AskF5 article: SOL17398: Configuring the High Speed Logging traffic distribution
method.
Logging daemons
BIG-IP AFM uses two daemons to display and populate log data:
Mgmt_acld = mgmt_acld is primarily responsible for maintaining statistics, logging, and reporting of Management Port
BIG-IP AFM Rules. In addition, it also periodically updates the statistics counters for management port rules.
Mysqld is the database server storing data for reporting/charts and event logs reports.
For more detailed information on BIG-IP AFM daemons, see AskF5 article: SOL14387: Overview of BIG-IP AFM daemons
Logging destinations
BIG-IP AFM messages can be logged directly on the BIG-IP system or to an external system. In making the decision, consider
performance impact factors such as disk space use, log retention policy, number of logs being sent, logging while under attack,
and log throttling.
F5 recommends using the IPFIX logging format for external logging for the fastest performance.
BIG-IPsystems can be configured to log messages locally or to remote high-speed log servers.
80
MONITORING AND LOGGING BIG-IP AFM
BIG-IP AFM logging
For more detailed information on configuring log destination, see Configuring Remote High-Speed Logging in External Monitoring
of BIG-IP Systems: Implementations.
Local logging
If logging locally, events are logged to local-db-publisher if using the Security >> Event Logs viewer and local-syslog-
publisher for local syslog.
Local logging of BIG-IP AFM and DoS events are useful for initial setup and testing but the recommended practice is to use only
remote logging for production level traffic. This is because there is a performance cost associated with local log destinations due
to I/O constraints and other factors. Turning on local logging as a troubleshooting step may also be useful.
If using local-db-publisher, be aware that the number of messages stored is capped at 1.25 million. This may not be adequate
for some environments.
Tip Firewall rules can be created from a log entry when using the Security
Event Logs viewer for network firewall events. Check the log entry and click
create rule. This will open the New Rule editing page. This is useful in
situations where there is a need to quickly resolve an issue or create a
temporary rule in place for traffic being blocked or allowed.
Remote logging
In order to log remotely, a pool of servers must be created and added to an unformatted destination which can in turn be added
to a formatted destination. The first consideration is the pool. This is a standard LTM pool that contains all of the remote servers
that should receive logs. By default, messages will go to the first pool member selected until either the rate of the HSL traffic
exceeds what the remote log server is capable of accepting or the HSL connection to the remote log server is lost.
For more detailed information on remote logging, see AskF5 article: SOL17398: Configuring the High Speed Logging traffic
distribution method.
Another consideration with log destinations, is the format to send the logs in. Firewall and DoS logs can both be forwarded in
ArcSight, Splunk, Syslog or IPFIX formats. If logs do not appear as expected on the remote device, it may be that the formatted
destination does not match the remote server.
IPFIX has been observed to have the fastest performance. Its a binary format and so is not directly human readable.
81
MONITORING AND LOGGING BIG-IP AFM
BIG-IP AFM logging
Syslog: provides just the field data which is harder to read but leaner. Most log consoles can provide the fields for human
readability.
Splunk
ArcSight
IPFIX
The BIG-IP system may be configured to log BIG-IP AFM events over the IPFIX protocol. These are binary-encoded strings that
are defined by IPFIX templates. These strings are then sent off-box to an external IPFIX collector.
For more detailed information on configuring IPFIX, see Logging Network Firewall Events to IPFIX Collectors in External
Monitoring of BIG-IP Systems: Implementations.
Debug logging
The BIG-IP systems default logging levels are set to capture useful information about BIG-IP system events while maintaining
minimal impact on system resources.
If the default logging level does not provide enough detail, and you can enable debug logging to gather more detailed diagnostic
information. The greater logging detail the debug level logs places added demand on BIG-IP system processor and hard disk
space.
With the BIG-IP system, you can configure the level of information that the system logs for events related to traffic management.
For more detailed information on levels of logging, see AskF5 article: SOL5532: Configuring the level of information logged for
TMM-specific events.
Disk space
Disk space use can increase significantly with local logging, debug log level, and a runaway tcpdump process.
For more detailed information on disk space maintenance on a BIG-IP system, see AskF5 article: SOL14403: Maintaining disk
space on the BIG-IP system (11.x - 12.x).
82
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
TROUBLESHOOTING
BIG-IP AFM logging
Troubleshooting
In order to troubleshoot issues related to BIG-IP AFM, a solid understanding is needed of the traffic flow process and the internal
structure of BIG-IP. This chapter will give an introduction to the packet flow process and the tools needed for troubleshooting.
Traffic flow
BIG-IP uses traffic flows to efficiently pass traffic from a client to internal resources. BIG-IP AFM works with TMOS to manage the
access control process which includes flow management. When a packet arrives at BIG-IP, TMOS first examines whether the
packet received belongs to an already existing flow or the first packet is a new flow.
For a detailed understanding of the packet processing flow, see Packet flow.
L4 mirroring
While F5 recommends enabling connection mirroring only for essential connections, a BIG-IP system can be configured to mirror
84
TROUBLESHOOTING
BIG-IP AFM logging
The Connection Mirroring checkbox is apparent when creating a virtual server and is only functional when a high availability
group is properly configured and has at least two members in it.
When configuring mirroring, F5 recommends using a dedicated VLAN and physical interface for all session mirroring traffic.
Environments requiring a significant amount of connection mirroring may impact BIG-IP performance. F5 recommends using the
following settings for optimal performance:
Set tm.fastl4_ack_mirror db setting to Disable to increase overall system performance when mirroring traffic for fastL4 virtual
servers.
If you do not disable this setting, you may experience HA Table disconnects under heavy load.
1. Log in to iHealth.
4. Click ha_stat.
Hits in these fields indicate that the buffer setting for statemirror.queuelen may be set too low for the volume of mirroring
required (min:1048576, max:268435456, default: 8388608). To fix this problem, raise the queue length value until the
overflows reported are near zero.
To modify the statemirror.queuelen value using TMSH from the command line
To check the statemirror.queuelen value using tmctl from the command line
85
TROUBLESHOOTING
BIG-IP AFM network firewall modes
Another optimization for mirroring performance requires setting Nagle in the mirroring connection key.
To set Nagle in the mirroring connection key using TMSH from the command line
ADC mode
By default, the network firewall is configured in ADC mode. In ADC mode all traffic destined for a virtual server is allowed unless
an ACL specifically denies it. In this mode, all traffic is allowed to virtual servers and self IPs on the system, and any traffic you
want to block must be explicitly specified. This applies only to the virtual server and self IP contexts on the system.
To configure the network firewall in ADC mode using the Configuration utility
If you have changed the firewall setting to Firewall mode, you can configure the BIG-IPnetwork firewall back to ADC mode.
1. Go toSecurity>>Options:Network Firewall.
2. From theVirtual Server & Self IP Contextslist, select the default actionAcceptfor the self IP and virtual
server contexts.
3. ClickUpdate.
Firewall mode
You can configure the BIG-IPnetwork firewall to drop or reject all traffic not explicitly allowed. Firewall mode applies a default
deny policy to all self IPs and virtual servers.
To configure the Network Firewall in Firewall mode using the Configuration utility
1. Go to Security>>Options:Network Firewall.
2. From theVirtual Server & Self IP Contextslist, select the default action for the self IP and virtual server
contexts.
86
TROUBLESHOOTING
BIG-IP AFM network firewall modes
3. SelectDropto silently drop all traffic to virtual servers and self IPs unless specifically allowed.
4. SelectRejectto drop all traffic to virtual servers and self IPs unless specifically allowed, and to send the
appropriate reject message for the protocol.
5. ClickUpdate.
Note Management port traffic is not handled by the global rule. Management
port rules must be explicitly defined for the management port context.
1. Go toSecurity>>Options:Network Firewall.
2. From theGlobal Contextlist, select the default action for the global rule, when the traffic matches no other
rule.
4. SelectRejectto drop traffic, and send the appropriate reject message for the protocol.
5. ClickUpdate.
87
TROUBLESHOOTING
BIG-IP AFM network firewall modes
To explicitly allow this traffic, you can use the built-in firewall rules_sys_self_
allow_defaults or _sys_self_allow_management, applied to the specific
self IPs used for connectivity between the device group.
Management and troubleshooting varies, depending on the type of traffic as well as the interface processing the traffic.
Traffic sourced from a BIG-IP self IP is allowed outbound of the BIG-IP AFM and does not require any ACL policy to correctly
operate for cases such as outbound log traffic or network high availability. Inbound self IP traffic must be allowed through a
firewall context, to be accepted and processed by the self IP and TMOS.
The BIG-IP management port is out of band, accessed from the control plane and is not affected by global firewall rules. This
port can be secured through a management port firewall policy that explicitly allows or denies traffic to and from the
management port. Unlike self IPs, outbound traffic allowed by the management port policy does not create a stateful flow.
Therefore, return traffic must be allowed into the management port when appropriate.
For example, a policy may allow traffic from remote authentication servers into the BIG-IP system using a management policy
rule in response to the outbound authentication request.
Flow eviction
As part of BIG-IP device security, TMOS manages the memory used by the system, including the connection table resources. If
a BIG-IP system becomes memory-constrained due to network events such as a DDoS attack or other high traffic condition,
TMOS evicts older idle flows in order to protect the BIG-IP from memory exhaustion using adaptive connection reaping.
Eviction policies for adaptive reaping can be configured at the global, route domain, and virtual server contexts. This allows
granular control over how resources are preserved during these memory-constrained conditions.
If BIG-IP AFM is experiencing memory constraint, idle flows can be removed, forcing connections to be re-established when they
are no longer idle. This might interrupt client traffic.
88
TROUBLESHOOTING
BIG-IP AFM network firewall modes
There are four eviction strategies which can influence the reapers selection of connections to remove from the connection table:
Bias Idle specifies that the system biases flow removal toward the existing flows that have been idle, with no payload
bytes, for the longest.
Bias Oldest specifies that the system biases flow removal toward the oldest existing flows.
Bias Bytes evicts traffic flows that show less data being transferred. Configure the grace period to allow new
connections to be spun up.
Low Priority specifies objects identified as lower priority by the flow removal strategy:
Route-domains
Virtual servers
Countries
To create an eviction policy at the global context using TMSH at the command line
To apply the eviction policy to the virtual server context (based on connection counts) using TMSH at the
command line
To apply the eviction policy to the route domain context (based on connection counts) using TMSH at the
command line
To apply the eviction policy to the global context (based on connection counts) using TMSH at the com-
mand line
89
TROUBLESHOOTING
BIG-IP AFM network firewall modes
To validate the eviction policy performance in any context using TMSH at the command line
For more detailed information on flow eviction, see AskF5 article: SOL15740: Overview of adaptive connection reaping (11.6.0
and later).
BIG-IP AFM uses slow flow monitoring to mitigate this class of attack. Slow flow monitoring directs the sweeper to detect and
optionally delete connections that are not moving data below the defined threshold rate. The sweeper routinely goes through the
connection table and evicts expired connections, so this additional task does not have a significant performance impact.
The slow flow monitoring settings allow some flexibility to allow applications that require slow connections. These settings
include the following:
Grace Period provides connections with time to ramp up before they are required to move data above the threshold
rate.
Slow Flow Throttling Percent permits a configured percentage of slow connections to remain.
Slow Flow Throttling Absolute permits the configured number of slow connections to remain.
For more detailed information on eviction policies eviction, see AskF5 article: SOL15821: Overview of eviction policies.
90
TROUBLESHOOTING
Policy compilation
Rule actions
During troubleshooting, you may find that rule actions adversely affect the behavior of BIG-IP AFM.
For example, if a rule is set at a global context with an action of Accept but when troubleshooting the connection flow traffic
does not flow from source to destination, it may be that rule actions are at issue. If so, you can troubleshoot by finding the
answers to the following questions:
Is there a route domain policy in place that could be dropping the traffic?
Is there a virtual server policy in place that could be dropping the traffic?
If running in Firewall mode (and not ADC mode) is the virtual server default action dropping the traffic?
If the firewall rules expected behavior is to accept and create a new flow rather than continue evaluating the connection in
another firewall context, use Accept Decisively as the rule action.
If the firewall rules expected behavior is to accept but continue evaluating other contexts, make sure there are appropriate accept
rules to match all contexts to allow the traffic to pass.
One way to resolve the issue described in the example is to use a staged policy and track ACL matches there.
For more detailed information on firewall rule actions, see AskF5 article: Network Firewall Implementation Guide.
Policy compilation
BIG-IP AFM contains control plane processes that compile and instantiate firewall policies to TMOS. When a configuration
change is made, the following processes implement the new firewall policy:
For detailed information of BIG-IP AFM processes, see AskF5 article: SOL14387: Overview of BIG-IP AFM daemons.
Firewall compiler
Depending on the size of the policy, pccd can take up to a couple minutes to compile and serialize a new policy.
2. Select the context you want to view (in this case, Global, Route Domain, or Virtual Server).
91
TROUBLESHOOTING
Logging
process-mem is amount of transient memory required to build the policy: pccd uses control plane memory to compile
policies
rule-count is the number of rules in the compiled policy.Each change in firewall policy (including scheduled rules) results
in a compilation and serialization of the policy. For historical data, see /var/log/ltm for local syslog entries of the pccd
daemon or remote syslog storage of these messages.
Logging
Default rule logging
By default, implicit default firewall actions are not logged unless configured to do so. For example, when troubleshooting a flow
that is dropped by the default rule, no drop log messages are generated.
To turn on logging for implicit firewall actions using TMSH at the command line
92
TROUBLESHOOTING
Logging
Turning on logging for these actions may be helpful when troubleshooting dropped traffic; however, having these actions logged
during normal operation may create an unnecessary amount of logging.
Another method to troubleshoot the dropped traffic is to add an explicit Default Deny rule at the end of the applied policy in
BIG-IP AFM and configure the log attribute in the rule to avoid modifying the system db variables listed above.
Audit
By default, BIG-IP is configured to log audit information of the BIG-IP system configuration to /var/log/audit. Audit logging
information includes configuration changes and the users that made them.
When audit logging is disabled, BIG-IP does not log configuration changes.
Note In BIG-IP TMOS 11.6 and earlier, audit logging is disabled by default. If
you upgrade to a newer version, this setting may be retained.
For more detailed information, see AskF5 article: SOL16304: Audit logging for BIG-IP configuration changes is now enabled by
default.
Tip BIG-IQ Security offers more comprehensive Firewall Policy auditing and
may be used to centrally manage Firewall Policies, allowing auditing of
multiple BIG-IP firewalls.
When the Default Firewall Action under Security >> Options : Network Firewall is set to Reject, BIG-IP AFM
generates a TCP reset message.
When a flow connection idles out, by default the BIG-IP system sends a TCP reset to both the client and server side of the
connection flow. (This behavior can be enabled or disabled as a function of the relevant FastL4 profile).
The BIG-IP system supports logging TCP reset causes within the packet, logging the TCP reset reason through Syslog, or both.
Depending on system load, local logging may not capture all resets due to rate limiting.
Enabling packet-level TCP reset provides a mechanism to view the TCP reset cause from a packet capture by using tcpdump
and the F5 Wireshark dissector.
For more detailed information on BIG-IP TCP reset logging and configuration settings, see AskF5 article: SOL13223: Configuring
93
TROUBLESHOOTING
Statistics
Logging troubleshooting
To view local log files in the /var/log/ directory from the command line
If logging is not apparent locally or on your remote logging destination, verify that the log publisher has been
assigned.
To view DoS logging for the current configuration using TMSH at the command line
To view the current firewall logging settings using TMSH at the command line
To view DoS logging or to apply a log-publisher for BIG-IP AFM logging using TMSH at the command line
For additional information of creating or modifying logging configuration see: Configuring Remote High-Speed Logging.
Statistics
Statistics are an important aspect of maintaining and troubleshooting BIG-IP AFM. When trying to identify whether or not a
firewall rule is working or how a DOS attack was detected and mitigated, it is helpful to know where to look.
Statistics are available in the Configuration utility on the Security >> Overview tab and the Reporting tab.
Overview provides a high-level view of what is going on with the system. It is divided into the following sections:
94
TROUBLESHOOTING
Statistics
Protocol displays HTTPS and DNS protocol security statistics (if configured for HTTPS and DNS security).
Reporting provides detailed views of statistics for protocol, network, and DoS.
For more detailed information on BIG-IP AFM Security >> Overview : Summary screen, see SOL16703: The BIG-IP AFM
Overview Summary screen has two Add Widget links that offer different sets of display methods
Rules
To view statistics for firewall rules using the Configuration utility
This screen displays important details on active rules, including rule hit counters and time stamps for the last time
the rule was hit.
2. The statistics for firewall rules only updates counters for active rules and not accumulated counters for a rule
list.
To view statistics for firewall rules using TMSH at the command line
You can use TMSH to show security firewall matching-rule matches, tuples/arguments with the existing rule base, and outputs
rules that can match the arguments.
In the following example, the TMSH command requires all filters for match criteria. Wildcards are not supported for VLAN or
protocol.
tmsh show /security firewall matching-rule source-addr 10.1.1.245 source-
port any dest-addr 192.168.1.245 dest-port 80 vlan /Common/internal-vlab
protocol tcp
Firewall Matching Rule:
-------------------------------------------------------------------
Context Type Context Name Policy Name Rule Name Action
95
TROUBLESHOOTING
Common troubleshooting tasks
-------------------------------------------------------------------
Route Domain /Common/0 /Common/rd _ policy accept _ defer Accept
Total records returned: 1
When viewing Device DoS statistics, the TMSH command show security dos displays statistics of all global attack vectors. This
information can then be analyzed and is useful for additional tuning of DoS settings.
For each DoS attack type, the detection threshold signifies the packet rate at which BIG-IP DoS logs attack-detected messages,
based on the one-minute average, which is a value displayed in the security DoS output.
By watching the averages of typical system traffic, you can determine a baseline of expected packet rates and use that baseline
to set the detection thresholds and optionally the rate limit for DoS attack types.
96
TROUBLESHOOTING
Common troubleshooting tasks
The interface to the connection table is provided by TMSH. You can use the connection table to see if a flow exists.
The following are best practices when working with the connection table:
Always use IP address/Port/Protocol filters to narrow the return set of a show /sys connection command.
Use the all-properties option to view detailed statistics of a flow including idle time, bits in/out.
The following shows a sample filter used to search for connection table entries:
tmsh show /sys conn cs-client-addr <IP ADDRESS> cs-server-port 4353
tmsh show /sys conn cs-client-addr 192.168.142.248 cs-server-port 4353
Sys::Connections
192.168.142.248:50241 10.10.20.245:4353 192.168.142.248:50241 10.10.20.245:4353
tcp 1 (tmm: 0) none
Total records returned: 1
The following shows a more granular filter example with detailed output:
tmsh show /sys connection cs-client-addr <IP ADDRESS> cs-server-port 4353
all-properties
tmsh show /sys connection cs-client-addr 192.168.142.248 cs-server-port
4353 all-properties
Sys::Connections
192.168.142.248:50241 - 10.10.20.245:4353 - 192.168.142.248:50241 -
10.10.20.245:4353
---------------------------------------------------------------------------
----------
TMM 0
Type self
Acceleration none
Protocol tcp
Idle Time 1
Idle Timeout 5
Unit ID 0
Lasthop /Common/internal 00:0c:29:e5:eb:6c
Virtual Path 10.10.20.245:4353
Conn Id 0
97
TROUBLESHOOTING
Common troubleshooting tasks
ClientSide ServerSide
Client Addr 192.168.142.248:50241 192.168.142.248:50241
Server Addr 10.10.20.245:4353 10.10.20.245:4353
Bits In 1.9K 0
Bits Out 0 1.9K
Packets In 4 0
Packets Out 0 4
Total records returned: 1
tcpdump
The tcpdump utility is a command-line packet sniffer with many features and options. Use tcpdump to capture (to a file) or view
(to the console) packets for a tagged VLAN on the BIG-IP system. The syntax, flags, and options specified when performing a
packet capture using tcpdump determine what information is included in the packet capture file and/or displayed to the console.
F5 recommends filtering to limit the capture packet count and filter what is being captured since running tcpdump can impact
system performance.
-w used to save to a file for review later requires <filename> to save data
-r used to view a binary capture file requires <filename> to read capture file
src port <port#> displays packets that are sourced from a port.
98
TROUBLESHOOTING
Common troubleshooting tasks
When using tcpdump with BIG-IP AFM, the directional flags of traffic can show traffic passing ACL evaluation or potentially not
passing ACL evaluation.
Consider the two following examples (remember to set the snaplen=0 to capture the full packet to see the f5 trailers).
tcpdump -nni 0.0 -s0 -c100 port 80 and host 10.1.1.245
16:26:38.726579 IP 192.168.245.245.1268 > 10.1.1.245.80: S
1614628366:1614628366(0) win 5792 <mss 1460,nop,nop,timestamp 1079282556 0>
in tmm0 lis=
16:26:38.727625 IP 192.168.245.245.6017 > 10.1.1.245.80: S
3728806033:3728806033(0) win 5792 <mss 1460,nop,nop,timestamp 1079290603 0>
in tmm1 lis=
16:26:37.217835 IP 192.168.245.245.58367 > 10.1.1.245.80: S
2430396241:2430396241(0) win 5792 <mss 1460,nop,nop,timestamp 1079290927 0>
in tmm4 lis=
This second example illustrates the flow ingress and egress the BIG-IP.
tcpdump -nni 0.0 -s0 -c100 and host 10.1.1.245
16:13:56.946046 IP 192.168.245.245.6422 > 10.1.1.245.80: S
2181731049:2181731049(0) win 5792 <mss 1460,nop,nop,timestamp 3586289132 0>
in tmm8 lis=
16:13:56.946079 IP 192.168.245.245.6422 > 10.1.1.245.80: S
2181731049:2181731049(0) win 5792 <mss 1460,nop,nop,timestamp 3586289132 0>
out tmm8 lis=/Common/Forwarding _ TCP _ VS
In the first example, TCP SYNs are received and show coming in to the BIG-IP AFM, but there is no egress packet. This may
indicate that an ACL is blocking this traffic. If logging drops, this can be validated through syslog analysis.
In the second example, the TCP SYN shows out of the BIG-IP AFM as well as matching a listener Forwarding_TCP_VS. For this
to occur, the initial connection must pass ACL evaluation.
For detailed information on tcpdump utility, see AskF5 article: SOL411: Overview of packet tracing with the tcpdump utility.
ePVA Acceleration
When using BIG-IP platforms that support ePVA, tcpdump does not show all packets using the default configuration.
If a packet capture is required on the BIG-IP, modify the fastl4 profile to change the offload state or fully disable ePVA as
required. Disabling ePVA acceleration may cause a performance impact, so F5 recommends careful consideration before
making these changes.
99
TROUBLESHOOTING
Troubleshooting using BIG-IQ
To minimize the effect of disabling acceleration when using network listeners, or on heavily utilized network traffic virtual servers,
create a host or network specific listener to apply the changes to for the troubleshooting session.
For detailed information on the ePVA feature, see AskF5 article: SOL12837: Overview of the ePVA feature and AskF5 article:
SOL6546: Recommended methods and limitations for running tcpdump on a BIG-IP system.
BIG-IQ requires TCP 443 and 22 ports bi-directionally to ensure all functionality. When troubleshooting BIG-IQ connectivity
issues on the BIG-IP system, make sure that these ports are open using tools such as SSH, telnet, and curl.
SSH command:
ssh -l root <IP ADDRESS>
SSH example:
ssh l root 192.168.3.158
The authenticity of host 192.168.3.158 (192.168.3.158) cant be
established.
RSA key fingerprint is 6e:35:a8:8c:39:3f:24:90:c1:91:97:a9:3e:ed:23:99.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 192.168.3.158 (RSA) to the list of known
hosts.
Password:
Last login: Wed Mar 30 21:29:54 2016 from 192.168.3.1
[root@big-ip:Active:Standalone] config
Curl command:
curl Ik <IP ADDRESS> | grep HTTP/1.1
100
TROUBLESHOOTING
Troubleshooting using BIG-IQ
REST version
BIG-IQ maintains REST packages on all managed BIG-IPs through an update process that deploys the requisite RPM files from
BIG-IP system to BIG-IP system.
To check the version installed on the BIG-IP using the advanced shell
If the version is not correct, or some packages are incorrect, consider re-deploying the REST framework from BIG-IQ to the
managed BIG-IP system.
For more detailed information on troubleshooting BIG-IQ, refer to the following resource see AskF5 article: SOL16307:
Troubleshooting BIG-IQ device discovery failures.
101
TROUBLESHOOTING
DoS statistics output
REST processes
The BIG-IP REST interface contains several control plane daemons to provide API framework to manage and control the BIG-IP
system. Occasionally the processes may require troubleshooting or restarting to resolve REST communication issues.
For more detailed information on how to restart processes via the advanced shell, see AskF5 article: SOL14736: BIG-IQ
daemons.
Device certificate
BIG-IQ uses the device certificate to communicate to BIG-IP systems. If the BIG-IP device certificate has expired or there is a
problem with the certificate contributing to an authentication issue, a possible solution is to generate a new device certificate on
the BIG-IP, followed by re-discovering the BIG-IP into BIG-IQ.
For more detailed information device certificates, see AskF5 article: SOL15664: Overview of BIG-IP device certificates.
If the network design requires stateful failover of long-lived connections, connection mirroring must be used to maintain the
default stateful inspection behavior of BIG-IP. When failover from an active to standby BIG-IP occurs, mirrored connection table
entries are available for flow lookup and do not require an additional ACL match to be re-applied.
To determine whether or not this process will effect an existing flow, perform a filtered show sys connection command on both
the active and standby BIG-IP. If the connection flow exists on both, the flow is mirrored and failover will be stateful.
For more detailed information on stateful failover using connection mirroring, see AskF5 article: SOL13478: Overview of
connection and persistence mirroring (11.x - 12.x).
Attack Detected toggles from 0 to 1 when a potential attack is detected. This will be reset back to 0 once the attack is
stopped.
Attack Count represents the total number of times the attack was detected for this BIG-IP AFM DoS vector since the
last boot.
The stats_1h_samples counter tracks the number of samples that BIG-IP AFM received in the previous hour. BIG-IP AFM
sampling rate is once per second. Each second it increases by one until an attack is detected.
Once the stats_1h_samples reaches 3600, BIG-IP AFM uses a detection threshold percent value for that DoS vector for attack
detection.
102
TROUBLESHOOTING
IP intelligence
BIG-IP AFM uses the lower value between detection threshold pps and detection threshold percent value for attack
detection when it has an enough number of samples.
Stats: Total number of packets that BIG-IP/AFM-DoS vector received since the last boot.
Stats Rate: Packets that BIG-IP AFM-DoS vector saw in last second.
Stats 1m: Packets that BIG-IP AFM-DoS vector saw in last minute.
Stats 1h: Packets that BIG-IP AFM-DoS vector saw in last hour until attack detected.
Drops: Total number of packets dropped by BIG-IP AFM-DoS vector since the last boot.
Drops Rate: Total number of packets dropped by BIG-IP AFM-DoS vector in the last second.
Drops 1m: Total number of packets dropped by BIG-IP AFM-DoS vector in last minute.
Drops 1h: Total number of packets dropped by BIG-IP AFM-DoS vector in last hour until attack detected.
Int Drops: Total number of packets dropped in hardware since the last boot.
Int Drops Rate: Total number of packets dropped by BIG-IP AFM-DoS vector in hardware in last second.
Int Drops 1m: Total number of packets dropped by BIG-IP AFM-DoS vector in hardware in last minute.
Int Drops 1h: Total number of packets dropped by BIG-IP AFM-DoS vector in hardware in last hour.
Whitelist Hits: Total number of packets that hit the Whitelist for a particular BIG-IP AFM-DoS vector since the last boot.
This includes total packet count that match the Whitelist for that vector both in software and hardware.
IP intelligence
IP intelligence statistics can be viewed from TMSH. TMSH commands show the current status of the IP reputation feed as well as
IP intelligence information.
IP reputation status shows the current timestamp of the IP intelligence update and the address count from the most recent
update.
tmsh show /sys iprep-status
-----------------------------------------------------------------------
Sys::IP Reputation Database Status
-----------------------------------------------------------------------
Last time the server was contacted for updates 03/14/2016 17:41:10
Last time an update was received 03/14/2016 16:50:34
Total number of IP Addresses in the database 8243809
Number of IP Addresses received in the last update 34766
103
TROUBLESHOOTING
IP intelligence
If Last time an update was received returns a value of failed or none verify a BIG-IP DNS resolver is configured.
To show an IP address in the IP Intelligence database using TMSH at the command line
104
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
OPTIMIZE THE SUPPORT EXPERIENCE
F5 technical support commitment
Customers are treated with respect and given every consideration possible.
Some technical support issues arise from configuration errors, either within the BIG-IP system or with other devices in the
network. In other cases, a misunderstanding of BIG-IP capabilities can lead to support questions and issues. Although F5 does
everything possible to prevent defects in BIG-IP hardware and software, these issues may still arise periodically. Regardless of
the root cause of a problem, the goal is to resolve any issues quickly.
The F5 Standard and Premium support offerings include remote assistance from F5 network support engineers, both online and
over the phone.
Premium Plus customers receive priority status at F5, with fast, easy access to a dedicated team of senior-level, F5-certified
network support engineers and a service delivery manager.
Professional services
Take advantage of the full range of F5 consulting services to help you design, customize, and implement a solution that is right for
your IT infrastructure and supports your business goals.
Consulting Services (f5.com/support/professional-services) provides information about a wide range of F5 Professional Services
offerings and Professional Services Partners. You can use our online forms to request Consulting On Demand for custom, shorter
scope consulting engagements, or iRules OnDemand to get fast access to iRules scripts tailored to your specific needs.
106
OPTIMIZE THE SUPPORT EXPERIENCE
F5 certification
You can request specific support services by using either of the following forms:
See F5 GUARDIAN Professional Service Partners (http://f5.com/support/professional-services - guardian) for a complete list of
partners.
F5 certification
F5 Certified exams test the skills and knowledge necessary to be successful when working with todays application delivery
challenges. Our technically relevant and appropriate exams deliver consistently reproducible results that guarantee excellence in
those that achieve certification.
Certification levels
The F5 certification program is progressive with the four levelsAdministrator, Specialist, Expert, and Professionalbuilding on
the skills and knowledge demonstrated on previous exams.
The starting point for all certifications; a certified BIG-IP Administrator has basic network and application knowledge to be
successful in application delivery.
The Technology Specialist certification assures employers that candidates are fully qualified to design, implement, and maintain a
specific product and its advanced features.
The Solution Expert certification focuses on how F5 technologies combine with industry technology to create real-world business
solutions.
The Application Delivery Engineer certification exam and requirements are still under development.
107
OPTIMIZE THE SUPPORT EXPERIENCE
Review self-help options
The Application Delivery Architect certification exam and requirements are still under development.
Certificate expiration
F5 certifications are valid for two (2) years. Three months before the expiration date, the certificate holder becomes eligible for
recertification and can register for the necessary exam. Only the last exam in the highest level certification achieved needs to be
retaken.
Get involved
There are a several ways to get involved with the F5 certification beta program:
Beta participation. Interested in taking our beta exams? Send an email to us at F5Certification@f5.com to learn more.
Exam development. Contact us at F5Certification@f5.com if youre interested in helping us create our F5 Certified exams.
LinkedIn community. Join us on LinkedIn (this link sends you to an external site) for answers to frequently asked questions,
community developed resources, and more.
Downloads
Security Updates
BIG-IP iHealth
TechNews
RSS Feeds
DevCentral
108
OPTIMIZE THE SUPPORT EXPERIENCE
Review self-help options
AskF5
AskF5 (support.f5.com) provides thousands of articles to help you manage your F5 products more effectively, and should be the
first resource you choose when you need support. Step-by-step instructions, downloads, and links to additional resources give
you the means to solve known issues quickly and without delay, and to address potential issues before they become reality.
Whether you want to search the knowledge base to research an issue, or you need the most recent news on your F5 products,
AskF5 is your source for:
General articles
Known issues
Security advisories
Recommended practices
Troubleshooting tips
How-to documents
Changes in behavior
Hotfix information
Downloads
Downloads are available from the F5 website. It is highly recommended that your F5 software is kept up-to-date, including
hotfixes, security updates, OPSWAT updates, and geolocation database updates. All software downloads are available from F5
Downloads (downloads.f5.com).
Security updates
You can receive timely security updates from F5. When remote vulnerabilities are discovered, F5 implements, tests, and releases
security hotfixes for any vulnerable supported version, and sends an email alert to the F5 Security mailing list. F5 encourages
customers with an active support account to subscribe to this list.
109
OPTIMIZE THE SUPPORT EXPERIENCE
Review self-help options
For more detailed information, see AskF5 article: SOL4602: Overview of the F5 security vulnerability response policy.
BIG-IP iHealth
BIG-IP iHealth (iHealth.f5.com) is among the most important preventative tools to verify the proper operation of your BIG-IP
system. This diagnostic viewer ensures that hardware and software are functioning at peak efficiency, and helps detect and
address issues that may potentially affect F5 systems. iHealth is not integrated within the BIG-IP system. This tool is hosted by F5
at iHealth.f5.com and can be accessed with any web browser.
F5 recommends you generate an iHealth qkview file on the BIG-IP system and upload it to iHealth on a weekly basis in order to
benefit from the many regularly occurring diagnostic updates. Uploading qkview files to iHealth also provides F5 technical
support with access to your qkview files if you open a support case.
By reviewing iHealth output, many of the issues commonly experienced by customers can be resolved without the need for
opening a support case with F5.
For more detailed information on running iHealth diagnostics, see BIG-IP iHealth in the TMOS Operations Guide.
TechNews
AskF5 provides two TechNews email publications to help keep administrators up-to-date on various F5 updates and other
offerings:
TechNews Weekly eNewsletters include timely information about known issues, product releases, hotfix releases,
updated and new articles, and new feature notices.
TechNews Notifications are sent any time a product or hotfix is released. This information is also included in the next
weekly TechNews email.
To sign up for the TechNews mailing lists, go to AskF5 Publications Preference Center and select Subscribe: Mailing Lists from
the Self-Help menu. Provide your contact information and select TechNews Weekly Newsletter and/or TechNews
Notifications.
The Recent Additions and Updates list is also published over RSS. You can configure feeds that pertain to specific products,
product versions, and/or document sets. You can also aggregate multiple feeds into your RSS Reader to display one unified list
of all selected documents.
110
OPTIMIZE THE SUPPORT EXPERIENCE
F5 global training services
Go to AskF5 Knowledge Base and select Subscribe: RSS from the Self-Help menu.
For more detailed information, see AskF5 article: SOL9957: Creating a custom RSS feed to view new and updated
documents.
DevCentral
DevCentral (devcentral.f5.com) is an online forum of F5 employees and customers that provides technical documentation,
discussion forums, blogs, and media, related to application delivery networking. DevCentral is a resource for education and
advice on F5 technologies, and is especially helpful for iRules and iApps developers. Access to DevCentral is free, but
registration is required.
Contribute to wikis.
In-person courses
F5 courses are available in multiple training facilities across five continents. Each course combines instructor presentations,
classroom discussions, and interactive labs. The hands-on learning environment helps provide a fast track to accomplishing your
goals.
111
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
F5 Training Programs and Education (f5.com/education/training) provides links to course schedules, pricing, and registration
details. This page also has information about alternative training solutions such as virtual and web-based training for those who
cannot attend training in person. Links to more information are provided in F5 Professional Certification or a non-accredited
Application Delivery Networking Certificate through F5.
Engage support
F5 Technical Support is designed to provide support for specific break-fix issues for customers with active support contracts. For
more information about F5 scope of support, see Support Policies on F5.com.
Online: You can open a support case at the F5 WebSupport Portal. Click Register for an Account to access to the
WebSupport Portal.
By phone: Phone numbers are provided in the General contact numbers section below. It is strongly recommended that
you contact F5 by phone if you have a Sev1 or Sev2 case, as defined in F5 Networks - Support Quick Reference Guide.
Contact numbers
Standard, Premium, and Premium Plus Support customers can open and manage cases by calling one of the contact numbers
listed below.
North America
112
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
Outside North America, Universal Toll-Free: +800 11 ASK 4 F5 or (800 11275 435)
Egypt: 0800-000-0537
Greece: 00-800-11275435
Indonesia: 001-803-657-904
Israel: 972-37630516
Malaysia: 1-800-814994
Philippines: 1-800-1-114-2564
Singapore: 6411-1800
Taiwan: 00-800-11275435
Thailand: 001-800-12-0666763
Vietnam: 120-11585
113
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
Before opening a support case with F5, you should consult the following resources:
Deployment guides and white papers provide information about specific deployment configurations.
AskF5 Knowledge Base provides many articles including known issues, how-to guides, security advisories, release notes,
and general information about products. Many of the issues customers encounter are already documented on AskF5.
BIG-IP iHealth enables customers to upload qkview configuration snapshots in order to verify operation of any BIG-IP
system. Gather information to open a support case.
If your issue cannot be solved using the resources listed, and you need to open a support case, you must first gather several
pieces of important information about your issue. Providing full and accurate information helps speed the path to resolution. The
required information for the majority of situations is summarized below:
The serial number or base registration key of the specific BIG-IP system requiring support.
For more detailed information on finding the serial number, see AskF5 article: SOL917: Finding the serial number or
registration key of your F5 device.
A full description of the issue: A clear problem statement is the best tool in helping to troubleshoot issues. Your description
should include as much of the following information as you can provide.
Occurrences and changes: The date and times of initial and subsequent recurrences. Did this issue arise at
implementation or later? Were there any changes or updates made to the BIG-IP system prior to the issue arising? If so,
what were they?
Symptoms: Ensuring your list of symptoms is as detailed as possible will give more information for support personnel to
correlate with.
Scope of the problem: Note whether the problem is system-wide or limited to a particular configuration feature, service, or
element (such as VLAN, interface, application service, virtual server, pool, and others).
BIG-IP component: The feature, configuration element, or service being used when the problem occurred (such as portal
access, network access, authentication services, VDI, Exchange).
Steps to reproduce: The steps to reproduce the problem as accurately and in as much detail as possible. Include
expected behavior (what should happen) as well as actual behavior (what does happen).
Environment: Current usage of the system. (Is this unit in production? If so, is there currently a workaround in place?)
114
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
Changes: System changes made immediately prior to the problems first occurrence. This may include upgrades,
hardware changes, network maintenance, and other upgrades. Have any changes been made to resolve the problem? If
so, what were they?
Issue Severity: A description of the impact the issue is having on your site or Case severity
Severity 1: Software or hardware conditions on your F5 device are preventing the execution of critical business
activities. The device will not power up or is not passing traffic.
Severity 2: Software or hardware conditions on your F5 device are preventing or significantly impairing high-level
commerce or business activities.
Severity 3: Software or hardware conditions on your F5 device are creating degradation of service or functionality in
normal business or commerce activities.
Severity 4: Questions regarding configurations (how to), troubleshooting non-critical issues, or requests for product
functionality that are not part of the current product feature set.
Contact and availability information including alternate contacts authorized to work on the problem with F5 Technical
Support. When there are more personnel available to work with F5 Technical Support, the resolution of your issue may be
expedited.
A qkview file obtained while problem symptoms are manifesting. A qkview file of the system before the occurrence is also
useful. F5 recommends archiving qkview files regularly. For more information, see BIG-IP iHealth in the TMOS Operations
Guide.
Platform and system. Version and provisioned software modules of the affected system.
To locate platform and system information using tmsh from the command line
115
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
Highlight and copy the output information and include it with your support case.
116
OPTIMIZE THE SUPPORT EXPERIENCE
Engage support
Highlight and copy the output information and include it with your support case.
Use of the WebSupport Portal requires a current support contract and registration on the F5 website (login.f5.com).
Once registered, you will receive an email within 24 hours letting you know your account has been enabled with WebSupport
Portal access.
1. Go to F5 WebSupport Portal.
4. Complete the Contact information portion of the page and then select I have a support contract and
need access to WebSupport.
117
OPTIMIZE THE SUPPORT EXPERIENCE
Send information to Support
For example, if joeuser@example.com has opened ticket C123456, he would log in to the dropbox.f5.com site using the following
information:
Username: C123456
Password: joeuser@example.com
If joeuser@example.com has opened ticket 1-12345678, he would log in to the dropbox.f5.com site using the following
information:
Username: 1-12345678
Password: joeuser@example.com
118
OPTIMIZE THE SUPPORT EXPERIENCE
Send information to Support
For more detailed information regarding uploading and downloading files using dropbox.f5.com, see AskF5 article: SOL2486:
Providing files to F5 Technical Support.
Example
In the following example, a Google Chrome browser is used to save a single HTTP request into a cURL command and the entire
transaction into an HAR archive file.
4. Copy as cURL.
119
OPTIMIZE THE SUPPORT EXPERIENCE
Send information to Support
SOL6546: Recommended methods and limitations for running tcpdump on a BIG-IP system
SOL7227: Considerations when using the tcpdump utility with tagged VLAN traffic
For more detailed information on UCS archives, see AskF5 article: SOL4423: Overview of UCS archives.
120
OPTIMIZE THE SUPPORT EXPERIENCE
Send information to Support
121
Help improve this guide
Please help F5 improve this guide by responding to a few questions about this chapter.
(Note: You must be viewing this document in Adobe Acrobat Reader or similar to use this form.)
Did this chapter answer all of your questions about the subject matter? Yes No
Did you find any errors pertaining to subject matter in this chapter? Yes No
If yes, please copy and paste the paragraph containing the error here:
Did you find non-subject-matter errors in this chapter (spelling, etc.)? Yes No
If yes, please copy and paste the paragraph containing the error here:
Send
LEGAL NOTICES
Send information to Support
Legal Notices
Publication date
This document was published on July 2016.
Copyright
Copyright 2013-2016, F5 Networks, Inc. All rights reserved.
F5 Networks, Inc. (F5) believes the information it furnishes to be accurate and reliable. However, F5 assumes no responsibility for
the use of this information, nor any infringement of patents or other rights of third parties which may result from its use. No
license is granted by implication or otherwise under any patent, copyright, or other intellectual property right of F5 except as
specifically described by applicable user licenses. F5 reserves the right to change specifications at any time without notice.
Trademarks
AAM, Access Policy Manager, Advanced Client Authentication, Advanced Firewall Manager, Advanced Routing, AFM, APM,
Application Acceleration Manager, Application Security Manager, ARX, AskF5, ASM, BIG-IP, BIG-IQ, Cloud Extender,
CloudFucious, Cloud Manager, Clustered Multiprocessing, CMP, COHESION, Data Manager, DevCentral, DevCentral [DESIGN],
DNS Express, DSC, DSI, Edge Client, Edge Gateway, EdgePortal, ELEVATE, EM, Enterprise Manager, ENGAGE, F5, F5[DESIGN],
F5 Certified [DESIGN], F5 Networks, F5SalesXchange [DESIGN], F5Synthesis, f5Synthesis, F5Synthesis[DESIGN], F5
TechXchange [DESIGN], Fast Application Proxy, Fast Cache, FirePass, Global Traffic Manager, GTM, GUARDIAN, iApps, IBR,
Intelligent Browser Referencing, Intelligent Compression, IPv6 Gateway, iControl, iHealth, iQuery, iRules, iRules OnDemand,
iSession, L7 RateShaping, LC, Link Controller, Local Traffic Manager, LTM, LineRate, LineRate Systems [DESIGN], LROS, LTM,
Message Security Manager, MSM, OneConnect, Packet Velocity, PEM, Policy Enforcement Manager, Protocol Security Manager,
PSM, Real Traffic Policy Builder, SalesXchange, ScaleN, Signalling Delivery Controller, SDC, SSL Acceleration, software designed
applications services, SDAC (except in Japan), StrongBox, SuperVIP, SYN Check, TCP Express, TDR, TechXchange, TMOS,
TotALL, Traffic Management Operating System, Traffix Systems, Traffix Systems (DESIGN), Transparent Data Reduction, UNITY,
VAULT, vCMP, VE F5 [DESIGN], Versafe, Versafe [DESIGN], VIPRION, Virtual Clustered Multiprocessing, WebSafe, and
ZoneRunner, are trademarks or service marks of F5 Networks, Inc., in the U.S. and other countries, and may not be used without
express written consent.
All other product and company names herein may be trademarks of their respective owners.
Patents
This product may be protected by one or more patents. See the F5 Patents page (http://www.f5.com/about/guidelines-policies/
patents).
123
LEGAL NOTICES
Notice
Notice
THE SOFTWARE, SCRIPTING, AND COMMAND EXAMPLES ARE PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE, SCRIPTING AND COMMAND EXAMPLES, OR THE
USE OR OTHER DEALINGS WITH THE SOFTWARE, SCRIPTING, AND COMMAND EXAMPLES.
124