When troubleshooting an issue on any device where network connectivity is impacted, a packet capture is still king. This is not to say every 403 Forbidden error needs to be examined in Wireshark, but when a user reports their browsing saying the connection was reset or a timeout was hit, it might be time to fire up tcpdump.

My focus in this article will be the normal tcpdump, but for F5 BIG-IP devices certain extra flags can be enabled which will enormously improve your troubleshooting experience. This is by no stretch of the imagination a full explanation of all tcpdump flags, but simply the most common use cases.

Basic connectivity check

You have the customer on the line, they say nothing is passing through, you’re not familiar with this exact setup and you want to see if this is visible on the network level: this is why we have -q (quick/quiet).

tcpdump -qeni any host <ip of user or destination host here>

A quick overview of the options used here:
-q Quick mode, only show timestamp, IP addresses, port numbers and packet size

-e Shows ethernet level headers (MAC addresses), extremely useful to verify where traffic is originating from on L2.

-n No resolving of IP addresses or services

-i <interface> Capture on the provided interface. Any is acceptable on devices without a ton of traffic, otherwise specify the interface.

host <ip> A simple filter. This matches the IP address in both source and destination.

Capturing for the purposes of reviewing in Wireshark

If a more in-depth inspection is required, you’ll want to write the capture to a file (.cap/.pcap) that can be opened in Wireshark. 

tcpdump -v -s 0 -i eth0 -w ~/descriptive_name.pcap host <ip> and port <port>

In addition to the previously explained flags, following options are vital for a decent analysis Wireshark:

-s 0 Snaplength. By default tcpdump will only capture 68 bytes of data per packet, this will make your capture look extremely puzzling. It will feel like you’re hitting some weird packet loss or MTU issues.

-w <filename> Self-descriptive, write to the specified file. This can be retrieved typically using SFTP or SCP.

-v Verbose. This might seem non-intuitive at first as you’re not really looking for any extra information to be printed to CLI, but this makes tcpdump show you how much packets have been captured in realtime. 

F5 BIG-IP specifics

F5 has modified tcpdump to add something that you’ll never want to go without again on other devices. This command is only available from bash (“advanced shell”), not from tmsh.

tcpdump -i 0.0:nnnp -w /var/tmp/awesomecapture.pcap -v host x.x.x.x

You’ll notice these flags aren’t added by dashes but on the capture interface itself. The :nnn means you want in depth analysis of which Virtual Server was used, original IP information, TCP RST analysis, … These fields become visible in Wireshark 2.6.0 automatically, you’ll need a plugin in older versions. You can use fewer n flags but in general I find there is no need.

The p flag automatically adds the matching back-end flow, as well as internal traffic towards for example the APM module. This means you can get both the external encrypted flow as well as the internal decrypted flow for example, from just a client source address. Using these flags will be an enormous help in troubleshooting F5 issues.